Python and webcam capture delay?

[jack catcher (nick)]

Hi,

I'm thinking of using Python for capturing and showing live webcam
stream simultaneously between two computers via local area network.
Operating system is Windows. I'm going to begin with VideoCapture
extension, no ideas about other implementation yet. Do you have any
suggestions on how short delay I should hope to achieve in showing the
video? This would be part of a psychological experiment, so I would need
to deliver the video stream with a reasonable delay (say, below 100ms).

 

[jack catcher (nick)]

Tim Roberts kirjoitti:

You need to do the math on this. Remember that a full 640x480 RGB stream
at 30 frames per second runs 28 megabytes per second. That's more than
twice what a 100 megabit network can pump.

You can probably use Python to oversee this, but you might want to consider
using lower-level code to control the actual hardware. If you are
targeting Windows, for example, you could write a DirectShow graph to pump
into a renderer that transmits out to a network, then another graph to
receive from the network and display it.

You can manage the network latency by adding a delays in the local graph.

Thanks Tim, you're correct about the math. What is your main point about
DirectShow: that it is generally faster and more reliable than doing the
job high-level, or that one could use coding/decoding in DirectShow to
speed up the transmission? I think the latter would be a great idea if
the latency were tolerable. On the other hand, I'd like to keep things
simple and do all the programming in Python. I've got no experience with
DirectShow, but I guess the filters need to be programmed in C++ and
called from Python?

Another option might be to use resolution 320x240@15fps.

 

[Stef Mientki]

Hi,

I'm thinking of using Python for capturing and showing live webcam
stream simultaneously between two computers via local area network.
Operating system is Windows. I'm going to begin with VideoCapture
extension, no ideas about other implementation yet. Do you have any
suggestions on how short delay I should hope to achieve in showing the
video? This would be part of a psychological experiment, so I would
need to deliver the video stream with a reasonable delay (say, below
100ms).

I would first check if video capture extension works anyway.
I couldn't get it working, ...
.... it seems to start ok
.... but moving the captured window,
.... or sometimes even just moving the mouse hangs the program :-(

So I'm still looking for a good / open source workiing video capture in
Python.
I can make a good working capture in delphi,
make a DLL or ActiveX from that and use it in Python,
but I'm not allowed to ditribute these.

cheers,
Stef

 

[Rhodri James]

Tim Roberts kirjoitti:

Thanks Tim, you're correct about the math. What is your main point about
DirectShow: that it is generally faster and more reliable than doing the
job high-level, or that one could use coding/decoding in DirectShow to
speed up the transmission? I think the latter would be a great idea if
the latency were tolerable. On the other hand, I'd like to keep things
simple and do all the programming in Python. I've got no experience with
DirectShow, but I guess the filters need to be programmed in C++ and
called from Python?

Another option might be to use resolution 320x240@15fps.

Does the webcam just deliver frames, or are you getting frames out of
a decoder layer? If it's the latter, you want to distribute the encoded
video, which should be much lower bandwidth. Exactly how you do that
depends a bit on what format the webcam claims to deliver.

 

[jack catcher (nick)]

Rhodri James kirjoitti:

Does the webcam just deliver frames, or are you getting frames out of
a decoder layer? If it's the latter, you want to distribute the encoded
video, which should be much lower bandwidth. Exactly how you do that
depends a bit on what format the webcam claims to deliver.

Well, getting already encoded video from the webcam sounds almost like a
free lunch (which it probably is not). At least I wouldn't want to get
too long a delay because of the encoding.

I haven't got the webcam(s) yet, and I guess I can basically purchase
any ones I find suitable for getting the job done. Any recommendations?

 

[Nobody]

Well, getting already encoded video from the webcam sounds almost like a
free lunch (which it probably is not). At least I wouldn't want to get
too long a delay because of the encoding.

I haven't got the webcam(s) yet, and I guess I can basically purchase
any ones I find suitable for getting the job done. Any recommendations?

The webcam is bound to do some encoding; most of them use USB "full speed"
(12Mbit/sec), which isn't enough for raw 640x480x24bpp@30fps data.

Chroma subsampling and JPEG compression will both reduce the bandwidth
without introducing noticable latency (the compression time will be no
more than one frame).

Temporal compression will reduce the bandwidth further. Using B-frames
(frames which contain the differences from a predicted frame which is
based upon both previous and subsequent frames) will provide more
compression, but increases the latency significantly. For this reason, the
compression built into video cameras normally only uses P-frames (frames
which contain the differences from a predicted frame which is based only
upon previous frames).

The biggest issue is likely to be finding latency specifications, followed
by finding a camera which meets your latency requirement.

Also, any "read frame, write frame" loops will add an additional frame of
latency, as you can't start writing the first byte of the frame until
after you've read the last byte of the frame. Video APIs which let you get
rows as they're decoded are rare.

 

[Rhodri James]

Rhodri James kirjoitti:

Well, getting already encoded video from the webcam sounds almost like a
free lunch (which it probably is not). At least I wouldn't want to get
too long a delay because of the encoding.

Not so unlikely as you might think, since there are very basic M-JPEG and
MPEG-2 on-chip encoders available, and the webcam will have to do some
compression to get the video data into the computer. USB is not as fast
as it would like to pretend.

I haven't got the webcam(s) yet, and I guess I can basically purchase
any ones I find suitable for getting the job done. Any recommendations?

Sorry, no. I'm used to getting my video feeds down fibre-optic cables

 

[jack catcher (nick)]

Nobody kirjoitti:

The webcam is bound to do some encoding; most of them use USB "full speed"
(12Mbit/sec), which isn't enough for raw 640x480x24bpp@30fps data.

Chroma subsampling and JPEG compression will both reduce the bandwidth
without introducing noticable latency (the compression time will be no
more than one frame).

Temporal compression will reduce the bandwidth further. Using B-frames
(frames which contain the differences from a predicted frame which is
based upon both previous and subsequent frames) will provide more
compression, but increases the latency significantly. For this reason, the
compression built into video cameras normally only uses P-frames (frames
which contain the differences from a predicted frame which is based only
upon previous frames).

The biggest issue is likely to be finding latency specifications, followed
by finding a camera which meets your latency requirement.

Thanks for the comments. Unfortunately, such specifications aren't easy
to find, even in reviews. Fortunately several newer webcams seem at
least to use usb2.

Regarding Python, I'll have to check whether the ImageCapture extension
works at all, as someone suggested earlier, and the possibility of using
DirectShow. I'm still considering Matlab as an option for Python here.

 

[Nobody]

Thanks for the comments. Unfortunately, such specifications aren't easy
to find, even in reviews. Fortunately several newer webcams seem at
least to use usb2.

Supporting USB-2 doesn't mean that the camera necessarily uses high-speed
(480Mbit/sec).

AFAIK, the only real difference between "USB-1 conformant" and "USB-2
conformant" is that the latter actually passed a test of its ability to
say "no, I can't do high-speed", while the former didn't. The check for
high-speed capability was designed such that it shouldn't cause problems
for USB-1 devices, but individual USB-1 devices weren't actually tested
for this.

 

[Dennis Lee Bieber]

AFAIK, the only real difference between "USB-1 conformant" and "USB-2
conformant" is that the latter actually passed a test of its ability to
say "no, I can't do high-speed", while the former didn't. The check for
high-speed capability was designed such that it shouldn't cause problems
for USB-1 devices, but individual USB-1 devices weren't actually tested
for this.

USB-1 or USB-1.1? As I recall the latter is capable of 11Mbps
whereas the original USB spec was much lower... (of course, USB 2 at top
is 480Mbps)

--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Nobody]

USB-1 or USB-1.1? As I recall the latter is capable of 11Mbps
whereas the original USB spec was much lower... (of course, USB 2 at top
is 480Mbps)

USB 1.0 supports 1.5Mbps (low-speed) and 12MBps (full-speed). Hosts and
hubs must support both, functions (devices) can use either. USB 1.1 was
a bug-fix release which solved some interoperability issues arising from
ambiguities in the USB 1.0 standard.

 

[Nobody]

That's not true. Most of the web cams made in the last 5 years or so run
at high speed, 480 Mbps. Full speed only gets you 1 fps at 640x480
uncompressed, so it's really only useful for the most primitive video
conference cams.

The very earliest models typically only did 320x200, while later USB-1
webcams used onboard compression to get a decent framerate.

For internet use, 12Mbps isn't that much of an obstacle; the internet
connection's upload speed is more likely to be the limiting factor. Faster
speeds are more useful for things like LAN-based CCTV systems.