[Sugar-devel] GSoC Proposal: Multimedia Broadcasting

[Benjamin M. Schwartz]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Geza Kovacs wrote:
> Actually, during the fall semester I had access to a few XOs from the
> Media Lab, and happened to try this very thing out (use gstreamer to
> capture video from webcam and stream it live to an icecast server). With
> VGA resolution with Theora, I was indeed getting horrible framerates,
> roughly 0.5 frames per second. At QVGA, framerates improved
> considerably, going around 10 frames per second; however I suppose QVGA
> is too low-resolution to be acceptable. I had slightly better
> performance with some older codecs, I believe either MPEG-1 or MJPEG,
> with VGA streaming at roughly 3 fps. I didn't try out H.261 or H.263,
> but I believe those should also be much less expensive than Theora to
> encode and decode, as they're used quite extensively in embedded
> devices. 

They're used quite extensively in embedded devices because those devices
have special purpose hardware to enc/dec these codecs.  That doesn't help
us here.

> In this case, since all traffic is going through the local
> wireless network, bandwidth isn't the primary bottleneck, but instead
> encoding and decoding video is, therefore using a less cpu-intensive
> codec is likely to be advantageous; I believe that if I use the
> appropriate encoding options and codec combinations to make compression
> as fast as possible then I should be able to stream VGA video at
> acceptable frame rates.

Perhaps it might be possible using MJPEG... but VNC is substantially more
efficient than MJPEG in both CPU and bandwidth.

> Since VNC itself, unlike X11, uses a
> raster-based format, I don't believe it's going to be any more efficient
> CPU-wise than a very primitive video codec like MJPEG or MPEG-1 at
> fastest compression settings in any context, so I don't believe I should
> even bother with VNC.

I strongly feel that you are wrong here.  VNC is actually quite
sophisticated.  No less than Adam Jackson, the Red Hat X maintainer, told
me that VNC's network protocol is substantially more efficient than the
X11 protocol, and my experience agrees.

For example,

x11vnc -shared -viewonly -xdamage -scale 1/2 -scrollcopyrect -cursorshape
- -cursorpos -ncache 2

Will: create a VNC session with an unlimited number of read-only
participants; use XDAMAGE to receive notifications when screen regions
change, so that only changes need to be read out, encoded, and
transmitted; scale the screen down by a factor of 2 (to VGA on an XO); use
CopyRect commands to avoid resending the entire area when windows are
scrolled; transmit the cursor shape and position out-of-band, so that
almost no data must be sent when the cursor moves; and tell the client to
allocate two screens-worth of offscreen cache, so that if the session
switches between two windows, they can both be cached locally, instead of
constantly being sent over the network.

VNC also uses JPEG compression by default.  It's like MJPEG, but with a
huge number of optimizations specific to the problem of transmitting the
contents of a screen.

For more info, check out
http://www.karlrunge.com/x11vnc/faq.html#faq-slow-link

The biggest difficulty with using VNC here is that you really want
link-local multicast.  The closest thing I've found to a Multicast VNC is
http://teleteaching.uni-trier.de/ttt.en.html ... which actually sounds
like it might be a great place for you to start.

- --Ben
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10 (GNU/Linux)

iEYEARECAAYFAknWI1oACgkQUJT6e6HFtqQHywCeMFKhTDMymmqgZUGqWccKpntI
FlUAn27B/D8MruNDyyWmGPQ4oBaxYs/m
=YDAX
-----END PGP SIGNATURE-----