[sugar] Cairo tile engine, and accessing 565 buffers from cairo and C

[Dan Williams]

On Mon, 2007-04-16 at 10:47 -0700, Carl Worth wrote:
> On Sun, 15 Apr 2007 01:30:55 -0400, Dan Williams wrote:
> > On Sat, 2007-04-14 at 17:10 -0700, Don Hopkins wrote:
> > > I've read on the mailing list that Cairo supports 565 ("begrudgingly", 
> > > whatever that implies).
> > 
> > It means that cairo supports 565 only as a compatibility option, but
> > will _not_ allow you to create new surfaces as 565.  Basically, if you
> > hand it a surface or pixel data that is in 565 format, it can use that
> > data.  But you cannot create 565 surfaces manually.  Cairo's native
> > format is 32 bit RGB with alpha.
> 
> I don't know where these rumors got started, but there's a bunch of
> misinformation in the above. Apparently I've done a poor job of
> explaining things somewhere, so I'm hoping I can straighten some of it
> out now.

Well, there's a slight omission on my part, but it's not really
misinformation.  I don't expect cairo to support all the bit depths and
image formats that exist.  But the problem is that Cairo just doesn't
have very good support for 16bpp right now _with_alpha_.

For starters, from cairo-deprecated.h:


So you can certainly get 16bpp 565 surfaces manually as a _side_ effect
of create_similar on a 16bpp Xlib window if X is running in 16bpp
mode.  

But here's where the problems start...  Given a 16bpp 565 xlib surface:

    source = cairo_surface_create_similar (target,
			CAIRO_CONTENT_COLOR,
			width, height);

gives you a 565 surface.

    source = cairo_surface_create_similar (target,
			CAIRO_CONTENT_COLOR_ALPHA,
			width, height);

gives you a 32-bit ARGB surface.

I can understand why; but in our case it's undesirable.  We'd rather
have 16bpp, but _some_ kind of alpha.  Like fontconfig property binding
with strong and weak.  The current code ensures that all the colors
possible in the target are possible in 'source', but we'd rather keep
the bit depth at 16bpp and get 1 or 4 bits of alpha rather than step up
to 32bpp.

There are a few issues here.

1) Since we're running in 565 in X, obviously pixel smashing has to
happen if there's any alpha in the pixmaps we're sending to X.  And
that's what's killing us right now.

2) Even if cairo did hand back a 1555 or 4444 surface, we'd still have
to composite it and smash it down to 565.  And pixman doesn't have
MMX-accelerated code for that anyway.

3) Maybe we should be running in 1555 or 4444 mode instead of 565, so
that we could get some native alpha without having to smash pixels.  But
of course we don't have MMX acceleration in pixman for these formats
either.

Ideally we could ask Cairo to create a surface of _any_ type we want,
either 1555, 4444, 0565, 8888, 0888, 0444, etc.  But cairo can't do that
AFAIK.  That's why I said "grudgingly supports 565" and "it's native
format is really 32bpp".  When I say "manually create" a surface, I mean
telling cairo _exactly_ what format of surface you want.

> Here are some facts about cairo and the support it has for X visuals
> of various depths and image formats of various depths.
> 
> First, understand that cairo provides different backends that allow
> the user to explicitly create a surface with one backend or
> another. Cairo also provides a way to create what it calls a "similar"
> surface, where the application does not explicitly select the backend
> surface type, but just passes an existing surface and asks cairo to
> create a surface of the most suitable type.
> 
> I think the two cairo backends of interest in the current discussion
> are the "xlib" and "image" backends. The xlib backend is used for
> drawing to an X11 Drawable (a Window or Pixmap). The image backend is
> used for drawing to a buffer of data in the application's memory
> space. I'll discuss the capabilities of each, and then the issues of
> using the "create_similar" interface.
> 
> xlib
> ----
> For the xlib backend, if you have the X Render extension in your X
> server, then you can use cairo to draw to an X11 Drawable of any depth
> supported by the X server. You can do this by passing either an
> XRenderPictFormat or a Visual describing the desired depth to one of
> the following functions:
> 
> 	cairo_xlib_surface_create_with_xrender_format
> 
> 	cairo_xlib_surface_create
> 
> So, on the OLPC, where you definitely have a server that supports the
> Render extension, you should have no problem using cairo to draw to an
> X11 Drawable with a 565 format.
> 
> (Note: There are known bugs in cairo in drawing to an X server without
> the Render extension and targeting an xlib surface with a format that
> does not also exist as a supported format in the image backend. These
> bugs should not affect OLPC as the Render extension should be present
> in the X server. And, we are working to eliminate these bugs before
> cairo 1.6 is released.)
> 
> image
> -----
> The image backend (cairo_image_surface_create) is used when the
> application wants to read/write to a buffer of data directly as well
> as drawing to it with cairo. If this isn't the case, this backend
> really should not be selected by the application.
> 
> The image backend does currently support only a very limited set of
> formats, (ARGB32 and RGB24), both of which are 32-bits-per-pixel
> formats.
> 
> It's conceptually possible to extend the list of formats supported by
> the image backend, but not very appealing until we have something
> besides humans to generate the combinatorial explosion of code needed
> to handle each format.
> 
> In the meantime, if you are really concerned about the performance
> when targeting an xlib surface, then you should really be using xlib
> surfaces for your source data, not image surfaces.
> 
> create_similar
> --------------
> In addition to the cairo_<backend>_surface_create functions discussed
> above which involve the application explicitly selecting a backend to
> use, cairo provides one additional interface:
> 
> 	cairo_surface_create_similar
> 
> With this interface, the application is not selecting the backend to
> use, but instead cairo will select one for the application to best
> meet its needs. The user expresses these needs by passing both an
> existing surface and a description of the kind of "content" that the
> new surface will hold, (whether color only, alpha only, or both color
> and alpha). And cairo does the best it can to meet these needs.
> 
> I haven't seen details of the code that is being worked with here, but
> I'm guessing that if 32-bit image surfaces are being generated in
> undesirable situations, it might be through this interface. I'm
> guessing that based on a statement such as:
> 
> > > But the surfaces it's handing my C code are 32 bit (RGB or ARGB).
> 
> That doesn't sound like somebody directly passed CAIRO_FORMAT_ARGB32
> or CAIRO_FORMAT_RGB24 to cairo_image_surface_create, (in which case
> the result wouldn't be surprising).
> 
> So, let's imagine the user has an X11 Drawable with a 565 visual, and
> has done cairo_xlib_surface_create to create a surface named "target"
> from that. Then, let's imagine the user wants to create an
> intermediate surface that will be used as a source to draw to this
> target.
> 
> If the user does not need to store any alpha content to the surface,
> then the user would call:
> 
>     source = cairo_surface_create_similar (target,
> 					   CAIRO_CONTENT_COLOR,
> 					   width, height);
> 
> And in this case, I fully expect cairo to return a new xlib surface
> based on a Drawable with a 565 visual. If you're seeing anything else
> in a situation like this, please let me know so that we can fix it
> right away.
> 
> Now, if the user actually needs to store alpha and color to the source
> surface, the user would call:
> 
>     source = cairo_surface_create_similar (target,
> 					   CAIRO_CONTENT_COLOR_ALPHA,
> 					   width, height);
> 
> And what should happen in this case?
> 
> From the problem reports, I'm guessing that this is the problematic
> code path, and that the result being returned here is an xlib surface
> with a 32-bit depth visual. If so, then drawing with this source to
> the original target will involve the X server doing 8888 -> 565
> conversion.
> 
> What other option is there? Is there some other visual that the X
> server provides that can store color and alpha and would work more
> efficiently with a 565 target surface? If so, it should be a trivial
> matter to fix cairo_surface_create_similar to use that visual in the
> surface that it creates. Please let me know.
> 
> > > How can I get ahold of the actual 16 bit 565 buffer that X can directly 
> > > and efficiently draw on the screen?
> 
> Now, this looks like a totally separate question. If you've got an X11
> Drawable with a 565 visual, then any 16-bit buffer that exists is
> accessible by the X server, and not by cairo, (which is client-side by
> definition).
> 
> I don't know if the X server has any means to allow the client to get
> access to that buffer, (in general it might exist in video-card memory
> and not be readily accessible by the CPU anyway). Maybe some of the X
> experts hacking on the OLPC know some trick, (XShm or something?). But
> regardless, tricks like that would be outside the scope of cairo
> anyway.
> 
> Meanwhile, I still don't completely understand what problem it is that
> you are trying to solve by imagining a cairo surface that allows both
> the application and the X server do have direct access to a common
> buffer. If you need direct-application access to modify the buffer,
> then wouldn't you be happy controlling its contents yourself? That is,
> do you really need cairo to be able to draw to that same buffer as
> well? If not, perhaps something outside the scope of cairo would suit
> you better anyway?
> 
> I'd be glad to answer further questions on these topics, and help out
> wherever possible, (including augmenting cairo as needed). But I
> definitely would need some help understanding the problems better,
> (because so far I haven't understood anything that could be changed in
> cairo to help with any of the problems being described).
> 
> Thanks,
> 
> -Carl