The "objects and translators" model, as described, does not have any easy way to retar a bundle; nor is it easy to figure out how to assign different mime types to different "kinds" of tar files (this one is a web page, that one is an activity bundle). Note that the "rebundle" problem is not so trivial - what do you do with the metadata on the subfiles? Let me try to sketch out the problem of bundles, with everything a good solution should enable. 1. In journal, the "most default" view shows only bundles, is not cluttered with all subfiles. 2. Bundles have their own metadata, and a mime-type that reflects their holistic nature (a set of photos, a web page, an activity) 3. Subfiles inherit (some?) bundle metadata, but can also have their own metadata. 4. Some subfiles, which make no sense on their own, are hidden by default (dotted files) 5. Possible to grab a piece of a bundle and start treating it as a separate top-level object. 6. Possible to edit a piece of a bundle as if it were a top-level object, yet keep it as part of the bundle. This creates a new version of the whole bundle, with the edited file. 7. Possible to take a top-level object and add it to a bundle. 8. Possible to remove (delete) an object from a bundle. 9. Possible to import/ export bundles as single files (zip/tar) to external media/over the net 10. Possible to import/export bundles as directories to external media. 11. For easier porting of legacy apps: ability to open a subfile in context, with the directory structure of the bundle extracted, so that relative paths work. ... I think it is clear from the above list that moving back and forth from bundle to file view should be as transparent as possible. Internal storage could be as separate files or as zipped/tarred groups; either way, there would need to be a storage of stubs/pointers to stand in for the other model. Here's one proposal to kick things off: External format: metadata for a traditional file is represented on external media by an invisible file in the same directory, with a related name, which contains json. bundles are represented on external media by a zip, which politely extracts itself to a subdirectory. metadata for bundles is represented by an invisible file next to the zip. This file is a zip which extracts to the same subdirectory, putting global metadata in an invisible file in the root, and subfile metadata in dotfiles as above. Internal format: Same as external format. Glucose knows how to unpack and repack the bundles. There is no pretense that Glucose maintains any integrity/signatures for the bundles. When editing as in option 6 above, the bundle is extracted (see option 11 above) and then repacked on *each save* of the subfile. Each non-hidden subfile is also represented by a metadata-only datastore pointer into the bundle, for journal searches.   Note: this proposal is less than ideal in some ways. For one instance: small changes to large files inside a zip are not good for delta-based storage. For another, zip has no way to include empty directories, which might be important for directory structure in some legacy cases. It assumes relatively large atomic saves in any subfile. Jameson <div class="gmail_quote">On Wed, Jun 11, 2008 at 9:37 AM, Benjamin M. Schwartz <<a href="mailto:bmschwar@fas.harvard.edu">bmschwar@fas.harvard.edu</a>> wrote: <blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I have a proposal for how to manage objects in the datastore, to resolve the various issues we have discussed.  I call it the "Objects and Translators" model.  The fundamental idea is this: the datastore is a flat, non-hierarchical listing of objects (versioned, but that does not come into play here).  The datastore provides these objects for use by Activities.  However, the datastore (or perhaps the Journal) also contains a new element: Translators. A Translator is a non-interactive object processing machine that takes an object as input and returns a set of objects as output.  Each Translator has a specified set of input types on which it can act.  The user, having selected a particular object, should be provided with a list of all compatible Translators.  Clicking on a Translator should show the list of output objects that Translator would generate for this input object, and the user may repeat this process on each of those objects as well. This model allows us to reproduce the usual sorts of directory hierarchies, simply by using a group object (I think of it as a .tar file) and an untar Translator.  To the user, this looks very much like a directory hierarchy.  However, because each "directory" is in fact a first-class object, it can also be associated with Activity instances, or passed from one XO to another over the network, etc. The Translator system is appealing to me because it can provide far more than a simple storage hierarchy.  For example, I have often had to extract a particular image from a PDF file, in order to include it in a class project.  I, and most people, have done this by blowing up the image and performing "Print Screen", or perhaps opening the PDF in the GIMP, which rasterizes it.  However, there is a much better solution, provided by xpdf's command-line utils, which allows one to split a PDF into its component images and text, losslessly. Almost nobody uses these tools, because they are command-line only, and obscure.  It would be trivial to wrap them up into a Translator, though, and then every PDF, in addition to showing options for viewing in Read, would show an option for splitting into it components, for further editing.  Nothing could be more in keeping with our goal of creating an editable world. You can easily think of many other interesting Translators.  For example, one could also provide a PDF->PNG rasterizer, a ODT->PDF renderer, or a Theora->JPG frames decompressor.  The APIs should be designed so that Translators can be lazy, computing the contents of output objects only when necessary. Translators provide tremendously powerful object management, with a simple path to implementation. - --Ben -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - <a href="http://enigmail.mozdev.org" target="_blank">http://enigmail.mozdev.org</a> iEYEARECAAYFAkhP8SkACgkQUJT6e6HFtqRQrACfcd5ntZ1ZXovZtTVI5k4LEG96 oDwAnjswhuiV1uKJ5cWhVV9N6uFNgT6E =x7ie -----END PGP SIGNATURE----- <div><div></div><div class="Wj3C7c">_______________________________________________ Sugar mailing list <a href="mailto:Sugar@lists.laptop.org">Sugar@lists.laptop.org</a> <a href="http://lists.laptop.org/listinfo/sugar" target="_blank">http://lists.laptop.org/listinfo/sugar</a> </div></div></blockquote></div>