[sugar] Datastore and Nautilus, an amusing ponder

[Benjamin M. Schwartz]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


It seems to me that the points of controversy at Scott's datastore talk
primarily concerned the way data is stored.  In particular, the Journal
requires the storage of a certain amount of metadata about each file, and
the way we store this metadata constrains how we can store the data.

The metadata we wish to store takes several forms, including a preview, a
comment, and tags.  As I thought about these properties I remembered that
Nautilus has some curious little-known features, and so I fired up
Nautilus and clicked "Properties" from the drop-down menu on the first
file I found.  Sure enough, Nautilus provides both tagging and commenting
for every file on the system.  The tags are called "Emblems" (or
"keywords" in the code).  They are unfortunately limited to a fixed set by
the UI, though these helpfully come with distinctive icons.  The comments
are called "Notes".  Also, Nautilus keeps thumbnails according to the
Thumbnail Managing Standard (http://jens.triq.net/thumbnail-spec/index.html).

The metadata is stored in the ~/.nautilus/metafiles directory.  That
directory has one XML file for each subdirectory in which a file has been
tagged.  Each file contains a list of entries, and each entry encodes the
complete metadata for one file.  Also, thumbnails are stored in
~/.thumbnails by the MD5 hash of the filename.

With the exception of versioning, it seems to me that the Journal could
easily use Nautilus's storage system for Object metadata.  This might be
attractive to those who value the reuse of widely-used techniques.
However, it is worth noting that Nautilus's Emblems and Notes are so
little used that their wide deployment does not provide a strong guarantee
about their robustness.

Using ~/.nautilus/metafiles could potentially provide us with a very
distinctive, if bizarre, advantage.  When Nautilus moves a file from one
directory to another, it preserves metadata.  If the Datastore works by
simply indexing the (recursive) contents of ~/Desktop according to the
Nautilus metadata, then Nautilus and the Journal can happily coexist,
looking at the same underlying filesystem.  Users could spend most of
their time in the Journal, but occasionally switch to Nautilus to make
directory trees and move their files around within those trees.

Unfortunately, I don't think this idea is actually workable.  Most
obviously, accessing the files using command-line tools will still result
in a corrupted Datastore.  For example, renaming a file using "mv" in the
Terminal will result in Nautilus being unable to associate any metadata
with it, which will cause the Journal to lose track of it entirely.  The
Journal could perhaps detect the presence of files that lack metadata and
prompt the user to fix the metadata, but this seems pretty disgusting.

The Journal permits duplicate names, and perhaps even empty names, but
file systems do not.  The Journal also carries mime type as explicit
metadata, whereas Nautilus guesses it from the file's extension and
contents.  Thus, files created in the Journal would have to have their
names munged in some way on disk.  Present datastore designs solve this by
using a cryptographic hash as the file name, but this would not be
acceptable if users were expected to interact with the files directly.
Munging filenames is not so hard, but recovering correctly when the user
changes the name of a file to something which cannot be inverse-munged is
an obvious tarpit.

There are many other issues here, not least of which is the potential for
unexpected behaviors in Nautilus to corrupt the Datastore for the Journal.
~ The gain, of being able to use Nautilus and the Journal but no other file
management tools, is of questionable value.

I'm really not sure why I wrote this e-mail.  Maybe it's to say that even
our easiest options for interoperability are actually quite complex,
fragile, and dangerous.

- --Ben
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkj2ftAACgkQUJT6e6HFtqSGVQCfZ8NO4iXbbwE4QSxWEp8bDbHk
g3wAn20KWTtCqU2Zx5EpESbfcZIu1EYi
=cFHa
-----END PGP SIGNATURE-----