[IAEP] fixing etoys

[Yoshiki Ohshima]

At Tue, 24 Jun 2008 14:16:22 -0400,
Albert Cahalan wrote:
> 
> Here are some ideas that might help you fix some of the problems
> with start-up performance, shut-down performance, open source,
> and software engineering practices.

  First, Etoys' start up time is very fast.  And shutdown/saving could
be very fast if we can save the whole image.  What is slow is scanning
relevant objects from the image upon saving, due to lack of modularity
in the image.

> You're trying to do a persistant system image on an OS that wasn't
> really designed for it. If you were on an exotic system with a
> persistant image (like EROS, KeyKOS, or OS/400), you might be able
> to cut the power at any time without losing more than a few seconds
> of work. The key here is that the persistant system image OSes
> continuously write out changes to disk. They do this atomicly,
> and in small chunks, rather than overwriting everything at once in
> a dangerous non-atomic operation.
>
> Start with that. Break each object out into a separate file.
> Your database becomes a directory rather than a big blob file.
> When an in-memory object changes, you write a copy to a fresh
> new file on disk. You keep the old on-disk copy around until
> the new one has been synched. (fsync or sync probably, but be
> very careful to avoid doing this more than once every few
> seconds) You may need subdirectories for better performance;
> it is very unwise to rely on Reiserfs-like performance when
> having large directories.

  Saving the Squeak image to disk is not slow.  (Back then, Ted
Kaehler had a memory system called OOZE for older Smalltalk that does
the checkpointing and constant object-grained swapping.  ALTO had a
memory problem that caused the system crashes once or twice a day
without any reasons.  OOZE kept people from losing their work except
last 30 seconds or so.)

> On-demand loading is required for start-up performance.

  Again, start up time is not a problem.  Etoys start up looks a bit
slow on XO, but that is because the DBus communication that has to be
done.  Otherwise, all it needs to do is reading a continuous 20MB file
into a continuous memory region and scanning the memory once.

> Inheritance from a read-only image will cut per-instance disk size.
> On a Debian system, install the read-only image under /usr/share
> and the per-instance changes somewhere under $HOME. On the XO,
> the read-only image stays in the activity and the per-instance
> part goes into the journal. For older XO system software which
> does not support grouping multiple files into single Journal entries,
> you'll have to do it yourself with a standard archive format.
> There are three to choose from: tar, cpio, pax.
> 
> To make things maintainable outside of the walled garden, store the
> objects in text form. Make them be like nicely formatted source code.
> Be permissive in parsing them, and try to preserve any comments that
> might be added by users with regular text editors. Try to maximize
> compatibility with GNU Smalltalk.

  But code and objects are different.  Vast majority of incremental
changes including object creation and setting preferences are already
maintained outside of the walled garden.  The latest image is
(essentially) just a serial applications of updates available at:

http://tinlizzie.org/updates/etoys/updates/

from an early image.

> If you have bulk data that would not reasonably be editable with
> a text editor, such as PNG images, then leave that part in binary.
> 
> To cut down on dirty anonymous pages in memory, and thus greatly
> improve the situation on low-memory swapless systems like the XO,
> you could do mmap(...,PROT_WRITE|PROT_READ,MAP_PRIVATE,...) on the
> binary blobs at startup. This should just be things like PNG images.

  I don't say there is a problem, such as the problem of modularity,
and suggestions similar to yours have been made before.  But it seems
that you really don't know how it works and what are the real
problems.

-- Yoshiki