[Sugar-devel] new journal and datastore maintainer

[Tomeu Vizoso]

On Fri, Sep 18, 2009 at 09:51, Martin Langhoff
<martin.langhoff at gmail.com> wrote:
> On Fri, Sep 18, 2009 at 2:10 AM, James Cameron <quozl at laptop.org> wrote:
>> By branching at the 0.82 point as deployed, determining from evidence
>
> Good theoretical re-statement. You post may clarify things for other
> readers -- as for me, I know very well what I am requesting. And I do
> respect that it is not easy.
>
> It is just... very important.
>
>> I've reviewed the changes just now from 0.82 to 0.84.  It was certainly
>> an entire rewrite, so what you are in effect asking is for someone to
>> support the 0.82 DS to fix a problem that is not yet well defined.
>
> The _root cause_ is not well defined, although Tomeu has pointed out
> likely culprits. The results are very well known, and it is possible
> recover the situation without surgery. Read:
>
>  http://lists.laptop.org/pipermail/devel/2008-May/014601.html
>
> And here you can find "steps to repro" and a workaround if you want to
> see it in action:
>
>  http://dev.laptop.org/ticket/7719
>
> (Uy users probably triggering the DS problem in a different way)
>
> As for the code that controls this?
>
> in shell/view/Shell.py:
>
>    def _start_journal_idle(self):
>        # Mount the datastore in internal flash
>        ds_path = env.get_profile_path('datastore')
>        try:
>            datastore.mount(ds_path, [], timeout=120 * \
>                                         DBUS_PYTHON_TIMEOUT_UNITS_PER_SECOND)
>        except:
>            # Don't explode if there's corruption; move the data out of the way
>            # and attempt to create a store from scratch.
>            shutil.move(ds_path, os.path.abspath(ds_path) + str(time.time()))
>            datastore.mount(ds_path, [], timeout=120 * \
>                                         DBUS_PYTHON_TIMEOUT_UNITS_PER_SECOND)
>
>
> The good news: if recent Sugars have retained this logic, even if we
> hit issues with newer datastore code (hope not! :-) ), the same
> "recovery" tool will help for newer Sugar.

No, what we do instead is making the index discardable so it's not our
weak point any more. For the rest of the data we use files and
directories, and as for every operation we only read the files we
need, a missing or unexpected file won't impact all operations, like
did the mount() step in the old DS.

Even when files are missing, the data loss should be contained to that entry.

Regards,

Tomeu

-- 
«Sugar Labs is anyone who participates in improving and using Sugar.
What Sugar Labs does is determined by the participants.» - David
Farning