[sugar] On improving Sugar performance

[Tomeu Vizoso]

On Mon, Apr 28, 2008 at 3:17 PM, Bernie Innocenti <bernie at codewiz.org> wrote:
> Hello,
>
>  me, tomeu and Marco engaged in a discussion about Sugar performance
>  off-list, but at some point we thought it would be better material
>  for a public discussion.
>
>  Each one of us has been interested in performance for a long time,
>  working in different areas and with different perspectives.  To my
>  knowledge, Michael, Chris and Scott have also been measuring and
>  contributing in the past.
>
>  We have of course different ideas and proposals, with Tomeu being
>  in a better position to give us hard numbers and informed opinions
>  because of the truly aggressive performance work he has been doing
>  lately.
>
>  When we say generically "performance", we aggregate several
>  related issues:
>
>  - time to perform specific actions (such as refreshing the screen)
>  - memory footprint of the Sugar shell and associated services
>  - overhead common to all activities (the "hello world" memory usage)
>  - memory footprint of specific activities (such as the browser)
>  - startup time of activities
>  - boot time of the OS
>
>  To simplify the discussion, I'd like to set aside, for now these
>  other areas:
>
>  - filesystem (datastore) performance and scalability
>  - networking performance and scalability
>
>  I'd like to introduce our discussion in the form of a platonic
>  dialog with Tomeu:

Fine ;)

>  - What have you been doing so far?

- Profiled Shell startup.
- Profiled Activity startup https://dev.laptop.org/ticket/5228
- Profiled Activity switching.
- Profiled DataStore general performance and measured scalability.
- Measured how often the UI elements were refreshed and how much time that took.

>  - What are the benefits you obtained so far?

- J5 (if I remember well) took out some bottlenecks from activity and
shell startup, long ago.

- Python modules use to do early initialization, and as python
activities are given support from the sugar API for several high-level
functionality items (presence, UI, datastore, etc) lots of modules
need to be initialized before nothing is drawn in the screen during
activity launch. Using a python launcher process that reuses a
pre-initialized python shell solved this and also brought considerable
memory savings from page sharing between processes. In joyride builds
now.

- Experimented with redirecting the activity windows to an off-screen
pixmap (using composition). This prevented many redraws and improved
substantially UI feedback (activity switching, frame slide-in/out and
palettes and menus popdown). There are some issues to solve yet before
this can be considered deployable. This approach increases memory
usage by 4MB per activity, but could be minimized to a total of 4MB
with occasional 8MB peaks by using a smart composition manager. In
'faster' builds now.

- The DataStore was brought to "good enough" performance and
scalability by improving how metadata is stored and indexed. We can
still improve considerably here without switching to a different
full-text engine.

- Much work was done in the Journal in order to cache data from the DS
and make scrolling quick enough. The results are not yet satisfactory
and a new UI design is planned that will make this issue much easier
to solve.

>  - Which are the areas where we could gain the most?

- Python activity startup in joyride is much faster, but still can be
improved greatly. The next bottleneck is believed to be reading from
the DS or creating the initial entry when launching.

- Currently, all files that get into the filesystem are compressed
on-the-fly by the jffs2 driver. This can result in a big amount of
work being performed inside the kernel, slowing considerably other
simultaneous operations (activity startup, switching and stopping, for
example). I think that we have now support in the driver for honoring
a flag for not doing this compression. Most files saved by activities
are already compressed formats (png, odt, pdf, xo, etc) so don't
benefit from this on-the-fly compression. We should start using it.

>  - Do you see low-hanging fruit we could pick in the short term?

Don't know currently of other areas other than the ones described
above. In my opinion, what we should do now is:

- consolidate the benefits we have already implemented by testing,
fixing any issues and putting into images that can be tested

- get feedback about other parts of Sugar that need performance love

- profile, measure, etc

Having some kind of automated tests for detecting regressions would be
very important, in my opinion.

>  - How much further can we improve by dropping specific unnecessary
>   or redundant work and similar pessimizations?

Lots ;) That's the best path for the time being, and I'm confident
we'll be able to get excellent performance just by doing that.

>  - How much further can we improve by refactoring subsystems and
>   APIs to do less work?

I consider this the same as above, isn't it?

>  - How much further can we improve by rewriting some
>   performance-critical parts in a low-level language?

AFAIK, we are not spending any significant amount of time doing useful
calculations in python.

>  - How much further can we improve by replacing inefficient
>   frameworks and support modules with different implementations
>   available off the shelf? (such as "use

Is this unfinished?

>  - How much further can we improve by compromising on the feature
>   set we are providing?  (such as "disable anti-aliasing" or
>   "simplify SVG icons or convert to bitmap")

This is interesting, but I don't know of any concrete case. We may
want to explore this path after we identify the next areas that need
improvement.

>  - How much further can we improve by pruning from feature set?
>   (such as "drop rounded edges from the Sugar theme").

Same as above.

>  Anyone is of course welcome to join us with questions, answers and
>  proposals.  To keep the discussion relaxed, we do not require backing
>  each and every idea with benchmark results, which in some cases are
>  hard to obtain.  Nevertheless, measuring is a useful tool, especially
>  when there there's disagreement on how effective each solution might be.

Sure, but rather than a useful tool, I would call measuring as the
only possible base on which decide actual work that needs to be done.
We could be refactoring and recoding for years and don't get any
noticeable improvement, if we don't measure.

Thanks,

Tomeu