[Sugar-devel] XO-1 vs Sugar 0.104 performance, and swap to NAND Flash

[James Cameron]

Browse is one of the most heavily used activities when internet or
local content is available.

Tests were run over many hours on several XO-1 laptops.  The XO-1 is
an old design which is slow enough to give useful statistics.

The results show a continued improvement to startup time over the
recent versions of Sugar, and a very small advantage to using swap
memory.

--

The first test was to reboot, wait for Sugar to start, then
automatically start the Browse activity, and time how long it took to
start.  Then the results of hundreds of tests were averaged.

Browse-140 on 12.1.0 with Sugar 0.94 took 25 seconds.

Browse-149.4 on 13.2.1 with Sugar 0.98 took 23 seconds.

Browse-157 on 13.2.4 with Sugar 0.104 and no swap took 21 seconds.

Browse-157 on 13.2.4 with Sugar 0.104 and NAND swap took 20 seconds.

This shows continued improvement to Browse startup time, in the
scenario where the libraries have to be loaded into memory.

(Reference: test #8, and #9)

--

Another test started and stopped the Browse activity 25 times without
rebooting.  Then the results were averaged.

Browse-140 on 12.1.0 with Sugar 0.94 took 14 seconds.

Browse-149.4 on 13.2.1 with Sugar 0.98 took 15 seconds.

Browse-157 on 13.2.4 with Sugar 0.104 and NAND swap took 13 seconds.

This shows some improvement to Browse startup time, in the scenario
where the needed libraries are already loaded into memory.

(Reference: test #6)

--

The same test also started and stopped most of the other activities
25 times without rebooting.  Then the results were averaged.

For Sugar 0.96 the average startup time was 15 seconds the first time,
and 11 seconds each subsequent time.

For Sugar 0.98 the average startup time was 17 seconds the first time,
and 13 seconds each subsequent time.

For Sugar 0.104 the average startup time was 14 seconds the first
time, and 11 seconds each subsequent time.

Detailed results by activity below.  The key for these tables is:

	cold = startup time for first start after sugar restart.
	warm = average of startup time for subsequence starts.
	std = population standard deviation for warm starts.
	ratio = a ratio comparing warm start to cold start times.
	tests = number of warm start tests recorded.

For Sugar 0.96 the results by activity were:

                        bundle_id    cold    warm    std  ratio  tests

             com.garycmartin.Moon  10.595  10.643  0.531  1.005  24
       com.jotaro.ImplodeActivity   6.691   6.486  0.045  0.969  24
       org.laptop.AbiWordActivity  19.474  14.459  0.804  0.743  24
       org.laptop.AcousticMeasure  11.984   7.761  0.045  0.648  24
             org.laptop.Calculate   9.809   9.560  0.065  0.975  24
          org.laptop.HelpActivity  19.487  11.342  0.688  0.582  24
       org.laptop.MeasureActivity  12.478  10.246  0.085  0.821  24
              org.laptop.Memorize  16.229  13.243  0.539  0.816  24
               org.laptop.Oficina  10.421   9.490  0.431  0.911  24
                 org.laptop.Pippy   6.421   6.150  0.050  0.958  24
        org.laptop.RecordActivity  12.563  11.179  0.346  0.890  24
            org.laptop.TamTamMini  16.676  14.414  0.338  0.864  24
           org.laptop.WebActivity  23.335  14.260  0.241  0.611  24
                 tv.alterna.Clock   8.782   8.631  0.067  0.983  24
                 vu.lux.olpc.Maze  11.699   8.731  0.269  0.746  24
                vu.lux.olpc.Speak  15.187  11.460  0.261  0.755  24

For Sugar 0.98 the results by activity were:

                        bundle_id    cold    warm    std  ratio  tests

             com.garycmartin.Moon  12.946  11.039  0.372  0.853  24
       com.jotaro.ImplodeActivity  11.494  11.352  0.499  0.988  24
       org.laptop.AbiWordActivity  26.611  21.501  1.041  0.808  24
       org.laptop.AcousticMeasure  14.865  12.949  0.351  0.871  24
             org.laptop.Calculate  12.063  10.220  0.207  0.847  24
          org.laptop.HelpActivity  18.378  11.101  0.311  0.604  24
       org.laptop.MeasureActivity  19.566  13.791  0.308  0.705  24
              org.laptop.Memorize  20.977  14.462  0.791  0.689  24
               org.laptop.Oficina  14.216  13.948  0.246  0.981  24
                 org.laptop.Pippy  11.793  10.983  0.141  0.931  24
        org.laptop.RecordActivity  18.459  13.165  0.514  0.713  24
         org.laptop.sugar.Jukebox  16.346  11.466  0.292  0.701  24
    org.laptop.sugar.ReadActivity  16.407  13.253  0.597  0.808  24
            org.laptop.TamTamMini  20.218  16.475  0.638  0.815  24
           org.laptop.WebActivity  22.427  14.587  0.329  0.650  24
                 tv.alterna.Clock   9.943   7.377  0.264  0.742  24
                 vu.lux.olpc.Maze  14.243  10.198  0.785  0.716  24
                vu.lux.olpc.Speak  22.032  13.281  1.150  0.603  24

For Sugar 0.104 the results by activity were:

                        bundle_id    cold    warm    std  ratio  tests

             com.garycmartin.Moon  11.928  10.294  0.492  0.863  24
       com.jotaro.ImplodeActivity   9.084   9.017  0.498  0.993  24
       org.laptop.AbiWordActivity  20.868  16.862  0.376  0.808  24
       org.laptop.AcousticMeasure  12.330  10.513  0.137  0.853  24
             org.laptop.Calculate  11.591   9.920  0.120  0.856  24
          org.laptop.HelpActivity  14.654   8.981  0.329  0.613  24
       org.laptop.MeasureActivity  16.381  11.364  0.135  0.694  24
              org.laptop.Memorize  17.961  14.550  0.183  0.810  24
               org.laptop.Oficina  12.231  12.029  0.351  0.983  24
                 org.laptop.Pippy   8.752   8.202  0.128  0.937  24
        org.laptop.RecordActivity  16.956  12.652  0.145  0.746  24
         org.laptop.sugar.Jukebox   9.666   8.986  0.108  0.930  24
    org.laptop.sugar.ReadActivity  11.296  10.477  0.165  0.928  24
            org.laptop.TamTamMini  18.628  14.734  0.576  0.791  24
           org.laptop.WebActivity  19.425  12.527  0.217  0.645  24
                 tv.alterna.Clock  10.061   7.124  0.123  0.708  24
                 vu.lux.olpc.Maze   8.366   8.265  0.120  0.988  24
                vu.lux.olpc.Speak  24.555  12.075  0.208  0.492  24

(Reference: test #6)

--

The testing scripts can be made available if anybody else would like
to replicate the results.

-- 
James Cameron
http://quozl.linux.org.au/