[Sugar-devel] Language section debugging, use ICU?

[Manuel Quiñones]

I have ongoing work on polishing the Language section of the Sugar
settings panel.  I'm sharing my findings to open discussion, to start
bringing back discussions to the mailing list, and to encourage
testing of my patches.

First, an introduction of the issues, ordered by priority as I understand:

1. list of languages: how could I select my language if it is
displayed in another language?
   http://bugs.sugarlabs.org/ticket/51

2. list of languages: if there are two languages and English is the
first one, the list is displayed as the second one
   http://bugs.sugarlabs.org/ticket/4327

3. many language names are not translated
   http://bugs.sugarlabs.org/ticket/4449#comment

4. the section takes a lot of time to start, we should display a
watch/busy cursor while it is loading
   https://bugs.sugarlabs.org/ticket/245

The issues are interconnected as you will find if you try to read the
many comments, which mix them all through the years :)

A brief of the current implementation:

A. we parse the output of 'locale -av' command and create a list of
(language name, territory name, locale code)
   https://git.sugarlabs.org/sugar/mainline/blobs/master/extensions/cpsection/language/model.py#line33

B. we call gettext with ISO-639 to translate the language name
   https://git.sugarlabs.org/sugar/mainline/blobs/master/extensions/cpsection/language/view.py#line28

C. we call gettext with ISO-3166 to translate the territory/country name
   https://git.sugarlabs.org/sugar/mainline/blobs/master/extensions/cpsection/language/view.py#line29

Number 2 gets obsolete if we solve no. 1.  The problem in 2. is that
we should not use gettext to translate strings to English, as they are
already in English.  It is iliustrated in this comment:
http://bugs.sugarlabs.org/ticket/4327#comment:6

Number 3 is a flaw of the current implementation: the output of
'locale -av' does not match 100% the strings in the po files for the
given gettext domains.  See comment by Chris Leonard on this:
http://bugs.sugarlabs.org/ticket/4449#comment:9

Number 4 has a general patch that works for all sections, except for
Modem section.  We actually have code that displays a busy cursor, but
it is shown only for an instant, because (fortunatly) the UI doesn't
block the program.  The patch wraps the initialization of the section
in a GObject.idle_add call, to make it work.  As said before, this
conflicts with the current implementation of the Modem section.  So..
still work to be done on this one.

By the way, number 4. gets less relevant if we speed up the section.
That leads me to talk about my findings on issue number 1:  I have
investigated alternatives to our current gettext implementation.

- using gettext domain ISO 639-3 instead of ISO 639 for translating
the language name
- using external libraries Babel and PyICU

Here is a table of my results:

https://docs.google.com/spreadsheet/ccc?key=0Auk55vVISSpndEdnbDA4OHJCelRwUHlKbmFNQVBsSkE#gid=0

And here is the result of profiling them:

http://bugs.sugarlabs.org/ticket/4449#comment:12

So, it looks like the ICU project is very fast and provides good
output.  And reading the project homepage it looks on shape too.  Can
we consider a switch to it?

-- 
.. manuq ..