[Dextrose] [PATCH RFC sugar] Replace activity updater with OLPC microformat compatible one

Sascha Silbe silbe at activitycentral.com
Sun Jun 26 15:04:13 EDT 2011


This patch replaces the Sugar activity updater with one that supports
the HTML based OLPC activity microformat.

New features:
- installing new activities (not just updating existing ones)
  - uses the optional olpc-activity-name attribute if available and falls back
    to a heuristic based on the bundle id (olpc-activity-id)
- configurable update URL
- extensive error handling (connection loss, disk full, etc.)

The download size is derived from the optional olpc-activity-size attribute
with a fallback to HTTP HEAD requests. Empty bundles get ignored.

Because of major changes to the inner workings the activities.sugarlabs.org
format backend has been removed. a.sl.o supports the OLPC microformat now and
the a.sl.o backend didn't make any use of information other than what the
microformat contains as well.

Co-Authored-by: Akash Gangil <akashg1611 at gmail.com>
Co-Authored-by: Anish Mangal <anish at sugarlabs.org>
Co-Authored-by: Sascha Silbe <silbe at activitycentral.com>
Signed-off-by: Sascha Silbe <silbe at activitycentral.com>
---

I've significantly reworked the microformat updater shipped by Dextrose.
There are a few places where I'm still unsure if we should keep them
as-is or rework some more. Comments?

Regular operation (update + install) has been tested, but I'd appreciate
help testing all the different error paths.

Once we're done with the FIXMEs and testing, I will post to sugar-devel
for the final review.

 data/sugar.schemas.in                              |   12 +
 extensions/cpsection/updater/backends/Makefile.am  |    2 +-
 extensions/cpsection/updater/backends/aslo.py      |  164 -------------
 .../cpsection/updater/backends/microformat.py      |  197 ++++++++++++++++
 extensions/cpsection/updater/model.py              |  247 +++++++++++---------
 extensions/cpsection/updater/view.py               |  114 +++++-----
 6 files changed, 406 insertions(+), 330 deletions(-)

diff --git a/data/sugar.schemas.in b/data/sugar.schemas.in
index b13f746..fe9cfc0 100644
--- a/data/sugar.schemas.in
+++ b/data/sugar.schemas.in
@@ -62,6 +62,18 @@
     </schema>

     <schema>
+      <key>/schemas/desktop/sugar/updater_url</key>
+      <applyto>/desktop/sugar/updater_url</applyto>
+      <owner>sugar</owner>
+      <type>string</type>
+      <default>http://activities-testing.sugarlabs.org/services/micro-format.php?collection_nickname=fructose</default>
+      <locale name="C">
+        <short>Activity updater URL.</short>
+        <long>This key contains the url which the microformat compatible activity updater will search for activity updates.</long>
+      </locale>
+    </schema>
+
+    <schema>
       <key>/schemas/desktop/sugar/backup_url</key>
       <applyto>/desktop/sugar/backup_url</applyto>
       <owner>sugar</owner>
diff --git a/extensions/cpsection/updater/backends/Makefile.am b/extensions/cpsection/updater/backends/Makefile.am
index e280a07..e9c1284 100644
--- a/extensions/cpsection/updater/backends/Makefile.am
+++ b/extensions/cpsection/updater/backends/Makefile.am
@@ -1,5 +1,5 @@
 sugardir = $(pkgdatadir)/extensions/cpsection/updater/backends

 sugar_PYTHON = 		\
-	aslo.py		\
+	microformat.py	\
 	__init__.py
diff --git a/extensions/cpsection/updater/backends/aslo.py b/extensions/cpsection/updater/backends/aslo.py
deleted file mode 100644
index 6504e9e..0000000
--- a/extensions/cpsection/updater/backends/aslo.py
+++ /dev/null
@@ -1,164 +0,0 @@
-#!/usr/bin/python
-# Copyright (C) 2009, Sugar Labs
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program; if not, write to the Free Software
-# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA
-
-"""Activity information microformat parser.
-
-Activity information is embedded in HTML/XHTML/XML pages using a
-Resource Description Framework (RDF) http://www.w3.org/RDF/ .
-
-An example::
-
-<?xml version="1.0" encoding="UTF-8"?>
-<RDF:RDF xmlns:RDF="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
-        xmlns:em="http://www.mozilla.org/2004/em-rdf#">
-<RDF:Description about="urn:mozilla:extension:bounce">
-    <em:updates>
-        <RDF:Seq>
-            <RDF:li resource="urn:mozilla:extension:bounce:7"/>
-        </RDF:Seq>
-    </em:updates>
-</RDF:Description>
-
-<RDF:Description about="urn:mozilla:extension:bounce:7">
-    <em:version>7</em:version>
-    <em:targetApplication>
-        <RDF:Description>
-            <em:id>{3ca105e0-2280-4897-99a0-c277d1b733d2}</em:id>
-            <em:minVersion>0.82</em:minVersion>
-            <em:maxVersion>0.84</em:maxVersion>
-            <em:updateLink>http://foo.xo</em:updateLink>
-            <em:updateSize>7</em:updateSize>
-            <em:updateHash>sha256:816a7c43b4f1ea4769c61c03ea4..</em:updateHash>
-        </RDF:Description>
-    </em:targetApplication>
-</RDF:Description></RDF:RDF>
-"""
-
-import logging
-from xml.etree.ElementTree import XML
-import traceback
-
-import gio
-
-from sugar.bundle.bundleversion import NormalizedVersion
-from sugar.bundle.bundleversion import InvalidVersionError
-
-from jarabe import config
-
-_FIND_DESCRIPTION = \
-        './/{http://www.w3.org/1999/02/22-rdf-syntax-ns#}Description'
-_FIND_VERSION = './/{http://www.mozilla.org/2004/em-rdf#}version'
-_FIND_LINK = './/{http://www.mozilla.org/2004/em-rdf#}updateLink'
-_FIND_SIZE = './/{http://www.mozilla.org/2004/em-rdf#}updateSize'
-
-_UPDATE_PATH = 'http://activities.sugarlabs.org/services/update-aslo.php'
-
-_fetcher = None
-
-
-class _UpdateFetcher(object):
-
-    _CHUNK_SIZE = 10240
-
-    def __init__(self, bundle, completion_cb):
-        # ASLO knows only about stable SP releases
-        major, minor = config.version.split('.')[0:2]
-        sp_version = '%s.%s' % (major, int(minor) + int(minor) % 2)
-
-        url = '%s?id=%s&appVersion=%s' % \
-                (_UPDATE_PATH, bundle.get_bundle_id(), sp_version)
-
-        logging.debug('Fetch %s', url)
-
-        self._completion_cb = completion_cb
-        self._file = gio.File(url)
-        self._stream = None
-        self._xml_data = ''
-        self._bundle = bundle
-
-        self._file.read_async(self.__file_read_async_cb)
-
-    def __file_read_async_cb(self, gfile, result):
-        try:
-            self._stream = self._file.read_finish(result)
-        except:
-            global _fetcher
-            _fetcher = None
-            self._completion_cb(None, None, None, None, traceback.format_exc())
-            return
-
-        self._stream.read_async(self._CHUNK_SIZE, self.__stream_read_async_cb)
-
-    def __stream_read_async_cb(self, stream, result):
-        xml_data = self._stream.read_finish(result)
-        if xml_data is None:
-            global _fetcher
-            _fetcher = None
-            self._completion_cb(self._bundle, None, None, None,
-                    'Error reading update information for %s from '
-                    'server.' % self._bundle.get_bundle_id())
-            return
-        elif not xml_data:
-            self._process_result()
-        else:
-            self._xml_data += xml_data
-            self._stream.read_async(self._CHUNK_SIZE,
-                                    self.__stream_read_async_cb)
-
-    def _process_result(self):
-        document = XML(self._xml_data)
-
-        if document.find(_FIND_DESCRIPTION) is None:
-            logging.debug('Bundle %s not available in the server for the '
-                'version %s', self._bundle.get_bundle_id(), config.version)
-            version = None
-            link = None
-            size = None
-        else:
-            try:
-                version = NormalizedVersion(document.find(_FIND_VERSION).text)
-            except InvalidVersionError:
-                logging.exception('Exception occured while parsing version')
-                version = '0'
-
-            link = document.find(_FIND_LINK).text
-
-            try:
-                size = long(document.find(_FIND_SIZE).text) * 1024
-            except ValueError:
-                logging.exception('Exception occured while parsing size')
-                size = 0
-
-        global _fetcher
-        _fetcher = None
-        self._completion_cb(self._bundle, version, link, size, None)
-
-
-def fetch_update_info(bundle, completion_cb):
-    """Queries the server for a newer version of the ActivityBundle.
-
-       completion_cb receives bundle, version, link, size and possibly an error
-       message:
-
-       def completion_cb(bundle, version, link, size, error_message):
-    """
-    global _fetcher
-
-    if _fetcher is not None:
-        raise RuntimeError('Multiple simultaneous requests are not supported')
-
-    _fetcher = _UpdateFetcher(bundle, completion_cb)
diff --git a/extensions/cpsection/updater/backends/microformat.py b/extensions/cpsection/updater/backends/microformat.py
new file mode 100644
index 0000000..0349df2
--- /dev/null
+++ b/extensions/cpsection/updater/backends/microformat.py
@@ -0,0 +1,197 @@
+#!/usr/bin/python
+#
+# Copyright (C) 2011, Anish Mangal <anish at sugarlabs.org>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA
+
+import logging
+from HTMLParser import HTMLParser
+import re
+
+import gconf
+import gio
+import glib
+import gobject
+
+from sugar.bundle.bundleversion import NormalizedVersion
+from jarabe import config
+
+
+_ACTIVITIES_LIST = {}
+_CHUNK_SIZE = 65536
+
+
+class MicroformatParser(HTMLParser):
+
+    _KNOWN_KEYS = ['olpc-activity-id', 'olpc-activity-version',
+                   'olpc-activity-name', 'olpc-activity-size',
+                   'olpc-activity-url']
+
+    def __init__(self, completion_cb):
+        HTMLParser.__init__(self)
+        self.reset()
+        self._completion_cb = completion_cb
+        self._current_activity = {}
+        self._active_key = None
+        self._activity_block_tag = None
+
+    def handle_starttag(self, tag, attrs):
+        for attribute, value in attrs:
+            if value == 'olpc-activity-info':
+                self._activity_block_tag = tag
+                self._active_key = None
+
+        if tag == 'span':
+            for attribute, value in attrs:
+                if value in self._KNOWN_KEYS:
+                    self._active_key = value
+
+        elif tag == 'a':
+            if self._active_key != 'olpc-activity-url':
+                return
+
+            for attribute, value in attrs:
+                if attribute == 'href':
+                    self._current_activity['url'] = value
+                    self._active_key = None
+                    return
+
+    def handle_data(self, data):
+        if self._active_key == 'olpc-activity-version':
+            self._current_activity['version'] = NormalizedVersion(data)
+
+        elif self._active_key == 'olpc-activity-id':
+            self._current_activity['bundle_id'] = data
+
+        elif self._active_key == 'olpc-activity-name':
+            self._current_activity['name'] = data
+
+        elif self._active_key == 'olpc-activity-size':
+            self._current_activity['size'] = int(data)
+
+        else:
+            return
+
+        self._active_key = None
+
+    def handle_endtag(self, tag):
+        if tag == self._activity_block_tag:
+            self._activity_block_tag = None
+            info = self._current_activity
+            self._current_activity = {}
+
+            bundle_id = info.pop('bundle_id')
+            if 'url' not in info or 'version' not in info:
+                return
+
+            _ACTIVITIES_LIST[bundle_id] = info
+
+        elif tag == 'body':
+            self._postprocess()
+
+    def _postprocess(self):
+        logging.debug('Postprocessing %d activity listings',
+                      len(_ACTIVITIES_LIST))
+        for bundle_id, info in _ACTIVITIES_LIST.items():
+            if not info.get('name'):
+                # Heuristics to derive activity name from bundle_id
+                activity_name = re.split('\.', bundle_id)[-1]
+                activity_name = re.sub('^[\s|\t]*', '', activity_name)
+                activity_name = re.sub('[\s|\t]*$', '', activity_name)
+                activity_name = re.sub('[A|a]ctivity$', '', activity_name)
+                info['name'] = activity_name
+
+            if info.get('size'):
+                continue
+
+            try:
+                gfile = gio.File(info['url'])
+                info['size'] = gfile.query_info('standard::size').get_size()
+
+            except glib.GError, exception:
+                logging.error('Could not query information for URL %r: %r',
+                              unicode(info['url']), unicode(exception))
+
+            if not info['size']:
+                logging.error('Bundle for %s reported as empty. Excluding from'
+                              ' update list.', bundle_id)
+                del _ACTIVITIES_LIST[bundle_id]
+
+        self._completion_cb(_ACTIVITIES_LIST, None)
+
+
+class UpdateFetcher(gobject.GObject):
+
+    __gsignals__ = {
+        'progress': (gobject.SIGNAL_RUN_FIRST, gobject.TYPE_NONE,
+                     # action, name or description, current, total
+                     [str, str, float, int]),
+    }
+
+    def __init__(self, completion_cb):
+        gobject.GObject.__init__(self)
+        # ASLO knows only about stable SP releases
+        major, minor = config.version.split('.')[0:2]
+        sp_version = '%s.%s' % (major, int(minor) + int(minor) % 2)
+        self._completion_cb = completion_cb
+        self._file = None
+        self._parser = MicroformatParser(self._completion_cb)
+
+    def download_bundle_updates(self):
+        self.emit('progress', 'check', '', 0, 2)
+
+        client = gconf.client_get_default()
+        url = client.get_string('/desktop/sugar/updater_url')
+        if '?' in url:
+            url += '&'
+        else:
+            url += '?'
+        #~ url += 'sugar=' + '.'.join(config.version.split('.')[0:2])
+        url += 'sugar=0.90'
+
+        self._file = gio.File(url)
+        logging.debug('Fetching %s', url)
+        self._file.read_async(self.__read_async_cb)
+
+    def __read_async_cb(self, gfile, result):
+        try:
+            stream = gfile.read_finish(result)
+        except glib.GError, exception:
+            self._completion_cb({}, unicode(exception))
+            return
+
+        stream.read_async(_CHUNK_SIZE, self.__stream_read_cb)
+
+    def __stream_read_cb(self, stream, result):
+        try:
+            data = stream.read_finish(result)
+        except glib.GError, exception:
+            self._completion_cb({}, unicode(exception))
+            self._close_stream(stream, ignore_error=True)
+            return
+
+        if not data:
+            self._close_stream(stream)
+            return
+
+        self._parser.feed(data)
+        stream.read_async(_CHUNK_SIZE, self.__stream_read_cb)
+
+    def _close_stream(self, stream, ignore_error=False):
+        try:
+            stream.close()
+        except glib.GError, exception:
+            if not ignore_error:
+                self._completion_cb({}, unicode(exception))
diff --git a/extensions/cpsection/updater/model.py b/extensions/cpsection/updater/model.py
index 7ea445f..57bb384 100755
--- a/extensions/cpsection/updater/model.py
+++ b/extensions/cpsection/updater/model.py
@@ -25,10 +25,10 @@ import os
 import logging
 import tempfile
 from urlparse import urlparse
-import traceback

 import gobject
 import gio
+import glib

 from sugar import env
 from sugar.datastore import datastore
@@ -37,76 +37,79 @@ from sugar.bundle.bundleversion import NormalizedVersion

 from jarabe.model import bundleregistry

-from backends import aslo
+from backends.microformat import UpdateFetcher
+
+
+_CHUNK_SIZE = 65536  # 64KB


 class UpdateModel(gobject.GObject):
     __gtype_name__ = 'SugarUpdateModel'

     __gsignals__ = {
-        'progress': (gobject.SIGNAL_RUN_FIRST,
-                     gobject.TYPE_NONE,
-                     ([int, str, float, int])),
+        'progress': (gobject.SIGNAL_RUN_FIRST, gobject.TYPE_NONE,
+                     # action, name or description, current, total
+                     [str, str, float, int]),
     }

-    ACTION_CHECKING = 0
-    ACTION_UPDATING = 1
-    ACTION_DOWNLOADING = 2
-
     def __init__(self):
         gobject.GObject.__init__(self)

         self.updates = None
-        self._bundles_to_check = None
         self._bundles_to_update = None
+        self._current_bundles = {}
         self._total_bundles_to_update = 0
         self._downloader = None
+        self._fetcher = None
         self._cancelling = False

+    def __progress_cb(self, model, action, description, current, total):
+        self.emit('progress', action, description, current, total)
+
     def check_updates(self):
         self.updates = []
-        self._bundles_to_check = list(bundleregistry.get_registry())
-        self._check_next_update()
-
-    def _check_next_update(self):
-        total = len(bundleregistry.get_registry())
-        current = total - len(self._bundles_to_check)
-
-        if not self._bundles_to_check:
-            return False
-
-        bundle = self._bundles_to_check.pop()
-        self.emit('progress', UpdateModel.ACTION_CHECKING, bundle.get_name(),
-                  current, total)
-
-        aslo.fetch_update_info(bundle, self.__check_completed_cb)
-
-    def __check_completed_cb(self, bundle, version, link, size, error_message):
+        self._current_bundles = {}
+        for bundle in bundleregistry.get_registry():
+            self._current_bundles[bundle.get_bundle_id()] = bundle
+        self._fetcher = UpdateFetcher(self.__bundle_info_fetched_cb)
+        self._fetcher.connect('progress', self.__progress_cb)
+        gobject.idle_add(self._fetcher.download_bundle_updates)
+
+    def __bundle_info_fetched_cb(self, new_bundles, error_message):
         if error_message is not None:
-            logging.error('Error getting update information from server:\n'
-                          '%s' % error_message)
+            logging.error('Error getting update information from server: %s',
+                          error_message)

-        if version is not None and \
-                version > NormalizedVersion(bundle.get_activity_version()):
-            self.updates.append(BundleUpdate(bundle, version, link, size))
+        self.emit('progress', 'check', '', 1, 2)

         if self._cancelling:
             self._cancel_checking()
-        elif self._bundles_to_check:
-            gobject.idle_add(self._check_next_update)
-        else:
-            total = len(bundleregistry.get_registry())
-            if bundle is None:
-                name = ''
+            return
+
+        for bundle_id, new_info in new_bundles.items():
+            if bundle_id in self._current_bundles:
+                bundle = self._current_bundles[bundle_id]
+                old_version = NormalizedVersion(bundle.get_activity_version())
+                if new_info['version'] <= old_version:
+                    continue
+
+                update = BundleUpdate('update', bundle_id, new_info['name'],
+                                      new_info['version'], new_info['url'],
+                                      new_info['size'], old_version,
+                                      bundle.get_icon())
             else:
-                name = bundle.get_name()
-            self.emit('progress', UpdateModel.ACTION_CHECKING, name, total,
-                      total)
+                update = BundleUpdate('install', bundle_id, new_info['name'],
+                                      new_info['version'], new_info['url'],
+                                      new_info['size'])
+
+            self.updates.append(update)
+
+        self.emit('progress', 'check', '', 2, 2)

     def update(self, bundle_ids):
         self._bundles_to_update = []
         for bundle_update in self.updates:
-            if bundle_update.bundle.get_bundle_id() in bundle_ids:
+            if bundle_update.bundle_id in bundle_ids:
                 self._bundles_to_update.append(bundle_update)

         self._total_bundles_to_update = len(self._bundles_to_update)
@@ -122,12 +125,13 @@ class UpdateModel(gobject.GObject):
         total = self._total_bundles_to_update * 2
         current = total - len(self._bundles_to_update) * 2 - 2

-        self.emit('progress', UpdateModel.ACTION_DOWNLOADING,
-                  bundle_update.bundle.get_name(), current, total)
+        self.emit('progress', 'download', bundle_update.name, current, total)

         self._downloader = _Downloader(bundle_update)
         self._downloader.connect('progress', self.__downloader_progress_cb)
         self._downloader.connect('error', self.__downloader_error_cb)
+        self._downloader.connect('finished', self.__downloader_finished_cb)
+        return False

     def __downloader_progress_cb(self, downloader, progress):
         logging.debug('__downloader_progress_cb %r', progress)
@@ -139,14 +143,13 @@ class UpdateModel(gobject.GObject):
         total = self._total_bundles_to_update * 2
         current = total - len(self._bundles_to_update) * 2 - 2 + progress

-        self.emit('progress', UpdateModel.ACTION_DOWNLOADING,
-                  self._downloader.bundle_update.bundle.get_name(),
+        self.emit('progress', 'download', self._downloader.bundle_update.name,
                   current, total)

-        if progress == 1:
-            self._install_update(self._downloader.bundle_update,
-                                 self._downloader.get_local_file_path())
-            self._downloader = None
+    def __downloader_finished_cb(self, downloader):
+        self._install_update(downloader.bundle_update,
+                             downloader.get_local_file_path())
+        self._downloader = None

     def __downloader_error_cb(self, downloader, error_message):
         logging.error('Error downloading update:\n%s', error_message)
@@ -157,27 +160,25 @@ class UpdateModel(gobject.GObject):

         total = self._total_bundles_to_update
         current = total - len(self._bundles_to_update)
-        self.emit('progress', UpdateModel.ACTION_UPDATING, '', current, total)
+        # FIXME: should we really do this?
+        self.emit('progress', 'update', '', current, total)

         if self._bundles_to_update:
             # do it in idle so the UI has a chance to refresh
             gobject.idle_add(self._download_next_update)

     def _install_update(self, bundle_update, local_file_path):
-
         total = self._total_bundles_to_update
         current = total - len(self._bundles_to_update) - 0.5

-        self.emit('progress', UpdateModel.ACTION_UPDATING,
-                  bundle_update.bundle.get_name(),
-                  current, total)
+        self.emit('progress', 'update', bundle_update.name, current, total)

         # TODO: Should we first expand the zip async so we can provide progress
         # and only then copy to the journal?
         jobject = datastore.create()
         try:
-            title = '%s-%s' % (bundle_update.bundle.get_name(),
-                               bundle_update.version)
+            title = '%s-%s' % (bundle_update.name,
+                               bundle_update.new_version)
             jobject.metadata['title'] = title
             jobject.metadata['mime_type'] = ActivityBundle.MIME_TYPE
             jobject.file_path = local_file_path
@@ -185,32 +186,31 @@ class UpdateModel(gobject.GObject):
         finally:
             jobject.destroy()

-        self.emit('progress', UpdateModel.ACTION_UPDATING,
-                  bundle_update.bundle.get_name(),
-                  current + 0.5, total)
+        self.emit('progress', 'update', bundle_update.name, current + 0.5,
+                  total)

         if self._bundles_to_update:
             # do it in idle so the UI has a chance to refresh
             gobject.idle_add(self._download_next_update)

+        return False
+
     def cancel(self):
         self._cancelling = True

     def _cancel_checking(self):
         logging.debug('UpdateModel._cancel_checking')
         total = len(bundleregistry.get_registry())
-        current = total - len(self._bundles_to_check)
-        self.emit('progress', UpdateModel.ACTION_CHECKING, '', current,
-                  current)
-        self._bundles_to_check = None
+        current = total - len(self._current_bundles)
+        self.emit('progress', 'check', '', current, current)
+        self._current_bundles = None
         self._cancelling = False

     def _cancel_updating(self):
         logging.debug('UpdateModel._cancel_updating')
         current = (self._total_bundles_to_update -
                    len(self._bundles_to_update) - 1)
-        self.emit('progress', UpdateModel.ACTION_UPDATING, '', current,
-                  current)
+        self.emit('progress', 'update', '', current, current)

         if self._downloader is not None:
             self._downloader.cancel()
@@ -226,22 +226,25 @@ class UpdateModel(gobject.GObject):

 class BundleUpdate(object):

-    def __init__(self, bundle, version, link, size):
-        self.bundle = bundle
-        self.version = version
+    def __init__(self, action, bundle_id, name, new_version, link, size,
+                 old_version=None, icon=None):
+        self.bundle_id = bundle_id
+        self.name = name
+        self.old_version = old_version
+        self.new_version = new_version
         self.link = link
         self.size = size
+        # Specify whether installing a new bundle or updating an
+        # existing one
+        self.action = action
+        self.icon = icon


 class _Downloader(gobject.GObject):
-    _CHUNK_SIZE = 10240  # 10K
     __gsignals__ = {
-        'progress': (gobject.SIGNAL_RUN_FIRST,
-                     gobject.TYPE_NONE,
-                     ([float])),
-        'error': (gobject.SIGNAL_RUN_FIRST,
-                  gobject.TYPE_NONE,
-                  ([str])),
+        'progress': (gobject.SIGNAL_RUN_FIRST, gobject.TYPE_NONE, [float]),
+        'finished': (gobject.SIGNAL_RUN_FIRST, gobject.TYPE_NONE, []),
+        'error': (gobject.SIGNAL_RUN_FIRST, gobject.TYPE_NONE, [str]),
     }

     def __init__(self, bundle_update):
@@ -267,46 +270,63 @@ class _Downloader(gobject.GObject):

         try:
             self._input_stream = self._input_file.read_finish(result)
-        except:
-            self.emit('error', traceback.format_exc())
+        except glib.GError, exception:
+            self._handle_error(exception, 'Could not open %r', gfile.get_uri())
+            return
+
+        try:
+            temp_file_path = self._get_temp_file_path(self.bundle_update.link)
+        except EnvironmentError, exception:
+            self._handle_error(exception, 'Could not create temporary file')
             return

-        temp_file_path = self._get_temp_file_path(self.bundle_update.link)
         self._output_file = gio.File(temp_file_path)
-        self._output_stream = self._output_file.create()
+        try:
+            self._output_stream = self._output_file.create()
+        except glib.GError, exception:
+            self._handle_error(exception, 'Could not create output file %r',
+                               temp_file_path)
+            return

-        self._input_stream.read_async(self._CHUNK_SIZE, self.__read_async_cb,
+        self._input_stream.read_async(_CHUNK_SIZE, self.__read_async_cb,
                                       gobject.PRIORITY_LOW)

     def __read_async_cb(self, input_stream, result):
         if self._cancelling:
             return

-        data = input_stream.read_finish(result)
+        try:
+            data = input_stream.read_finish(result)
+        except glib.GError, exception:
+            self._handle_error(exception, 'Error while reading %r',
+                               self._input_file.get_uri())
+            return

-        if data is None:
-            # TODO
-            pass
-        elif not data:
-            logging.debug('closing input stream')
+        if not data:
+            logging.debug('Finished reading')
             self._input_stream.close()
             self._check_if_finished_writing()
         else:
             self._pending_buffers.append(data)
-            self._input_stream.read_async(self._CHUNK_SIZE,
+            self._input_stream.read_async(_CHUNK_SIZE,
                                           self.__read_async_cb,
                                           gobject.PRIORITY_LOW)

         self._write_next_buffer()

-    def __write_async_cb(self, output_stream, result, user_data):
+    def __write_async_cb(self, output_stream, result):
         if self._cancelling:
             return

-        count = output_stream.write_finish(result)
+        try:
+            count = output_stream.write_finish(result)
+        except glib.GError, exception:
+            self._handle_error(exception, 'Error while writing %r',
+                               self._output_file.get_uri())
+            return

         self._downloaded_size += count
-        progress = self._downloaded_size / float(self.bundle_update.size)
+        progress = float(self._downloaded_size) / self.bundle_update.size
         self.emit('progress', progress)

         self._check_if_finished_writing()
@@ -317,22 +337,16 @@ class _Downloader(gobject.GObject):
     def _write_next_buffer(self):
         if self._pending_buffers and not self._output_stream.has_pending():
             data = self._pending_buffers.pop(0)
-            # TODO: we pass the buffer as user_data because of
-            # http://bugzilla.gnome.org/show_bug.cgi?id=564102
             self._output_stream.write_async(data, self.__write_async_cb,
-                                            gobject.PRIORITY_LOW,
-                                            user_data=data)
+                                            gobject.PRIORITY_LOW)

     def _get_temp_file_path(self, uri):
-        # TODO: Should we use the HTTP headers for the file name?
-        scheme_, netloc_, path, params_, query_, fragment_ = \
-                urlparse(uri)
-        path = os.path.basename(path)
+        path = urlparse(uri)[2]
+        base_name, extension_ = os.path.splitext(os.path.basename(path))

         if not os.path.exists(env.get_user_activities_path()):
             os.makedirs(env.get_user_activities_path())

-        base_name, extension_ = os.path.splitext(path)
         fd, file_path = tempfile.mkstemp(dir=env.get_user_activities_path(),
                 prefix=base_name, suffix='.xo')
         os.close(fd)
@@ -344,11 +358,34 @@ class _Downloader(gobject.GObject):
         return self._output_file.get_path()

     def _check_if_finished_writing(self):
-        if not self._pending_buffers and \
-                not self._output_stream.has_pending() and \
-                self._input_stream.is_closed():
+        if (self._pending_buffers or self._output_stream.has_pending() or
+            not self._input_stream.is_closed()):
+            return

-            logging.debug('closing output stream')
+        logging.debug('Finished writing')
+        try:
             self._output_stream.close()
+        except glib.GError, exception:
+            self._handle_error(exception, 'Error closing output file %r',
+                               self._output_file.get_path())
+            return

-            self.emit('progress', 1.0)
+        self.emit('finished')
+
+    def _handle_error(self, exception, message, *args):
+        """Report error to UI and clean up."""
+        logging.error(message + ': %s', *(args + [unicode(exception)]))
+        self.emit('error', unicode(exception).encode('utf-8'))
+        self._close_all_noerror()
+
+    def _close_all_noerror(self):
+        """Close all open streams and ignore errors.
+
+        Used to clean up after an error occured.
+        """
+        for stream in [self._input_stream, self._output_stream]:
+            try:
+                if not stream.is_closed():
+                    stream.close()
+            except glib.GError, exception:
+                logging.warning('Ignored error %r', unicode(exception))
diff --git a/extensions/cpsection/updater/view.py b/extensions/cpsection/updater/view.py
index 814658f..a98eb0a 100644
--- a/extensions/cpsection/updater/view.py
+++ b/extensions/cpsection/updater/view.py
@@ -17,14 +17,15 @@

 from gettext import gettext as _
 from gettext import ngettext
-import locale
 import logging

+import glib
 import gobject
 import gtk

 from sugar.graphics import style
 from sugar.graphics.icon import Icon, CellRendererIcon
+from sugar.util import format_size

 from jarabe.controlpanel.sectionview import SectionView

@@ -59,9 +60,9 @@ class ActivityUpdater(SectionView):
         bottom_label.set_line_wrap(True)
         bottom_label.set_justify(gtk.JUSTIFY_LEFT)
         bottom_label.props.xalign = 0
-        bottom_label.set_markup(
-                _('Software updates correct errors, eliminate security ' \
-                  'vulnerabilities, and provide new features.'))
+        text = _('Software updates correct errors, eliminate security'
+                 ' vulnerabilities, and provide new features.')
+        bottom_label.set_text(text)
         self.pack_start(bottom_label, expand=False)
         bottom_label.show()

@@ -114,37 +115,41 @@ class ActivityUpdater(SectionView):
             self._update_box = None

     def __progress_cb(self, model, action, bundle_name, current, total):
-        if current == total and action == UpdateModel.ACTION_CHECKING:
-            self._finished_checking()
-            return
-        elif current == total:
-            self._finished_updating(int(current))
+        logging.debug('__progress_cb %r %r %r %r', action, bundle_name,
+                      current, total)
+        if current == total:
+            if action == 'check':
+                self._finished_checking()
+            else:
+                self._finished_updating(int(current))
+
             return

-        if action == UpdateModel.ACTION_CHECKING:
-            message = _('Checking %s...') % bundle_name
-        elif action == UpdateModel.ACTION_DOWNLOADING:
-            message = _('Downloading %s...') % bundle_name
-        elif action == UpdateModel.ACTION_UPDATING:
-            message = _('Updating %s...') % bundle_name
+        if action == 'check' and not bundle_name:
+            message = _('Fetching update information')
+        elif action == 'check':
+            message = _('Checking %s...') % (bundle_name, )
+        elif action == 'download':
+            message = _('Downloading %s...') % (bundle_name, )
+        elif action == 'update':
+            message = _('Updating %s...') % (bundle_name, )

         self._switch_to_progress_pane()
         self._progress_pane.set_message(message)
         self._progress_pane.set_progress(current / float(total))

     def _finished_checking(self):
-        logging.debug('ActivityUpdater._finished_checking')
         available_updates = len(self._model.updates)
         if not available_updates:
             top_message = _('Your software is up-to-date')
         else:
-            top_message = ngettext('You can install %s update',
-                                   'You can install %s updates',
+            top_message = ngettext('You can install %d update',
+                                   'You can install %d updates',
                                    available_updates)
-            top_message = top_message % available_updates
+            top_message = top_message % (available_updates, )
             top_message = gobject.markup_escape_text(top_message)

-        self._top_label.set_markup('<big>%s</big>' % top_message)
+        self._top_label.set_markup('<big>%s</big>' % (top_message, ))

         if not available_updates:
             self._clear_center()
@@ -156,13 +161,13 @@ class ActivityUpdater(SectionView):
         self._refresh()

     def _refresh(self):
-        top_message = _('Checking for updates...')
-        self._top_label.set_markup('<big>%s</big>' % top_message)
+        top_message = glib.markup_escape_text(_('Checking for updates...'))
+        self._top_label.set_markup('<big>%s</big>' % (top_message, ))
         self._model.check_updates()

     def __install_button_clicked_cb(self, button):
-        text = '<big>%s</big>' % _('Installing updates...')
-        self._top_label.set_markup(text)
+        text = glib.markup_escape_text(_('Installing updates...'))
+        self._top_label.set_markup('<big>%s</big>' % (text, ))
         self._model.update(self._update_box.get_bundles_to_update())

     def __cancel_button_clicked_cb(self, button):
@@ -170,11 +175,11 @@ class ActivityUpdater(SectionView):

     def _finished_updating(self, installed_updates):
         logging.debug('ActivityUpdater._finished_updating')
-        top_message = ngettext('%s update was installed',
-                               '%s updates were installed', installed_updates)
-        top_message = top_message % installed_updates
-        top_message = gobject.markup_escape_text(top_message)
-        self._top_label.set_markup('<big>%s</big>' % top_message)
+        top_message = ngettext('%d update was installed',
+                               '%d updates were installed', installed_updates)
+        top_message = top_message % (installed_updates, )
+        top_message = glib.markup_escape_text(top_message)
+        self._top_label.set_markup('<big>%s</big>' % (top_message, ))
         self._clear_center()

     def undo(self):
@@ -273,8 +278,8 @@ class UpdateBox(gtk.VBox):
             if row[UpdateListModel.SELECTED]:
                 total_size += row[UpdateListModel.SIZE]

-        markup = _('Download size: %s') % _format_size(total_size)
-        self._size_label.set_markup(markup)
+        text = _('Download size: %s') % (format_size(total_size), )
+        self._size_label.set_text(text)

     def _update_install_button(self):
         for row in self._update_list.props.model:
@@ -357,35 +362,24 @@ class UpdateListModel(gtk.ListStore):

         for bundle_update in model.updates:
             row = [None] * 5
-            row[self.BUNDLE_ID] = bundle_update.bundle.get_bundle_id()
+            row[self.BUNDLE_ID] = bundle_update.bundle_id
             row[self.SELECTED] = True
-            row[self.ICON_FILE_NAME] = bundle_update.bundle.get_icon()
-
-            details = _('From version %(current)s to %(new)s (Size: %(size)s)')
-            details = details % \
-                    {'current': bundle_update.bundle.get_activity_version(),
-                     'new': bundle_update.version,
-                     'size': _format_size(bundle_update.size)}
-
-            row[self.DESCRIPTION] = '<b>%s</b>\n%s' % \
-                    (bundle_update.bundle.get_name(), details)
-
+            row[self.ICON_FILE_NAME] = bundle_update.icon
+
+            if bundle_update.action == 'update':
+                details = _('From version %(current)s to %(new)s (Size:'
+                            ' %(size)s)')
+                details = details % {'current': bundle_update.old_version,
+                                     'new': bundle_update.new_version,
+                                     'size': format_size(bundle_update.size)}
+            elif bundle_update.action == 'install':
+                details = _('Installing new activity version %(new)s (Size:'
+                            ' %(size)s)')
+                details = details % {'new': bundle_update.new_version,
+                                     'size': format_size(bundle_update.size)}
+
+            row[self.DESCRIPTION] = '<b>%s</b>\n%s' % (
+                glib.markup_escape_text(bundle_update.name),
+                glib.markup_escape_text(details))
             row[self.SIZE] = bundle_update.size
-
             self.append(row)
-
-
-def _format_size(size):
-    """Convert a given size in bytes to a nicer better readable unit"""
-    if size == 0:
-        # TRANS: download size is 0
-        return _('None')
-    elif size < 1024:
-        # TRANS: download size of very small updates
-        return _('1 KB')
-    elif size < 1024 * 1024:
-        # TRANS: download size of small updates, e.g. '250 KB'
-        return locale.format_string(_('%.0f KB'), size / 1024.0)
-    else:
-        # TRANS: download size of updates, e.g. '2.3 MB'
-        return locale.format_string(_('%.1f MB'), size / 1024.0 / 1024)
--
1.7.2.5



More information about the Dextrose mailing list