[Sugar-devel] [PATCH v3 sugar-datastore 2/2] Add API to check/wait for index rebuild to finish (SL#1160)

[Sascha Silbe]

The data store (re)builds the index in the background (idle loop), but
has no API to wait for this to finish. While it rebuilds the index, some API
functions work differently than they do during regular operation. In
particular, the find() function returns all entries, rather than just matching
ones. The requested sort order isn't respected, either.

This fact isn't mentioned in any public API description, so very few clients
are designed with it in mind. Every client would need to duplicate an important
subset of the data store functionality: filtering and sorting. The one
advantage of this approach is that, at least in theory, the data store is ready
to operate right away, allowing the user to start working. In practice, the
advantage isn't quite as clear: Migration happens synchronously [2] and
rebuilding is either fast (removing the need for it to happen in the
background) or takes significant system resources, reducing the usefulness of
the system while the rebuild is happening.

For these reasons, for most clients it's better to just wait for the data store
to finish rebuilding. We're adding API to allow them to do just that, in both
synchronous and asynchronous ways. wait_ready() blocks until the data store is
ready, whereas the Ready signal and the check_ready() function allow
implementing a race-free, asynchronous check.

Clients can now choose (and the Activity framework should probably do so by
default) to either wait for the rebuild to complete or cope with whatever the
data store does differently during rebuild.

The additional API functions don't affect the rest of the API in any way. The
queue that we add as part of this patch handles only the outstanding
wait_ready() calls. Other invocations get handled immediately.

It could be argued that index rebuilds should happen rarely in the field and we
should just always block until the data store is ready for normal operation. If
rebuilds are happening often in the field, we should analyse and fix the reason
for this, rather than covering up by making rebuild asynchronous and providing
inconsistent API. This patch provides a reasonable middle ground that we can
build on. If or when data store rebuilds happen rarely in the field, we can
still switch to blocking at start-up.

[1] https://bugs.sugarlabs.org/ticket/1160
[2] https://bugs.sugarlabs.org/ticket/1546

Signed-off-by: Sascha Silbe <silbe at activitycentral.com>
---
v2->v3: Significantly enhanced description; no code changes

 src/carquinyol/datastore.py |   38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/src/carquinyol/datastore.py b/src/carquinyol/datastore.py
index de79500..3c0c294 100644
--- a/src/carquinyol/datastore.py
+++ b/src/carquinyol/datastore.py
@@ -52,6 +52,7 @@ class DataStore(dbus.service.Object):
     """
 
     def __init__(self, **options):
+        self._wait_ready_queue = []
         bus_name = dbus.service.BusName(DS_SERVICE,
                                         bus=dbus.SessionBus(),
                                         replace_existing=False,
@@ -164,6 +165,12 @@ class DataStore(dbus.service.Object):
             self._index_store.flush()
             self._index_updating = False
             logging.debug('Finished updating index.')
+
+            self.Ready()
+            for callback in self._wait_ready_queue:
+                callback()
+
+            self._wait_ready_queue = []
             return False
         else:
             return True
@@ -443,3 +450,34 @@ class DataStore(dbus.service.Object):
     @dbus.service.signal(DS_DBUS_INTERFACE, signature="a{sv}")
     def Unmounted(self, descriptor):
         pass
+
+    @dbus.service.method(DS_DBUS_INTERFACE,
+                         in_signature='',
+                         out_signature='b')
+    def check_ready(self):
+        """Check whether datastore is ready for processing (regular) API
+        calls.
+
+        Return True if datastore is fully operational.
+        """
+        return not self._index_updating
+
+    @dbus.service.method(DS_DBUS_INTERFACE,
+             in_signature='',
+             out_signature='',
+             async_callbacks=('async_cb', 'async_err_cb'))
+    def wait_ready(self, async_cb, async_err_cb):
+        """Block until the datastore is ready for processing (regular) API
+        calls.
+        """
+        if not self._index_updating:
+            async_cb()
+            return
+
+        self._wait_ready_queue.append(async_cb)
+
+    @dbus.service.signal(DS_DBUS_INTERFACE, signature='')
+    def Ready(self):
+        """Signal emitted after datastore is ready for processing API calls.
+        """
+        pass
-- 
1.7.9.5