[Sugar-devel] Searching for datastore entries by title

Tomeu Vizoso tomeu at sugarlabs.org
Wed Sep 16 08:11:59 EDT 2009


On Wed, Sep 16, 2009 at 14:05, Bert Freudenberg <bert at freudenbergs.de> wrote:
>
> On 16.09.2009, at 13:48, Tomeu Vizoso wrote:
>
>> On Wed, Sep 16, 2009 at 12:09, Bert Freudenberg <bert at freudenbergs.de>
>> wrote:
>>>
>>> On 16.09.2009, at 03:11, Gary C Martin wrote:
>>>
>>>> Hi Bert,
>>>>
>>>> On 16 Sep 2009, at 00:51, Bert Freudenberg wrote:
>>>>
>>>>> I want to find a datastore entry with a specific title and mime_type.
>>>>> But it returns *all* entries with that mimetype instead. Here's the
>>>>> log:
>>>>>
>>>>> 1253058031.600170 DEBUG root: datastore.find
>>>>> dbus.Dictionary({dbus.String(u'mime_type'):
>>>>> dbus.String(u'application/
>>>>> x-squeak-project'), dbus.String(u'title'): dbus.String(u'The Hole')},
>>>>> signature=dbus.Signature('ss'))
>>>>> 1253058031.600731 DEBUG root: parse_query
>>>>> dbus.Dictionary({dbus.String(u'mime_type'):
>>>>> dbus.String(u'application/
>>>>> x-squeak-project'), dbus.String(u'title'): dbus.String(u'The Hole')},
>>>>> signature=dbus.Signature('ss')) None
>>>>> 1253058031.600886 WARNING root: Unknown term:
>>>>> dbus.String(u'title')=dbus.String(u'The Hole')
>>>>> 1253058031.601042 WARNING root: Unknown term(s):
>>>>> {dbus.String(u'mime_type'): dbus.String(u'application/x-squeak-
>>>>> project'), dbus.String(u'title'): dbus.String(u'The Hole')}
>>>>> 1253058031.601170 DEBUG root: queries:
>>>>> ['Xapian::Query(FMapplication/x-
>>>>> squeak-project)']
>>>>> 1253058031.602399 DEBUG org.laptop.sugar.DataStore: find():
>>>>> 0.0017540454864501953
>>>>>
>>>>> What does "unknown term" mean? It seams to get ignored. But it's in
>>>>> the log quite often, not only from my query, that does not sound
>>>>> right, something seems broken:
>>>>>
>>>>> 1253057661.236064 DEBUG root: datastore.find
>>>>> dbus.Dictionary({dbus.String(u'limit'): dbus.Int32(5,
>>>>> variant_level=1), dbus.String(u'order_by'):
>>>>> dbus.Array([dbus.String(u'+timestamp')],
>>>>> signature=dbus.Signature('s'), variant_level=1),
>>>>> dbus.String(u'activity'): dbus.String(u'org.laptop.Pippy',
>>>>> variant_level=1)}, signature=dbus.Signature('sv'))
>>>>> 1253057661.236569 DEBUG root: parse_query
>>>>> dbus.Dictionary({dbus.String(u'activity'):
>>>>> dbus.String(u'org.laptop.Pippy', variant_level=1)},
>>>>> signature=dbus.Signature('sv')) None
>>>>> 1253057661.236753 WARNING root: Unknown term(s):
>>>>> {dbus.String(u'activity'): dbus.String(u'org.laptop.Pippy',
>>>>> variant_level=1)}
>>>>> 1253057661.236886 DEBUG root: queries:
>>>>> ['Xapian::Query(FAorg.laptop.Pippy)']
>>>>> 1253057661.240903 DEBUG org.laptop.sugar.DataStore: find():
>>>>> 0.0045418739318847656
>>>>>
>>>>> My test program (run in Pippy) and its output was this:
>>>>>
>>>>> =================================
>>>>> def unwrap(x):
>>>>>   if isinstance(x, list): return map(unwrap, x)
>>>>>   if isinstance(x, tuple): return tuple(map(unwrap, x))
>>>>>   if isinstance(x, dict): return dict([(unwrap(k), unwrap(v)) for
>>>>> k, v in x.iteritems()])
>>>>>   for t in [unicode, str, long, int, float, bool]:
>>>>>       if isinstance(x, t): return t(x)
>>>>>   return x
>>>>>
>>>>> import dbus
>>>>> datastore =
>>>>> dbus.SessionBus().get_object('org.laptop.sugar.DataStore',
>>>>> '/org/laptop/sugar/DataStore')
>>>>> (items,count) = datastore.find(
>>>>>   {
>>>>>     'title':'The Hole',
>>>>
>>>> Hmm FWIW, from my copy-from-journal experience, I though title was
>>>> no longer supported (I opened a ticket about in 0.84 and it was
>>>> closed as 'not a bug' and the copy-from-journal script changed). I
>>>> think you search with 'query' instead now... Might have happened as
>>>> part of Tomeu's (much welcomed) re-write of the data-store for
>>>> robustness, back in 0.84 I guess.
>>>
>>>
>>> 'query' searches for (sub-) strings in all text fields. I want an
>>> exact title match.
>>
>> Several features were dropped from the DS when it was rewritten,
>> because non-fundamental features were getting in the way of
>> fundamental ones (like retaining the users' data).
>>
>> I thought this had been discussed back then (almost a year ago).
>
> If so I forgot that title search was removed.
>
>> If there's interest, now should be easier to add more features but we
>> should remember that the DS is a critical component and we don't have
>> too many resources to make sure we don't regress in robustness.
>
>
> I wasn't really complaining, just asking ;)
>
> Where do I find what searching options are supported now?

Right now we only index what the journal needs, see the _PREFIX_
constants below:

http://git.sugarlabs.org/projects/sugar-datastore/repos/mainline/blobs/master/src/carquinyol/indexstore.py#line32

> And what about the many "Unknown term(s)" warnings? Just ignore? Shouldn't
> this not be used anymore if it is not supported?

That message warns that you are trying to filter by a term that we
don't index, but the ones that are supported should work.

Regards,

Tomeu

-- 
«Sugar Labs is anyone who participates in improving and using Sugar.
What Sugar Labs does is determined by the participants.» - David
Farning


More information about the Sugar-devel mailing list