[Sugar-devel] Plain Query Format proposal

[Sascha Silbe]

On Sun, Jul 19, 2009 at 01:11:42AM +0000, Aleksey Lim wrote:

> I'm going to implement [1] for 0.86.
Have you taken a look at my datastore redesign proposal [3]? I'm 
currently busy implementing the new API in the old datastore. Or rather 
busy adapting the Python side (several parts of Sugar each interface 
directly with the datastore via DBus, I'm trying to unify it) - the 
datastore is already mostly done, though not thoroughly tested for 
obvious reasons. Have already found a number of (pre-existing) bugs in 
the Journal, though. ;)

Added today:
- list and range queries
- prefixes in Xapian (string) queries

Still missing:
- support for (filtering by) arbitrary metadata
    - quite hard to implement: range matches are only supported on Xapian
      Values, not Xapian Terms. Values are accessed by a 32bit number and
      I'm not sure how storage space will be affected by "holes" in the
      numbering.
    - probably won't be implemented for current datastore
- regular expression search (requires full scan)
    - probably won't be implemented for current datastore
- match inversion ("!" prefix)


Long story short: AFAICT most of your proposal is already supported by 
the textsearch() API call. The "few" metadata keys you have proposed can 
be added to (=hardcoded in) the DS, avoiding the "arbitrary metadata" 
problem for now.
The proposed unit tests would be quite nice to have, of course. :)

What I don't understand about your proposal, though, is why you intend 
to _replace_ the query dictionary instead of supplementing it like in my 
proposal (and already supported in the old datastore by passing 'query': 
'whatever' as part of the query dictionary). In what way is formatting 
(including quoting/escaping!) a string easier than creating a 
dictionary?

> * let experienced users use system terms in Journal search bar
FWIW, the current Journal already supports this, albeit with a very 
limited numbers of terms and single-character prefixes (no colon):

_PREFIX_UID = 'Q'
_PREFIX_ACTIVITY = 'A'
_PREFIX_ACTIVITY_ID = 'I'
_PREFIX_MIME_TYPE = 'M'
_PREFIX_KEEP = 'K'

My prototype also supports prefixes now (e.g. "mime_type:text/plain").

>   * existed implementation has hard-coded logic for example in case of 
> having several mime_types in query(all mime_types will be ORed despite 
> what user wants).
Sorry, I fail to see your point.


What both your and my proposal don't address, BTW, are stemming and 
spelling corrections - those need to know what language a given string 
is in, so are not that easy to handle.


> [1] http://wiki.sugarlabs.org/go/Features/Plain_Query_Format
[3] http://git.sugarlabs.org/projects/versionsupport-project/repos/mainline/blobs/master/datastore-redesign.html (follow "raw blob" link)

CU Sascha

-- 
http://sascha.silbe.org/
http://www.infra-silbe.de/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 489 bytes
Desc: Digital signature
Url : http://lists.sugarlabs.org/archive/sugar-devel/attachments/20090719/310ac489/attachment.pgp