[Sugar-devel] Plain Query Format proposal

[Aleksey Lim]

On Sun, Jul 19, 2009 at 02:02:14PM +0200, Sascha Silbe wrote:
> On Sun, Jul 19, 2009 at 01:11:42AM +0000, Aleksey Lim wrote:
> 
> > I'm going to implement [1] for 0.86.
> Have you taken a look at my datastore redesign proposal [3]? I'm 
> currently busy implementing the new API in the old datastore. Or rather 
> busy adapting the Python side (several parts of Sugar each interface 
> directly with the datastore via DBus, I'm trying to unify it) - the 
> datastore is already mostly done, though not thoroughly tested for 
> obvious reasons. Have already found a number of (pre-existing) bugs in 
> the Journal, though. ;)
> 
> Added today:
> - list and range queries
> - prefixes in Xapian (string) queries
> 
> Still missing:
> - support for (filtering by) arbitrary metadata
>     - quite hard to implement: range matches are only supported on Xapian
>       Values, not Xapian Terms. Values are accessed by a 32bit number and
>       I'm not sure how storage space will be affected by "holes" in the
>       numbering.
>     - probably won't be implemented for current datastore
> - regular expression search (requires full scan)
>     - probably won't be implemented for current datastore
> - match inversion ("!" prefix)
> 
> 
> Long story short: AFAICT most of your proposal is already supported by 
> the textsearch() API call. The "few" metadata keys you have proposed can 
> be added to (=hardcoded in) the DS, avoiding the "arbitrary metadata" 
> problem for now.
> The proposed unit tests would be quite nice to have, of course. :)
> 
> What I don't understand about your proposal, though, is why you intend 
> to _replace_ the query dictionary instead of supplementing it like in my 
> proposal (and already supported in the old datastore by passing 'query': 
> 'whatever' as part of the query dictionary). In what way is formatting 
> (including quoting/escaping!) a string easier than creating a 
> dictionary?
> 
> > * let experienced users use system terms in Journal search bar
> FWIW, the current Journal already supports this, albeit with a very 
> limited numbers of terms and single-character prefixes (no colon):
> 
> _PREFIX_UID = 'Q'
> _PREFIX_ACTIVITY = 'A'
> _PREFIX_ACTIVITY_ID = 'I'
> _PREFIX_MIME_TYPE = 'M'
> _PREFIX_KEEP = 'K'
> 
> My prototype also supports prefixes now (e.g. "mime_type:text/plain").
> 
> >   * existed implementation has hard-coded logic for example in case of 
> > having several mime_types in query(all mime_types will be ORed despite 
> > what user wants).
> Sorry, I fail to see your point.
> 
> 
> What both your and my proposal don't address, BTW, are stemming and 
> spelling corrections - those need to know what language a given string 
> is in, so are not that easy to handle.
> 
> 
> > [1] http://wiki.sugarlabs.org/go/Features/Plain_Query_Format
> [3] http://git.sugarlabs.org/projects/versionsupport-project/repos/mainline/blobs/master/datastore-redesign.html (follow "raw blob" link)
> 
> CU Sascha
> 
> -- 
> http://sascha.silbe.org/
> http://www.infra-silbe.de/

The question is are you targeting your features for 0.86 or 0.88?
Plain_Query_Format is just 50-60 lines of xapian related code(for
current trunk), so its not a problem to implement it in 0.86

-- 
Aleksey