[Sugar-devel] How to implement TTS + highlighting

[Benjamin M. Schwartz]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

James Simmons wrote:
> As you point out in your proposal, highlighting the word as it is spoken 
> is a big part of the benefit of what you're proposing.  If all you 
> wanted to do was capture some highlighted text in the clipboard and have 
> it spoken in a voice you can configure in a control panel, that would be 
> easy, even trivial.  It's the highlighting that's difficult.
...
> 
> If I had to write a facility that did what Read Etexts does outside of 
> the Activity I wouldn't know how to do it.  It seems to me that 
> highlighting is best done by the Activity itself.  I can't deny that it 
> would be useful to have all this work done as you have described without 
> the Activity knowing anything about it, but it doesn't seem feasible.  
> You'd have to have something that could work with gtk textareas, the 
> evince component Read uses, Abiword, and everything else that came along.

What if we forget about showing the highlighting "in place"?  Instead, the
TTS button can pop up a small window or palette showing the text from the
"clipboard" (actually the PRIMARY selection[1]).  This window would use
the widget from Read Etexts or Listen and Spell, to provide active
highlighting.  As soon as TTS completes, the window disappears.

That gives us "karaoke" highlighting with any activity that supports
selection.

> Another thing you'd have to deal with is PDFs composed of scanned in 
> book pages.  There are a lot of these around (the Internet Archive is 
> full of them) and somehow the kid trying to select words on a scanned in 
> page would have to be clued in that these words are not selectable.

The interface already does this, by not allowing selections unless evince
can extract text from the PDF.  (Note that this works even with some of
those scanned books, which often have a layer of OCR text aligned with the
scanned images, invisible until you start a selection.)

- --Ben

[1] http://en.wikipedia.org/wiki/X_Window_selection#Clipboard
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAknUP7kACgkQUJT6e6HFtqTPegCdEVWr99KHkb/VSErJKB6NC9s8
XlIAn33utSJRQ9VwF9yv3hh+q0JAywg7
=7JrQ
-----END PGP SIGNATURE-----