[Sugar-devel] Problems downloading PG titles without a web browser

[Daniel Narvaez]

Actually it seems like you might be able to do it with urllib too.

http://docs.python.org/2/library/urllib.html

Search for urlopener, there is an example.

On Monday, 11 February 2013, Daniel Narvaez wrote:

>
> http://git.sugarlabs.org/sugar-toolkit-gtk3/sugar-toolkit-gtk3/blobs/master/src/sugar3/network.py
>
> That code *looks* trivial to port to urllib2. You could do that, submit
> the patch and in the meantime just copy and paste the ported code in your
> activity (it doesn't seem to have dependencies on other toolkit stuff).
> Well, unless it's a gtk2 activity, in that case you can skip the "submit
> the patch" step :)
>
> On Monday, 11 February 2013, James Simmons wrote:
>
>> Marcello (and everyone else who replied):
>>
>> It looks like the mirrors have the same policy.  I'm currently using this
>> to download from the URL:
>>
>>
>> http://doc.sugarlabs.org/epydocs/sugar.network.GlibURLDownloader-class.html
>>
>> I'm not seeing a way to put a user agent in using this code.  It looks
>> like urllib2 supports it.  I used the other because I was already using it
>> for collaboration and it gives me a way to do a progress report on the
>> download.  I appreciate everyone's detective work and suggestions.
>>
>> James Simmons
>>
>>
>> On Sun, Feb 10, 2013 at 1:22 PM, Marcello Perathoner <
>> marcello at perathoner.de> wrote:
>>
>>> On 02/10/2013 06:46 PM, James Simmons wrote:
>>>
>>>> I had created an Activity for the One Laptop Per Child project which
>>>> downloads and reads the Plain Text version of PG titles.  I use the
>>>> offline
>>>> catalogue and some Python code to list available titles in the catalogue
>>>> that match a search string.  For instance the child might enter "Twain"
>>>> and
>>>> get a list of all the books by and about Twain.  Then the child can
>>>> download the book using the same program.
>>>>
>>>> This has been working well for years but lately it has stopped working.
>>>> When I try to debug it I see it trying to download this URL for
>>>> instance:
>>>>
>>>> http://www.gutenberg.org/dirs/**1/1/119/119.zip<http://www.gutenberg.org/dirs/1/1/119/119.zip>
>>>>
>>>
>>>
>>> Supply a user-agent that clearly defines your app and provides a way to
>>> contact you eg.
>>>
>>>   OLPCReader/1.0; +http://www.olpc-reader.org/**app-info.html<http://www.olpc-reader.org/app-info.html>
>>>
>>>   OLPCReader/1.0; +mailto:me at example.com
>>>
>>> The standard Python-urllib user agent will not do!
>>>
>>>
>>> or
>>>
>>>
>>> use a mirror site. See: www.gutenberg.org/MIRRORS.ALL
>>>
>>>
>>>
>>>
>>>
>>> Regards
>>>
>>> --
>>> Marcello Perathoner
>>> webmaster at gutenberg.org
>>>
>>
>>
>
> --
> Daniel Narvaez
>
>

-- 
Daniel Narvaez
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sugarlabs.org/archive/sugar-devel/attachments/20130211/7baee21d/attachment.html>