[ASLO] ASLO build and deployment process (Jatin Dhankhar)
Tony Anderson
tony_anderson at usa.net
Wed Apr 12 20:20:10 EDT 2017
Hi Jatin
At the moment, I think you should focus on the 25 activity test set and
getting it working with django. I find it is much more productive to
move from one working version to another with added capability. If we
can arrange to host the development on a Sugar Labs server, it should
make both the repository itself (download.sugarlabs.org) and the ASLO
metadata directly accessible.
My IRC nickname is tony37. However, I am never on #sugar (a twelve hour
time difference so the normal participants are in bed). Last year,
Utkarsh and I met whenever he felt the need to discuss a technical issue
in detail. We limited the meetings to one hour. That proved quite
productive.
Tony
On 04/13/2017 01:14 AM, Jatin Dhankhar wrote:
> Hi,
>
> I think we need to agree on the use of IRC. If you want to
> communicate with members of the community, you must go where they
> are (#sugar). If you want a one-to-one meeting on IRC with me, I
> would suggest #sugar-newbies. It is normally dormant but works
> well and saves a log for later review. It worked well for meetings
> with Utkarsh Tiwari during last year's GSOC.
>
> Sure, whatever works :). What is your IRC nickname ?
>
> There are two things that you will need to have local to the
> django project. First is the directory download.sugarlabs.org
> <http://download.sugarlabs.org/> which has the Sugar activity bundles
>
> Do I need to mirror the whole setup/directory ?
>
> When talking about scraping you probably meant
> http://activities.sugarlabs.org instead of
> http://download.sugarlabs.org/, right ?
> Also for scraping, Scrapy <https://scrapy.org/> seems to more popular
> than beautifulsoup ?
>
> Also a big thanks for including Walter in the discussion :D
>
> - Jatin Dhankhar
>
> On Wed, Apr 12, 2017 at 6:30 AM, Tony Anderson <tony_anderson at usa.net
> <mailto:tony_anderson at usa.net>> wrote:
>
> Hi, Jatin
>
> I think we need to agree on the use of IRC. If you want to
> communicate with members of the community, you must go where they
> are (#sugar). If you want a one-to-one meeting on IRC with me, I
> would suggest #sugar-newbies. It is normally dormant but works
> well and saves a log for later review. It worked well for meetings
> with Utkarsh Tiwari during last year's GSOC.
>
> There are two things that you will need to have local to the
> django project. First is the directory download.sugarlabs.org
> <http://download.sugarlabs.org> which has the Sugar activity
> bundles. The second is the 'metadata' in the mysql db. For
> scraping, I would recommend BeautifulSoup (bs4). The trick will be
> to decide what data we want to capture and add to the json.
>
> The json fields in activities.json are ones I chose for the
> minimal system. You may want to include other information such as
> the number of downloads, which collections (should be entered as
> tags in a tag-field) and so on. One item I have referred to as
> flags (I marked some as X but don't remember what that meant, oh
> well). The intent is to record the platforms where the activity
> works. We also should provide links to the homepage, repository
> page, and update page (whatever that is). I think if you have a
> working scrape tool, the data it collects can be expanded as
> needed (assuming the tool runs in a reasonable time).
>
> Naturally, it would be easier if you have access to the db directly.
>
> Tony
>
>
> On 04/12/2017 01:40 AM, Jatin Dhankhar wrote:
>>
>> One thing you could look at. On activities.sugarlabs,org, can
>> you determine from Remora where the metadata is stored? I
>> assume a db. Currently I am thinking to use BeautifulSoup to
>> scrape the site to get that data, but it would be much easier
>> to access the data directly.
>>
>> As per wiki
>> https://wiki.mozilla.org/Update:Remora_Server_Requirements#SVN.2C_DB_and_app_config
>> <https://wiki.mozilla.org/Update:Remora_Server_Requirements#SVN.2C_DB_and_app_config> data
>> is stored in mysql database. I don't have access to the
>> production server where ASLO is currently running, following file
>> https://github.com/sugarlabs/aslo/blob/master/aslo/db-update.sh#L9 <https://github.com/sugarlabs/aslo/blob/master/aslo/db-update.sh#L9> confirms
>> that data is stored in a mysql db. However it would be
>> interesting and fun to scrape the data from live site. I would do
>> that.
>>
>> Thanks will poke around the code, looks to me it's a django app
>> and I have to mount it on my django project, thanks :)
>>
>> If you are talking about IRC as a place to meet Sugar
>> community members, use the freenode #sugar. This is probably
>> most active from 8-17 EST (UTC-5). I am currently in the
>> Philippines which is UTC+ 7.
>>
>>
>> Yes, tried that. https://gitter.im fits in naturally with Github
>> (really sorry for suggesting a new mode of communication everyday) 😅
>>
>> - Jatin Dhankhar
>>
>>
>> On Tue, Apr 11, 2017 at 6:14 AM, Tony Anderson
>> <tony_anderson at usa.net <mailto:tony_anderson at usa.net>> wrote:
>>
>> If you are talking about IRC as a place to meet Sugar
>> community members, use the freenode #sugar. This is probably
>> most active from 8-17 EST (UTC-5). I am currently in the
>> Philippines which is UTC+ 7.
>>
>> Localization of Python activities is done by Pootle, when
>> implemented by the developer. The developer does something
>> like the following:
>>
>> from gettext import gettext as _
>>
>> self.copy.set_tooltip(_('Copy'))
>>
>> In this way, all text displayed is taken from a po file based
>> on the locale (e.g. en.po or hi.po). This is a simplification
>> as the actual file is compressed: en.mo, hi.mo. These files
>> are in the activity bundle. The detail is that when a new
>> version is released, there is a master file: Paint.pot from
>> which the local language files are built. This needs to be
>> submitted to translate.sugarlabs.org
>> <http://translate.sugarlabs.org> which maintains a copy.
>> However, then the localized version needs to be added back to
>> the bundle. However, the localizations can take months for
>> 100 languages so how synchronize the po directory with the
>> activity release is difficult.
>>
>> The sugar3 vs sugar issue is decided. The community wants to
>> move to sugar3 (gtk3). The problem is that less that 20% of
>> the activities have been converted.
>> The ones that have been converted are low hanging fruit. The
>> unconverted ones may require intensive work (gimp which
>> developed gtk originally has not made the conversion).
>>
>> One thing you could look at. On activities.sugarlabs,org, can
>> you determine from Remora where the metadata is stored? I
>> assume a db. Currently I am thinking to use BeautifulSoup to
>> scrape the site to get that data, but it would be much easier
>> to access the data directly.
>>
>> Yesterday afternoon, the ISP restored service. Last time it
>> died after two days, but I am keeping my fingers crossed. I
>> am attaching the django stuff.
>>
>> Tony
>>
>>
>> On 04/11/2017 01:36 AM, Jatin Dhankhar wrote:
>>> Hi Tony,
>>>
>>> Normally, we use http://chat.sugarlabs.orgor on
>>> freenode: sugar-meeting or sugar-newbies. These are
>>> logged sites so that there is a record. The second is
>>> more appropriate since sugar-meeting is used for SLOB
>>> meetings and the like. The real problem with IRC is time
>>> zones. Email has the advantage that either party can
>>> send or receive at any time. Last year with a GSOC
>>> mentee we used sugar-newbies by arranging a specific
>>> meeting time in advance.
>>>
>>> Yes, that is correct, main issue in communication barrier is
>>> due to timezone issues. Since most of the people are
>>> familiar and are available on IRC, it's seems to be the
>>> primary channel of communication along with mailing lists
>>> and email. But since you said we can use anything else,
>>> giving Slack a try won't hurt (if issue is about not using
>>> closed source software then IRC is fine, or we can try
>>> Mattermost <https://about.mattermost.com/>).
>>>
>>> Another part of the process is how to update
>>> 'translate.sugarlabs.org
>>> <http://translate.sugarlabs.org/>' with the
>>> corresponding POT file to enable localization. We can
>>> get help from Chris Leonard on this.
>>>
>>>
>>> I am not aware on how localization works. Do we need to
>>> download relevant files and bundle them with the acitvity
>>> before making it available on ASLO ?
>>>
>>> I have my Django version available - but the internet
>>> problems here are still unresolved. The technician is
>>> supposed to make another visit today to see what is
>>> wrong with our connection. Let me know if and when you
>>> think this will be useful to you.
>>>
>>> Let me know when your connection is stable and I would start.
>>> What are the things you need me to do in the meantime ?
>>>
>>> One open issue is sugar3 vs sugar. Currently two
>>> versions of Sugar are released. The sugar version
>>> supports gtk while sugar3 supports gtk3. Unfortunately,
>>> gtk3 was developed to be totally incompatible with gtk.
>>> For example, incorporation of one gtk3 feature requires
>>> that all direct and indirect references to gtk be
>>> removed or the activity will throw an exception. Several
>>> of the gtk3 conversions failed to meet this test and so
>>> fail. The issue is whether curated activities be limited
>>> to ones converted to gtk3. The positive is that Sugar
>>> could revert to releasing and maintaining only a single
>>> version. The downside is that 100 or more activities
>>> will no longer be available. Specifically, in our
>>> implementation of ASLO, we need to show which versions
>>> of an activity work on which versions of Sugar (e.g.
>>> i86, arm, amd64, sugar or sugar3, and so on). We also
>>> need to show which ones support localization. There are
>>> many English activities and many Spanish activities that
>>> make no provision for localization. Luckily there are
>>> many that have no language component. However, for many
>>> of these, some kind of help is needed to convey the way
>>> the activity works.
>>>
>>>
>>> Some people believe GTK3 is slightly better
>>> <https://www.reddit.com/r/linux/comments/3e3q8n/is_there_a_technical_reason_why_gtk3_is_better/> and
>>> I think GTK3 will stay but that should be asked in community
>>> and voted upon and taking in considerations cost of
>>> development and porting, only a discussion will help in this
>>> one.
>>>
>>> You are wading into a deep and vast body of water!
>>>
>>> As long as I have something to hold onto, I will not drown 😅
>>>
>>> Thanks,
>>> Jatin Dhankhar
>>>
>>> On Mon, Apr 10, 2017 at 9:11 AM, Tony Anderson
>>> <tony_anderson at usa.net <mailto:tony_anderson at usa.net>> wrote:
>>>
>>> Hi, Jatin
>>>
>>> Normally, we use http://chat.sugarlabs.org or on
>>> freenode: sugar-meeting or sugar-newbies. These are
>>> logged sites so that there is a record. The second is
>>> more appropriate since sugar-meeting is used for SLOB
>>> meetings and the like. The real problem with IRC is time
>>> zones. Email has the advantage that either party can
>>> send or receive at any time. Last year with a GSOC
>>> mentee we used sugar-newbies by arranging a specific
>>> meeting time in advance.
>>>
>>> I haven't heard from Walter, but my preference would be
>>> to use the Sugarlabs server since the content is largely
>>> already there and it would be easier to make it the
>>> official site if that were decided. So in the short run,
>>> I think you should do whatever is best for your own
>>> development process.
>>>
>>> I am not sure how CI fits into this. If the activity
>>> development is done on GitHub, then the deployment model
>>> is to run setup.py to create an xo bundle and then copy
>>> that bundle to the appropriate location in the
>>> download.sugarlabs.org <http://download.sugarlabs.org>
>>> tree. Assuming the update results from a PR, the
>>> deployer would need to update the activity information
>>> on ASLO appropriately. However, that process depends on
>>> where that data (metadata) is stored. Another part of
>>> the process is how to update 'translate.sugarlabs.org
>>> <http://translate.sugarlabs.org>' with the corresponding
>>> POT file to enable localization. We can get help from
>>> Chris Leonard on this.
>>>
>>> I have my Django version available - but the internet
>>> problems here are still unresolved. The technician is
>>> supposed to make another visit today to see what is
>>> wrong with our connection. Let me know if and when you
>>> think this will be useful to you.
>>>
>>> I have now tested most of the activities (~400). I was
>>> optimistic in the number that work out of the box.
>>> However, a part of this is running them in the Ubuntu
>>> version of Sugar (amd64). There are many activities
>>> which launch object code (mostly c) which is dependent
>>> on the architecture. I am now trying to repeat the tests
>>> on an XO-1.75. One issue on Ubuntu is that many
>>> activities assume a 1200x900 screen and so on a 1024X768
>>> screen overflow. This makes some of the games unusable
>>> since part of the controls are off the screen. Because
>>> of the internet problems, the untested activities tend
>>> to be new ones since I was using my local repository
>>> which is a snapshot taken several months ago. The other
>>> group are the GCompris activities (about 70).
>>>
>>> My intent is to build a 'curated' repository of
>>> activities known to work and be usable on the XO and on
>>> Ubuntu (or such other platform that Sugar may choose to
>>> support). Most of the currently not work activities have
>>> software dependencies no longer included in the current
>>> Sugar release. So the curated library will grow as
>>> activities are repaired over time.
>>>
>>> One open issue is sugar3 vs sugar. Currently two
>>> versions of Sugar are released. The sugar version
>>> supports gtk while sugar3 supports gtk3. Unfortunately,
>>> gtk3 was developed to be totally incompatible with gtk.
>>> For example, incorporation of one gtk3 feature requires
>>> that all direct and indirect references to gtk be
>>> removed or the activity will throw an exception. Several
>>> of the gtk3 conversions failed to meet this test and so
>>> fail. The issue is whether curated activities be limited
>>> to ones converted to gtk3. The positive is that Sugar
>>> could revert to releasing and maintaining only a single
>>> version. The downside is that 100 or more activities
>>> will no longer be available. Specifically, in our
>>> implementation of ASLO, we need to show which versions
>>> of an activity work on which versions of Sugar (e.g.
>>> i86, arm, amd64, sugar or sugar3, and so on). We also
>>> need to show which ones support localization. There are
>>> many English activities and many Spanish activities that
>>> make no provision for localization. Luckily there are
>>> many that have no language component. However, for many
>>> of these, some kind of help is needed to convey the way
>>> the activity works.
>>>
>>> You are wading into a deep and vast body of water!
>>>
>>> Tony
>>>
>>>
>>> On 04/10/2017 12:00 AM, Jatin Dhankhar wrote:
>>>> Hi,
>>>> Sorry for the delay. I went through the polls tutorials
>>>> and I think I am getting hang of Django. I have one
>>>> query that is out of context, what is your IRC setup ?
>>>> IRC doesn't allow message to be delivered or stored
>>>> once either party is offline, people login through a
>>>> external server for IRC's to maintain their
>>>> availability in a channel. May I suggest something like
>>>> Slack or Flock for communication. IRC is good for quick
>>>> and fast connection but Slack and alternatives allow
>>>> easy communication. (Just a suggestion, though)
>>>>
>>>> Should I deploy the same polls app on DigitalOcean
>>>> along with CI pipeline and branching model in the
>>>> meantime with code hosted on Github ?
>>>>
>>>> Thanks,
>>>> Jatin Dhankhar
>>>>
>>>> On Sat, Apr 8, 2017 at 6:28 AM, Tony Anderson
>>>> <tony_anderson at usa.net <mailto:tony_anderson at usa.net>>
>>>> wrote:
>>>>
>>>> Hi, Jatin
>>>>
>>>> I abbreviated this thread because I am getting
>>>> complaints from the list that the emails are too long.
>>>>
>>>> Yesterday, I was able to complete a set of 25
>>>> activities to use as a test base. The Django app
>>>> now has two views. First is a display of all of the
>>>> activities with the icon and name. A link for each
>>>> opens the full activity page. The data for the
>>>> views is in a file which is a list of jsons, one
>>>> json per line.
>>>> The views.py reads this file and builds the view
>>>> from the jsons.
>>>>
>>>> So the app consists of:
>>>> urls.py
>>>> views.py
>>>> templates/
>>>> base.html
>>>> list.html
>>>> activity.html
>>>>
>>>> with the data:
>>>> fixtures/activities.json
>>>> icons/
>>>> xo/
>>>>
>>>> The icons directory has the activity icons (*.svg)
>>>> The xo directory has the activity bundles (*.xo)
>>>>
>>>> Meanwhile, I am having more troubles with the ISP
>>>> (a new one). When that gets resolved, I should be
>>>> able to send the above to you.
>>>>
>>>> I have also mentioned our dialog to Walter Bender
>>>> suggesting that you could set up your working model
>>>> as activities3.sugarlabs.org
>>>> <http://activities3.sugarlabs.org>. This would give
>>>> you a permanent place to do the development and
>>>> make your project visible to the community so that
>>>> we can obtain feedback. I aslo suggested that the
>>>> project repository could be placed on GitHub so
>>>> that again the community can comment and
>>>> potentially contribute.
>>>>
>>>> Tony
>>>> On 04/07/2017 12:23 AM, Jatin Dhankhar wrote:
>>>>> Hi,
>>>>>
>>>>> I am setting up Django now and going through some
>>>>> tutorials. I will setup the code with a very basic
>>>>> and barebones version of what we want to achieve
>>>>> and put it in a private repo. We can even host it
>>>>> online, since I have some DigitalOcean credit
>>>>> left. Will keep you posted.
>>>>>
>>>>> Thanks.
>>>>> Jatin Dhankhar
>>>>>
>>>>> On Thu, Apr 6, 2017 at 7:11 AM, Tony Anderson
>>>>> <tony_anderson at usa.net
>>>>> <mailto:tony_anderson at usa.net>> wrote:
>>>>>
>>>>> Hi, Jatin
>>>>>
>>>>> In setting up Django, I think you will do it
>>>>> on your computer not on a separate server.
>>>>> Django handles that through its own server and
>>>>> has sgqlite as a database built-in. A good
>>>>> first start would be to set up the tutorial
>>>>> app (https://www.djangoproject.com/).
>>>>> Essentially you will need to install django
>>>>> and then set up the tutorial app 'poll'.
>>>>> Working through this tutorial will help a lot
>>>>> later on. The key point is that after initial
>>>>> setup, there are three important elements:
>>>>> url.py which defines the urls to access the
>>>>> application, views.py which is the python code
>>>>> that responds to a request by accessing the db
>>>>> and delivering the relevant information to a
>>>>> template. The template is an html file with
>>>>> variables of the form {{ activity.name
>>>>> <http://activity.name> }} transferring
>>>>> information from the view to the template.
>>>>>
>>>>> What I have is an application: aslo. Once you
>>>>> have Django installed and have run the
>>>>> tutorial app through Django's server, you'll
>>>>> be ready. Essentially, you will only need to
>>>>> add aslo as a second app in the settings. One
>>>>> possible confusion is that DJango lives in a
>>>>> project - essentially a directory containing
>>>>> its manage.py admin interface. In the
>>>>> directory is another directory of the same
>>>>> name with the settings.py and url.py. The poll
>>>>> app is a directory in the top-level alongside
>>>>> the inner directory with the project name.
>>>>>
>>>>> For example, my project is schoolsite. So my
>>>>> setup looks like:
>>>>>
>>>>> /library/schoolsite/schoolsite
>>>>> /library/schoolsite/aslo
>>>>>
>>>>> Meanwhile my code links directly to an
>>>>> activity page with no index. I'll add an index
>>>>> so the essential structure will be there. I'll
>>>>> also include the 'fixtures' to set up a 25
>>>>> activity capability. Fixtures are csv files
>>>>> from which the database (metadata) can be loaded.
>>>>>
>>>>> Tony
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sugarlabs.org/archive/aslo/attachments/20170413/0849d2d6/attachment-0001.html>
More information about the ASLO
mailing list