[ASLO] ASLO build and deployment process (Jatin Dhankhar)
Tony Anderson
tony_anderson at usa.net
Tue Apr 11 21:00:11 EDT 2017
Hi, Jatin
I think we need to agree on the use of IRC. If you want to communicate
with members of the community, you must go where they are (#sugar). If
you want a one-to-one meeting on IRC with me, I would suggest
#sugar-newbies. It is normally dormant but works well and saves a log
for later review. It worked well for meetings with Utkarsh Tiwari during
last year's GSOC.
There are two things that you will need to have local to the django
project. First is the directory download.sugarlabs.org which has the
Sugar activity bundles. The second is the 'metadata' in the mysql db.
For scraping, I would recommend BeautifulSoup (bs4). The trick will be
to decide what data we want to capture and add to the json.
The json fields in activities.json are ones I chose for the minimal
system. You may want to include other information such as the number of
downloads, which collections (should be entered as tags in a tag-field)
and so on. One item I have referred to as flags (I marked some as X but
don't remember what that meant, oh well). The intent is to record the
platforms where the activity works. We also should provide links to the
homepage, repository page, and update page (whatever that is). I think
if you have a working scrape tool, the data it collects can be expanded
as needed (assuming the tool runs in a reasonable time).
Naturally, it would be easier if you have access to the db directly.
Tony
On 04/12/2017 01:40 AM, Jatin Dhankhar wrote:
>
> One thing you could look at. On activities.sugarlabs,org, can you
> determine from Remora where the metadata is stored? I assume a db.
> Currently I am thinking to use BeautifulSoup to scrape the site to
> get that data, but it would be much easier to access the data
> directly.
>
> As per wiki
> https://wiki.mozilla.org/Update:Remora_Server_Requirements#SVN.2C_DB_and_app_config data
> is stored in mysql database. I don't have access to the production
> server where ASLO is currently running, following file
> https://github.com/sugarlabs/aslo/blob/master/aslo/db-update.sh#L9 confirms
> that data is stored in a mysql db. However it would be interesting and
> fun to scrape the data from live site. I would do that.
>
> Thanks will poke around the code, looks to me it's a django app and I
> have to mount it on my django project, thanks :)
>
> If you are talking about IRC as a place to meet Sugar community
> members, use the freenode #sugar. This is probably most active
> from 8-17 EST (UTC-5). I am currently in the Philippines which is
> UTC+ 7.
>
>
> Yes, tried that. https://gitter.im fits in naturally with Github
> (really sorry for suggesting a new mode of communication everyday) 😅
>
> - Jatin Dhankhar
>
>
> On Tue, Apr 11, 2017 at 6:14 AM, Tony Anderson <tony_anderson at usa.net
> <mailto:tony_anderson at usa.net>> wrote:
>
> If you are talking about IRC as a place to meet Sugar community
> members, use the freenode #sugar. This is probably most active
> from 8-17 EST (UTC-5). I am currently in the Philippines which is
> UTC+ 7.
>
> Localization of Python activities is done by Pootle, when
> implemented by the developer. The developer does something like
> the following:
>
> from gettext import gettext as _
>
> self.copy.set_tooltip(_('Copy'))
>
> In this way, all text displayed is taken from a po file based on
> the locale (e.g. en.po or hi.po). This is a simplification as the
> actual file is compressed: en.mo, hi.mo. These files are in the
> activity bundle. The detail is that when a new version is
> released, there is a master file: Paint.pot from which the local
> language files are built. This needs to be submitted to
> translate.sugarlabs.org <http://translate.sugarlabs.org> which
> maintains a copy. However, then the localized version needs to be
> added back to the bundle. However, the localizations can take
> months for 100 languages so how synchronize the po directory with
> the activity release is difficult.
>
> The sugar3 vs sugar issue is decided. The community wants to move
> to sugar3 (gtk3). The problem is that less that 20% of the
> activities have been converted.
> The ones that have been converted are low hanging fruit. The
> unconverted ones may require intensive work (gimp which developed
> gtk originally has not made the conversion).
>
> One thing you could look at. On activities.sugarlabs,org, can you
> determine from Remora where the metadata is stored? I assume a db.
> Currently I am thinking to use BeautifulSoup to scrape the site to
> get that data, but it would be much easier to access the data
> directly.
>
> Yesterday afternoon, the ISP restored service. Last time it died
> after two days, but I am keeping my fingers crossed. I am
> attaching the django stuff.
>
> Tony
>
>
> On 04/11/2017 01:36 AM, Jatin Dhankhar wrote:
>> Hi Tony,
>>
>> Normally, we use http://chat.sugarlabs.orgor on freenode:
>> sugar-meeting or sugar-newbies. These are logged sites so
>> that there is a record. The second is more appropriate since
>> sugar-meeting is used for SLOB meetings and the like. The
>> real problem with IRC is time zones. Email has the advantage
>> that either party can send or receive at any time. Last year
>> with a GSOC mentee we used sugar-newbies by arranging a
>> specific meeting time in advance.
>>
>> Yes, that is correct, main issue in communication barrier is due
>> to timezone issues. Since most of the people are familiar and are
>> available on IRC, it's seems to be the primary channel of
>> communication along with mailing lists and email. But since you
>> said we can use anything else, giving Slack a try won't hurt (if
>> issue is about not using closed source software then IRC is fine,
>> or we can try Mattermost <https://about.mattermost.com/>).
>>
>> Another part of the process is how to update
>> 'translate.sugarlabs.org <http://translate.sugarlabs.org/>'
>> with the corresponding POT file to enable localization. We
>> can get help from Chris Leonard on this.
>>
>>
>> I am not aware on how localization works. Do we need to download
>> relevant files and bundle them with the acitvity before making it
>> available on ASLO ?
>>
>> I have my Django version available - but the internet
>> problems here are still unresolved. The technician is
>> supposed to make another visit today to see what is wrong
>> with our connection. Let me know if and when you think this
>> will be useful to you.
>>
>> Let me know when your connection is stable and I would start.
>> What are the things you need me to do in the meantime ?
>>
>> One open issue is sugar3 vs sugar. Currently two versions of
>> Sugar are released. The sugar version supports gtk while
>> sugar3 supports gtk3. Unfortunately, gtk3 was developed to be
>> totally incompatible with gtk. For example, incorporation of
>> one gtk3 feature requires that all direct and indirect
>> references to gtk be removed or the activity will throw an
>> exception. Several of the gtk3 conversions failed to meet
>> this test and so fail. The issue is whether curated
>> activities be limited to ones converted to gtk3. The positive
>> is that Sugar could revert to releasing and maintaining only
>> a single version. The downside is that 100 or more activities
>> will no longer be available. Specifically, in our
>> implementation of ASLO, we need to show which versions of an
>> activity work on which versions of Sugar (e.g. i86, arm,
>> amd64, sugar or sugar3, and so on). We also need to show
>> which ones support localization. There are many English
>> activities and many Spanish activities that make no provision
>> for localization. Luckily there are many that have no
>> language component. However, for many of these, some kind of
>> help is needed to convey the way the activity works.
>>
>>
>> Some people believe GTK3 is slightly better
>> <https://www.reddit.com/r/linux/comments/3e3q8n/is_there_a_technical_reason_why_gtk3_is_better/> and
>> I think GTK3 will stay but that should be asked in community and
>> voted upon and taking in considerations cost of development and
>> porting, only a discussion will help in this one.
>>
>> You are wading into a deep and vast body of water!
>>
>> As long as I have something to hold onto, I will not drown 😅
>>
>> Thanks,
>> Jatin Dhankhar
>>
>> On Mon, Apr 10, 2017 at 9:11 AM, Tony Anderson
>> <tony_anderson at usa.net <mailto:tony_anderson at usa.net>> wrote:
>>
>> Hi, Jatin
>>
>> Normally, we use http://chat.sugarlabs.org or on freenode:
>> sugar-meeting or sugar-newbies. These are logged sites so
>> that there is a record. The second is more appropriate since
>> sugar-meeting is used for SLOB meetings and the like. The
>> real problem with IRC is time zones. Email has the advantage
>> that either party can send or receive at any time. Last year
>> with a GSOC mentee we used sugar-newbies by arranging a
>> specific meeting time in advance.
>>
>> I haven't heard from Walter, but my preference would be to
>> use the Sugarlabs server since the content is largely already
>> there and it would be easier to make it the official site if
>> that were decided. So in the short run, I think you should do
>> whatever is best for your own development process.
>>
>> I am not sure how CI fits into this. If the activity
>> development is done on GitHub, then the deployment model is
>> to run setup.py to create an xo bundle and then copy that
>> bundle to the appropriate location in the
>> download.sugarlabs.org <http://download.sugarlabs.org> tree.
>> Assuming the update results from a PR, the deployer would
>> need to update the activity information on ASLO
>> appropriately. However, that process depends on where that
>> data (metadata) is stored. Another part of the process is how
>> to update 'translate.sugarlabs.org
>> <http://translate.sugarlabs.org>' with the corresponding POT
>> file to enable localization. We can get help from Chris
>> Leonard on this.
>>
>> I have my Django version available - but the internet
>> problems here are still unresolved. The technician is
>> supposed to make another visit today to see what is wrong
>> with our connection. Let me know if and when you think this
>> will be useful to you.
>>
>> I have now tested most of the activities (~400). I was
>> optimistic in the number that work out of the box. However, a
>> part of this is running them in the Ubuntu version of Sugar
>> (amd64). There are many activities which launch object code
>> (mostly c) which is dependent on the architecture. I am now
>> trying to repeat the tests on an XO-1.75. One issue on Ubuntu
>> is that many activities assume a 1200x900 screen and so on a
>> 1024X768 screen overflow. This makes some of the games
>> unusable since part of the controls are off the screen.
>> Because of the internet problems, the untested activities
>> tend to be new ones since I was using my local repository
>> which is a snapshot taken several months ago. The other group
>> are the GCompris activities (about 70).
>>
>> My intent is to build a 'curated' repository of activities
>> known to work and be usable on the XO and on Ubuntu (or such
>> other platform that Sugar may choose to support). Most of the
>> currently not work activities have software dependencies no
>> longer included in the current Sugar release. So the curated
>> library will grow as activities are repaired over time.
>>
>> One open issue is sugar3 vs sugar. Currently two versions of
>> Sugar are released. The sugar version supports gtk while
>> sugar3 supports gtk3. Unfortunately, gtk3 was developed to be
>> totally incompatible with gtk. For example, incorporation of
>> one gtk3 feature requires that all direct and indirect
>> references to gtk be removed or the activity will throw an
>> exception. Several of the gtk3 conversions failed to meet
>> this test and so fail. The issue is whether curated
>> activities be limited to ones converted to gtk3. The positive
>> is that Sugar could revert to releasing and maintaining only
>> a single version. The downside is that 100 or more activities
>> will no longer be available. Specifically, in our
>> implementation of ASLO, we need to show which versions of an
>> activity work on which versions of Sugar (e.g. i86, arm,
>> amd64, sugar or sugar3, and so on). We also need to show
>> which ones support localization. There are many English
>> activities and many Spanish activities that make no provision
>> for localization. Luckily there are many that have no
>> language component. However, for many of these, some kind of
>> help is needed to convey the way the activity works.
>>
>> You are wading into a deep and vast body of water!
>>
>> Tony
>>
>>
>> On 04/10/2017 12:00 AM, Jatin Dhankhar wrote:
>>> Hi,
>>> Sorry for the delay. I went through the polls tutorials and
>>> I think I am getting hang of Django. I have one query that
>>> is out of context, what is your IRC setup ? IRC doesn't
>>> allow message to be delivered or stored once either party is
>>> offline, people login through a external server for IRC's to
>>> maintain their availability in a channel. May I suggest
>>> something like Slack or Flock for communication. IRC is good
>>> for quick and fast connection but Slack and alternatives
>>> allow easy communication. (Just a suggestion, though)
>>>
>>> Should I deploy the same polls app on DigitalOcean along
>>> with CI pipeline and branching model in the meantime with
>>> code hosted on Github ?
>>>
>>> Thanks,
>>> Jatin Dhankhar
>>>
>>> On Sat, Apr 8, 2017 at 6:28 AM, Tony Anderson
>>> <tony_anderson at usa.net <mailto:tony_anderson at usa.net>> wrote:
>>>
>>> Hi, Jatin
>>>
>>> I abbreviated this thread because I am getting
>>> complaints from the list that the emails are too long.
>>>
>>> Yesterday, I was able to complete a set of 25 activities
>>> to use as a test base. The Django app now has two views.
>>> First is a display of all of the
>>> activities with the icon and name. A link for each opens
>>> the full activity page. The data for the views is in a
>>> file which is a list of jsons, one json per line.
>>> The views.py reads this file and builds the view from
>>> the jsons.
>>>
>>> So the app consists of:
>>> urls.py
>>> views.py
>>> templates/
>>> base.html
>>> list.html
>>> activity.html
>>>
>>> with the data:
>>> fixtures/activities.json
>>> icons/
>>> xo/
>>>
>>> The icons directory has the activity icons (*.svg)
>>> The xo directory has the activity bundles (*.xo)
>>>
>>> Meanwhile, I am having more troubles with the ISP (a new
>>> one). When that gets resolved, I should be able to send
>>> the above to you.
>>>
>>> I have also mentioned our dialog to Walter Bender
>>> suggesting that you could set up your working model as
>>> activities3.sugarlabs.org
>>> <http://activities3.sugarlabs.org>. This would give you
>>> a permanent place to do the development and make your
>>> project visible to the community so that we can obtain
>>> feedback. I aslo suggested that the project repository
>>> could be placed on GitHub so that again the community
>>> can comment and potentially contribute.
>>>
>>> Tony
>>> On 04/07/2017 12:23 AM, Jatin Dhankhar wrote:
>>>> Hi,
>>>>
>>>> I am setting up Django now and going through some
>>>> tutorials. I will setup the code with a very basic and
>>>> barebones version of what we want to achieve and put it
>>>> in a private repo. We can even host it online, since I
>>>> have some DigitalOcean credit left. Will keep you posted.
>>>>
>>>> Thanks.
>>>> Jatin Dhankhar
>>>>
>>>> On Thu, Apr 6, 2017 at 7:11 AM, Tony Anderson
>>>> <tony_anderson at usa.net <mailto:tony_anderson at usa.net>>
>>>> wrote:
>>>>
>>>> Hi, Jatin
>>>>
>>>> In setting up Django, I think you will do it on
>>>> your computer not on a separate server. Django
>>>> handles that through its own server and has sgqlite
>>>> as a database built-in. A good first start would be
>>>> to set up the tutorial app
>>>> (https://www.djangoproject.com/). Essentially you
>>>> will need to install django and then set up the
>>>> tutorial app 'poll'. Working through this tutorial
>>>> will help a lot later on. The key point is that
>>>> after initial setup, there are three important
>>>> elements: url.py which defines the urls to access
>>>> the application, views.py which is the python code
>>>> that responds to a request by accessing the db and
>>>> delivering the relevant information to a template.
>>>> The template is an html file with variables of the
>>>> form {{ activity.name <http://activity.name> }}
>>>> transferring information from the view to the template.
>>>>
>>>> What I have is an application: aslo. Once you have
>>>> Django installed and have run the tutorial app
>>>> through Django's server, you'll be ready.
>>>> Essentially, you will only need to add aslo as a
>>>> second app in the settings. One possible confusion
>>>> is that DJango lives in a project - essentially a
>>>> directory containing its manage.py admin interface.
>>>> In the directory is another directory of the same
>>>> name with the settings.py and url.py. The poll app
>>>> is a directory in the top-level alongside the inner
>>>> directory with the project name.
>>>>
>>>> For example, my project is schoolsite. So my setup
>>>> looks like:
>>>>
>>>> /library/schoolsite/schoolsite
>>>> /library/schoolsite/aslo
>>>>
>>>> Meanwhile my code links directly to an activity
>>>> page with no index. I'll add an index so the
>>>> essential structure will be there. I'll also
>>>> include the 'fixtures' to set up a 25 activity
>>>> capability. Fixtures are csv files from which the
>>>> database (metadata) can be loaded.
>>>>
>>>> Tony
>>>>
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sugarlabs.org/archive/aslo/attachments/20170412/79a82c78/attachment-0001.html>
More information about the ASLO
mailing list