[ASLO] ASLO build and deployment process (Jatin Dhankhar)

[Jatin Dhankhar]

Hi,

Can you tell me which version of Django and Python you are using? I am
using the latest stable version and some things like patterns were removed
from Django. Also the code you sent me only contains the app aslo and not
the whole django web project ?

Thanks,
Jatin Dhankhar

On Thu, Apr 13, 2017 at 11:24 PM, Jatin Dhankhar <dhankhar.jatin at gmail.com>
wrote:

> At the moment, I think you should focus on the 25 activity test set and
>> getting it working with django.
>
> Okay, on it.
>
>
>>  we can arrange to host the development on a Sugar Labs server, it should
>> make both the repository itself (download.sugarlabs.org) and the ASLO
>> metadata directly accessible.
>
> In the meantime we can host it on DigitalOcean, I have credits for
> DigitalOcean which I got as part of Student pack.
>
> My IRC nickname is tony37. However, I am never on #sugar (a twelve hour
>> time difference so the normal participants are in bed). Last year, Utkarsh
>> and I met whenever he felt the need to discuss a technical issue in detail.
>> We limited the meetings to one hour. That proved quite productive.
>
>
> Yes we can do the same.
>
> Thanks,
> Jatin Dhankhar
>
> On Thu, Apr 13, 2017 at 5:50 AM, Tony Anderson <tony_anderson at usa.net>
> wrote:
>
>> Hi Jatin
>>
>> At the moment, I think you should focus on the 25 activity test set and
>> getting it working with django. I find it is much more productive to move
>> from one working version to another with added capability. If we can
>> arrange to host the development on a Sugar Labs server, it should make both
>> the repository itself (download.sugarlabs.org) and the ASLO metadata
>> directly accessible.
>>
>> My IRC nickname is tony37. However, I am never on #sugar (a twelve hour
>> time difference so the normal participants are in bed). Last year, Utkarsh
>> and I met whenever he felt the need to discuss a technical issue in detail.
>> We limited the meetings to one hour. That proved quite productive.
>>
>> Tony
>>
>>
>> On 04/13/2017 01:14 AM, Jatin Dhankhar wrote:
>>
>> Hi,
>>
>> I think we need to agree on the use of IRC. If you want to communicate
>>> with members of the community, you must go where they are (#sugar). If you
>>> want a one-to-one meeting on IRC with me, I would suggest #sugar-newbies.
>>> It is normally dormant but works well and saves a log for later review. It
>>> worked well for meetings with Utkarsh Tiwari during last year's GSOC.
>>
>> Sure, whatever works :). What is your IRC nickname ?
>>
>> There are two things that you will need to have local to the django
>>> project. First is the directory download.sugarlabs.org which has the
>>> Sugar activity bundles
>>
>> Do I need to mirror the whole setup/directory ?
>>
>> When talking about scraping you probably meant
>> <http://activities.sugarlabs.org>http://activities.sugarlabs.org instead
>> of http://download.sugarlabs.org/, right ?
>> Also for scraping, Scrapy <https://scrapy.org/> seems to more popular
>> than beautifulsoup ?
>>
>> Also a big thanks for including Walter in the discussion :D
>>
>> - Jatin Dhankhar
>>
>>
>> On Wed, Apr 12, 2017 at 6:30 AM, Tony Anderson <tony_anderson at usa.net>
>> wrote:
>>
>>> Hi, Jatin
>>>
>>> I think we need to agree on the use of IRC. If you want to communicate
>>> with members of the community, you must go where they are (#sugar). If you
>>> want a one-to-one meeting on IRC with me, I would suggest #sugar-newbies.
>>> It is normally dormant but works well and saves a log for later review. It
>>> worked well for meetings with Utkarsh Tiwari during last year's GSOC.
>>>
>>> There are two things that you will need to have local to the django
>>> project. First is the directory download.sugarlabs.org which has the
>>> Sugar activity bundles. The second is the 'metadata' in the mysql db. For
>>> scraping, I would recommend BeautifulSoup (bs4). The trick will be to
>>> decide what data we want to capture and add to the json.
>>>
>>> The json fields in activities.json are ones I chose for the minimal
>>> system. You may want to include other information such as the number of
>>> downloads, which collections (should be entered as tags in a tag-field) and
>>> so on. One item I have referred to as flags (I marked some as X but don't
>>> remember what that meant, oh well). The intent is to record the platforms
>>> where the activity works. We also should provide links to the homepage,
>>> repository page, and update page (whatever that is). I think if you have a
>>> working scrape tool, the data it collects can be expanded as needed
>>> (assuming the tool runs in a reasonable time).
>>>
>>> Naturally, it would be easier if you have access to the db directly.
>>>
>>> Tony
>>>
>>>
>>> On 04/12/2017 01:40 AM, Jatin Dhankhar wrote:
>>>
>>> One thing you could look at. On activities.sugarlabs,org, can you
>>>> determine from Remora where the metadata is stored? I assume a db.
>>>> Currently I am thinking to use BeautifulSoup to scrape the site to get that
>>>> data, but it would be much easier to access the data directly.
>>>
>>> As per wiki https://wiki.mozilla.org/Update:Remora_Server_Requirements#S
>>> VN.2C_DB_and_app_config data is stored in mysql database. I don't have
>>> access to the production server where ASLO is currently running, following
>>> file https://github.com/sugarlabs/aslo/blob/master/aslo/db-update.sh#L9 confirms
>>> that data is stored in a mysql db. However it would be interesting and fun
>>> to scrape the data from live site. I would do that.
>>>
>>> Thanks will poke around the code, looks to me it's a django app and I
>>> have to mount it on my django project, thanks :)
>>>
>>> If you are talking about IRC as a place to meet Sugar community members,
>>>> use the freenode #sugar. This is probably most active from 8-17 EST
>>>> (UTC-5). I am currently in the Philippines which is UTC+ 7.
>>>
>>>
>>>  Yes, tried that.  https://gitter.im fits in naturally with Github
>>> (really sorry for suggesting a new mode of communication everyday) 😅
>>>
>>> - Jatin Dhankhar
>>>
>>>
>>> On Tue, Apr 11, 2017 at 6:14 AM, Tony Anderson < <tony_anderson at usa.net>
>>> tony_anderson at usa.net> wrote:
>>>
>>>> If you are talking about IRC as a place to meet Sugar community
>>>> members, use the freenode #sugar. This is probably most active from 8-17
>>>> EST (UTC-5). I am currently in the Philippines which is UTC+ 7.
>>>>
>>>> Localization of Python activities is done by Pootle, when implemented
>>>> by the developer. The developer does something like the following:
>>>>
>>>>         from gettext import gettext as _
>>>>
>>>>         self.copy.set_tooltip(_('Copy'))
>>>>
>>>> In this way, all text displayed is taken from a po file based on the
>>>> locale (e.g. en.po or hi.po). This is a simplification as the actual file
>>>> is compressed: en.mo, hi.mo. These files are in the activity bundle. The
>>>> detail is that when a new version is released, there is a master file:
>>>> Paint.pot from which the local language files are built. This needs to be
>>>> submitted to translate.sugarlabs.org which maintains a copy. However,
>>>> then the localized version needs to be added back to the bundle. However,
>>>> the localizations can take months for 100 languages so how synchronize the
>>>> po directory with the activity release is difficult.
>>>>
>>>> The sugar3 vs sugar issue is decided. The community wants to move to
>>>> sugar3 (gtk3). The problem is that less that 20% of the activities have
>>>> been converted.
>>>> The ones that have been converted are low hanging fruit. The
>>>> unconverted ones may require intensive work (gimp which developed gtk
>>>> originally has not made the conversion).
>>>>
>>>> One thing you could look at. On activities.sugarlabs,org, can you
>>>> determine from Remora where the metadata is stored? I assume a db.
>>>> Currently I am thinking to use BeautifulSoup to scrape the site to get that
>>>> data, but it would be much easier to access the data directly.
>>>>
>>>> Yesterday afternoon, the ISP restored service. Last time it died after
>>>> two days, but I am keeping my fingers crossed. I am attaching the django
>>>> stuff.
>>>>
>>>> Tony
>>>>
>>>>
>>>> On 04/11/2017 01:36 AM, Jatin Dhankhar wrote:
>>>>
>>>> Hi Tony,
>>>>
>>>>
>>>>> Normally, we use  <http://chat.sugarlabs.org>http://chat.sugarlabs.org
>>>>>  or on freenode: sugar-meeting or sugar-newbies. These are logged
>>>>> sites so that there is a record. The second is more appropriate since
>>>>> sugar-meeting is used for SLOB meetings and the like. The real problem with
>>>>> IRC is time zones. Email has the advantage that either party can send or
>>>>> receive at any time. Last year with a GSOC mentee we used sugar-newbies by
>>>>> arranging a specific meeting time in advance.
>>>>
>>>> Yes, that is correct, main issue in communication barrier is due to
>>>> timezone issues. Since most of the people are familiar and are available on
>>>> IRC, it's seems to be the primary channel of communication along with
>>>> mailing lists and email. But since you said we can use anything else,
>>>> giving Slack a try won't hurt (if issue is about not using closed source
>>>> software then IRC is fine, or we can try Mattermost
>>>> <https://about.mattermost.com/>).
>>>>
>>>>  Another part of the process is how to update 'translate.sugarlabs.org'
>>>>> with the corresponding POT file to enable localization. We can get help
>>>>> from Chris Leonard on this.
>>>>
>>>>
>>>> I am not aware on how localization works. Do we need to download
>>>> relevant files and bundle them with the acitvity before making it available
>>>> on ASLO ?
>>>>
>>>> I have my Django version available - but the internet problems here are
>>>>> still unresolved. The technician is supposed to make another visit today to
>>>>> see what is wrong with our connection. Let me know if and when you think
>>>>> this will be useful to you.
>>>>
>>>> Let me know when your connection is stable and I would start.
>>>>  What  are the things you need me to do in the meantime ?
>>>>
>>>> One open issue is sugar3 vs sugar. Currently two versions of Sugar are
>>>>> released. The sugar version supports gtk while sugar3 supports gtk3.
>>>>> Unfortunately, gtk3 was developed to be totally incompatible with gtk. For
>>>>> example, incorporation of one gtk3 feature requires that all direct and
>>>>> indirect references to gtk be removed or the activity will throw an
>>>>> exception. Several of the gtk3 conversions failed to meet this test and so
>>>>> fail. The issue is whether curated activities be limited to ones converted
>>>>> to gtk3. The positive is that Sugar could revert to releasing and
>>>>> maintaining only a single version. The downside is that 100 or more
>>>>> activities will no longer be available. Specifically, in our implementation
>>>>> of ASLO, we need to show which versions of an activity work on which
>>>>> versions of Sugar (e.g. i86, arm, amd64, sugar or sugar3, and so on). We
>>>>> also need to show which ones support localization. There are many English
>>>>> activities and many Spanish activities that make no provision for
>>>>> localization. Luckily there are many that have no language component.
>>>>> However, for many of these, some kind of help is needed to convey the way
>>>>> the activity works.
>>>>
>>>>
>>>> Some people believe GTK3 is slightly better
>>>> <https://www.reddit.com/r/linux/comments/3e3q8n/is_there_a_technical_reason_why_gtk3_is_better/> and
>>>> I think GTK3 will stay but that should be asked in community and voted upon
>>>> and taking in considerations cost of development and porting, only a
>>>> discussion will help in this one.
>>>>
>>>> You are wading into a deep and vast body of water!
>>>>
>>>> As long as I have something to hold onto, I will not drown 😅
>>>>
>>>> Thanks,
>>>> Jatin Dhankhar
>>>>
>>>> On Mon, Apr 10, 2017 at 9:11 AM, Tony Anderson <
>>>> <tony_anderson at usa.net>tony_anderson at usa.net> wrote:
>>>>
>>>>> Hi, Jatin
>>>>>
>>>>> Normally, we use <http://chat.sugarlabs.org>http://chat.sugarlabs.org
>>>>> or on freenode: sugar-meeting or sugar-newbies. These are logged sites so
>>>>> that there is a record. The second is more appropriate since sugar-meeting
>>>>> is used for SLOB meetings and the like. The real problem with IRC is time
>>>>> zones. Email has the advantage that either party can send or receive at any
>>>>> time. Last year with a GSOC mentee we used sugar-newbies by arranging a
>>>>> specific meeting time in advance.
>>>>>
>>>>> I haven't heard from Walter, but my preference would be to use the
>>>>> Sugarlabs server since the content is largely already there and it would be
>>>>> easier to make it the official site if that were decided. So in the short
>>>>> run, I think you should do whatever is best for your own development
>>>>> process.
>>>>>
>>>>> I am not sure how CI fits into this. If the activity development is
>>>>> done on GitHub, then the deployment model is to run setup.py to create an
>>>>> xo bundle and then copy that bundle to the appropriate location in the
>>>>> download.sugarlabs.org tree. Assuming the update results from a PR,
>>>>> the deployer would need to update the activity information on ASLO
>>>>> appropriately. However, that process depends on where that data (metadata)
>>>>> is stored. Another part of the process is how to update '
>>>>> translate.sugarlabs.org' with the corresponding POT file to enable
>>>>> localization. We can get help from Chris Leonard on this.
>>>>>
>>>>> I have my Django version available - but the internet problems here
>>>>> are still unresolved. The technician is supposed to make another visit
>>>>> today to see what is wrong with our connection. Let me know if and when you
>>>>> think this will be useful to you.
>>>>>
>>>>> I have now tested most of the activities (~400). I was optimistic in
>>>>> the number that work out of the box. However, a part of this is running
>>>>> them in the Ubuntu version of Sugar (amd64). There are many activities
>>>>> which launch object code (mostly c) which is dependent on the architecture.
>>>>> I am now trying to repeat the tests on an XO-1.75. One issue on Ubuntu is
>>>>> that many activities assume a 1200x900 screen and so on a 1024X768 screen
>>>>> overflow. This makes some of the games unusable since part of the controls
>>>>> are off the screen. Because of the internet problems, the untested
>>>>> activities tend to be new ones since I was using my local repository which
>>>>> is a snapshot taken several months ago. The other group are the GCompris
>>>>> activities (about 70).
>>>>>
>>>>> My intent is to build a 'curated' repository of activities known to
>>>>> work and be usable on the XO and on Ubuntu (or such other platform that
>>>>> Sugar may choose to support). Most of the currently not work activities
>>>>> have software dependencies no longer included in the current Sugar release.
>>>>> So the curated library will grow as activities are repaired over time.
>>>>>
>>>>> One open issue is sugar3 vs sugar. Currently two versions of Sugar are
>>>>> released. The sugar version supports gtk while sugar3 supports gtk3.
>>>>> Unfortunately, gtk3 was developed to be totally incompatible with gtk. For
>>>>> example, incorporation of one gtk3 feature requires that all direct and
>>>>> indirect references to gtk be removed or the activity will throw an
>>>>> exception. Several of the gtk3 conversions failed to meet this test and so
>>>>> fail. The issue is whether curated activities be limited to ones converted
>>>>> to gtk3. The positive is that Sugar could revert to releasing and
>>>>> maintaining only a single version. The downside is that 100 or more
>>>>> activities will no longer be available. Specifically, in our implementation
>>>>> of ASLO, we need to show which versions of an activity work on which
>>>>> versions of Sugar (e.g. i86, arm, amd64, sugar or sugar3, and so on). We
>>>>> also need to show which ones support localization. There are many English
>>>>> activities and many Spanish activities that make no provision for
>>>>> localization. Luckily there are many that have no language component.
>>>>> However, for many of these, some kind of help is needed to convey the way
>>>>> the activity works.
>>>>>
>>>>> You are wading into a deep and vast body of water!
>>>>>
>>>>> Tony
>>>>>
>>>>>
>>>>> On 04/10/2017 12:00 AM, Jatin Dhankhar wrote:
>>>>>
>>>>> Hi,
>>>>> Sorry for the delay. I went through the polls tutorials and I think I
>>>>> am getting hang of Django. I have one query that is out of context, what is
>>>>> your IRC setup ? IRC doesn't allow message to be delivered or stored once
>>>>> either party is offline, people login through a external server for IRC's
>>>>> to maintain their availability in a channel. May I suggest something like
>>>>> Slack or Flock for communication. IRC is good for quick and fast connection
>>>>> but Slack and alternatives allow easy communication. (Just a suggestion,
>>>>> though)
>>>>>
>>>>> Should I  deploy the same polls app on DigitalOcean along with CI
>>>>> pipeline and branching model in the meantime with code hosted on Github ?
>>>>>
>>>>> Thanks,
>>>>> Jatin Dhankhar
>>>>>
>>>>> On Sat, Apr 8, 2017 at 6:28 AM, Tony Anderson <
>>>>> <tony_anderson at usa.net>tony_anderson at usa.net> wrote:
>>>>>
>>>>>> Hi, Jatin
>>>>>>
>>>>>> I abbreviated this thread because I am getting complaints from the
>>>>>> list that the emails are too long.
>>>>>>
>>>>>> Yesterday, I was able to complete a set of 25 activities to use as a
>>>>>> test base. The Django app now has two views. First is a display of all of
>>>>>> the
>>>>>> activities with the icon and name. A link for each opens the full
>>>>>> activity page. The data for the views is in a file which is a list of
>>>>>> jsons, one json per line.
>>>>>> The  views.py reads this file and builds the view from the jsons.
>>>>>>
>>>>>> So the app consists of:
>>>>>>     urls.py
>>>>>>     views.py
>>>>>>     templates/
>>>>>>         base.html
>>>>>>         list.html
>>>>>>         activity.html
>>>>>>
>>>>>> with the data:
>>>>>>         fixtures/activities.json
>>>>>>         icons/
>>>>>>         xo/
>>>>>>
>>>>>> The icons directory has the activity icons (*.svg)
>>>>>> The xo directory has the activity bundles (*.xo)
>>>>>>
>>>>>> Meanwhile, I am having more troubles with the ISP (a new one). When
>>>>>> that gets resolved, I should be able to send the above to you.
>>>>>>
>>>>>> I have also mentioned our dialog to Walter Bender suggesting that you
>>>>>> could set up your working model as activities3.sugarlabs.org. This
>>>>>> would give you a permanent place to do the development and make your
>>>>>> project visible to the community so that we can obtain feedback. I aslo
>>>>>> suggested that the project repository could be placed on GitHub so that
>>>>>> again the community can comment and potentially contribute.
>>>>>>
>>>>>> Tony
>>>>>> On 04/07/2017 12:23 AM, Jatin Dhankhar wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I am setting up Django now and going through some tutorials. I will
>>>>>> setup the code with a very basic and barebones version of what we want to
>>>>>> achieve and put it in a private repo. We can even host it online, since I
>>>>>> have some DigitalOcean credit left. Will keep you posted.
>>>>>>
>>>>>> Thanks.
>>>>>> Jatin Dhankhar
>>>>>>
>>>>>> On Thu, Apr 6, 2017 at 7:11 AM, Tony Anderson <
>>>>>> <tony_anderson at usa.net>tony_anderson at usa.net> wrote:
>>>>>>
>>>>>>> Hi, Jatin
>>>>>>>
>>>>>>> In setting up Django, I think you will do it on your computer not on
>>>>>>> a separate server. Django handles that through its own server and has
>>>>>>> sgqlite as a database built-in. A good first start would be to set up the
>>>>>>> tutorial app ( <https://www.djangoproject.com>
>>>>>>> https://www.djangoproject.com/). Essentially you will need to
>>>>>>> install django and then set up the tutorial app 'poll'. Working through
>>>>>>> this tutorial will help a lot later on. The key point is that after initial
>>>>>>> setup, there are three important elements: url.py which defines the urls to
>>>>>>> access the application, views.py which is the python code that responds to
>>>>>>> a request by accessing the db and delivering the relevant information to a
>>>>>>> template. The template is an html file with variables of the form {{
>>>>>>> activity.name }} transferring information from the view to the
>>>>>>> template.
>>>>>>>
>>>>>>> What I have is an application: aslo. Once you have Django installed
>>>>>>> and have run the tutorial app through Django's server, you'll be ready.
>>>>>>> Essentially, you will only need to add aslo as a second app in the
>>>>>>> settings. One possible confusion is that DJango lives in a project -
>>>>>>> essentially a directory containing its manage.py admin interface. In the
>>>>>>> directory is another directory of the same name with the settings.py and
>>>>>>> url.py. The poll app is a directory in the top-level alongside the inner
>>>>>>> directory with the project name.
>>>>>>>
>>>>>>> For example, my project is schoolsite. So my setup looks like:
>>>>>>>
>>>>>>> /library/schoolsite/schoolsite
>>>>>>> /library/schoolsite/aslo
>>>>>>>
>>>>>>> Meanwhile my code links directly to an activity page with no index.
>>>>>>> I'll add an index so the essential structure will be there. I'll also
>>>>>>> include the 'fixtures' to set up a 25 activity capability. Fixtures are csv
>>>>>>> files from which the database (metadata) can be loaded.
>>>>>>>
>>>>>>> Tony
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sugarlabs.org/archive/aslo/attachments/20170414/928bb255/attachment-0001.html>