[ASLO] ASLO build and deployment process (Jatin Dhankhar)

[Walter Bender]

On Wed, Apr 19, 2017 at 9:09 PM, Tony Anderson <tony_anderson at usa.net>
wrote:

> I get the following messages when trying to create a new repository in
> sugarlabs:
>
> You don’t appear to have permission to create repositories for this
> organization. Sorry about that.
>

That is by design. Only members can create repositories on the project.

Quoting from
https://github.com/sugarlabs/sugar-docs/blob/master/contributing.md

"For new activities, make a new repository in your GitHub account, put the
source code in it, then ask the systems@ list to move it to the sugarlabs
organisation."

You can also of course ask to join the sugarlabs project on GitHub.

-walter


Tony
>
> On 04/20/2017 03:56 AM, Samuel Cantero wrote:
>
> On Wed, Apr 19, 2017 at 1:40 AM, Tony Anderson <tony_anderson at usa.net>
> wrote:
>
>> Hi Walter
>>
>> We haven't heard from Sam C.
>>
>
> Hi everyone! I'm sorry I haven't replied before. I have been very busy
> these days. I don't know much about ASLO architecture. I just have helped
> to keep it working. Aleksey is the correct guy here.
>
>>
>> I am thinking that rather than keep the metadata regarding Sugar
>> activities, it might be better to include it in activity.info (e.g.
>> developers, summary, description, what works, release notes). This would
>> enable ASLO to generate its screens from the bundle.
>>
>> Jatin now has a working minimal prototype of the Django version. It would
>> be helpful if it were on the Sugar servers supporting ASLO.
>>
>
> We can configure a dev environment in our server and enable CI. It would
> be good to keep main repo in GitHub, inside sugarlabs organization. This
> will give us more chance to encourage other people to help. Where is the
> prototype right now? I would like to take a look.
>
>
>> I would like to add some activities from ASLO to the github repository.
>> Currently I am a 'member'. Is that sufficient to enable adding a new
>> activity?
>>
>
> Try and tell me.
>
>>
>> I have posted issues to each of the activities I tested on the
>> github/sugarlabs. Several of the activities can be fixed by simple code
>> changes. More importantly, some order is needed in the assignment of
>> version numbers and releasing the updated activities to ASLO. While I am a
>> developer on ASLO, I don't have the ability to release new versions of
>> activities in general.
>>
>
>> I would appreciate your help in setting up the authorizations needed.
>>
>
>> Thanks,
>>
>> Tony
>>
>>
>>
>> On 04/13/2017 01:40 AM, Walter Bender wrote:
>>
>> Let's try to get Sam C., who currently maintains ASLO into the loop. I
>> think he'll have lots of good advice for us.
>>
>> regards.
>>
>> -walter
>>
>> On Wed, Apr 12, 2017 at 1:14 PM, Jatin Dhankhar <
>> <dhankhar.jatin at gmail.com>dhankhar.jatin at gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I think we need to agree on the use of IRC. If you want to communicate
>>>> with members of the community, you must go where they are (#sugar). If you
>>>> want a one-to-one meeting on IRC with me, I would suggest #sugar-newbies.
>>>> It is normally dormant but works well and saves a log for later review. It
>>>> worked well for meetings with Utkarsh Tiwari during last year's GSOC.
>>>
>>> Sure, whatever works :). What is your IRC nickname ?
>>>
>>> There are two things that you will need to have local to the django
>>>> project. First is the directory download.sugarlabs.org which has the
>>>> Sugar activity bundles
>>>
>>> Do I need to mirror the whole setup/directory ?
>>>
>>> When talking about scraping you probably meant
>>> <http://activities.sugarlabs>http://activities.sugarlabs.org instead of
>>> http://download.sugarlabs.org/, right ?
>>> Also for scraping, Scrapy <https://scrapy.org/> seems to more popular
>>> than beautifulsoup ?
>>>
>>> Also a big thanks for including Walter in the discussion :D
>>>
>>> - Jatin Dhankhar
>>>
>>>
>>> On Wed, Apr 12, 2017 at 6:30 AM, Tony Anderson < <tony_anderson at usa.net>
>>> tony_anderson at usa.net> wrote:
>>>
>>>> Hi, Jatin
>>>>
>>>> I think we need to agree on the use of IRC. If you want to communicate
>>>> with members of the community, you must go where they are (#sugar). If you
>>>> want a one-to-one meeting on IRC with me, I would suggest #sugar-newbies.
>>>> It is normally dormant but works well and saves a log for later review. It
>>>> worked well for meetings with Utkarsh Tiwari during last year's GSOC.
>>>>
>>>> There are two things that you will need to have local to the django
>>>> project. First is the directory download.sugarlabs.org which has the
>>>> Sugar activity bundles. The second is the 'metadata' in the mysql db. For
>>>> scraping, I would recommend BeautifulSoup (bs4). The trick will be to
>>>> decide what data we want to capture and add to the json.
>>>>
>>>> The json fields in activities.json are ones I chose for the minimal
>>>> system. You may want to include other information such as the number of
>>>> downloads, which collections (should be entered as tags in a tag-field) and
>>>> so on. One item I have referred to as flags (I marked some as X but don't
>>>> remember what that meant, oh well). The intent is to record the platforms
>>>> where the activity works. We also should provide links to the homepage,
>>>> repository page, and update page (whatever that is). I think if you have a
>>>> working scrape tool, the data it collects can be expanded as needed
>>>> (assuming the tool runs in a reasonable time).
>>>>
>>>> Naturally, it would be easier if you have access to the db directly.
>>>>
>>>> Tony
>>>>
>>>>
>>>> On 04/12/2017 01:40 AM, Jatin Dhankhar wrote:
>>>>
>>>> One thing you could look at. On activities.sugarlabs,org, can you
>>>>> determine from Remora where the metadata is stored? I assume a db.
>>>>> Currently I am thinking to use BeautifulSoup to scrape the site to get that
>>>>> data, but it would be much easier to access the data directly.
>>>>
>>>> As per wiki
>>>> <https://wiki.mozilla.org/Update:Remora_Server_Requirements#SVN.2C_DB_and_app_config>
>>>> https://wiki.mozilla.org/Update:Remora_Server_Requirements#SVN.2C_DB_and_app_config data
>>>> is stored in mysql database. I don't have access to the production server
>>>> where ASLO is currently running, following file
>>>> <https://github.com/sugarlabs/aslo/blob/master/aslo/db-update.sh#L9>
>>>> https://github.com/sugarlabs/aslo/blob/master/aslo/db-update.sh#L9 confirms
>>>> that data is stored in a mysql db. However it would be interesting and fun
>>>> to scrape the data from live site. I would do that.
>>>>
>>>> Thanks will poke around the code, looks to me it's a django app and I
>>>> have to mount it on my django project, thanks :)
>>>>
>>>> If you are talking about IRC as a place to meet Sugar community
>>>>> members, use the freenode #sugar. This is probably most active from 8-17
>>>>> EST (UTC-5). I am currently in the Philippines which is UTC+ 7.
>>>>
>>>>
>>>>  Yes, tried that.   <https://gitter.im>https://gitter.im fits in
>>>> naturally with Github (really sorry for suggesting a new mode of
>>>> communication everyday) 😅
>>>>
>>>> - Jatin Dhankhar
>>>>
>>>>
>>>> On Tue, Apr 11, 2017 at 6:14 AM, Tony Anderson <
>>>> <tony_anderson at usa.net>tony_anderson at usa.net> wrote:
>>>>
>>>>> If you are talking about IRC as a place to meet Sugar community
>>>>> members, use the freenode #sugar. This is probably most active from 8-17
>>>>> EST (UTC-5). I am currently in the Philippines which is UTC+ 7.
>>>>>
>>>>> Localization of Python activities is done by Pootle, when implemented
>>>>> by the developer. The developer does something like the following:
>>>>>
>>>>>         from gettext import gettext as _
>>>>>
>>>>>         self.copy.set_tooltip(_('Copy'))
>>>>>
>>>>> In this way, all text displayed is taken from a po file based on the
>>>>> locale (e.g. en.po or hi.po). This is a simplification as the actual file
>>>>> is compressed: en.mo, hi.mo. These files are in the activity bundle. The
>>>>> detail is that when a new version is released, there is a master file:
>>>>> Paint.pot from which the local language files are built. This needs to be
>>>>> submitted to translate.sugarlabs.org which maintains a copy. However,
>>>>> then the localized version needs to be added back to the bundle. However,
>>>>> the localizations can take months for 100 languages so how synchronize the
>>>>> po directory with the activity release is difficult.
>>>>>
>>>>> The sugar3 vs sugar issue is decided. The community wants to move to
>>>>> sugar3 (gtk3). The problem is that less that 20% of the activities have
>>>>> been converted.
>>>>> The ones that have been converted are low hanging fruit. The
>>>>> unconverted ones may require intensive work (gimp which developed gtk
>>>>> originally has not made the conversion).
>>>>>
>>>>> One thing you could look at. On activities.sugarlabs,org, can you
>>>>> determine from Remora where the metadata is stored? I assume a db.
>>>>> Currently I am thinking to use BeautifulSoup to scrape the site to get that
>>>>> data, but it would be much easier to access the data directly.
>>>>>
>>>>> Yesterday afternoon, the ISP restored service. Last time it died after
>>>>> two days, but I am keeping my fingers crossed. I am attaching the django
>>>>> stuff.
>>>>>
>>>>> Tony
>>>>>
>>>>>
>>>>> On 04/11/2017 01:36 AM, Jatin Dhankhar wrote:
>>>>>
>>>>> Hi Tony,
>>>>>
>>>>>
>>>>>> Normally, we use  <http://chat.sugarlabs.org>
>>>>>> http://chat.sugarlabs.org or on freenode: sugar-meeting or
>>>>>> sugar-newbies. These are logged sites so that there is a record. The second
>>>>>> is more appropriate since sugar-meeting is used for SLOB meetings and the
>>>>>> like. The real problem with IRC is time zones. Email has the advantage that
>>>>>> either party can send or receive at any time. Last year with a GSOC mentee
>>>>>> we used sugar-newbies by arranging a specific meeting time in advance.
>>>>>
>>>>> Yes, that is correct, main issue in communication barrier is due to
>>>>> timezone issues. Since most of the people are familiar and are available on
>>>>> IRC, it's seems to be the primary channel of communication along with
>>>>> mailing lists and email. But since you said we can use anything else,
>>>>> giving Slack a try won't hurt (if issue is about not using closed source
>>>>> software then IRC is fine, or we can try Mattermost
>>>>> <https://about.mattermost.com/>).
>>>>>
>>>>>  Another part of the process is how to update 'translate.sugarlabs.org'
>>>>>> with the corresponding POT file to enable localization. We can get help
>>>>>> from Chris Leonard on this.
>>>>>
>>>>>
>>>>> I am not aware on how localization works. Do we need to download
>>>>> relevant files and bundle them with the acitvity before making it available
>>>>> on ASLO ?
>>>>>
>>>>> I have my Django version available - but the internet problems here
>>>>>> are still unresolved. The technician is supposed to make another visit
>>>>>> today to see what is wrong with our connection. Let me know if and when you
>>>>>> think this will be useful to you.
>>>>>
>>>>> Let me know when your connection is stable and I would start.
>>>>>  What  are the things you need me to do in the meantime ?
>>>>>
>>>>> One open issue is sugar3 vs sugar. Currently two versions of Sugar are
>>>>>> released. The sugar version supports gtk while sugar3 supports gtk3.
>>>>>> Unfortunately, gtk3 was developed to be totally incompatible with gtk. For
>>>>>> example, incorporation of one gtk3 feature requires that all direct and
>>>>>> indirect references to gtk be removed or the activity will throw an
>>>>>> exception. Several of the gtk3 conversions failed to meet this test and so
>>>>>> fail. The issue is whether curated activities be limited to ones converted
>>>>>> to gtk3. The positive is that Sugar could revert to releasing and
>>>>>> maintaining only a single version. The downside is that 100 or more
>>>>>> activities will no longer be available. Specifically, in our implementation
>>>>>> of ASLO, we need to show which versions of an activity work on which
>>>>>> versions of Sugar (e.g. i86, arm, amd64, sugar or sugar3, and so on). We
>>>>>> also need to show which ones support localization. There are many English
>>>>>> activities and many Spanish activities that make no provision for
>>>>>> localization. Luckily there are many that have no language component.
>>>>>> However, for many of these, some kind of help is needed to convey the way
>>>>>> the activity works.
>>>>>
>>>>>
>>>>> Some people believe GTK3 is slightly better
>>>>> <https://www.reddit.com/r/linux/comments/3e3q8n/is_there_a_technical_reason_why_gtk3_is_better/> and
>>>>> I think GTK3 will stay but that should be asked in community and voted upon
>>>>> and taking in considerations cost of development and porting, only a
>>>>> discussion will help in this one.
>>>>>
>>>>> You are wading into a deep and vast body of water!
>>>>>
>>>>> As long as I have something to hold onto, I will not drown 😅
>>>>>
>>>>> Thanks,
>>>>> Jatin Dhankhar
>>>>>
>>>>> On Mon, Apr 10, 2017 at 9:11 AM, Tony Anderson <
>>>>> <tony_anderson at usa.net>tony_anderson at usa.net> wrote:
>>>>>
>>>>>> Hi, Jatin
>>>>>>
>>>>>> Normally, we use <http://chat.sugarlabs.org>http://chat.sugarlabs.org
>>>>>> or on freenode: sugar-meeting or sugar-newbies. These are logged sites so
>>>>>> that there is a record. The second is more appropriate since sugar-meeting
>>>>>> is used for SLOB meetings and the like. The real problem with IRC is time
>>>>>> zones. Email has the advantage that either party can send or receive at any
>>>>>> time. Last year with a GSOC mentee we used sugar-newbies by arranging a
>>>>>> specific meeting time in advance.
>>>>>>
>>>>>> I haven't heard from Walter, but my preference would be to use the
>>>>>> Sugarlabs server since the content is largely already there and it would be
>>>>>> easier to make it the official site if that were decided. So in the short
>>>>>> run, I think you should do whatever is best for your own development
>>>>>> process.
>>>>>>
>>>>>> I am not sure how CI fits into this. If the activity development is
>>>>>> done on GitHub, then the deployment model is to run setup.py to create an
>>>>>> xo bundle and then copy that bundle to the appropriate location in the
>>>>>> download.sugarlabs.org tree. Assuming the update results from a PR,
>>>>>> the deployer would need to update the activity information on ASLO
>>>>>> appropriately. However, that process depends on where that data (metadata)
>>>>>> is stored. Another part of the process is how to update '
>>>>>> translate.sugarlabs.org' with the corresponding POT file to enable
>>>>>> localization. We can get help from Chris Leonard on this.
>>>>>>
>>>>>> I have my Django version available - but the internet problems here
>>>>>> are still unresolved. The technician is supposed to make another visit
>>>>>> today to see what is wrong with our connection. Let me know if and when you
>>>>>> think this will be useful to you.
>>>>>>
>>>>>> I have now tested most of the activities (~400). I was optimistic in
>>>>>> the number that work out of the box. However, a part of this is running
>>>>>> them in the Ubuntu version of Sugar (amd64). There are many activities
>>>>>> which launch object code (mostly c) which is dependent on the architecture.
>>>>>> I am now trying to repeat the tests on an XO-1.75. One issue on Ubuntu is
>>>>>> that many activities assume a 1200x900 screen and so on a 1024X768 screen
>>>>>> overflow. This makes some of the games unusable since part of the controls
>>>>>> are off the screen. Because of the internet problems, the untested
>>>>>> activities tend to be new ones since I was using my local repository which
>>>>>> is a snapshot taken several months ago. The other group are the GCompris
>>>>>> activities (about 70).
>>>>>>
>>>>>> My intent is to build a 'curated' repository of activities known to
>>>>>> work and be usable on the XO and on Ubuntu (or such other platform that
>>>>>> Sugar may choose to support). Most of the currently not work activities
>>>>>> have software dependencies no longer included in the current Sugar release.
>>>>>> So the curated library will grow as activities are repaired over time.
>>>>>>
>>>>>> One open issue is sugar3 vs sugar. Currently two versions of Sugar
>>>>>> are released. The sugar version supports gtk while sugar3 supports gtk3.
>>>>>> Unfortunately, gtk3 was developed to be totally incompatible with gtk. For
>>>>>> example, incorporation of one gtk3 feature requires that all direct and
>>>>>> indirect references to gtk be removed or the activity will throw an
>>>>>> exception. Several of the gtk3 conversions failed to meet this test and so
>>>>>> fail. The issue is whether curated activities be limited to ones converted
>>>>>> to gtk3. The positive is that Sugar could revert to releasing and
>>>>>> maintaining only a single version. The downside is that 100 or more
>>>>>> activities will no longer be available. Specifically, in our implementation
>>>>>> of ASLO, we need to show which versions of an activity work on which
>>>>>> versions of Sugar (e.g. i86, arm, amd64, sugar or sugar3, and so on). We
>>>>>> also need to show which ones support localization. There are many English
>>>>>> activities and many Spanish activities that make no provision for
>>>>>> localization. Luckily there are many that have no language component.
>>>>>> However, for many of these, some kind of help is needed to convey the way
>>>>>> the activity works.
>>>>>>
>>>>>> You are wading into a deep and vast body of water!
>>>>>>
>>>>>> Tony
>>>>>>
>>>>>>
>>>>>> On 04/10/2017 12:00 AM, Jatin Dhankhar wrote:
>>>>>>
>>>>>> Hi,
>>>>>> Sorry for the delay. I went through the polls tutorials and I think I
>>>>>> am getting hang of Django. I have one query that is out of context, what is
>>>>>> your IRC setup ? IRC doesn't allow message to be delivered or stored once
>>>>>> either party is offline, people login through a external server for IRC's
>>>>>> to maintain their availability in a channel. May I suggest something like
>>>>>> Slack or Flock for communication. IRC is good for quick and fast connection
>>>>>> but Slack and alternatives allow easy communication. (Just a suggestion,
>>>>>> though)
>>>>>>
>>>>>> Should I  deploy the same polls app on DigitalOcean along with CI
>>>>>> pipeline and branching model in the meantime with code hosted on Github ?
>>>>>>
>>>>>> Thanks,
>>>>>> Jatin Dhankhar
>>>>>>
>>>>>> On Sat, Apr 8, 2017 at 6:28 AM, Tony Anderson <
>>>>>> <tony_anderson at usa.net>tony_anderson at usa.net> wrote:
>>>>>>
>>>>>>> Hi, Jatin
>>>>>>>
>>>>>>> I abbreviated this thread because I am getting complaints from the
>>>>>>> list that the emails are too long.
>>>>>>>
>>>>>>> Yesterday, I was able to complete a set of 25 activities to use as a
>>>>>>> test base. The Django app now has two views. First is a display of all of
>>>>>>> the
>>>>>>> activities with the icon and name. A link for each opens the full
>>>>>>> activity page. The data for the views is in a file which is a list of
>>>>>>> jsons, one json per line.
>>>>>>> The  views.py reads this file and builds the view from the jsons.
>>>>>>>
>>>>>>> So the app consists of:
>>>>>>>     urls.py
>>>>>>>     views.py
>>>>>>>     templates/
>>>>>>>         base.html
>>>>>>>         list.html
>>>>>>>         activity.html
>>>>>>>
>>>>>>> with the data:
>>>>>>>         fixtures/activities.json
>>>>>>>         icons/
>>>>>>>         xo/
>>>>>>>
>>>>>>> The icons directory has the activity icons (*.svg)
>>>>>>> The xo directory has the activity bundles (*.xo)
>>>>>>>
>>>>>>> Meanwhile, I am having more troubles with the ISP (a new one). When
>>>>>>> that gets resolved, I should be able to send the above to you.
>>>>>>>
>>>>>>> I have also mentioned our dialog to Walter Bender suggesting that
>>>>>>> you could set up your working model as activities3.sugarlabs.org.
>>>>>>> This would give you a permanent place to do the development and make your
>>>>>>> project visible to the community so that we can obtain feedback. I aslo
>>>>>>> suggested that the project repository could be placed on GitHub so that
>>>>>>> again the community can comment and potentially contribute.
>>>>>>>
>>>>>>> Tony
>>>>>>> On 04/07/2017 12:23 AM, Jatin Dhankhar wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am setting up Django now and going through some tutorials. I will
>>>>>>> setup the code with a very basic and barebones version of what we want to
>>>>>>> achieve and put it in a private repo. We can even host it online, since I
>>>>>>> have some DigitalOcean credit left. Will keep you posted.
>>>>>>>
>>>>>>> Thanks.
>>>>>>> Jatin Dhankhar
>>>>>>>
>>>>>>> On Thu, Apr 6, 2017 at 7:11 AM, Tony Anderson <
>>>>>>> <tony_anderson at usa.net>tony_anderson at usa.net> wrote:
>>>>>>>
>>>>>>>> Hi, Jatin
>>>>>>>>
>>>>>>>> In setting up Django, I think you will do it on your computer not
>>>>>>>> on a separate server. Django handles that through its own server and has
>>>>>>>> sgqlite as a database built-in. A good first start would be to set up the
>>>>>>>> tutorial app ( <https://www.djangoproject.com>
>>>>>>>> https://www.djangoproject.com/). Essentially you will need to
>>>>>>>> install django and then set up the tutorial app 'poll'. Working through
>>>>>>>> this tutorial will help a lot later on. The key point is that after initial
>>>>>>>> setup, there are three important elements: url.py which defines the urls to
>>>>>>>> access the application, views.py which is the python code that responds to
>>>>>>>> a request by accessing the db and delivering the relevant information to a
>>>>>>>> template. The template is an html file with variables of the form {{
>>>>>>>> activity.name }} transferring information from the view to the
>>>>>>>> template.
>>>>>>>>
>>>>>>>> What I have is an application: aslo. Once you have Django installed
>>>>>>>> and have run the tutorial app through Django's server, you'll be ready.
>>>>>>>> Essentially, you will only need to add aslo as a second app in the
>>>>>>>> settings. One possible confusion is that DJango lives in a project -
>>>>>>>> essentially a directory containing its manage.py admin interface. In the
>>>>>>>> directory is another directory of the same name with the settings.py and
>>>>>>>> url.py. The poll app is a directory in the top-level alongside the inner
>>>>>>>> directory with the project name.
>>>>>>>>
>>>>>>>> For example, my project is schoolsite. So my setup looks like:
>>>>>>>>
>>>>>>>> /library/schoolsite/schoolsite
>>>>>>>> /library/schoolsite/aslo
>>>>>>>>
>>>>>>>> Meanwhile my code links directly to an activity page with no index.
>>>>>>>> I'll add an index so the essential structure will be there. I'll also
>>>>>>>> include the 'fixtures' to set up a 25 activity capability. Fixtures are csv
>>>>>>>> files from which the database (metadata) can be loaded.
>>>>>>>>
>>>>>>>> Tony
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>> _______________________________________________
>>> ASLO mailing list
>>> ASLO at lists.sugarlabs.org
>>> http://lists.sugarlabs.org/listinfo/aslo
>>>
>>>
>>
>>
>> --
>> Walter Bender
>> Sugar Labs
>> http://www.sugarlabs.org
>>
>>
>>
>
>


-- 
Walter Bender
Sugar Labs
http://www.sugarlabs.org
<http://www.sugarlabs.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sugarlabs.org/archive/aslo/attachments/20170419/0cc50113/attachment-0001.html>