[Sugar-devel] Proposal to build app store for python 3 activities

[sugar at radii.dev]

Thank You James for the feedback. This has greatly clarified the requirements of project and now I can work towards building it. I have looked at the Tony Anderson's work, and understood that website should work similar to his work i.e. work completely without any backend including search functionality similar to python offline documentation. Since, activities would remain in range of hundreds to thousands, we can easily load entire search index for it to user's browser even when app store is hosted remotely.

I will work on building a prototype showcasing offline search functionality of the app store this weekend and get back for weekly feedback.

I will come to other aspects that you have talked such as syncing or pointing to download.sugarlabs.org and counting downloads from server logs when we reach to those points.

Regards,
Manish


Apr 13, 2020, 15:15 by quozl at laptop.org:

> G'day Manish,
>
> Thanks for your time on this.
>
> Sugarizer has all activities preloaded, as far as I know, (I'm no
> expert), so no need to include it.
>
> Users of a Sugar activities app store also include teachers and
> assistants carrying USB drives, SD cards, Raspberry Pi servers, and
> cell phones.  This is why I've specified static HTML5 with JavaScript
> so that the content of the app store can be used from a variety of
> physical sources.  I can foresee adding the whole app store to
> laptops.
>
> You may be interested in Tony Anderson's work on this two years ago;
> https://github.com/tony37/Sugaractivities
>
> Original app store is here;
> https://github.com/sugarlabs/aslo
> it is a derivative of Remora, the Mozilla plugin download manager used
> by Firefox.  It is written in PHP.
>
> Answering some of your questions;
>
> Yes, Sugar can run JavaScript activities; these can be found in .xo
> bundle files on activities.sugarlabs.org, on git.sugarlabs.org, or in
> github.com/sugarlabs.  Some of these activities have been further
> developed inside github.com/llaske/sugarizer
>
> Releasing an activity to the app store should be by uploading the .xo
> bundle to a directory on download.sugarlabs.org, or by adding a
> symlink to the existing upload.  Trusted developer would first check
> that the activity works on the platform before doing this.  The number
> of releases per month is so small, you see.
>
> Search box could be like python.org docs search; use a JSON file and
> implement the search on client side; the number of activities is in
> the hundreds and isn't expected to change that much now.  So no need
> for databases.
>
> Automatic update should use the uploaded files; the frequency of
> release is too low to warrant automation via github, and some day we
> may move to gitlab or another git web hosting provider.
>
> We can parse the server log files to get download counts; there's
> no need to do it within the app store.  Download counts are needed
> though, because without them we don't know how frequently the app
> store will be used.
>
> On your additional questions;
>
> 1.  yes, a few trusted developers can upload via SSH now, and we may
> add developers to that list if they begin to release activities.
>
> 2.  future considerations; at this stage growth is not expected beyond
> a few hundred activities.
>
> 3.  perhaps Python 3 for the tools that create the static content;
> JSON, HTML5, and JavaScript.  We have several instances of web servers
> that we can place the content on, and add a redirect or virtual host.
> Our content is also automatically mirrored; see mirror.aarnet.edu.au
>
> Thanks for such a comprehensive post.
>
> You may find it more efficient to build a prototype and ask for
> feedback, rather than to use a classical software development
> lifecycle.  We have few people willing to work on requirements
> specifications.
>
> On Sun, Apr 12, 2020 at 03:06:31PM +0200, sugar at radii.dev wrote:
>
>> Unified App Store
>>
>> Once the app store is built with all the specified requirements, we can with
>> minimal effort turn it into the unified app store for all activities including
>> python 2 (though all would be ported to python 3), JavaScript and those (if
>> any) written in other languages. Based on user-agent we can decide:
>>
>>  • Sugar OS - all activities including JS written for Sugar and Sugarizer
>>  • Old Sugar OS - python 2 activities (was there any support to run JS written
>>  activities?)
>>  • All other - Sugarizer activities unless toggled to show all
>>
>> This combined with building native app store (again with minor modification to
>> web app store) which is geared only towards students for searching, browsing,
>> installing and uninstalling apps would provide single app store codebase for
>> entire platform.
>>
>> Please provide your comments on the idea. If idea is supported by the community
>> than I can work towards building it.
>>
>> Regards,
>> Manish
>>
>> Apr 12, 2020, 01:18 by sugar at radii.dev:
>>
>>  Hello everyone,
>>
>>  I wish to build [1]app store for python 3 activities. For this I would like
>>  to inquire exact requirements of the project from the mentors and community
>>  at large.
>>
>>  Below is my current understanding of project. Please read through it and
>>  correct where these are wrong. After it are some questions to answer to
>>  further improve my understanding. Some of these are merely more elaborate
>>  rewording of requirements specified on [2]website. It is meant to detect
>>  possible misunderstanding that I may have from reading very brief one-line
>>  requirements.
>>
>>  After few responses for this mail, a draft requirements document will be
>>  mailed for feedback before final requirements specification.
>>
>>  Objective
>>
>>  □ The objective of app store is to provide a one-stop platform for
>>  students to browse or search and download activities to their system.
>>  It is expected than once the app store is built, students would not
>>  have to search for activities over internet, on GitHub or on wiki and
>>  download from there.
>>  □ Sugar will use app store to automatically upgrade newer version of
>>  activities.
>>
>>  Users of app store
>>
>>  □ Students: search and browse app store for activities and download them
>>  if not already installed.
>>  □ Trusted developers: add and modify activities to app store via SSH and
>>  than check them on app store. These are trusted developers and not all
>>  the developers. Their role would be similar to package managers for
>>  distros. That is, most of the times, the developer who built activity
>>  would not be the one who uploads it but request trusted developer to
>>  grant access or upload themselve.
>>  □ Sugar desktop environment/OS: check for current version on app store
>>  and if new version found than download and upgrade activities on local
>>  machine.
>>
>>  User-interface
>>
>>  □ The user interface/frontend of app store would be catering to students
>>  primarily and therefore must be built keeping them in mind rather than
>>  developers. But developers would still use it (or not?), therefore
>>  there would be some consideration of them and providing information
>>  relevant to them such as version of app, size, detailed/complete
>>  release note, website/link to repository, license etc.
>>  □ Special attention needs to be made towards user experience and
>>  usability. Such as web app should be responsive, delightful to use by
>>  students. For this, some artwork also needs to be built such as logo of
>>  app store, button icons, background etc.
>>
>>  Functionalities
>>
>>  □ Upload activities to app store using SSH: Trusted developers with
>>  access granted/account created for this specific purpose can upload
>>  activities to app store and have parameters to specify such as markdown
>>  supported detailed description of activities etc. but information which
>>  is available in activity.info will not be asked/cannot be specified in
>>  parameters while uploading, instead it will be processed directly by
>>  parsing activity.info file.
>>  □ Search box: There will be a single search input field in which user
>>  enters search query. Thee are no complicated search query features such
>>  as selecting category, no. of entries per page etc. At backend, based
>>  on query keywords either a) a simple keyword match is performed and
>>  results are returned in order in which received. b) some search
>>  improvement processing is done such as a rank based search prioritizing
>>  keyword matches in activities tile over activities description. There
>>  will/will not (please inform) functionality to sort search results
>>  based on activities last updated or added, no. of downloads etc.
>>  □ Redirect older Sugar to [3]python 2 app store: Redirect only Fedora 18
>>  or earlier version based Suger 0.112 or earlier version to [4]
>>  activities.sugarlabs.org. Suger 0.112 or earlier desktop environment is
>>  not running on any other distribution other than Fedora 18 or earlier
>>  or not needed to be checked for.
>>  □ List activities: There will be a category in browse section of app
>>  store named “All activities”, using it user can browse through all
>>  activities and not just in some specific activity.
>>  □ Support for Sugar’s microformat software upgrade feature: Embed CSS
>>  selectors and HTML data on activity bundles page and activity bundle
>>  list page, to allow microformats based Sugar software upgrade feature
>>  to check and automatically update activities when new version available
>>  on app store. Should there be a page listing all activity bundles where
>>  Sugar software upgrade feature will check for updates or should I
>>  follow the same tree structure as [5]on wiki or should there be a way
>>  to let know Sugar of dedicated page of each activity bundle to check
>>  for update (but modification to activity.info is not allowed)?
>>  □ Content-Type: When user downloading activity bundle from app store,
>>  server sends Content-Type header in response with the content-type of
>>  sugar activities that is:
>>  ☆ application/vnd.olpc-sugar for Sugar activity bundle (extension
>>  .xo)
>>  ☆ application/vnd.olpc-content for Sugar content bundle (extension
>>  .xol).
>>  ☆ application/vnd.olpc-journal-entry for Sugar Journal entry bundle
>>  (extension .xoj)
>>  ☆ application/vnd.olpc-journal-backup for Sugar Journal backup files
>>  (extension .xob)
>>
>>  There would not be any other file than .xo but better to have information
>>  for all file types used by Sugar in app store software.
>>
>>  □ Automatically update activities from activities repository specified in
>>  activity.info file: A standard recipe/script is to be made for server/
>>  app store to periodically check for newer release under ‘releases’
>>  section of activities hosted on GitHub and automatically create its
>>  bundle and add it to the app store. However, it will not be done for
>>  all activities which specify its website/repository as GitHub
>>  repository but only for those which are whitelisted by website admin/
>>  trusted developers as trusted repositories for security reasons and all
>>  GitHub repositories starting with [6]https://github.com/sugarlabs/ i.e.
>>  hosted by Sugar labs organization.
>>  □ Inform if activity is already installed: Wherever on app store, there
>>  will be download buttons for activities, A check will be performed if
>>  activity bundles are already installed on system. If so, than download
>>  button colour will be changed and “already installed” will be displayed
>>  below it. But, user can still download it. This need to be accomplished
>>  without modifying activities and should work for all activities
>>  available on app store. So, options which rely on modifying app itself
>>  are out of option but Sugar can be modified to support this
>>  functionality such supporting something similar to
>>  getInstalledRelatedApps() in Android.
>>  □ Download count: Every activity bundle should display no. of times it is
>>  downloaded. It should include download count of all previous version as
>>  well.
>>
>>  Development
>>
>>  □ Use general database such as MySQL/MariaDB for storing all textual data
>>  such as app title, description, author name etc. and use their in-built
>>  search functionality to perform search (after sanitizing input).
>>  □ The website will not be using JavaScript to make requests and update
>>  page dynamically.
>>
>>  Constraints
>>
>>  □ Activities itself won’t be modified including their metadata file
>>  located at activity/activity.info in their source tree for supporting
>>  any of the required feature. Request for modifying metadata should not
>>  be made such as for adding detailed description and instead should be
>>  fetched from their repository or options should be provided for
>>  specifying description while uploading activity bundle via SSH.
>>
>>  Security
>>
>>  □ Only developers which have been provided access in advance can upload
>>  activities to app store. Some may have admin access, so they can add
>>  other developers as well. A simple CLI script should be provided to
>>  easily add new developers and their access level as ‘can upload new
>>  activities’, ‘can modify specified activity’ and admin access.
>>
>>  Additional Questions
>>
>>  1. Developer uploading activities via SSH? Will access to uploading
>>  activities be with few trusted developers (my assumption as of now) or
>>  anyone can upload activities to app store once approved.
>>  2. Future considerations: how far is the app store expected to grow in
>>  future in terms of its scope and functionalities. What considerations
>>  should be kept in mind if app store functionalities and codebase is
>>  expected to be a lot bigger in future.
>>  3. Is programming language to be used for backend is per-decided to be
>>  Python or can be JavaScript as well? I am comfortable with using any of
>>  the two. My suggested framework for Python is Flask and if JavaScript
>>  is used than Express.js. I also suggest using MySQL database similar to
>>  [7]aslo for backend for storing and looking up activities metadata but
>>  store activities bundles as normal files on disk. Is their any reverse
>>  proxy used on server where it will be installed such as Nginx or
>>  Apache?
>>
>>  Please give your comments on these questions.
>>
>>  Regards,
>>  Manish
>>
>> References:
>>
>> [1] https://github.com/sugarlabs/GSoC/blob/master/Ideas-2020.md#sugar-app-store-for-python-3-activities-aslov4
>> [2] https://github.com/sugarlabs/GSoC/blob/master/Ideas-2020.md#sugar-app-store-for-python-3-activities-aslov4
>> [3] https://activities.sugarlabs.org/en-US/sugar/
>> [4] https://activities.sugarlabs.org/
>> [5] http://wiki.laptop.org/go/Activities/
>> [6] https://github.com/sugarlabs/
>> [7] https://github.com/sugarlabs/aslo
>>
>> _______________________________________________
>> Sugar-devel mailing list
>> Sugar-devel at lists.sugarlabs.org
>> http://lists.sugarlabs.org/listinfo/sugar-devel
>>
>
>
> -- 
> James Cameron
> http://quozl.netrek.org/
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sugarlabs.org/archive/sugar-devel/attachments/20200413/b1df3c73/attachment-0001.htm>