[Sugar-devel] Updates: Python3 app store (aslov4) : Sugar App Store

Srevin Saju srevinsaju at sugarlabs.org
Wed Jun 24 08:48:34 EDT 2020


Hello,

/(PS: If you received the email earlier, please ignore this. For some 
reason, my previous email was marked as spam, so most of them might not 
have actually received it. This is an updated email of the previously 
sent 'Updates: Python3 app store (aslov4) : Sugar App Store'; I hope 
this email is also not classified as spam, //¯\_(ツ)_/¯ )/

Hope all are having a safe week amid nCoV.

I and Manish (sugar at radii.dev) had been working on the GSoC project 
which did *not* get a slot (Python3 app store aslov4), and I guess we 
have made good progress in it. /We decided to collaborate on the project 
because a frontend and backend developer combo helps to get the best 
features of both ends :D./

/To avoid misconceptions and misinterpretations, please read the entire 
email before replying. Thanks!   :P/

*Need Statement*:

In short, the most important points

  * /Create the simplest possible app store for Sugar activities, where
    each activity included has been (a) ported to Python 3 and released,
    and (b) tested on Sugar Live Build./
  * /We used to have an app store for Sugar activities, but because we
    can't seem to attract any PHP developers the app store has failed to
    keep up with development./
  * /We now use activities.sugarlabs.org for Python 2 activities only./
  * /We have tried to make a replacement for activities.sugarlabs.org
    <https://activities.sugarlabs.org> three times, and each time the
    features we need were not finished. These projects have been too
    ambitious and have not been supported collectively by the Sugar Labs
    community./

/~ from 
//https://github.com/sugarlabs/GSoC/blob/master/Ideas-2020.md#sugar-app-store-for-python-3-activities-aslov4/

This made us rethink the idea of a redesigned app store for sugarlabs. 
This, would be a collection of all the activities in sugarlabs 
organization which can be /*successfully* built/ without /errors/.

The *new app store *is special in a lot of ways, let me bring out 
the/most important points:/

 1. The appstore is built in _pure static HTML_, Vanilla Javascript
    <https://developer.mozilla.org/en-US/docs/Web/JavaScript> and JQuery
    <https://jquery.com/>. Anyone with /basic Javascript knowledge/ can
    easily add features. Theming and customization are provided by
    Bootstrap4 <https://getbootstrap.com>. The theme, scripts are
    completely _independent_ of each other. Changes to the /will not
    /affect the generation of the appstore.
 2. *Search function* is provided by a powerful Javascript library
    called minisearch <https://github.com/lucaong/minisearch> which is
    an open source search indexer. All processing is therefore done on
    client side. The benefits of using an external library are many, as
    you know,
 3. The App Store is based on *JSON* (/Javascript Object Notation/). an
    "index.json" is produced by the generator helper script so that
    search indexing, activity filtering can be easily done on the client
    side.
 4. Python scripts <https://github.com/sugarlabs-appstore/sugarappstore>
    are used to generate static HTML files of the app bundles and
    *index.json*. The activity's unique HTML document is produced
    without style / css, and are externally loaded, so that it can be
    customized / removed whenever necessary (for example on very low
    bandwidth connections, or for adding another theme)
 5. Sugar Appstore depends on *static HTML*s also because, it can be
    useful for SEOs, and to provide an external link to the activity
    from a third party website. Dynamically filtered search results
    might not be static. Providing a correct link to the bundle helps
    another user to share a specific activity bundle with another
 6. All the activities on https://github.com/sugarlabs
    <https://github.com/sugarlabs>are*indexed automatically *</see
    /*(**9**)*>//and built using *Continuous Integration* (GitHub
    Actions) over here
    <https://github.com/srevisaju/sugar-activity-build>. Out of 300+
    bundles, *199* and /still counting /have successfully built a *.xo
    bundle and are continuously uploaded to CDN.
 7. The*appstore could be downloaded for Offline use*, or can be hosted
    on a local server if a user would not like to use the online one.
    This is specifically useful, if a school would host, the sugar
    appstore on its local server, so students could download activities
    without accessing the internet. The entire app store (/batteries/
    and /bundles/ included) weighs approximately 613 MB. See /this
    <https://github.com/srevinsaju/sugar-activity-build/releases/tag/latest>/
    for a detailed idea.
 8. In case a user would like to add_another bundle _to the sugar
    appstore, previously he/she had to have a SugarLabs shell account
    (AFAIK, and /as far as I understand/). This might limit the
    accessibility to some users. This is hence solved by a new system of
    adding bundles to the activity. _*Pull Request to Add Applications *
    _is system which I adopted from the AppImage Hub
    (https://github.com/appimage/appimage.github.io
    <https://github.com/appimage/appimage.github.io>) where a similar
    app store <https://appimage.github.io> is built based on the same
    concept. Every PR is tested by continuous integration, and then all
    successfully merged PRs are added to the website, and the static
    website rebuild is triggered on each commit. A similar idea is
    implemented in Sugarlabs Appstore. This ensures that a .xo can be
    built on another host also using the same commands. Improving the
    quality of the bundles. For information regarding how to add a
    bundle, see this
    <https://github.com/srevinsaju/sugar-activity-build#adding-your-bundles-xo>.
    For a sample PR which intends to add a bundle, see PR#6
    <https://github.com/srevinsaju/sugar-activity-build/pull/6> which
    intends to add Jupyter Activity. All /urls/ which can be cloned
    using /git/ <https://git-scm.com/> are accepted. You may look into
    how the build script works here
    https://github.com/srevinsaju/sugar-activity-build/blob/master/.github/workflows/bundle_test.yml
    <https://github.com/srevinsaju/sugar-activity-build/blob/master/.github/workflows/bundle_test.yml>.
    /For those users who are not interested in GitHub, its recommended
    if Sugarlabs would host a GitLab which also supports CI; As this
    issue has been discussed previously, I am not putting too much
    insight into the same; :D/
 9. Builds are refreshed *every 5th minute *of a day; (cron: "5 * * *
    *"). There is no specific reason why 5th minute is chosen, mostly
    because I can finish pushing changes between 0th minute and 5th
    minute (normally because my school classes finish at the 0th minute;
    LOL; )
10. *Builds are hosted to netlify CDN*. *Netlify* CDN is chosen because
    its free for open source, and it also has a capability to get form
    submissions without a server. This is a cheap implementation without
    a server. There is no other reason why netlify chosen. Netlify has a
    very permissive quota. Other static site hosting as such as
    github.io will make the repo huge; or will require payment.
    Moreover. build on netlify takes only 1minute and 5 seconds to
    complete, with older fast network speed.
11. *Responsive HTML design*. Makes it adaptable on all screen sizes, on
    small android phones to 1080p screens and much more. This is better
    for SEO, as well as accessibility.
12. *Lighthouse speed testing* shows better /cumulative/ speed score
    than activities.sugarlabs.org. On Desktop, the score for ASLOv4 is
    94/100; on mobile its 76/100 for ASLOv4. For
    activities.sugarlabs.org, its 89/100 on Desktop, 76/100 on mobile.
    Results may vary slightly from time to time; Repeat tests to get
    concurrent values. Results provided by Lighthouse
    <https://github.com/googlechrome/lighthouse> powered by Google.
    Result details for sugarstore.netlify.app :
    https://developers.google.com/speed/pagespeed/insights/?url=https%3A%2F%2Fsugarstore.netlify.app%2F&tab=desktop;
    Result details for activities.sugarlabs.org:
    https://developers.google.com/speed/pagespeed/insights/?url=https%3A%2F%2Factivities.sugarlabs.org&tab=desktop
13. Its possible to embed (keeping a local copy of) /JQuery/ and
    /Bootstrap/ for offline use, but I personally prefer to keep the cdn
    (i.e <script>), this is because; modern browsers automatically cache
    commonly used script files. So this would increase the performance
    with caching. This will not require the user to download the libs
    again. Moreover, if the user has visited any other site which uses
    the same library, it would be actually better.
14. It looks sophisticated to keep a download count on static websites,
    but that does not mean its impossible. Its nevertheless, not
    possible (afaik) to get download files with only static files (i.e
    without a server to depend on). I could
     1. send a POST request to another server, which would increment the
        download count on every click of the download button; along with
        a GET request from the server to get the download count. This
        would imply, for every activity; a GET request will be emitted,
        or preferably, a single GET request to get all the download
        counts and update each of the activity shown for display with
        the new
     2. Use Firebase Library to create a firebase server where I could
        use Cloud Firestore and send and receive download stats.
        However, this has a few cons: i.e increase the page load time
        from 1.1s to 2.3 seconds; as well as make it complex to protect
        the server from user manipulation of the download count. I have
        tried this, I will enable it, if necessary
     3. Add a download analytics with a one way download count (send
        only - no receive). By using Google Analytics, I can send custom
        event trigger, and administrators can analyze and filter data /
        sort data by download counts. I, however, did not find a direct
        method to get data from Google Analytics.

15. Future Additions:
     1. *Add Featured Activities*: Activities which are most worked on:
        i.e *MusicBlocks*, *TurtleArt*, *Physics* Activities, *Fructose*
        Set of Activities could get a wide card along with a feature
        image. This will act as a showcase to the developer and the
        activity; mostly based on GitHub Stargazers. /I would like to
        follow https://snapcraft.io as a reference from which Ubuntu
        Software is built./
     2. *Add a link to **Flatpak**Package [DONE: Activity pages now show
        Flatpak card if its available]*: this aslov4 could be the
        /final/ activity store for sugarlabs. Recently many
        contributions have diverged. Its important to unify them
        together. Some users might prefer flatpak over a .xo file.
        Activities that are packaged as Flatpaks are good because some
        users, follow a modern desktop system, or might not be
        interested to install Sugar OS to run sugar desktop. For those
        corner cases, providing Flatpak as an alternative would not let
        users turn away from using Sugar Activities. Primarily Sugar
        Activities are meant for improving a child's skills or
        understanding; which ever mode, an activity is distributed;
        i.e., the most accessible package.
     3. Add information to other packages of sugar activity bundle. This
        includes Ubuntu Packages. Fedora RPMs, and the Arch Linux
        Community Packages; I will need a way to identify each of them;
        But at the present, I would like to leave it as is. If any
        developers at sugar is interested to change some structure of
        Sugar Repository, this would make things faster.
     4. *Add a screenshot, description tag in activity.info*:
        Screenshots of the working activity on the appstore will
        probably give a clearer idea on how the activity works. The
        https://help.sugarlabs.org is not frequently updated with
        changes on activity's repository. If we could add another tag
        called screenshots (following the convention given by Martin on
        https://github.com/tchx84/sugarapp. See
        https://github.com/tchx84/sugarapp/blob/master/flatpak-guide.md#porting-an-application-with-sugarapp
        (point 7) on how an ideal "activity.info" could be created. This
        would help me out in creating a better appstore (faster). The
        new"activity.info" could include 'screenshot' and 'description'
        tag which is not currently used. Martin has also mentioned of
        the source of the new data in the link given below.
     5. Create a*Python3 minimal activity *which does not need to load
        the entire website for downloading an activity. The activity
        will just pull index.json, and parse it. As the extra HTML, CSS,
        and JS are unnecessary for a GTK3 based sugar-activity, it would
        help to save a lot of data, especially on countries where
        internet connection is extremely slow (/50kbps/ ?)

I would like to convey my thanks to Manish for providing me frequent 
suggestions and always pointing out important flaws ( High CPU usage bug 
in /Chromium/Chrome/ Browser: 65a7b8f3ea2593976ef21df7a2592b030501c6d8 
<https://github.com/sugarlabs-appstore/sugarappstore-static/commit/65a7b8f3ea2593976ef21df7a2592b030501c6d8> 
, Accessibility ), and also the base repository, i.e., 
https://github.com/free-libre-software/sugarappstore as a template and 
idea provider for the /sugarlabs app store/ to take this shape.

________________________________________________

*Logo*:

The logo is designed under Creative Commons License Attribution Share 
Alike 4.0. You may get the source code of the logo here:

/*source*/: https://github.com/sugarlabs-appstore/appstore-assets

The logo has been designed with modern conventions of app design - 
symmetry and vibrance. Maybe its similar to the colorful icon design 
proposal for GSoC : 
/https://github.com/sugarlabs/GSoC/blob/master/Ideas-2020.md#colored-desktop-and-activity-icons/

________________________________________________

*Generator* (tool used to generate static webpages):

The generator is script which automates build process of bundles and 
writes static HTML files. Primarily written on Python Programming 
Language, can be customized with alternative script files which are 
supported on Linux. Generator, aka (internally called), SaaSBuild (Build 
tool for Sugarlabs Activity App Store), is designed to be multi platform 
supported, although tested on Linux only. Its possible to run it on 
other OSes too.

/*source*/: https://github.com/sugarlabs-appstore/sugarappstore

*NOTE* [IMPORTANT]: Current development goes on in PR#4 
(/https://github.com/sugarlabs-appstore/sugarappstore/pull/4/). You 
might need to checkout *oop-ss* if you want to test out the latest 
features. The *master* branch contains the work by Manish before the 
official start of GSoC. /The PR is waiting for a review./

________________________________________________

*Static HTML, CSS, JS*

These are files which can be run without the need of the generated 
index.json. Using JQuery, the data is dynamically read, and updated on 
the index.html.

/Q) Why was Static parts of the website not merged with the generator 
(SaaSBuild)?/

Its because of yet another few reasons

  * Another developer could manually add his own themes / html styleset
    with, maybe another JS Framework, for example React, Angular or
    Vue.js later. He/She would not have to rewrite the Generator class
    again for changing the frontend part.
  * Developers who are willing to contribute to HTML / Web design / Js
    can only work on the "static" repository. This will help to
    (probably) increase contributors. A developer who is only good at JS
    would not be hence be discouraged to contribute to the current
    repository, if Python is also involved.

*/source/: *https://github.com/sugarlabs-appstore/sugarappstore-static

________________________________________________

*Continuous Integration*

Continuous Integration Downloads and sets up Sugar desktop, i.e 
/sugar-toolkit-gtk3/, /sugar/,/sugar-datastore/, /sugar-artwork/ and 
creates bundles and packages it for publish to CDN

/*source*/: https://github.com/srevinsaju/sugar-activity-build

________________________________________________

The *DEPLOYED WEBSITE:*

The _best_ part, check it out here: *https://sugarstore.netlify.app*

For a list of indexed apps in a portable format:

  * *https://sugarstore.netlify.app/index.json*
  * *https://github.com/srevinsaju/sugar-activity-build/releases/download/latest/index.json*
  * *https://github.com/srevinsaju/sugar-activity-build/releases/download/latest/sitemap.xml*

For bundles app store (portable - except cdn), see

*https://github.com/srevinsaju/sugar-activity-build/releases*

________________________________________________

*/Q) How to run the appstore?/*

Use any static hosting server; for testing I use:

  * python3 -m http.server 3000
  * gunicorn
  * php
  * npm install -g http-server; http-server
  * apache
  * ngrok
  * serveo

*Regarding whom to approach:*

We are working on two different things. Manish <mailto:sugar at radii.dev> 
knows a way lot more than me regarding HTTP Headers, Sphinx, CORS, 
User-Agent, and other web server related stuff.I 
<mailto:srevinsaju at sugarlabs.org>am working on design (css, html, js) 
and the generator script (python), and those related to sugar GTK shell, 
sugar activities. We both collaborate on working with the generator script.

=====================================

Hope everyone like this. All questions are welcome. Please note that the 
App Store has/not yet released v0.1,/ so its basically in /alpha/. (PS: 
v0.1.alpha to be released this week-end)

PS: If anyone is interested to contribute, PRs are welcome. Please reply 
me on this thread if you require access to 
https://github.com/sugarlabs-appstore/ organization. It was created so 
that Manish and I could collaborate. And also so that, development won't 
freeze in the absence of one.

PS: If you have some other /sweet/ suggestions for Sugarlabs App Store 
(name), please let me know. Just as /jarabe/, /carquinyol/, and 
/sucrose/ are used within sugar, it might be interesting for kids to see 
their favorite sweet on the screen. We have stuck with 
sugarlabs-appstore, as we did not get any other name!

Thanks if you read all the way till here!!

Stay safe! Take care!!

V/r Srevin Saju https://github.com/srevinsaju

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sugarlabs.org/archive/sugar-devel/attachments/20200624/c444d72b/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0x1007816766D390D7.asc
Type: application/pgp-keys
Size: 4849 bytes
Desc: not available
URL: <http://lists.sugarlabs.org/archive/sugar-devel/attachments/20200624/c444d72b/attachment-0001.key>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <http://lists.sugarlabs.org/archive/sugar-devel/attachments/20200624/c444d72b/attachment-0001.sig>


More information about the Sugar-devel mailing list