[Sugar-devel] Updates: Python3 app store (aslov4) : Sugar App Store

Srevin Saju srevinsaju at sugarlabs.org
Wed Jun 24 08:48:34 EDT 2020

Previous message (by thread): [Sugar-devel] Updates: Python3 app store (aslov4) : Sugar App Store
Next message (by thread): [Sugar-devel] Sugarizer Knowledge Activity Pack meeting [17 June]
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Hello,

/(PS: If you received the email earlier, please ignore this. For some
reason, my previous email was marked as spam, so most of them might not
have actually received it. This is an updated email of the previously
sent 'Updates: Python3 app store (aslov4) : Sugar App Store'; I hope
this email is also not classified as spam, //¯\_(ツ)_/¯ )/

Hope all are having a safe week amid nCoV.

I and Manish (sugar at radii.dev) had been working on the GSoC project
which did *not* get a slot (Python3 app store aslov4), and I guess we
have made good progress in it. /We decided to collaborate on the project
because a frontend and backend developer combo helps to get the best
features of both ends :D./

/To avoid misconceptions and misinterpretations, please read the entire
email before replying. Thanks! :P/

*Need Statement*:

In short, the most important points

* /Create the simplest possible app store for Sugar activities, where
each activity included has been (a) ported to Python 3 and released,
and (b) tested on Sugar Live Build./
* /We used to have an app store for Sugar activities, but because we
can't seem to attract any PHP developers the app store has failed to
keep up with development./
* /We now use activities.sugarlabs.org for Python 2 activities only./
* /We have tried to make a replacement for activities.sugarlabs.org
<https://activities.sugarlabs.org> three times, and each time the
features we need were not finished. These projects have been too
ambitious and have not been supported collectively by the Sugar Labs
community./

/~ from
//https://github.com/sugarlabs/GSoC/blob/master/Ideas-2020.md#sugar-app-store-for-python-3-activities-aslov4/

This made us rethink the idea of a redesigned app store for sugarlabs.
This, would be a collection of all the activities in sugarlabs
organization which can be /*successfully* built/ without /errors/.

The *new app store *is special in a lot of ways, let me bring out
the/most important points:/

1. The appstore is built in _pure static HTML_, Vanilla Javascript
<https://developer.mozilla.org/en-US/docs/Web/JavaScript> and JQuery
<https://jquery.com/>. Anyone with /basic Javascript knowledge/ can
easily add features. Theming and customization are provided by
Bootstrap4 <https://getbootstrap.com>. The theme, scripts are
completely _independent_ of each other. Changes to the /will not
/affect the generation of the appstore.
2. *Search function* is provided by a powerful Javascript library
called minisearch <https://github.com/lucaong/minisearch> which is
an open source search indexer. All processing is therefore done on
client side. The benefits of using an external library are many, as
you know,
3. The App Store is based on *JSON* (/Javascript Object Notation/). an
"index.json" is produced by the generator helper script so that
search indexing, activity filtering can be easily done on the client
side.
4. Python scripts <https://github.com/sugarlabs-appstore/sugarappstore>
are used to generate static HTML files of the app bundles and
*index.json*. The activity's unique HTML document is produced
without style / css, and are externally loaded, so that it can be
customized / removed whenever necessary (for example on very low
bandwidth connections, or for adding another theme)
5. Sugar Appstore depends on *static HTML*s also because, it can be
useful for SEOs, and to provide an external link to the activity
from a third party website. Dynamically filtered search results
might not be static. Providing a correct link to the bundle helps
another user to share a specific activity bundle with another
6. All the activities on https://github.com/sugarlabs
<https://github.com/sugarlabs>are*indexed automatically *</see
/*(**9**)*>//and built using *Continuous Integration* (GitHub
Actions) over here
<https://github.com/srevisaju/sugar-activity-build>. Out of 300+
bundles, *199* and /still counting /have successfully built a *.xo
bundle and are continuously uploaded to CDN.
7. The*appstore could be downloaded for Offline use*, or can be hosted
on a local server if a user would not like to use the online one.
This is specifically useful, if a school would host, the sugar
appstore on its local server, so students could download activities
without accessing the internet. The entire app store (/batteries/
and /bundles/ included) weighs approximately 613 MB. See /this
<https://github.com/srevinsaju/sugar-activity-build/releases/tag/latest>/
for a detailed idea.
8. In case a user would like to add_another bundle _to the sugar
appstore, previously he/she had to have a SugarLabs shell account
(AFAIK, and /as far as I understand/). This might limit the
accessibility to some users. This is hence solved by a new system of
adding bundles to the activity. _*Pull Request to Add Applications *
_is system which I adopted from the AppImage Hub
(https://github.com/appimage/appimage.github.io
<https://github.com/appimage/appimage.github.io>) where a similar
app store <https://appimage.github.io> is built based on the same
concept. Every PR is tested by continuous integration, and then all
successfully merged PRs are added to the website, and the static
website rebuild is triggered on each commit. A similar idea is
implemented in Sugarlabs Appstore. This ensures that a .xo can be
built on another host also using the same commands. Improving the
quality of the bundles. For information regarding how to add a
bundle, see this
<https://github.com/srevinsaju/sugar-activity-build#adding-your-bundles-xo>.
For a sample PR which intends to add a bundle, see PR#6
<https://github.com/srevinsaju/sugar-activity-build/pull/6> which
intends to add Jupyter Activity. All /urls/ which can be cloned
using /git/ <https://git-scm.com/> are accepted. You may look into
how the build script works here
https://github.com/srevinsaju/sugar-activity-build/blob/master/.github/workflows/bundle_test.yml
<https://github.com/srevinsaju/sugar-activity-build/blob/master/.github/workflows/bundle_test.yml>.
/For those users who are not interested in GitHub, its recommended
if Sugarlabs would host a GitLab which also supports CI; As this
issue has been discussed previously, I am not putting too much
insight into the same; :D/
9. Builds are refreshed *every 5th minute *of a day; (cron: "5 * * *
*"). There is no specific reason why 5th minute is chosen, mostly
because I can finish pushing changes between 0th minute and 5th
minute (normally because my school classes finish at the 0th minute;
LOL; )
10. *Builds are hosted to netlify CDN*. *Netlify* CDN is chosen because
its free for open source, and it also has a capability to get form
submissions without a server. This is a cheap implementation without
a server. There is no other reason why netlify chosen. Netlify has a
very permissive quota. Other static site hosting as such as
github.io will make the repo huge; or will require payment.
Moreover. build on netlify takes only 1minute and 5 seconds to
complete, with older fast network speed.
11. *Responsive HTML design*. Makes it adaptable on all screen sizes, on
small android phones to 1080p screens and much more. This is better
for SEO, as well as accessibility.
12. *Lighthouse speed testing* shows better /cumulative/ speed score
than activities.sugarlabs.org. On Desktop, the score for ASLOv4 is
94/100; on mobile its 76/100 for ASLOv4. For
activities.sugarlabs.org, its 89/100 on Desktop, 76/100 on mobile.
Results may vary slightly from time to time; Repeat tests to get
concurrent values. Results provided by Lighthouse
<https://github.com/googlechrome/lighthouse> powered by Google.
Result details for sugarstore.netlify.app :
https://developers.google.com/speed/pagespeed/insights/?url=https%3A%2F%2Fsugarstore.netlify.app%2F&tab=desktop;
Result details for activities.sugarlabs.org:
https://developers.google.com/speed/pagespeed/insights/?url=https%3A%2F%2Factivities.sugarlabs.org&tab=desktop
13. Its possible to embed (keeping a local copy of) /JQuery/ and
/Bootstrap/ for offline use, but I personally prefer to keep the cdn
(i.e <script>), this is because; modern browsers automatically cache
commonly used script files. So this would increase the performance
with caching. This will not require the user to download the libs
again. Moreover, if the user has visited any other site which uses
the same library, it would be actually better.
14. It looks sophisticated to keep a download count on static websites,
but that does not mean its impossible. Its nevertheless, not
possible (afaik) to get download files with only static files (i.e
without a server to depend on). I could
1. send a POST request to another server, which would increment the
download count on every click of the download button; along with
a GET request from the server to get the download count. This
would imply, for every activity; a GET request will be emitted,
or preferably, a single GET request to get all the download
counts and update each of the activity shown for display with
the new
2. Use Firebase Library to create a firebase server where I could
use Cloud Firestore and send and receive download stats.
However, this has a few cons: i.e increase the page load time
from 1.1s to 2.3 seconds; as well as make it complex to protect
the server from user manipulation of the download count. I have
tried this, I will enable it, if necessary
3. Add a download analytics with a one way download count (send
only - no receive). By using Google Analytics, I can send custom
event trigger, and administrators can analyze and filter data /
sort data by download counts. I, however, did not find a direct
method to get data from Google Analytics.

15. Future Additions:
1. *Add Featured Activities*: Activities which are most worked on:
i.e *MusicBlocks*, *TurtleArt*, *Physics* Activities, *Fructose*
Set of Activities could get a wide card along with a feature
image. This will act as a showcase to the developer and the
activity; mostly based on GitHub Stargazers. /I would like to
follow https://snapcraft.io as a reference from which Ubuntu
Software is built./
2. *Add a link to **Flatpak**Package [DONE: Activity pages now show
Flatpak card if its available]*: this aslov4 could be the
/final/ activity store for sugarlabs. Recently many
contributions have diverged. Its important to unify them
together. Some users might prefer flatpak over a .xo file.
Activities that are packaged as Flatpaks are good because some
users, follow a modern desktop system, or might not be
interested to install Sugar OS to run sugar desktop. For those
corner cases, providing Flatpak as an alternative would not let
users turn away from using Sugar Activities. Primarily Sugar
Activities are meant for improving a child's skills or
understanding; which ever mode, an activity is distributed;
i.e., the most accessible package.
3. Add information to other packages of sugar activity bundle. This
includes Ubuntu Packages. Fedora RPMs, and the Arch Linux
Community Packages; I will need a way to identify each of them;
But at the present, I would like to leave it as is. If any
developers at sugar is interested to change some structure of
Sugar Repository, this would make things faster.
4. *Add a screenshot, description tag in activity.info*:
Screenshots of the working activity on the appstore will
probably give a clearer idea on how the activity works. The
https://help.sugarlabs.org is not frequently updated with
changes on activity's repository. If we could add another tag
called screenshots (following the convention given by Martin on
https://github.com/tchx84/sugarapp. See
https://github.com/tchx84/sugarapp/blob/master/flatpak-guide.md#porting-an-application-with-sugarapp
(point 7) on how an ideal "activity.info" could be created. This
would help me out in creating a better appstore (faster). The
new"activity.info" could include 'screenshot' and 'description'
tag which is not currently used. Martin has also mentioned of
the source of the new data in the link given below.
5. Create a*Python3 minimal activity *which does not need to load
the entire website for downloading an activity. The activity
will just pull index.json, and parse it. As the extra HTML, CSS,
and JS are unnecessary for a GTK3 based sugar-activity, it would
help to save a lot of data, especially on countries where
internet connection is extremely slow (/50kbps/ ?)

I would like to convey my thanks to Manish for providing me frequent
suggestions and always pointing out important flaws ( High CPU usage bug
in /Chromium/Chrome/ Browser: 65a7b8f3ea2593976ef21df7a2592b030501c6d8
<https://github.com/sugarlabs-appstore/sugarappstore-static/commit/65a7b8f3ea2593976ef21df7a2592b030501c6d8>
, Accessibility ), and also the base repository, i.e.,
https://github.com/free-libre-software/sugarappstore as a template and
idea provider for the /sugarlabs app store/ to take this shape.

________________________________________________

*Logo*:

The logo is designed under Creative Commons License Attribution Share
Alike 4.0. You may get the source code of the logo here:

/*source*/: https://github.com/sugarlabs-appstore/appstore-assets

The logo has been designed with modern conventions of app design -
symmetry and vibrance. Maybe its similar to the colorful icon design
proposal for GSoC :
/https://github.com/sugarlabs/GSoC/blob/master/Ideas-2020.md#colored-desktop-and-activity-icons/

________________________________________________

*Generator* (tool used to generate static webpages):

The generator is script which automates build process of bundles and
writes static HTML files. Primarily written on Python Programming
Language, can be customized with alternative script files which are
supported on Linux. Generator, aka (internally called), SaaSBuild (Build
tool for Sugarlabs Activity App Store), is designed to be multi platform
supported, although tested on Linux only. Its possible to run it on
other OSes too.

/*source*/: https://github.com/sugarlabs-appstore/sugarappstore

*NOTE* [IMPORTANT]: Current development goes on in PR#4
(/https://github.com/sugarlabs-appstore/sugarappstore/pull/4/). You
might need to checkout *oop-ss* if you want to test out the latest
features. The *master* branch contains the work by Manish before the
official start of GSoC. /The PR is waiting for a review./

________________________________________________

*Static HTML, CSS, JS*

These are files which can be run without the need of the generated
index.json. Using JQuery, the data is dynamically read, and updated on
the index.html.

/Q) Why was Static parts of the website not merged with the generator
(SaaSBuild)?/

Its because of yet another few reasons

* Another developer could manually add his own themes / html styleset
with, maybe another JS Framework, for example React, Angular or
Vue.js later. He/She would not have to rewrite the Generator class
again for changing the frontend part.
* Developers who are willing to contribute to HTML / Web design / Js
can only work on the "static" repository. This will help to
(probably) increase contributors. A developer who is only good at JS
would not be hence be discouraged to contribute to the current
repository, if Python is also involved.

*/source/: *https://github.com/sugarlabs-appstore/sugarappstore-static

________________________________________________

*Continuous Integration*

Continuous Integration Downloads and sets up Sugar desktop, i.e
/sugar-toolkit-gtk3/, /sugar/,/sugar-datastore/, /sugar-artwork/ and
creates bundles and packages it for publish to CDN

/*source*/: https://github.com/srevinsaju/sugar-activity-build

________________________________________________

The *DEPLOYED WEBSITE:*

The _best_ part, check it out here: *https://sugarstore.netlify.app*

For a list of indexed apps in a portable format:

* *https://sugarstore.netlify.app/index.json*
* *https://github.com/srevinsaju/sugar-activity-build/releases/download/latest/index.json*
* *https://github.com/srevinsaju/sugar-activity-build/releases/download/latest/sitemap.xml*

For bundles app store (portable - except cdn), see

*https://github.com/srevinsaju/sugar-activity-build/releases*

________________________________________________

*/Q) How to run the appstore?/*

Use any static hosting server; for testing I use:

* python3 -m http.server 3000
* gunicorn
* php
* npm install -g http-server; http-server
* apache
* ngrok
* serveo

*Regarding whom to approach:*

We are working on two different things. Manish <mailto:sugar at radii.dev>
knows a way lot more than me regarding HTTP Headers, Sphinx, CORS,
User-Agent, and other web server related stuff.I
<mailto:srevinsaju at sugarlabs.org>am working on design (css, html, js)
and the generator script (python), and those related to sugar GTK shell,
sugar activities. We both collaborate on working with the generator script.

=====================================

Hope everyone like this. All questions are welcome. Please note that the
App Store has/not yet released v0.1,/ so its basically in /alpha/. (PS:
v0.1.alpha to be released this week-end)

PS: If anyone is interested to contribute, PRs are welcome. Please reply
me on this thread if you require access to
https://github.com/sugarlabs-appstore/ organization. It was created so
that Manish and I could collaborate. And also so that, development won't
freeze in the absence of one.

PS: If you have some other /sweet/ suggestions for Sugarlabs App Store
(name), please let me know. Just as /jarabe/, /carquinyol/, and
/sucrose/ are used within sugar, it might be interesting for kids to see
their favorite sweet on the screen. We have stuck with
sugarlabs-appstore, as we did not get any other name!

Thanks if you read all the way till here!!

Stay safe! Take care!!

V/r Srevin Saju https://github.com/srevinsaju

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sugarlabs.org/archive/sugar-devel/attachments/20200624/c444d72b/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0x1007816766D390D7.asc
Type: application/pgp-keys
Size: 4849 bytes
Desc: not available
URL: <http://lists.sugarlabs.org/archive/sugar-devel/attachments/20200624/c444d72b/attachment-0001.key>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <http://lists.sugarlabs.org/archive/sugar-devel/attachments/20200624/c444d72b/attachment-0001.sig>

Previous message (by thread): [Sugar-devel] Updates: Python3 app store (aslov4) : Sugar App Store
Next message (by thread): [Sugar-devel] Sugarizer Knowledge Activity Pack meeting [17 June]
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

More information about the Sugar-devel mailing list