[Sugar-devel] [ANNOUNCE] packagekit-backend-presolve as an attempt to provide installing activity dependencies on demand in order to support GNU/Linux distribution agnostic launch from app stores

[Peter Robinson]

Hi Aleksey,

Did the ever happen? Also was there plans for a control panel update
interface to package kit as well as the back end bits to support .xo
files with package kit?

Peter

On Sat, Sep 15, 2012 at 1:37 AM, Aleksey Lim <alsroot at sugarlabs.org> wrote:
> Hi all!
>
>
> This is an announce of new project that is intended to solve, as a part
> of more complex work, issue that prevents reliable launching of Sugar
> Activities from app stores like Activity Library or Sugar Network.
>
> The design document (mentioned below) url is:
>
>     http://wiki.sugarlabs.org/go/Platform_Team/packagekit-backend-presolve
>
> This is only design document, but the first implemlementation will be
> pretty trivial and will be ready for the next Sugar Network development
> release (later this month). Any ideas/suggestions are welcome.
>
>
> == Summary ==
>
> This is a [http://www.packagekit.org/ PackageKit] backend which is
> intended to be used on restricted systems like XO-1 laptops where
> regular package management routines are too heavy for limited amount of
> memory or CPU resources.
>
> == The reason ==
>
> The known, for the author of these lines, current practice is:
>
> * [http://wiki.laptop.org/go/Olpc-update
>   olpc-update]
>   [[wikipedia:Rsync|Rsync]] the full system.
>
> * [[Platform_Team/Server_Kit/sugar-client|sugar-client]]'s
>   [[Platform_Team/Server_Kit/sugar-client#Unattended_system_update|unattended
>   update]] in [[Dextrose]]
>   Regular update but only for particular,
>   much smaller than main ones, repository(ies).
>
> Both ways are useful for usecases they were created for. But there is a
> situation when they both become less useful. Having [[Activity Library]]
> (or its distributed version on regional or school server), XO users
> might download activities directly from the Internet (or distributed
> server). So, it is becoming hard to use:
>
> * olpc-update,
>   [[Activity Library]] might have non-standard
>   dependencies and it is either hard to keep olpc-update server
>   up-to-date, or, useless to install all possible dependencies on every
>   XO.
>
> * sugar-client,
>   activity dependencies are, in most cases, from
>   official repositories when sugar-client's 3rd party repository are
>   useless.
>
> In other words, there is a need to install particular packages from
> official Fedora repository and using existing methods is either
> impractical (olpc-update) or impossible (sugar-client). Fallback to
> using regular (like yum) ways is also not useful because they take too
> much memory (at the first) and CPU resources, it is a special problem on
> XO-1 laptops.
>
> The packagekit-backend-presolve is intended to solve this issue by
> providing standard way (it is PackageKit backend, i.e., sugar launching
> code will be common for all platforms, it will call PackageKit all time)
> to install activity dependencies and consuming as less as possible
> resources on XO laptops. The backend was designed to support reliable
> launch from resources like [[Activity Library]] and/or [[Sugar Network]]
> when there are activities that might have non-standard dependencies.
> This backend might be treated as a temporal solution, i.e., when either
> XO will be powerful enough or regular installation procedures will take
> less resources, it will be possible to switch to default PackageKit
> backend (without making changes in activity launching code).
>
> == Concept ==
>
> The general idea is to pre-resolve dependencies for a limited set of
> top-level packages and store this information on a server. Clients need
> to download only small amount of information (compared with entire list
> of available packages) to reuse it as-is (and avoid reading database
> into memory during solving process).
>
> Another important point is that pre-resolved dependency graphs don't
> include packages from the base system. In other words, a particular
> dependency tree doesn't end with the {{Code|glibc}} package. Instead, it
> ends with the {{Code|sugar}} package. This expedient should drastically
> reduce the size of trees.
>
> Finally, missing packages will be installed directly by using the
> {{Code|rpm -i}} command.
>
> === Upsides ===
>
> * Users download metadata on a package by package basis, thus avoiding
>   the need to download the full packages database;
>
> * After downloading, metadata will be reused as-is, i.e., after checking
>   what packages were already installed, system will download only missed
>   RPMs and install them directly by calling {{Code|rpm -i}} command;
>
> * Activities on resources like [[Activity Library]] or [[Sugar Network]]
>   will be distro-agnostic. That is, they will work unmodified on all
>   platforms that provide PackageKit (which means all mainstream
>   GNU/Linux distributions).
>
> === Downsides ===
>
> * The project is intended to install only limited number of top-level
>   packages, i.e., packages that have pre-resolved dependency trees on a
>   server; for example, it is not capable for full system update;
>
> * Need to support server-side generation of dependency trees.
>
> == Implementation ==
>
> === Premisses ===
>
> The implementation is following the premises:
>
> * Being targeted only to XO laptops (assuming that other platforms are
>   more powerful where standard PackageKit backend can be used);
>
> * Keep less work on XO side and more work on server side to make
>   installation process on client side as fast and reliable as possible;
>
> * Implement only first-needed functionality and provide more complex
>   features as required, e.g., current implementation doesn't support
>   updating already installed packages leave it to hard (and rare, e.g.,
>   yearly based) full system updates (re-flashes or calling olpc-update)
>   and incremental regular updates (sugar-client).
>
> === Overview ===
>
> * Clients keep a list of packages installed by the back-end;
>
> * After getting install request, the back-end will check if the
>   requested package is already installed;
>
> * If not, it downloads a file from know HTTP location with the requested
>   package's name;
>
> * Downloaded file contains RPM url for requested package and a list of
>   package-name/url pairs for all dependencies (starting from some known
>   point);
>
> * All missing packages will be downloaded and installed by directly
>   calling {{Code|rpm -i}};
>
> * Installed packages list will be updated correspondingly.
>
> == Usage examples ==
>
> [[Sugar Network]], in attempt to provide GNU/Linux distribution agnostic
> launches of activities it provides, supports packagekit-backend-presolve
> users.
>
> API server provides the following kinds of urls:
>
> * {{Code|<nowiki>http://api-devel.network.sugarlabs.org/packages</nowiki>}}
>   list all supported platforms;
>
> * {{Code|<nowiki>http://api-devel.network.sugarlabs.org/packages/</nowiki><PLATFORM>}}
>   list all supported packages for particular platform;
>
> * {{Code|<nowiki>http://api-devel.network.sugarlabs.org/packages/</nowiki><PLATFORM>/<PACKAGE>}}
>   pre-solved dependency graph for particular package.
>
> Where:
>
> * {{Code|PLATFORM}}
>   one of supported packagekit-backend-presolve
>   platforms, e.g., OLPC-13.0.1;
>
> * {{Code|PACKAGE}}
>   a name of the particular package.
>
> In order to support distribution agnostic launches, [[Sugar Network]]
> supports metadata ''database'' for all packages that are being used as
> dependencies for activities it provides (see ''Packages'' project using
> one of existing [[Sugar_Network#Try_it|Sugar Network clients]]). Every
> entry in the ''database'' contains a map of native package names for
> particular GNU/Linux distributions. So, activities mention only
> ''database'' entry name as a dependency. [[Sugar Network]] will
> automatically generate dependency graphs for packagekit-backend-presolve
> using ''metadata'' database. Afterwards, on launch side, ''database''
> entry name will be resolved to a package name according to local
> distribution and will be passed to the PackageKit (on XO laptops,
> packagekit-backend-presolve will be used; on regular platforms, default
> PackageKit back-end).
>
> == Sources ==
>
> Clone source repository from [http://git.sugarlabs.org/desktop
> Gitorious] project:
>
>  git clone
>  git://git.sugarlabs.org/desktop/packagekit-backend-presolve.git
>
> --
> Aleksey
> _______________________________________________
> Sugar-devel mailing list
> Sugar-devel at lists.sugarlabs.org
> http://lists.sugarlabs.org/listinfo/sugar-devel