[IAEP] [support-gang] [IIAB] [XSCE] RE: [UKids] Re: Taking OpenStreetMap Offline - DESIGN Call - Thur June 11, 10AM EDT / 2PM UTC

[Tim Moody]

I agree on the VM.  I use VBox and usually give it 1G.  I have started giving 100G of disk because it only takes what it needs.

 

I haven't gotten into vagrant, so I just create a 'Base' VM and then clone it, which takes about 2 minutes.  I put minimal fedora (I have both 21 and 22) plus my keys, git, nano, wget, ansible, and mkdir /opt/schoolserver into the base.

 

Now I just have to clone the vm, clone the xsce repo and either ./install-console or ./runansible.  I usually save this VM too, for future cloning to do tests.

 

From: support-gang [mailto:support-gang-bounces at lists.laptop.org] On Behalf Of Anish Mangal
Sent: Saturday, June 20, 2015 10:53 PM
To: Jérôme Gagnon-Voyer
Cc: xsce-devel; Internet In a Box Working Group; Braddock Gaskill; Unleash Kids!; Jaakko Helleranta; server-devel; iaep; Community Support Volunteers -- who help respond to help AT laptop.org
Subject: Re: [support-gang] [IIAB] [XSCE] RE: [UKids] Re: Taking OpenStreetMap Offline - DESIGN Call - Thur June 11, 10AM EDT / 2PM UTC

 

I guess a VM using a single core and 512 MB or 1GB RAM would be a reasonable starting point. A Rpi-2 would be great as well. You could just test from the latest master branch, and it *should* work. Currently, a role called 'Pathagar' is broken, so you may have to disable installing it in vars/default_vars.yml when you give it a go. 

Is using a VM, I often tend to use vagrant. You can use a fedora 21/22 minimal or centos minimal image. 

This is the image I used for testing my XSCE install, where things went smoothly - https://github.com/holms/vagrant-centos7-box/releases/download/7.1.1503.001/CentOS-7.1.1503-x86_64-netboot.box

If using centOS you will need to enable the epel repo - yum install epel-release

 

Hope this helps. 

Best,

Anish

 

On Sun, Jun 21, 2015 at 7:44 AM, Jérôme Gagnon-Voyer <gagnonje at gmail.com <mailto:gagnonje at gmail.com> > wrote:

Hi all

 

I wanted to do some testing of the search solutions on some kind of hardware/software that would look similar to what will be deployed in the field (to measure the performance impact).

 

Got this idea from this post where : http://www.openstreetmap.org/user/SimonPoole/diary/34857

 

Should I be testing with XSCE 5.1? XSCE 5.5 beta? Should I get some hardware like a Pi2, or just get some VM? Any VM recommendation?

 

Thanks

 

On Wed, Jun 17, 2015 at 3:15 PM, Tim Moody <tim at timmoody.com <mailto:tim at timmoody.com> > wrote:

Thanks, this is very useful information.  I didn't realize that slippy maps is a strategy and leaflet is a library for it that OSM itself uses.

 

A few questions.

 

Which of the filtered datasets did you use for populated places? 

 

Are they from http://wiki.openstreetmap.org/wiki/Planet.osm#Country_and_area_extracts ?

 

Is the directory structure such that separately rendered tiles for different regions can be merged by over-copying?

 

I did some sampling.

 

In Manhattan the maximum zoom is  http://xsce/iiab/maps/tile/15/9649/12318.png so level 15 (16th)

 

In Northern Ontario the maximum zoom is http://xsce/iiab/maps/tile/13/2048/2704.png level 13 (14th)

 

In the middle of the North Pacific http://xsce/iiab/maps/tile/14/1088/6190.png a surprising level 14

 

Moving slightly left I get http://xsce/iiab/maps/tile/14/1080/6191.png so I'm not quite clear on the generic blue tile point.

 

I'm also not clear on the point about the meta tiles.  From what I read in http://wiki.openstreetmap.org/wiki/Meta_tiles , meta files just concatenate the smaller tiles, so I'm not clear on how that saves space.  I do understand that it greatly reduces the number of nodes in the file system making retrieving them viable.

 

I looked at space usage by zoom level on the current IIAB OSM:

 


Level

Size (in K)

Cumulative (in K)

Cumulative (in G)

				

0

60

60

0


1

84

144

0


2

152

296

0


3

300

596

0


4

1,004

1,600

0


5

2,800

4,400

0


6

9,064

13,464

0


7

26,328

39,792

0


8

86,280

126,072

0


9

255,576

381,648

0


10

859,540

1,241,188

1


11

2,233,136

3,474,324

3


12

5,905,172

9,379,496

9


13

17,480,260

26,859,756

27


14

43,947,380

70,807,136

71


15

35,454,488

106,261,624

106

 

So 10G gets 13 levels.

 

 

From: unleashkids at googlegroups.com <mailto:unleashkids at googlegroups.com>  [mailto:unleashkids at googlegroups.com <mailto:unleashkids at googlegroups.com> ] On Behalf Of Braddock Gaskill
Sent: Monday, June 15, 2015 8:39 PM
To: Internet In a Box Working Group
Cc: xsce-devel; Unleash Kids!; Jaakko Helleranta; server-devel; iaep; Community Support Volunteers -- who help respond to help AT laptop.org <http://laptop.org> 
Subject: Re: [IIAB] [XSCE] RE: [UKids] Re: [support-gang] Taking OpenStreetMap Offline - DESIGN Call - Thur June 11, 10AM EDT / 2PM UTC

 

In IIAB we use a dataset of populated places to render at the highest resolution level, and render the rest of the world at a slightly lower resolution.  Our highest resolution is high enough to see individual buildings.  Our total dataset of rendered tiles for the world with detailed populated places coverage is 100GB.

 

Another optimization we made for IIAB that contributed considerable dataset size reduction and helped with filesystem efficiency - we use the OSM "meta" tiles, which normally store 64 tile png images, but we modify them to store 4096 png images per file, and we made an optimization so that identical tiles are removed (think of all the blue of the ocean cover 75% of the earth).

 

We use Leaflet as our javascript web front end, and it works quite well.

 

-braddock

 

 

On Sun, Jun 14, 2015 at 11:24 PM, Nick Doiron <ndoiron at mapmeld.com <mailto:ndoiron at mapmeld.com> > wrote:

I don't think there's any technical issues with rendering the world at 10 and specific countries at 16, other than the human knowing where they can and cannot zoom

 

-- Nick

 

On Sun, Jun 14, 2015 at 11:58 AM, Tim Moody <tim at timmoody.com <mailto:tim at timmoody.com> > wrote:

couple of observations:

 

As expected, the new tiles have a lot more detail.

 

There are more levels of zoom in the new ones.

 

Some names have changed - the old map had Lalitpur and the new one has Patan (both are used)

 

I don't see any boxes for unprintable characters, but there is a lot less Devanagari. (Google maps has more)

 

Is it possible to merge individually generated regional tiles?  for example if you rendered India and Nepal separately would you get both?

 

What happens if you render the world at level 10 and then specific countries at 16?

 

From: xsce-devel at googlegroups.com <mailto:xsce-devel at googlegroups.com>  [mailto:xsce-devel at googlegroups.com <mailto:xsce-devel at googlegroups.com> ] On Behalf Of Anish Mangal
Sent: Saturday, June 13, 2015 10:06 PM
To: Jérôme Gagnon-Voyer
Cc: xsce-devel; Community Support Volunteers -- who help respond to help AT laptop.org <http://laptop.org> ; Unleash Kids!; server-devel; iaep; Internet In a Box Working Group; Jaakko Helleranta
Subject: Re: [XSCE] RE: [UKids] Re: [support-gang] Taking OpenStreetMap Offline - DESIGN Call - Thur June 11, 10AM EDT / 2PM UTC

 

Hi,

Following from the skype call this week, I uploaded Nepal's OSM data in postgres and render tiles to see performance, and as expected, everything is blazing fast. 

The pbf file went into the postgress in less than 5 minutes, and tiles are being rendered pretty fast as well.

You can check it yourself by going to 
http://home.braddock.com:28112/osm/slippymap.html

* Select Mapnik

* Zoom out and center over Nepal

* Select Local tiles

* Zoom in

I don't know what this "proves" as the bottleneck still is doing this for the entire planet, which we need to figure out a way for anyway.

Best,

Anish

 

 

On Sun, Jun 14, 2015 at 6:02 AM, Anish Mangal <anishmg at umich.edu <mailto:anishmg at umich.edu> > wrote:

Hi Jérôme,

I incorporated one of your comments; as for the other I think Timm, Nick would be better suited for the discussion (python backend v/s frontend js). :-)

Best,

Anish

 

On Sun, Jun 14, 2015 at 3:22 AM, Jérôme Gagnon-Voyer <gagnonje at gmail.com <mailto:gagnonje at gmail.com> > wrote:

Hi Anish

 

Great document. I've added 2 comments to the document, feel free to incorporate into the document if that makes sense.

 

Also added a TODO for myself to do more research about the various search solutions. 

I've used Nominatim in the past (on the client side, not the server infrastructure) and it was overall very good, but I'll want to know more about the other ones.

 

On Sat, Jun 13, 2015 at 12:51 PM, Tim Moody <tim at timmoody.com <mailto:tim at timmoody.com> > wrote:

Thanks, Anish for an excellent start at getting this down on paper (so to speak).  I think this covers things pretty well and gives us the necessary hooks on which to hang the details as we begin fleshing out solutions to the requirements you documented.

 

Under issues I added one point about rendering non-Roman character sets.

 

From: unleashkids at googlegroups.com <mailto:unleashkids at googlegroups.com>  [mailto:unleashkids at googlegroups.com <mailto:unleashkids at googlegroups.com> ] On Behalf Of Anish Mangal
Sent: Saturday, June 13, 2015 12:15 PM
To: Community Support Volunteers -- who help respond to help AT laptop.org <http://laptop.org> 
Cc: Unleash Kids!; server-devel; xsce-devel; iaep; Internet In a Box Working Group; Jaakko Helleranta
Subject: [UKids] Re: [support-gang] Taking OpenStreetMap Offline - DESIGN Call - Thur June 11, 10AM EDT / 2PM UTC

 

Okay, so I tried to encapsulate whatever we discussed into a design document, which can hopefully serve as a base for reaching out to the OSM community and to better organize our own thoughts. 

https://docs.google.com/document/d/1LWsN-cPH3lvMuXS-f0Tk8IWVh-3X808WoIt0OJ-QNt8/edit# <https://docs.google.com/document/d/1LWsN-cPH3lvMuXS-f0Tk8IWVh-3X808WoIt0OJ-QNt8/edit> 

Please feel free to edit the information there as I may have missed some points, or interpreted them differently than as intended. 

Best,

Anish

 

On Fri, Jun 5, 2015 at 12:00 AM, Adam Holt <holt at laptop.org <mailto:holt at laptop.org> > wrote:

Who's attending http://stateofthemap.us at the UN in NYC this weekend?

Who's most serious about bringing OpenStreetMap's opportunities and rapid progress into the hands of the world's offline poor -- kids and all, American and Swahili?  Wanna Take This Map Outside, offline and off the grid, where we all belong/began?

http://Internet-in-a-Box.org and similar efforts have made a gigantic 1st step: in Ghana and Rwanda I could never have imagined better reviews to our 2015 deployments begun there in recent months.  Many are now changing the game increasingly for the Bottom BillionS, among many who've literally never seen a globe before, nevermind a map of their own towns.  Both in OLPC (school) contexts, in libraries, on Nepali hillsides where folks don't have proper homes, and far beyond --- perspectives (literally) are about to change.

But modern phones today contain more than enough gigabytes to display ALL OpenStreetMap map detail within most countries, and yet they do not yet, WHY?  Where are the Khan Academies and KA Lites of OSM (OpenStreetMap) bringing new classroom/journalistic rubrics, freeing everyone's "brain software" to explore and document our own communities in our own languages?

Who Will Take The Next Steps?
What engineering middleware, distribution vectors, community/economic models, and final field packagings will get us all there & beyond?

Will offline edit-contributions prove impossible, much like with Wikipedia in Peru, when offline kids edit overly stale OpenStreetMap/Wikipedia images a year before/later?


 

As such the wider OLPC community is hosting a DESIGN Call to bring forward ideas across the OpenStreetMap landscape, fertilizing our immediate work with school server projects like http://schoolserver.org, http://xsce.org, https://en.opensuse.org/openSUSE:Education-Li-f-e ETC.  Ministries of Education in India elsewhere are watching closely, expressly eager to help if we can point the way.  All giving our "2020 Vision" questions very practical and immediate "customers" well before 2020, much like Garmin GPS units fed a wonderful ecosystem of geo-specific "gmapsupp.img" offline map files over the past decade: http://wiki.openstreetmap.org/wiki/OSM_Map_On_Garmin/Download

That decade's now done: what IS our framework for the coming decade?  Please join us if you have strategic/partnership ideas towards making Offline OSM Designs happen, Thank You !!

 

10AM New York Time / 2PM UTC

Thursday, June 11th

RSVP with your Skype username, if we are more than 15-20 we'll use an industrial conference call system instead!


CONCLUSION: Could OpenStreetMap be the very ultimate in Constructionist Learning Projects, replacing OLPC in coming years, on a quickly shrinking/endangered but still green-in-parts planet?  Regardless, how to build the OLPC Movement's offline/civic learning successes, consciously learning from its community infrastructural mistakes?  Nick Doiron (http://mapmeld.com) and Anish Mangal (https://in.linkedin.com/in/anishmangal) who've spent years thinking about this topic will lead the discussion, agenda is entirely yours!  OPTIONAL: submit agenda items in advance right here:

https://docs.google.com/document/d/1o6QtzLb6e58YKWqMf_junux2XyBRLFm31un8YLcYslg

 

< forward invitation to Twitter/wherever as appropriate >



--

Unsung Heroes of OLPC, interviewed live @ http://unleashkids.org !


_______________________________________________
support-gang mailing list
support-gang at lists.laptop.org <mailto:support-gang at lists.laptop.org> 
http://lists.laptop.org/listinfo/support-gang





-- 
Unsung Heroes of OLPC, interviewed live @ http://unleashkids.org !
--- 
You received this message because you are subscribed to the Google Groups "Unleash Kids" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unleashkids+unsubscribe at googlegroups.com <mailto:unleashkids+unsubscribe at googlegroups.com> .
For more options, visit https://groups.google.com/d/optout.





 

-- 

Jérôme Gagnon-Voyer





-- 

 

 

 

 

 

-- 
Unsung Heroes of OLPC, interviewed live @ http://unleashkids.org !
--- 
You received this message because you are subscribed to the Google Groups "Unleash Kids" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unleashkids+unsubscribe at googlegroups.com <mailto:unleashkids+unsubscribe at googlegroups.com> .
For more options, visit https://groups.google.com/d/optout.





 

-- 

Jérôme Gagnon-Voyer

-- 
Unsung Heroes of OLPC, interviewed live @ http://unleashkids.org !
--- 
You received this message because you are subscribed to the Google Groups "Unleash Kids" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unleashkids+unsubscribe at googlegroups.com <mailto:unleashkids+unsubscribe at googlegroups.com> .
For more options, visit https://groups.google.com/d/optout.

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sugarlabs.org/archive/iaep/attachments/20150621/95411fac/attachment-0001.html>