I have read the patches.<div>The only one I don't like too much is the last one.</div><div>I prefer doing like in dataretriever.py</div><div><br></div><div><a href="https://github.com/cscott/wikiserver/blob/50446f21436c936f0aea3884401fc6cd1e8ae5d6/dataretriever.py#L179">https://github.com/cscott/wikiserver/blob/50446f21436c936f0aea3884401fc6cd1e8ae5d6/dataretriever.py#L179</a></div>
<div><br></div><div>I know, there are a few places with code duplication, I need improve that.</div><div><br></div><div>Gonzalo</div><div><br></div><div><br><div class="gmail_quote">On Sun, Feb 3, 2013 at 11:33 PM, Gonzalo Odiard <span dir="ltr"><<a href="mailto:godiard@gmail.com" target="_blank">godiard@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Thanks!<div>I don't know if I will have time this week, but will look at this and integrate the changes.</div><div>
<br></div><div>Gonzalo<div><div class="h5"><br><br><div class="gmail_quote">On Sun, Feb 3, 2013 at 5:25 PM, C. Scott Ananian <span dir="ltr"><<a href="mailto:cscott@laptop.org" target="_blank">cscott@laptop.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I did quite a bit of work on the Wikipedia activity this weekend (related to an HTML port I was working on). I've pushed a set of patches to:<br>
<a href="https://github.com/cscott/wikiserver" target="_blank">https://github.com/cscott/wikiserver</a><br>
based on the latest version at<br> ssh://<a href="http://dev.laptop.org/git/projects/wikiserver" target="_blank">dev.laptop.org/git/projects/wikiserver</a><br><br>Full list of patches is at the end of this email. The summary: I updated mwlib (used to generate HTML from mediawiki markup) from 0.6.2 to 0.8.5 to fix issues I had with infinite-loops in template rendering in the full simple English wikipedia. In the process we got the latest full complete list of translations for Image/Media/Template etc so greatly broadened the # of languages the Wikipedia activity will work with.<br>
<br>I also fixed a bug with #anchors in article titles which was causing multiple copies of the articles to be added to the database. This should decrease the size of the database for all languages. Another bug squashed involved different articles with the names differing only in case, for example Template:TOCright and Template:tocright. And there were some SQL escaping issues with article titles which contain quotation marks.<br>
<br>There were also some fixes to prevent the processing scripts from breaking when their output is redirected to a file (by default console uses utf-8 encoding, file uses ascii encoding, sigh).<br><br>Review, comment, merge if you can, enjoy!<br>
--scott<br><br clear="all">Full list of patches:<br><span style="font-family:courier new,monospace">f35f8b4c10fb6cddfb48af32ef1cc65f4d40fee2 Use smaller bzip2 block size to improve decompression time.<br>efdf324952e30260596a6a2092be5d59559c8e4c Update to mwlib 0.8.5.<br>
2773b52fa988a6c456f879a470e400eb87e060d2 Update mwlib to 0.8.4.<br>490c092afff6b6c697252f399b3ce3c3ad0b1cc3 Update mwlib to 0.8.3.<br>e0002405fda01b4a29fde0436d058d75ee7ca8db Update mwlib to 0.8.2.<br>9da2774ebdadd81484e29144856f0798df08b505 Update mwlib to 0.8.1.<br>
ceae189e3dc1631d5157c0c9335803521485276b Update list of localized namespaces to that in mediawiki 1.20.2.<br>d4d4c636dcf154a6c05874410d848a353ffd5f7b Update to copy of namespace_langs from mwlib 0.11.2.<br>3beec2ad23a8a5e3aa48197f968035ce16ec9e6e Update mwlib to 0.8.0.<br>
f2bca19b4d27a53ff5fe6e7f4f231f182b64c7eb Update mwlib to 0.7.1.<br>8bb74a6c4f4f3f919171759d9c9a042d0b96aba0 Update mwlib to 0.7.0.<br>c1d765f8638add17b582b41e1f970945fc86da73 Update mwlib to 0.6.2.<br>86e5bd672b2a23c2b38321e3ee502b7d1219c1a6 Trim #anchor from links to avoid putting multiple copies of articles in db.<br>
47642fe412d0e687995956f0c33abb1663d3078e Don't break if we redirect the output of make_selection.py.<br>bbb9892d7e77a7568b70edb82e4d06a4b30e2b16 Make expandtemplates.py script work even when processing --all articles.<br>
d7ddf77f555d9a67a3b19b7bbafb6700814f9213 Make normalize_title consistent; don't squash case-sensitivity.<br>48153400296aea2c048a4c23f8fe2ac733a088cd Fix RedirectParser and make it the same in the 3 places it is duplicated.<br>
6ba73792e5560d813075f313f8c1fe3443a5e0af Fix SQL escaping issues.<span><font color="#888888"><br><br></font></span></span><span><font color="#888888">-- <br> ( <a href="http://cscott.net" target="_blank">http://cscott.net</a> )
</font></span></blockquote></div><br><br clear="all"><div><br></div></div></div><span class="HOEnZb"><font color="#888888">-- <br>Gonzalo Odiard<br><br>
</font></span></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>Gonzalo Odiard<br><br>
</div>