[Sugar-devel] Browse and the move to WebKit

[Luke Kenneth Casson Leighton]

Daniel Drake <dsd <at> laptop.org> writes:

> 
> There have been various discussions in the past suggesting a move from
> mozilla to webkit for the Browse activity and related components, but
> I've never really been convinced: there is always a cost to switching,
> and convincing-looking numbers from webkit supporters tended to be
> countered with convincing-looking numbers from mozilla supporters.

daniel, hi,

it's just been brought to my attention just how f****ed up the mozilla
foundation's codebase has become.  i'm the lead developer of pyjamas,
and have been tracking the situation as it goes from bad to worse to
just completely f*****g useless for quite some time.

as of xulrunner-1.9.1, there was one outstanding bug which, if it had
been fixed as described in the bugreport just below, would have made
the use of hulahop absolutely perfect for pyjamas-desktop... ok, not
absolutely perfect, but passably so.
https://bugzilla.mozilla.org/show_bug.cgi?id=502234

as you can see, a decision was made to go with a "solution" which
made the code _worse_ to work with.  a band-aid, made out of ignorance
rather than intelligent evaluation and decision-making, that actually
broke python-xpcom.

there appears to be within the mozilla foundation an insanity that says
"we are gods, javascript is our lord and saviour, any other programming
language or the use of any other programming language can go take a
running jump".

i'm further deeply disappointed and angered to hear that you have found
evidence that embedding of the gecko / xulrunner engine is _also_ prevented,
prohibited and actively discouraged.

it begs the question, "what the bloody hell are the mozilla foundation playing
at??"

anyway - leaving that aside, i thought you should know that yes there is
an alternative option in the form of webkit, but that it is equally fraught.

all in all i think that the free software community is completely and utterly
failing to get with the picture with respect to browser engine technology,
and that it is pretty ironic that MSHTML (microsoft's web browser engine
technology) just works out-of-the-box and yet the U.S. Dept of Justice and
the E.U. Monopolies and Mergers Commission tried to force microsoft *not* to
have MSHTML pre-installed.

anyway - end that, allow me to outline what's possible with what's already
out there.


> === WebKit ===
> 
> What I'd like to do now is spec out a project for someone to take on,
> moving Browse to WebKit in a way that can be clearly justified for the
> community.
> 
> But, WebKit is totally new to me so I have many open questions. Can
> anyone help me answer them? I'll be collecting the results into a wiki
> page (to form the above spec).
> 
> 1. I've only made half the argument above. Mozilla is bad, but why is
> WebKit the solution? The key questions here are: is it embeddable?

 yes.

> Does it work well when embedded? Do the developers support it being
> embedded?

 no they do not.  mark rowe applied evasive dictatorship - bullying
 and "technical goal-post moving" that was shown later to be thoroughly
 hypocritical, allowing identical code which he earlier rejected with
 unreasonable arguments failing completely to even read counter-arguments
 or explanations as to why particular decisions had been taken.
 
 if you have the time, you can see evidence of his
 shockingly-bad behaviour here: http://bugs.webkit.org/show_bug.cgi?id=16401

 the outcome of what he did has had severe knock-on ramifications as the
 webkitgtk code which is now in the webkit main tree has severe technical
 flaws that i am "banned" i.e. "actively prevented and prohibited" from
 discussing, outlining and helping with, on both bugs.webkit.org and also
 the webkit-dev mailing list.

 i have been posting various descriptions to help give hints to help fix
 some of these issues under accounts named "webkit censorship bypass" but
 the level of technical expertise is sufficiently high in order to fix these
 issues that they are "dismissed" as "ramblings" and are completely ignored.

 the consequences are that there are fundamental flaws in both webkitgtk
 and pywebkitgtk which cannot be "fixed" without some serious redesign
 work that will take the inexperienced webkitgtk developers several weeks
 to understand, and several weeks to implement.


> Does anyone have experience here? 

 yes.

> The name "WebKit" makes it sound
> nice and modular, and the fact that WebKit itself isn't a browser
> would seem to support these arguments, but it would be nice being able
> to argue this on a more solid basis.

 ok.

 it's good... but... the code has been developed "piece-meal".  the mainline
 webkit tree has gobject bindings support, through which you can use gobject
 "introspection" to have python bindings on top of that...

 ... but the gobject bindings themselves (which are to the DOM) are... well,
 they're shit basically.  and i can say that, because i wrote them.  they
 were the "first version", and you know what "first version" code is like.
 it's where you learn how to do the 2nd version.

 unfortunately, the people who tried to help out very stupidly listened to
 what mark rowe, who was and still is completely ignorant of the subtleties
 of doing language bindings to DOM, and started *removing* critical code.
 by the time i pointed out the problems that this would cause it was too
 late, and they then added in a *wrongly-implemented* replacement that will
 cause race-conditions (segfaults), significant CPU usage and other unexpected
 behaviour.

 *on the other hand*, if you have absolutely no need to access the DOM, then
 the webkitgtk version *is* ok to use, as long as you get the version that Xan
 is maintaining.  xan releases versions of webkitgtk where he adds in the
 patches that he is maintaining, which the webkit team have not yet accepted.

 it's a big job, and you would do well to follow *only* the stable releases
 that Xan prepares.

 on top of that, pywebkitgtk is (or was, when i was looking at it 2+ years
 ago) "functional".


> 3. What is the safe of pywebkitgtk in F14, F15, F16?
> This is the backend library used by Surf, right? Is this still the
> right answer for creating a webkit-based app using Python + GTK?
> Does it work well on Fedora 14, or do we need a newer distro?
> 
> I remember at one point there being some pretty key problems with
> pywebkitgtk causing Surf development to halt. What were these issues
> and have they been overcome?
> 
> IIRC those pywebkitgtk-related problems were going to be solved with a
> move to GObject introspection, which wasn't mature back in that
> timeframe.

 this is IMPORTANT.

 UNDER NO CIRCUMSTANCES should you use the gobject introspection-based code.



> But it is mature and usable now.

 no - it is not.

 there are subtle issues which i have repeatedly stated which are repeatedly
 ignored by all developers associated with the code.

 one is the event handling.  the original code that i wrote for event handling
 was destroyed by mark rowe.  plus, i had written it in a workable but "naive"
 way which prioritises the GTK event loop (through the use of gobject signals)
 rather than prioritising and making use of the webkit internal event thread
 execution unit.  this flaw was something that i did not appreciate at the
 time.

 the *replacement* code, which has the same fundamental flaw, added
 *another* problem: it activated absolutely every single sodding
  event under the sun that any bit of DOM would ever have, on each and
 every _bit_ of DOM added and controlled through gobject.

 some of the DOM objects have 40+ possible events!  the memory usage not to
 mention CPU usage will just go through the roof!

 then, when you _get_ an event, you will find that its activation fires
 into the main GTK event loop, and then allows GTK's main thread to execute.

 the problem with this is that, like in gecko, webkit has its *own* execution
 thread(s), where protection of various data structures with semaphores is
 carefully carried out.

 by firing an event into the *GTK* thread, you've just completely bypassed all
 that.

 it gets even worse when you try to use g_object timers, and as for trying to
 use GTK popup windows and dialog boxes, fired up from events, you can
 flat-out forget it: memory corruption results and the application _will_
 lock up.


 solving all of these issues (aka "version 2" in the "if you don't plan
 to write 3 versions you will end up writing 3 anyway" plan)
 was why i wrote pythonwebkit.

 http://www.gnu.org/software/pythonwebkit

 in this version, i made sure that it was *webkit* that called into
 c-based callbacks (that then called into python functions), and i also
 made sure that even Timers called into c-based callbacks that then
 called python functions.  this solved one of the critical problems with
 the original code that i wrote, which was the race conditions caused by
 letting gobject timers and the main GTK loop access the webkit
 datastructures from *outside* of the main webkit infrastructure.

 to highlight why this is important: look at javascript.  it's single-threaded
 isn't it?  when you execute a timer event piece of code, everything else
 *stops* being executed until that timer-fired piece of code has completed.
 in this way you can guarantee that there will never be *any* race-conditions,
 and that there will be no "cross-interference".  no semaphores are needed
 (within the javascript code), no mutexes - it's a much simpler environment.

 now bypass that.

 worst case, you end up with the possibility where timers or events get fired
 multiple times by one piece of code, causing code-execution that may or may
 not complete (inside the GTK world) before the next timer or event is fired...

 ... do you understand now why the existing gobject bindings are so critically
 and fundamentally flawed?


 anyway.

 pythonwebkit as used in pyjamas-desktop doesn't *need* to be a GTK window,
 and for that reason, although i originally based the code on pywebkitgtk
 i made a decision to *remove* the blending of pygtk-window + webkit and
 to do things entirely in c, firing up *one* main GTK window (c-based)
 into which a (c-based) WebKitWebView was placed - pretty much exactly
 like in midori or a better example is "like in the webkit GtkLauncher
 demo program", because that's all, really, that's needed.

 you can see the results here, in WebView_init:

 http://git.savannah.gnu.org/cgit/pythonwebkit.git/tree/pywebkitgtk/gtk/webkitgtkmodule.c?h=python_codegen

 you can see that the sole purpose of this code is to fire up a single
 window, load a URI and then present to the developer the option to
 begin accessing the DOM (GetDOMDocument).

 however, it would be relatively straightforward to add in the code
 which already exists in pywebkitgtk, to provide python bindings for
 things like "history notification", "back button" etc. etc. and also
 to make this a proper pygtk window again.

 anyway.

 yes i am happy to help, and i will happily grant access to the savannah
 pythonwebkit repository to anyone who asks for it, but i will *not* do
 code development "for free".  you can have the benefit of my time and
 experience to fix issues, in the form of mentorship, but if you want me
 to actually write some code you will need to find some sponsorship and
 pay me some money.  sorry: i set this rule after doing significant work
 for the free software community and the benefit of free software users
 who then did *not* reward my efforts.  thus, the rule is now: a contract
 must now be agreed where i receive financial renumeration for work done,
 at the time that it is done.

 l.