[sugar] Education? Visual programming interfaces
Tue Mar 20 09:23:49 EDT 2007
Simon Forman's stuff about xerblin is fascinating, and I'm excited about
where it's heading, and how we can incorporate ideas from eToys into
Python! I like the idea of having visual meta-languages that are
compiled into Python, which avoids the problems of editing Python text
or parse trees directly, and can support simplified "kindergarten"
languages as well as more advanced forms.
The drag-and-drop stack and code outliner ideas work well with
PostScript, which is a stack based but lispy code=data dynamic language
that easily supports smalltalk-like object oriented programming via
PostScript's "dictionary stack". Python + Cairo is also a great platform
for implementing that kind of stuff, with dynamic layout of hybrid text
and outline graphics, which scales and zooms and supports direct
manipulation of data structures!
Here's a paper about PSIBER (PostScript Interactive Bug Eradication
Routines), a visual interface to the PostScript interpreter in NeWS, and
some links to video demos, too. Sorry about the flashing and poor
compression -- they're recorded off a hires Sun monitor whose refresh
rate was different than the camera, and I mercilessly compressed them a
few years ago when the Internet was slower.
The Shape of PSIBER Space:
PostScript Interactive Bug Eradication Routines
PSIBER Demo: (9434433 bytes)
Demo of the NeWS PSIBER Space Deck. Research performed under the
direction of Mark Weiser and Ben Shneiderman. Developed and documented
thanks to the support of John Gilmore and Julia Menapace. Developed and
demonstrated by Don Hopkins.
One problem with PSIBER was that it was too easy to make a mistake
dragging and dropping, and accidentally totally hose the internals of
the window system, since you were editing shared structures in the NeWS
server, like classes and canvases and event handler threads! It needed
some kind of read-only safety shield or edit mode switch. Like Emacs,
its main purpose in life was to develop and debug itself (and
secondarily other NeWS applications like HyperTIES and NeMACS)!
A regular hierarchal outliner like most gui toolkits support might be
too limiting for a visual programming language. Objects might have
several ways to "open" them, and links coming in as well as going out.
Any object might be at the intersection of several trees or sequences at
once (like the class hierarchy, and the window hierarchy, the set of
instances of the same class, and an ordered list of search results).
PSIBER supported "peripheral views" that let you attach embedded visual
editors and open objects in different ways. Good XML editors support a
branch for element attributes as well as a separate branch for
sub-elements and text. Check out the way 3D Studio Max has outlines with
two kinds of branching at each level of the 3d object tree (one branch
for animatable object properties, and another branch for attached
sub-objects), and the way it crosses a vertical outline with a
horizontal timeline. It would be nice to be able to view an object in
one or more hierarchies or sequences at once (like 3dsmax's
property/sub-object outline + timeline), and easily pivot the editor
between different hierarchies and sequences and alternative views
(narrowing it to just a timeline, or just a sub-object outline, or a
free-form graph view).
I can't remember what he called his system, but Steve Strassmann did
some cool stuff on Mac Common Lisp or Dylan with "butterfly diagrams"
that branched out in different directions, left for incoming links and
right for outgoing links.
The closest thing I could google about Strassmann's butterfly diagrams
was his infamous "Is There Toscanini's Ice Cream in Heaven?" flowchart:
Marc H. Brown and Robert Sedgewick at Brown University developed a cool
visual interface to Pascal called Balsa (named after a tree, of course),
which supported multiple synchronized views of Pascal programs (lexical
structure outline, Nassi-Shneiderman flowcharts, dynamic scope views,
pascal syntax graphs, algorithm animation, etc). But it was pretty
restrictive and ungainly about how you could input and edit a program
(you could not do anything that wasn't syntactically correct, and I
don't think it supported drag-and-drop), so you couldn't just type
Pascal code into a text editor and watch the code views update in real
Here's a paper by Brad Myers that mentions Balsa and lots of other cool
stuff like Henry Lieberman's "Tinker" Lisp programming by demonstration
Brad Myers: Taxonomies of Visual Programming and Program Visualization
Marc H. Brown, Robert Sedgewick: A system for algorithm animation
Henry Lieberman: Tinker: A Programming by Demonstration System for
Beginning Programmers, in Watch What I Do: Programming by Demonstration,
Allen Cypher, ed., MIT Press, 1993.
One problem with editing programs as text while trying to maintain a
visual representation, is that typing in and editing a program as text
puts the program through many syntactically incorrect states, before
you've closed all your parens and balanced all your blocks, and you have
a horrible correspondence problem mapping between changes in the text to
changes in the structure. So it's hard to have your cake and eat it too.
Even Emacs Electric-C Mode can get pretty annoying when it tries to
close your parens and reindent your program for you while you're typing,
if you're not trained to expect it. Of course it's much easier to
attempt with languages like Lisp and Python that have simple clean
syntax, rather than languages like Perl and C++ with complex byzantine
PS: Some weird videos:
Here's an incomprehensible video I recorded late at night, of the freaky
"PseudoScientific Visualizer" stuff:
Pseudo Scientific Visualizer Demo: (21431618 bytes)
Demo of the PseudoScientific Visualizer and NeWS PSIBER Space Deck.
Research performed under the direction of Mark Weiser and Ben
Shneiderman. Developed and documented thanks to the support of John
Gilmore and Julia Menapace. Developed and demonstrated by Don Hopkins.
HyperTIES Demo: (3562242 bytes)
University of Maryland Human Computer Interaction Lab HyperTIES Demo.
Research performed under the direction of Ben Shneiderman. HyperTIES
hypermedia browser developed by Ben Shneiderman, Bill Weiland, Catherine
Plaisant and Don Hopkins. Demonstrated by Don Hopkins.
NeMACS Demo: (3511315 bytes)
Demo of UniPress NeMACS running in the NeWS Window System. Emacs
development performed under the direction of Mike Gallaher. NeWS user
interface developed and demonstrated by Don Hopkins.
HyperLook SimCity Demo: (49816346 bytes)
Demonstration of SimCity running under the HyperLook user interface
development system, based on NeWS PostScript. Includes a demonstration
of editing HyperLook graphics and user interfaces, the HyperLook
Cellular Automata Machine, and the HyperLook Happy Tool. Also shows The
NeWS Toolkit applications PizzaTool and RasterRap. HyperLook developed
by Arthur van Hoff and Don Hopkins at the Turing Institute. SimCity
ported to Unix and HyperLook by Don Hopkins. HyperLook Cellular Automata
Machine, Happy Tool, The NeWS Toolkit, PizzaTool and Raster Rap
developed by Don Hopkins. Demonstration, transcript and close captioning
by Don Hopkins. Camera and interview by Abbe Don. Taped at the San
Even more weird videos:
More information about the Sugar-devel