Re: VPython for Graph Visualization

On Tue, Jul 10, 2012 at 4:29 PM, Mark Adam <[email protected]> wrote:
>
> I'm wondering if there's been anyone who's explored using the 3-d
> visualization extension of python for graph visualization?

ha, well, i'm actually giving a short presentation/demo of exactly
this at work tomorrow for ~40 or so other developers, and, in
retrospect ... all of you ;-)

i can't really share the code ATM (it belongs to me but currently
integrates with the production AMQP network, etc) but i do intend on
releasing at a later time.

tbh, it really is just a demo at this point, but still rather
impressive (many oohs and ahhs) considering a SLOC barely over 200 --
strongly hinting at vpython's power-thru-simplicity model.

the talk itself focuses on the natural intuitions behind generic 3D
interfaces, and self-organizing, real[ish]-time visualizations (sadly,
for both, sans some sweet HID equip + libs needed to move from dream
-> reality -- eg. Xbox Kinect sensors4all -- kbds and mice are
OBSOLETE! yuck! :-)

SOURCE: application binds to message queue, subscribing to real-time
phone-state queues (emitted by ~400 CSR agents in another facility,
across the country [US]). these are nothing more than (time, agent,
state); other properties are available but unused.

INTERFACE: outside "observers" (ie. users) may navigate this universe:
 - free-space navigation
 - click-to-mount camera (to an arbitrary sphere)
... or influence fundamental constants/properties as deemed useful:
 - gravity
 - intial vector
 - intial vector magnitude
 - various constants like growth/etc
... modifying the output + user perception/experience stream, and
[potentially] surface new insights. introspection/aggregation
[stats/avg/etc] on individual agents and/or logical groupings
contribute usefulness as a human interface (think current-get FPS
games, and the massive information I/O densities in effect).

SELF-ORGANIZING: to complement humans already superb pattern matching
abilities, the visualizer renders agents as simple, colored spheres
orbiting a gravity-well anchored to origin. state change == color
change. agents grow at a continuous pace while idling in one state;
any transition resets the radius back to 1. from just these basic
"truths", patterns emerge like:
 - "seems like ~40% in idle at any given time"
 - "why are so many busy/unavailable [red]?"
 - "why agent XYZ in red [busy], and 3x larger?"
 - "technical problem? everyone just went IDLE"
... what's truly incredible is the fact that near *everyone*
interprets the output similarly, by virtue of a shared experience,
that of life (another variant employs magnetic-like interactions
between agents)

[1/2] tl;dr, physics fits really well in this domain! it's something
we've practice, without choice, since day 1, and are pretty well
disciplined by now :-)

quick aside: at a previous company, we ran gltail [
http://www.fudgie.org/ ] all day, every day, and i can't even begin to
tell you how many times network or completely unrelated *problems*
were spotted by my peers throughout the office -- read: very
NON-TECHIE folk! -- on numerous occasion. often, IMO, the anomalies
were often subtle -- far from approaching boundary alarms a la
Nagios/Cactus/whatever, but still with [complex] negative implications
if left unbound -- these people simply noticed "the blobber feels
off/out-of-sync today".  it was phenomenal ... inspiring continued
interest, and soon after, leading me to [v]python :-)

moving along ... the impl is not too crazy-fancy, but worthy of
mention.  i run the vpython bits under uWSGI
[http://projects.unbit.it/uwsgi/ : absolutely fabulous, check it out]
within an offloaded/independent "--mule" process, avoiding detrimental
conflict with HTTP/WSGI request/response cycles. this is critical, and
allows painless interaction with the vpython/visualizer loop, from the
outside, via any one of the umpteen uWSGI methods. a single FX class
manages transitions thru staticmethod(generator) members.  each agent
sphere carries an "FXQ" [effects queue], filled with LIVE,
instantiated generators.  during each cycle/iteration/Hz, i simply
call next(generator) on each generator, in each queue, thereby
advancing the entire system one unit-over-Hz. FX behave asynchronously
(thx to generator-awesome-ness) and perform everything from applying
gravity -> velocity -> new_pos all the way to managing the camera
updates, timeouts, etc. etc. when a generator expires (raises
StopIteration), it's removed from it's respective FXQ.

the FXQ model its immensely simple, but i suspect not all that
efficient or scalable; better would be some
generic-as-possible-low-level-transform-engine parallelizing some
relationships, eg. via matrices or other transforms ... sadly, despite
insatiable fervor for mathematics and physics, i get paid to architect
web/system/business applications, my computational physics skills
surely suck ... pointer very much welcomed!

... hopefully you found what became a short story either interestingly
entertaining or entertainingly interesting, because The End, is ...
NOW!

* tune in next week for the shocking conclusion! *

-- 

C Anthony

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/