Dear all,
The workshop on "(Graphical) User Interfaces" is approaching
fast and we therefore would like to initiate a discussion on
topics related to the workshop. The idea is to cover as many
aspects as possible in advance to already have a broad overview
on possible aspects when the workshop takes place.
So let us start with some hints which appeared to be relevant
or interesting at our research institute. Our focus may be
neither representative nor may it be shared by others but it
is at least motivated by projects and we would like to ask
everybody to describe his needs / views / expectations in the
same way. This perhaps allows us to extract the really needed
aspects at the workshop.
Kind regards
Christian Schlegel, Boris Kluge
=============================================================
The following is tailored to 3D graphics, our 2D visualization
approach is included in the SmartSoft distribution and can be
looked at there (several painters, coordinate systems etc.)
o Visualization
- ordinary visualization server for distributed components
where everything from an arbitrary number of clients is
simply displayed
- User can zoom in / pan / scroll and select its viewpoint
without client interaction
- symbolic ("an instance of class GREEN-BOTTLE is at
position p") vs. subsymbolic information (laser scan,
force/torque, 3d vision depth map etc.).
Use Cases:
- widget with 3d model of manipulator which is controlled
and updated by one component, only visualization and no
feedback
- mobile platform in a world model showing in detail its
pose, manipulator state, pan tilt unit configuration etc.
=> symbolic model composed of high level primitives
=> updates of the graphic model by updating the kinematic
chain of the graphics model
- several clients paint into the same viewport (one client
is responsible for the manipulator, another for providing
the force/torque sensor values attached to a manipulator
joint) and the user can inspect the result by flying
through the combined visualization
=> several clients contribute to the same model, but the
client coordination should be completely separated
from the visualization server
- Visualization of 3d depth sensor data from vision or
from laser scanner
=> visualization server must be able to deal with
high volume sensor data
o Interaction
- feedback from the visualization server to client components
- integration of classical GUI controls like sliders and
buttons etc. where a client specifies the value range
and the user adjusts the parameters.
- interaction with objects:
the server detects with which objects the user interacts
(e.g. by moving / clicking / dragging the mouse on their
representation on the screen) and notifies the appropriate
client.
=> Compare it with the X window system where the user
interacts with the graphical representations of
distributed clients. The X server displays the
client's data and dispatches user events (mouse,
keyboard)
Use Cases:
- select a joint of a manipulator and use a slider to
set its new value.
=> transmit the final setting to the appropriate
client which then updates the graphic. The value
range is already checked at the server.
=> or transmit every intermediate value for interactive
pose control to the client
- select a link of a manipulator and move it
=> send every intermediate pose to the client which
is responsible for collision detection and the
inverse kinematic
- interaction with remote components via the 3d gui
(teleoperation)
=> low latencies and delays
=> visualization of states and sensor values
=> perhaps graphical highlighting of critical
distances etc.
- point to pose where mobile platform shall move to and
look at the visualization how progress is made towards
the goal
- select the enclosure of a mobile platform and rotate it
in the scene viewer and look how the robot executes that
simultaneously (telemanipulaton). Select the manipulator
or the pan tilt unit or whatever and command them in the
3D graphical representation
o Editors
- graphical editors for finite state automatons / execution
engine or to compose components in a LEGO way to behaviors
etc ...
o What have we done / tried / ... ?
- Combination of Open Inventor with Collision Checker
- forward the triangles generated by Open Inventor
to Swift / PQP collision checkers
- worked as hack, but nothing stable ...
- Visualization of LOOM knowledge base
- generic graphic models for all relevant objects of the
knowledge base provided as files for a visualization
server
- build up an open inventor scene description by
parameterizing the generic models with the current
knowledge out of the loom knowledge base
- successfully used to inspect the spatial knowledge
- Interaction with LOOM knowledge base
- extension of the above visualization
- allow handles on several displayed objects
- objects can be picked up and placed elsewhere
- knowledge base is used to check whether all constraints
are met at the final place (only one object per shelf
etc.) and new place is rejected if not compatible
- if object is placed in midair, object is moved downwards
till it hits a supporting object.
o Differences Robotics / Visualization / Game Engines
- Game Engines are highly efficient but normally provide
only very basic collision checkers based on enclosing
cylinders e.g.
- Visualization normally ignores interaction with
external components like collision checkers or
inverse kinematic models.
- Robotics seems to need a combination of an
interactive visualization with interaction
with external components (inverse kinematic
model, collision checker)
o Interesting topics
- Interaction with physical simulation, physical realism
- Interaction with collision checkers
- Interaction with kinematic models
- Level of abstraction for modelling
- Software reuse:
GUI components which are general (highly reusable)
but low-level
vs.
GUI components which are high-level (i.e. easy to
use) but very specific
- symbolic ("an instance of class GREEN-BOTTLE is at
position p") vs. subsymbolic information (laser scan,
force/torque, 3d vision depth map etc.).
- Patterns:
Model-View-Controler-Pattern (MVC):
"Visualization" is the View part, "Interaction" is
covered by the Controler aspect, and "Simulation" etc.
is represented by the Model part.
"Recursive" MVC Pattern:
Implement the "View" as an MVC pattern itself, i.e.
during interaction (control) the user receives visual
feedback (view) and his actions are validated (e.g.
collision detection), before any change to the (external)
model is committed.
o Summary
- for many robotic applications one needs a high level
object centered representation of to be displayed
scenes like provided in Open Inventor / Coin
- one needs easy to use manipulators which allow
interaction with a 3D scene
- one should also be able to display high volume
sensor data like laser range scans etc.
- if one uses high level object centered representations
one has to address the question of how those primitives
are transformed into models suitable for collision
detection
- one should strictly separate the visualization from
the service providing components, which means if
several components have to know about the collision
of their displayed objects that has to be communicated
between the clients and a collision checker and not
by the visualization component which is only responsible
for visualization. The visualization component is not
the common world model ...
- therefore no activity nodes as available in "open
inventor" where one can move objects along a trajectory
etc.
--
---------------------------------------------------------------------
Christian Schlegel
FAW Ulm (Research Institute for Applied Knowledge Processing)
PO Box 2060
D-89010 Ulm
Germany
Tel. : ++49 (731) 501 8949
Fax : ++49 (731) 501 999
email: [email protected]
WWW : http://www.faw.uni-ulm.de/
---------------------------------------------------------------------
lmpx.com only provides a reader for public news (NNTP) servers. It is not
affiliated with the servers or forums shown here and is not responsible for
the content of articles, which is written by their respective authors.