how to write snappy benchmark?

I'm interested in soliciting ideas about how to write this "snappy"
benchmark that's been referred to a few times. While I don't have time
to write the benchmarks, if I can get a clear picture of what will
help, I may be able to project manage and pay to have them
written. Here is how I would state the objective:

 To generate a set of numeric performance results for application and
 widget level operations which help predict end-user perception of
 responsiveness and speed. The operations used should provide a direct
 comparison across across multiple platforms and setups, including
 mutiple X11 setups, Windows, and MacOS.

I think achieving this objective is tricky, because it's hard to
figure out exactly what the user percieves. Is it the redraw time? Is
it the time from processing one event to another, independent of each
redraw? Is it something else entirely? Here are some brainstorm ideas
for operations to test, comments and additional suggestions welcome. I
think the goal is to come up with a set of useful tests, and then
simplify them into a small set of "tester applications" which can be
written.

1) application menu display time

When a mouse click event causes an application menu to display, the
time from when the mouse click event is sent (simulated) until the
last drawing operation for display of the menu is realized to the
screen.

  mouse -> display menu

2) application menu reaction time

When a mouse click event causes an application menu to display and a
immediate mouse move operation causes another menu to display, the
time from when the first mouse click is sent until the second menu is
realized to the screen.

  mouse, mouse -> display menu -> hide menu -> display menu 2

3) window expose redraw time

When a command causes an occluding window to disappear, the time from
when that command is sent until the occluded window in another
application is realized to the screen.

  hide window -> notify redraw -> redraw

4) multiple app window expose redraw time

When a command causes an occluding window to disappear, the time from
when that command is sent until a set of occluded windows, each in a
different application are all realized to the screen.

  hide window -> notify redraw * -> redraw *

5) mouse event in/out time

When an application is watching all mouse-move events to provide
hover-over behavior, the time from when a mouse move event is sent
(simulated) to when the specificed hoverover display is fully
realized.

  mouse move -> app mouse event -> draw

6) mouse event in/out reaction time

When an application is watching all mouse-move events to provide
hover-over behavior, when a mouse moves into and then immediately out
of an area of hover-over behavior, the time from when the two mouse
events are sent (simulated) until the hover state and then the last
display state are fully realized.

  mouse move, mouse move -> app mouse event -> draw 
      -> app mouse event -> draw

7) focus to event delivery time

When a button is clicked on in a non-focused window (click-to-focus),
the length of time between the sending an action event which requires
a focus change, until the newly focused application receives the
action event.

  mouse click -> application event reciept

8) live-window resize speed

When live-resizing a window whose contents are dependent on the window
size, but which can be computed quickly, time between a single mouse
move and the final window contents are realized to the screen.

9) live-window resize responsiveness

When live-resizing a window whose contents are dependent on the window
size, and which take reasonable cpu time to compute, the time between
a sequence of mouse move events and the final window contents being
realized to the screen; and the number of redraw events completed.

10) live-window drawing/move cross-effects

When dragging a window whose contents are being constantly changed,
the number of redraw events processed during a given move.

11) application window drag stall

When dragging a window whose application is too busy to frequently
process events, the number of unique window positions, measured as the
number of unique expose events delivered to an occluded window.

12) drag responsiveness

When dragging a window (application window or object), the length of
time between a mouse event which causes a move of the window until the
next mouse event is processed.

13) drag responsiveness across exposed regions

When dragging a window (application window or object) across other
application windows, the length of time between a mouse event which
causes a move of the window until the next mouse event is processed.

14) draw start responsiveness

When clicking to start a new drag operation (of an application icon),
the length of time between the mouse click and mouse move, and when
(a) the mouse move event is processed, (b) the drag object is realized
to the screen.

----

That feels like a good start. Let the onslaut begin. :)

- 
David Jeske (N9LCA) + http://www.chat.net/~jeske/ + [email protected]