RE: clipping planes bypassed in CG vertex shaders

I haven't had time to respond to this thread yet.  A few comments:

The fundamental problem is that user clipping semantics defined in the
spec are significantly tied to the fixed-function transformation
pipeline.  For user programs, the vertex transformation doesn't have any
necessary correspondence to the fixed-function pipe, so specifying what
happens there is more problematic.  

Many "real world" user programs actually do have this correspondence,
but inferring whether an arbitrary program does or does not have this
property in the driver is not a completely trivial task, especially in
the face of optimizations (e.g., you know you only have 3-component
input vertices, so might write your transformation accordingly).
Clearly defining whether a shader has this property in a spec is even
uglier, since there are so many different ways you can write the "same"
transformation.

So it's not a trivial problem.  However, there are a number of different
ways to get the desired clipping behavior.

* As the original question seems to refer to Cg, I'll start by saying
that Cg provides "CLP0" -- "CLP5" vertex result semantics for profiles
where the clip distance outputs (described below) are available.

* NV_vertex_program2 provides outputs o[CLP0] through o[CLP5] that
allows a vertex program to write "clip distances".  You evaluate the
clipping dot product in the vertex program in whatever space you'd like.
The hardware clips to the region where o[CLPx] >= 0.  Clip distances map
with clip planes.

* NV_vertex_program2_option (and NV_vertex_program3 for the GeForce 6
series) provide the same functionality with ARB-style programs.  I think
they're called "result.clip[0..5]".

* NV_vertex_program2 is supported on NV30 (GeForce FX) and better.

* If you need to make it work on NVIDIA parts prior to GeForce FX, the
texture shader approach would probably be needed.  That same set of
parts doesn't support fragment programs, so your best bet might be to
have one "basic" vertex program that writes either to a clip distance or
texture coordinate component.  You'll probably have to write different
fragment shaders regardless.

* Unextended ARB_vertex_program doesn't provide any explicit clip
distance support.  It does, however, support user clipping in
conjunction with position invariant programs.  In other words, if a
fixed-function position transform is used, fixed-function user clipping
works normally.  (NV_vertex_program1_1, which was the first extension
with position-invariant programs, doesn't do this -- this was added when
ARB_vertex_program was standardized.)

* GLSL has a "gl_ClipVertex" output where you basically pretend your
output vector gives the eye coordinates for the vertex and use them with
"standard" user clipping.  This involves computing the dot product with
the relevant clip planes, and then clipping to the region where the dot
product is >= 0.  If you use this functionality, keep in mind that the
clip planes are transformed by the current modelview matrix at the time
you specify them, so the values used for evaluating "clip vertex" dot
products may not be the four values you pass in to glClipPlane.

Hope this helps.

Pat


-----Original Message-----
Stephen J Baker
Sent: Monday, May 23, 2005 5:18 PM
To: Multiple recipients of list OPENGL-GAMEDEV-L

Well, I guess it pays to RTFM.

According to the nVidia vertex_program extension:

    "What part of OpenGL do vertex programs specifically bypass?

       Vertex programs bypass the following OpenGL functionality:

         o  Normal transformation and normalization
         o  Color material
         o  Per-vertex lighting
         o  Texture coordinate generation
         o  The texture matrix
         o  The normalization of AUTO_NORMAL evaluated normals
         o  The modelview and projection matrix transforms
         o  The per-vertex processing in EXT_point_parameters
         o  The per-vertex processing in NV_fog_distance
         o  Raster position transformation
===>    o  Client-defined clip planes"

...and it's right there on the last line.

A further question clarifies the reasoning:

    "Should client-defined clip planes operate when vertex program mode
is
     enabled?

       RESOLUTION.  No.

       OpenGL's client-defined clip planes are specified in eye-space.
       Vertex programs generate homogeneous clip space positions.
       Unlike the conventional OpenGL vertex transformation mode, vertex
       program mode requires no semantic equivalent to eye-space.

       Applications that require client-defined clip planes can simulate
       OpenGL-style client-defined clip planes by generating texture
       coordinates and using alpha testing or other per-fragment tests
       such as NV_texture_shader's CULL_FRAGMENT_NV program to discard
       fragments.  In many ways, these schemes provide a more flexible
       mechanism for clipping than client-defined clip planes.

       Unfortunately, vertex programs used in conjunction with selection
       or feedback will not have a means to support client-defined clip
       planes because the per-fragment culling mechanisms described in
the
       previous paragraph are not available in the selection or feedback
       render modes.  Oh well.

       Finally, as a practical concern, client-defined clip planes
       greatly complicate clipping for various hardware rasterization
       architectures."

...so the recommendation is to use the texture hack that older nVidia
hardware uses (urgh!).

I'm horrified.

-----------------------------------------------------------------------
The second law of Frisbee throwing states: "Never precede any maneuver
by a comment more predictive than "Watch this!"...it turns out that
this also applies to writing Fragment Shaders.
-----------------------------------------------------------------------
Steve Baker                      (817)619-2657 (Vox/Vox-Mail)
L3Com/Link Simulation & Training (817)619-2466 (Fax)
Work: [email protected]           http://www.link.com
Home: [email protected]       http://www.sjbaker.org

----- 
FAQ and OpenGL Resources at:
  http://www.geocities.com/SiliconValley/Hills/9956/OpenGL

-- 
Author: Stephen J Baker
  INET: [email protected]

Fat City Hosting, San Diego, California -- http://www.fatcity.com
---------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: [email protected] (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB OPENGL-GAMEDEV-L
(or the name of mailing list you want to be removed from).  You may
also send the HELP command for other information (like subscribing).
----- 
FAQ and OpenGL Resources at:
  http://www.geocities.com/SiliconValley/Hills/9956/OpenGL

-- 
Author: Pat Brown \(OpenGL\)
  INET: [email protected]

Fat City Hosting, San Diego, California -- http://www.fatcity.com
---------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: [email protected] (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB OPENGL-GAMEDEV-L
(or the name of mailing list you want to be removed from).  You may
also send the HELP command for other information (like subscribing).