Re: a better libtds

On Mon, 22 Aug 2011 15:43:14 -0400
Brian Bruns <[email protected]> wrote:

> Iteration is relatively easy with the addition of a couple more fields
> to the structure (function type which lets you know what kind of thing
> we are dealing with scalar or array, the element size and type, and so
> forth), 

We're thinking along the same lines.  But I'm not sure what tool to
use, if any.  As I looked more into RDP I realized not only it but most
lex/yacc tools are aimed at text files and are capable of handling much
more complex languages than is TDS.  Too little and too much!  

Can you show me, say, a Flex rule that parses a varchar (int16,
value N, followed by N characters)?  

> Wish I had more time to do some of this, it's an interesting problem.

Yes.  In my mind I see a database of TDS metadata, from which to
generate what we want:

1.  TDS primitives: integers of various sizes, counted strings of 1- or
2-byte characters, dates, decimals, etc.  For each primitive a reader.
Readers boil down to kinds: read_size (1,2,4,8 bytes), and read_value (N
bytes, where N might be zero).  libtds already has all the readers we
need, if not in the exact form we'll want.  

2.  For each TDS Token (marker), a list of primitives.  

3.  For each TDS Token, a list of valid successor tokens.  

All three need to be qualified by TDS version.  Both lists need
optionality specified.  (Not every Language token is followed by a
ROWFMT token, and not every ROWFMT by a ROW, etc.)

>From the above, we can generate:

1.  a C struct for every packet type containing 
	a.  every possible primitive
	b.  for each TDS version, an array of pairs of 
	    reader pointers and target member addresses

    I'm happier if we can avoid void* by using a parser-generator. 

2.  better TDS documentation. 

3.  input for a state diagram from graphviz. 

4.  packet-reading functions, one for every packet type, that iterates
over its array-of-primitives.  

5.  For each TDS version, a list of valid successor tokens for each
token and, for each one, whether or not it's terminal (causes a
return).  

And that, basically, *is* libtds.  As Frediano said, 

	"Once you remove code that package and decode
	 streams/special packets what's left into libTDS?"

> this big work would go much easier if we were all
> on git.  Anybody ready to make that a reality?

I'm ready.  It might be a little while before automatic patching and
nightly snapshots are available.  If there's a real emergency, we can
still use CVS for patches to 0.91, otherwise even a month's delay of
snapshots right after a release won't cause anyone great pain,
especially because we're disturbing the code quite a bit.  

Have we agreed on a git hub?  

Regards, 

--jkl