Re: a better libtds

This is Microsoft's TDS grammar:
http://msdn.microsoft.com/en-us/library/dd304582%28v=PROT.13%29.aspx

A grammar can be turned into a parser and tree walker with a tool like the Gold Parser:
http://www.devincook.com/goldparser/

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of [email protected]
Sent: Thursday, August 18, 2011 3:02 PM
To: FreeTDS Development Group
Subject: [freetds] a better libtds

I would like to re-engineer libtds from the ground up as a finite state machine.  

Fundamentally, libtds is a TDS parser.  Packets arrive in pre-determined sequences.  Packets, even variable-length fields within packets, are well defined.  I don't know if TDS is a *regular* language, but I suspect so.  It's definitely a parseable one.  


Why do this?  libtds is quite ad hoc; it reflects its origins as a discovery platform for understanding the TDS protocol.  Consequently, it's hard to understand and hasn't attracted a new developer in five years.  Clear principles and architecture would benefit the library and the project.  

Technically, I would put forward these goals:  

0.  Reduce TDS description to tabular form, to be used as input the parser-generator.  Can also be used to generate structs for each packet type (see #5).  Can also be used to improve TDS documentation.  

1.  More predictable and rational flow of control, essential for thread safety.  

2.  Clear factoring of protocol versions. 

3.  Better RAII, clearer memory ownership.  

4.  Removal of iconv from TDS layer.  libtds should be transparent. 
Charset issues are binding issues; client libraries need to support access to the raw data.  Cf. dbdata() and nvarchar columns.  

5.  Packet-based access to the network.  Data exchange between the client libraries and libtds should be by packet.  

    I do not mean that each client library should chase every packet 1:1 on the wire.  In particular, libtds has to scoop up the DONE_IN_PROC and compute packets after e.g. the final dbnextrow().  

6.  The same state machine can inform a server implementation.  

There are non-technical goals, too:

7.  Attract more developers by adopting a classic, computer-science approach.  I venture to say most programmers never get to work on a finite-state machine.  

8.  Make FreeTDS portable to other RDBMSs.  Cf.
http://freedb.schemamania.org.  

9.  Make client library maintenance easier.  Consider this from
dbnextrow():

const int mask = TDS_STOPAT_ROWFMT|TDS_RETURN_DONE|TDS_RETURN_ROW|TDS_RETURN_COMPUTE;
...
switch (tds_process_tokens(tds, &res_type, NULL, mask)) {

What would your mother say if she saw you writing C like that?  

Asynchronous calls comport with a by-packet interface design.  To implement dbpoll()/dbdataready(), db-lib would call something like:

	int tds_next_packet_type(TDS*);
	
which would return the TDS marker if a packet is available, or -1 if not.  The blocking calls would be:

	int tds_get_XXX(TDS*, struct XXX *); 

I once thought C++ was the answer to libtds's problems, and I could still be convinced to use C++.  But I think the real answer is a formal approach.  

I've been reluctant to undertake all this myself, though.  I see no point in writing a new libtds unless the client libraries are updated to use it, and I don't want to take on more than db-lib.  If I *did* write libtds2 *and* change db-lib to use it, and that's all that happened, I'd have split the project, probably making things worse, not better.  For the sake of the project, I really need agreement before I can start.  

--jkl
_______________________________________________
FreeTDS mailing list
[email protected]
http://lists.ibiblio.org/mailman/listinfo/freetds