Re: JAWIN Digest - 17 Aug 2004 to 18 Aug 2004 (#2004-77)

Morten,
        attached is a patch to the jawinuserguide_dll.html, containing an
overview (my understanding of it) of the instruction strings and the stack
size. When you get a chance, please review it. I am not 100% confident
that there are no inaccuracies in the explanation, and I tried not to make
it too detailed or involved. Next, I will probably try to go over all
instruction strings and document some examples of their usage. Along these
lines, I was wondering : why does the Transform.cpp use literals in the
process() method, instead of using the pre-defined values from
instructions.h ?

        I am also working on a Parameters class that would automate &
simplify most of the basic instruction string building, some of the
marshalling of ints, strings, buffers, etc, and stack size calculations. I
will provide that together with the generated structs from VJ++. For
handling structs, my idea is to basically  extend the Struct classes
created by VJ++ (which contain all data elements), and have the child
class implement a predefined interface (e.g. JawinCStruct), which will
define the operations that will be necessary for the Parameters class
(e.g. marshal, marshallOut, size, token, etc).

        Regarding my initial impression that JTB was able to generate
stubs for regular dll-s (without type info compiled in them) - I double
checked the updated documentation and it explicitly states that it is not
supported. So, I was either looking at an older version of the docs, or I
misunderstood - the current documentation is very clear about it.




> Date:    Wed, 18 Aug 2004 14:04:49 +0200
> From:    Morten Andersen <[email protected]>
> Subject: Re: Passing byte arrays into C functions
>
> Hi Alex
>
> A couple of quick notes, I will probably return with more later, when I
> find better time.
>
> Alex Kotchnev wrote:
>
> >
> >>Some of the things your are talking about are of interest.  If you
have
> >>a sourceforge account I would be happy to turn it on for Jawin and
work
> >>with you on your projects.
> >
> >
> >    My sourceforge username is polrtex.
>
> First, of course, welcome on board :-). Please let me know if you get
> any problems setting up your build environment for Jawin (it should be
> pretty straightforward as described in the docs/jawindeveloper.html
> document, but there could be some special issues that we have
overlooked).
>
> >
> > I was initially under the impression that the Jawin type browser was
able
> > to generate code from dll-s (I think that the documentation had
something
> > mentioning it).
>
> If it is a DLL containing any of the COM structures mentioned on the
> jawintypebrowser.html page, we are able to generate code for some/most
> structures.
>
> But if you are thinking of an "old" DLL with published "DLL Entry
> Points" (such as the Win32 API), we are not able to do it. I think this
> is what the documentation in the jawintypebrowser.html page says, but if
> you find the exact location where we have written it incorrectly, please
> feel free to correct it.
>
> (you would probably also want to consult the new jawinuserguide_dll.html
> file, with information about calling DLL entry points).
>
> >
> > I will collect examples as I work through my Win32 programming book. I
> > will have to write-up my summary understanding of the marshalling in
> > layman's terms the way I understand it(for me, it was quite a steep
curve
> > to figure out how the marshalling works and what it is all about - I
am
> > not all that good with C and I am not all that familiar with what goes
> > into the stack, how big it has to be, etc). I have constructed in my
head
> > an explanation of the whole marshalling business, I will do a write up
and
> > send for review to see if I understand things correctly.
>
> Sounds great, I assume you are still talking about the marshalling when
> calling DLL entry points? - Your writing could probably finish section
> 3.2 in the jawinuserguide_dll.html document, where i newer got finished
> on explaining the instruction string and stacksize (please feel free to
> ask on everything, no matter how trivial and simple the question feels,
> one could still waste _hours_ tumbling around in the marshalling code -
> and there is a big chance that either Roger, Robert or myself can answer
> the question quickly).
>
> >
> > I am not very certain what you are referring to here. The examples &
> > documentation will be an ongoing thing (no timeline). Currently, I
have
> > all the structs and constants for win32 generated (from VJ++) and they
> > compile. The only thing that I might have to do is remove some of the
MS
> > generated @dll.import javadoc tags, and I can send them in. The
generated
> > functions signatures don't compile yet.
>
> Sounds great. You should also check the content of the
> org.jawin.donated.win32 package (in the stubs/src folder) and the
> org.jawin.win32 package in the main src folder.
>
> The org.jawin.win32 package contains some Win32 API structures used
> internally in Jawin. The org.jawin.donated.win32 package contains some
> stubs I think Stuart wrote once, for manipulating with some of the Win32
> API's (eg. there is code for manipulating the registry).
>
> >
> > I also have stubs generated for MS Word and Excel that compile and
work -
> > as I had mentioned before, I am not sure how much value that has,
since
> > anyone who can run the Jawin browser can generate them on their own
(with
> > a few minor fixes). The value that I see in them is breaking them up
in a
> > separate project, that will come compiled & bundled with the Jawin
.dll,
> > so that Java developers that don't know anything about Jawin (and
don't
> > need to know anything about Jawin), can just drop the jar in their
> > classpath and have full access to the MS Office APIs.
>
> This is just what the donated packages are for. They are build into the
> jawin-stubs.jar file (this was not in the last binary release, so you
> will have to build "ant deploy" from the CVS-head yourself to get this
> file).
>
> Best Regards
> Morten
>
> ------------------------------
>
> Date:    Wed, 18 Aug 2004 16:01:03 -0400
> From:    Mukund Wassan <[email protected]>
> Subject: Re: Getting Out Parameter value in Jawin
>
> Hi Morten,
>
> Its a great new feature! that works great!
> Thank you very much for sparing the time for the reply.
> BTW, the jawin documentation update is fantastic.
>
> Thanks & Cheers to the Jawin team,
> Mukund
>
> ------------------------------
>
> End of JAWIN Digest - 17 Aug 2004 to 18 Aug 2004 (#2004-77)
> ***********************************************************


Regards,

Alex Kotchnev
Developer / Systems Analyst
Diversified Information Technologies

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
CONFIDENTIALITY NOTICE: If you have received this e-mail in error, please
immediately notify the sender by e-mail at the address shown.  This e-mail
transmission may contain confidential information.  This information is
intended only for the use of the individual(s) or entity to whom it is
intended even if addressed incorrectly.  Please delete it from your files
if you are not the intended recipient.  Thank you for your compliance.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

88a89
>                 <li><a href="#instructions_stacksize_overview">Understanding instruction strings and stack size </a></li>
324a326,327
>     <h3><a name="instructions_stacksize_overview">3.3. Understanding instruction strings and stack size</a></h3>
>     <p> Instruction strings </p>
326c329
< 		TODO: EXPLAIN CODE AND INSTRUCTION STRING AND STACKSIZE.
---
> 		Many C-functions receive pointers as parameters and other C-specific data types not present in Java;therefore, in order to invoke the functions in a dll, and to pass the parameters correctly to the C function, Jawin has to have a way of converting the parameters passed to it, to the data types required by the dll functions. In order to accomplish that, Jawin uses an "instruction string" (details of the meaning of each instruction character in the instruction string are specified in <a href="instruction_docs.html" title="Instruction String Explanations">The Instruction String Reference</a>.At this point the Instruction string reference is not complete, so additional information can be gleaned from cpp/jawin/instructions.h (where the letters in the instruction string are explained pretty well, but not complete), and in cpp/jawin/Transform.cpp (where the actual instructions are implemented).  
327a331,348
>     
>     
>     <p>
> 		Overall, the instruction string processing happens at three stages. If reading cpp/jawin/instructions.h, it often refers to "src" and "dest" and the references refer to different things depending on which part of the instruction string is being processed. 
>         
>         <ol>
>             <li>
>                 Converting the data passed into the byte stream as specified by the first section of the instruction string (e.g. the <code>org.jawin.io.NakedByteStream</code>) to the data types required by the dll function. At this stage, one has to pass instruction string that will convert the data from the input byte array from java, to the appropriate data types for the C call, and then pass the converted parameters to the C function. At this stage, in instructions.h, "src" refers to the byte array (or the content of the NakedByteStream) passed from Java into the FuncPtr.invoke() method, and "dest" refers to what is being passed to the dll function (which is also  a byte array with the size of StackSize). <br/>
>                 In many cases, the instructions passed here, create the appropriate pointers and place them as parameters to the dll function. The instructions here have to follow the same sequence in which the actual data is written to the byte array sent to the FuncPtr.invoke method. 
>             </li>
>             <li>
>                 Retrieving the retun value of the dll function and serializing it onto the return byte array (note, the FuncPtr.invoke() function returns a byte[] in the general case - that is because the de-serialization of the return values to the appropriate data types cannot be determined by FuncPtr itself. For example, if the dll function has a retun value of int, and the instruction string specified that the return value was an I, the first 4 bytes of the return byte array will contain the return value and can be retrieved by calling the readInt() method of the LittleEndianInputStream class). In another example, if the function returns a pointer to a piece of data that we want to use in Java, the instructions specify that the content of what the pointer points to, should be read and written to the return byte array. At this stage, "src" refers to the value returned from the function, and "dest" refers to the byte array returned from the FuncPtr.invoke method.
>             </li>
>             <li>
>                 Reading the "out" parameters that were passed to the dll function (as specified by the instruction string) and writing  them to the <code>byte[]</code> returned by the FuncPtr.invoke method. Once again, in order for the return data to be useful for the Java application, it has to be de-serialized into the appropriate Java data types. At this stage, "src" refers to the byte array that was passed to the function (with the size of StackSize, this is the same byte array, to which step 1 referred to as "dest"; however, this time with values populated ), and "dest" refers to the byte array returned from the FuncPtr.invoke() method. <br/>
>                 In many cases, the instructions passed here, either skip data passed as "in" parameters only, retrieve the data that was placed in the "out" parameters of the function call (by the dll function itself) and place it in the <code> byte[]</code> returned to Java. If there are no output parameters to be read, this section would be empty. In most cases, the content of this section will closely resemble the 1st stage of the input string, with instructions to skip the "in"-only parameters, and to read the "out" parameters in their sequence. However, that is not necessarily true in all cases, since the output instruction string can be totally independent of the input section and can arrange the content of the <code>byte[] </code>passed to Java in any way it wants. 
>             </li>
>         </ol> 
328a350,373
> 	</p>
>     <p> Stack Size </p>
>     <p> 
>         The stack size is important in the context of invoking the native method (I find it convenient to think about in terms of the size of the native function parameters between the function param parenthesis), because that is the size of the byte array that will be allocated to hold all parameters passed to a function. So, in essence, the stack size is the sum of the sizes of all the arguments passed to the native function. An important characteristic is that the stack size is a multiple of 4 bytes (the size of an int): e.g. although it is not impossible to specify that the stack size is not a multiple of 4, there is a good chance that something will go wrong. <br/>
>          
>         To calculate the stack size for the method call, add up the sizes of the arguments of the native method signature. While adding the argument sizes, if any of the arguments is shorter than sizeof(int), add sizeof(int) to the stack size instead of the actual argument size. For example, if the method accepts 2 integers and a pointer to a struct, then the stack size will be 2*4 + 4 (the pointer size is sizeof(int)). If the method accepts a byte array with a length of 100, a struct (and not a pointer to a struct) of size 12, and a boolean, then the stack becomes 100+12+4 (the size of the boolean is less than 4; however, 4 is the minimum increment allocated on the stack). <br/>
>         
>         Important sizes in calculating stack size are:
>         <ul>
>             <li> <code>int</code>: 4 </li>
>             <li> <code>double</code>: 8 </li>
>             <li> any kind of pointer, handle, etc: 4 </li>
>             <li> <code>string</code> (BSTR) : 4 (again, it is a pointer to str) </li>
>             <li> 
>                 <code>struct</code>: the sum of the sizes of all elements of the struct </li>
>             <li> <code>byte</code> buffer: 4 (since this is essentially a pointer to the buffer is on the stack, not the actual buffer ) </li>
>             
>         </ul>
>         - 
>     </p>
>     <p>
> 		TODO: EXPLAIN CODE AND INSTRUCTION STRING AND STACKSIZE.
> 	</p>
>