Re: Xalan Java XPath parser design

--_000_MN2PR12MB4221FD4544884A2F5367F6FE9E022MN2PR12MB4221namp_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Good analysis, Mukul, though perhaps a bit kinder to us than we deserve. Pr=
obably worth adding to our official documentation.

The xpath and XSLT intermediate representation was originally designed for =
interpretive Xalan. One of the reasons it is so compact is that PCs were mu=
ch smaller and slower at the time so storage and access efficiency was cruc=
ial. (The same reason DTM its designed essentially at the byte level for co=
mpactness.) But compactness can also buy performance advantages; it's as cl=
ose to C or ASM data models as Java let us get.

When Sun donated their xslt compiler to Apache, the IBM / Apache team adjus=
ted it to share this representation so it could be integrated with Xalan. I=
 don't recall how much the Xalan front end had to change to support that; n=
ot a great deal, I think.

There could certainly be better choices. I've mentioned some of them in pas=
t discussion of IBM Research's Xylem processor.


--
Joe (Typo'ed on my phone.)

Car Talk has been advised that the staff opticians, C. F. Eyecare, are now =
under new management. It's Ike and Zeke Leary, now; Lorraine is gone.
________________________________
From: Mukul Gandhi <[email protected]>
Sent: Sunday, May 17, 2026 12:47:48 PM
To: [email protected] <[email protected]>
Subject: Xalan Java XPath parser design

Hello Xalan dev list,
   I've always been curious to learn about the detailed design for
Xalan Java XPath parser.

I did find following information, a while ago on Google Gemini AI
software, and thought someone might be interested to know these
details.

Question : What is software design for Xalan Java class
org.apache.xpath.compiler.OpMap?

Answers (with minor grammatical improvements):
The org.apache.xpath.compiler.OpMap class in Apache Xalan-Java serves
as the structural foundation for XPath expression representation. It
is designed to act as an optimized, flat data structure (an array of
integers) to represent an XPath abstract syntax tree (AST),
prioritizing fast execution speeds and low memory overhead.

1. The Core Design Concept: The "OpMap"

Instead of building a traditional tree of objects (which introduces
heavy memory and allocation overhead), Xalan parses an XPath string
into a flat integer array called the Operations Map (OpMap).

1.1) Flat Array Representation: Each node and its parameters (like
operations, step operators, and literals) are stored sequentially in
an OpMapVector.

1.2) Opcode + Arguments: The array is traversed in chunks. A specific
opcode (e.g., OpCodes.OP_LOCATIONPATH) sits at a given position,
immediately followed by the length of the operation and the
lengths/pointers to its child nodes.

1.3) Helper Methods: OpMap is equipped with protected/utility methods
like getArgLength() and getFirstChildPos() to parse and navigate this
array structure without writing explicit, repetitive pointer math.

2. Primary Components

The OpMap encapsulates the basic state and memory structures required
to store a parsed XPath:

2.1) m_opMap: An OpMapVector (a custom array-based list) that holds
the actual tree of integer opcodes and their arguments.

2.2) m_tokenQueue: An ObjectVector containing the literal tokens
(e.g., string literals, node names, and numeric constants) referenced
by the opcodes in m_opMap.

2.3) m_currentPattern: Stores the raw XPath string, which is mostly
kept for diagnostic and debugging purposes.

3. Structural Hierarchy

3.1) Base Class (OpMap): Contains the data structures and basic
navigation algorithms for traversing the opcode array.

3.2) Derived Class (Compiler): Extends OpMap to actually iterate over
the flat integer map and build a proper executable expression tree
(e.g., Expression objects) that evaluates the XPath dynamically
against context nodes.

This design, for Xalan Java XPath parser creating a flat array (OpMap)
and a compiler traversing it was chosen to balance Xalan's need for
both interpretative tooling and fast, compiled runtime performance.



--
Regards,
Mukul Gandhi

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]


--_000_MN2PR12MB4221FD4544884A2F5367F6FE9E022MN2PR12MB4221namp_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
</head>
<body>
<div dir=3D"auto" style=3D"font-family: Aptos, Aptos_MSFontService, -apple-=
system, Roboto, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(3=
3, 33, 33);">
Good analysis, Mukul, though perhaps a bit kinder to us than we deserve. Pr=
obably worth adding to our official documentation.</div>
<div dir=3D"auto" style=3D"font-family: Aptos, Aptos_MSFontService, -apple-=
system, Roboto, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(3=
3, 33, 33);">
<br>
</div>
<div dir=3D"auto" style=3D"font-family: Aptos, Aptos_MSFontService, -apple-=
system, Roboto, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(3=
3, 33, 33);">
The xpath and XSLT intermediate representation was originally designed for =
interpretive Xalan. One of the reasons it is so compact is that PCs were mu=
ch smaller and slower at the time so storage and access efficiency was cruc=
ial. (The same reason DTM its designed
 essentially at the byte level for compactness.) But compactness can also b=
uy performance advantages; it's as close to C or ASM data models as Java le=
t us get.</div>
<div dir=3D"auto" style=3D"font-family: Aptos, Aptos_MSFontService, -apple-=
system, Roboto, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(3=
3, 33, 33);">
<br>
</div>
<div dir=3D"auto" style=3D"font-family: Aptos, Aptos_MSFontService, -apple-=
system, Roboto, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(3=
3, 33, 33);">
When Sun donated their xslt compiler to Apache, the IBM / Apache team adjus=
ted it to share this representation so it could be integrated with Xalan. I=
 don't recall how much the Xalan front end had to change to support that; n=
ot a great deal, I think.</div>
<div dir=3D"auto" style=3D"font-family: Aptos, Aptos_MSFontService, -apple-=
system, Roboto, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(3=
3, 33, 33);">
<br>
</div>
<div dir=3D"auto" style=3D"font-family: Aptos, Aptos_MSFontService, -apple-=
system, Roboto, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(3=
3, 33, 33);">
There could certainly be better choices. I've mentioned some of them in pas=
t discussion of IBM Research's Xylem processor.</div>
<div id=3D"ms-outlook-mobile-body-separator-line" data-applydefaultfontstyl=
es=3D"true" dir=3D"auto" style=3D"font-family: Aptos, Aptos_MSFontService, =
-apple-system, Roboto, Arial, Helvetica, sans-serif; font-size: 12pt;">
<div style=3D"font-family: Aptos, Aptos_MSFontService, -apple-system, Robot=
o, Arial, Helvetica, sans-serif; font-size: 12pt;">
<br>
</div>
</div>
<div id=3D"ms-outlook-mobile-signature" dir=3D"auto" style=3D"font-family: =
Aptos, Aptos_MSFontService, -apple-system, Roboto, Arial, Helvetica, sans-s=
erif; font-size: 12pt; color: rgb(33, 33, 33);">
<div dir=3D"auto" style=3D"font-family: Aptos, Aptos_MSFontService, -apple-=
system, Roboto, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(3=
3, 33, 33);">
<br>
</div>
<div dir=3D"auto" style=3D"font-family: Aptos, Aptos_MSFontService, -apple-=
system, Roboto, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(3=
3, 33, 33);">
--</div>
<div dir=3D"auto" style=3D"font-family: Aptos, Aptos_MSFontService, -apple-=
system, Roboto, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(3=
3, 33, 33);">
Joe (Typo'ed on my phone.)</div>
<div dir=3D"auto" style=3D"font-family: Aptos, Aptos_MSFontService, -apple-=
system, Roboto, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(3=
3, 33, 33);">
<br>
</div>
<div dir=3D"auto" style=3D"font-family: Aptos, Aptos_MSFontService, -apple-=
system, Roboto, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(3=
3, 33, 33);">
Car Talk has been advised that the staff opticians, C. F. Eyecare, are now =
under new management. It's Ike and Zeke Leary, now; Lorraine is gone.</div>
</div>
<hr style=3D"display:inline-block;width:98%" tabindex=3D"-1">
<div id=3D"divRplyFwdMsg" dir=3D"ltr"><font face=3D"Calibri, sans-serif" st=
yle=3D"font-size:11pt" color=3D"#000000"><b>From:</b> Mukul Gandhi &lt;gand=
[email protected]&gt;<br>
<b>Sent:</b> Sunday, May 17, 2026 12:47:48 PM<br>
<b>To:</b> [email protected] &lt;[email protected]&gt;<br>
<b>Subject:</b> Xalan Java XPath parser design</font>
<div>&nbsp;</div>
</div>
<div class=3D"BodyFragment"><font size=3D"2"><span style=3D"font-size:11pt;=
">
<div class=3D"PlainText">Hello Xalan dev list,<br>
&nbsp;&nbsp; I've always been curious to learn about the detailed design fo=
r<br>
Xalan Java XPath parser.<br>
<br>
I did find following information, a while ago on Google Gemini AI<br>
software, and thought someone might be interested to know these<br>
details.<br>
<br>
Question : What is software design for Xalan Java class<br>
org.apache.xpath.compiler.OpMap?<br>
<br>
Answers (with minor grammatical improvements):<br>
The org.apache.xpath.compiler.OpMap class in Apache Xalan-Java serves<br>
as the structural foundation for XPath expression representation. It<br>
is designed to act as an optimized, flat data structure (an array of<br>
integers) to represent an XPath abstract syntax tree (AST),<br>
prioritizing fast execution speeds and low memory overhead.<br>
<br>
1. The Core Design Concept: The &quot;OpMap&quot;<br>
<br>
Instead of building a traditional tree of objects (which introduces<br>
heavy memory and allocation overhead), Xalan parses an XPath string<br>
into a flat integer array called the Operations Map (OpMap).<br>
<br>
1.1) Flat Array Representation: Each node and its parameters (like<br>
operations, step operators, and literals) are stored sequentially in<br>
an OpMapVector.<br>
<br>
1.2) Opcode + Arguments: The array is traversed in chunks. A specific<br>
opcode (e.g., OpCodes.OP_LOCATIONPATH) sits at a given position,<br>
immediately followed by the length of the operation and the<br>
lengths/pointers to its child nodes.<br>
<br>
1.3) Helper Methods: OpMap is equipped with protected/utility methods<br>
like getArgLength() and getFirstChildPos() to parse and navigate this<br>
array structure without writing explicit, repetitive pointer math.<br>
<br>
2. Primary Components<br>
<br>
The OpMap encapsulates the basic state and memory structures required<br>
to store a parsed XPath:<br>
<br>
2.1) m_opMap: An OpMapVector (a custom array-based list) that holds<br>
the actual tree of integer opcodes and their arguments.<br>
<br>
2.2) m_tokenQueue: An ObjectVector containing the literal tokens<br>
(e.g., string literals, node names, and numeric constants) referenced<br>
by the opcodes in m_opMap.<br>
<br>
2.3) m_currentPattern: Stores the raw XPath string, which is mostly<br>
kept for diagnostic and debugging purposes.<br>
<br>
3. Structural Hierarchy<br>
<br>
3.1) Base Class (OpMap): Contains the data structures and basic<br>
navigation algorithms for traversing the opcode array.<br>
<br>
3.2) Derived Class (Compiler): Extends OpMap to actually iterate over<br>
the flat integer map and build a proper executable expression tree<br>
(e.g., Expression objects) that evaluates the XPath dynamically<br>
against context nodes.<br>
<br>
This design, for Xalan Java XPath parser creating a flat array (OpMap)<br>
and a compiler traversing it was chosen to balance Xalan's need for<br>
both interpretative tooling and fast, compiled runtime performance.<br>
<br>
<br>
<br>
-- <br>
Regards,<br>
Mukul Gandhi<br>
<br>
---------------------------------------------------------------------<br>
To unsubscribe, e-mail: [email protected]<br>
For additional commands, e-mail: [email protected]<br>
<br>
</div>
</span></font></div>
</body>
</html>

--_000_MN2PR12MB4221FD4544884A2F5367F6FE9E022MN2PR12MB4221namp_--