Re: Xindice 1.1: crash with NullPointerException

Olle,

Thanks for including database.xml, I was about to ask for it to see
the configuration.

I don't think it is a problem that you are missing some of the system
collections for Xindice 1.1 that you have in 1.0, as far as I remember
they are not used anymore (sorry, cannot check right now), difference
in the collection size should not matter either - collections reserve
space based on configuration settings, but system collections normally
do not need so much space, so most of it will be just empty.

What you listed under SysSymbols collections looks kind of normal,
except that it should have more documents. Based on your configuration
from database.xml, there should be at least two more documents in
/db/system/SysSymbols collection: w3c and w3c-local.

Does /db/system/SysSymbols have any other documents in it?

As for your question - Xindice configuration location is hard-coded,
it is always has the same structure (system/SysConfig and
system/SysSymbols). To function properly, Xindice needs two pieces of
information for compressed collections - binary data stored in the
collection file itself (.tbl file) plus system data to know how to
interpret that binary data. You can see the listing of the documents
in collections, so I assume that collection file is OK.

Regards,
Natalia

On Thu, Jun 10, 2010 at 8:27 AM, Olle Olsson <[email protected]> wrote:
> Nataila,
>
> Good suggestion, thanks.It helped me to see that I may face a more diffic=
ult
> problem. Some "symbol docs" cannot be found!
>
> The following hierarchy is what I see in the Ugly Browser (for 1.1) ...
>
> | =A0db
> | =A0 =A0system
> | =A0 =A0 =A0SysConfig
> | =A0 =A0 =A0 =A0database.xml =A0 =A0 =A0 (document)
> | =A0 =A0 =A0SysSymbols
> | =A0 =A0 =A0 =A0system_SysConfig =A0 (document)
> | =A0 =A0 =A0 =A0w3c-local_meta =A0 =A0 (document)
>
> Now, the part under SysSymbols does look strange!
>
> I revived the old Xindice 1.0 version I have kept around, to have somethi=
ng
> to compare to.
> And there I see that much data seems to be missing in my 1.1 system.
>
> Tragically, core data has gone away:
> =A0- system_SysAccess
> =A0- system_SysGroups
> =A0- system_SysObjects
> =A0- system_SysUsers
> But also the 100+ application data collections (only one such collection =
is
> seen)
>
> For completeness, further down I reproduce the contents of the three
> documents thar *can* be seen.
>
> To get some idea about whether actual files have disappeared in the file
> system, I looked at that,
> but could not find anything unexpected (i.e., existence and size looked O=
K)
>
> | =A0../../../opt/xindice-1.1/w3c/system/SysAccess:
> | =A0total 1
> | =A0-rwxr--r--+ 1 olleo sics 12288 2003-08-12 19:36 SysAccess.tbl
> |
> | =A0../../../opt/xindice-1.1/w3c/system/SysConfig:
> | =A0total 5
> | =A0-rw-rw-r--+ 1 olleo sics 45056 2010-06-09 10:56 SysConfig.tbl
> |
> | =A0../../../opt/xindice-1.1/w3c/system/SysGroups:
> | =A0total 1
> | =A0-rwxr--r--+ 1 olleo sics 12288 2003-08-12 19:36 SysGroups.tbl
> |
> | =A0../../../opt/xindice-1.1/w3c/system/SysObjects:
> | =A0total 1
> | =A0-rwxr--r--+ 1 olleo sics 12288 2003-08-12 19:36 SysObjects.tbl
> |
> | =A0../../../opt/xindice-1.1/w3c/system/SysSymbols:
> | =A0total 52
> | =A0-rw-rw-r--+ 1 olleo sics 4202496 2010-06-09 10:56 SysSymbols.tbl
> |
> | =A0../../../opt/xindice-1.1/w3c/system/SysUsers:
> | =A0total 1
> | =A0-rwxr--r--+ 1 olleo sics 12288 2003-08-12 19:36 SysUsers.tbl
>
> The for the old Xindice 1.0, exactly the same was seen, except that
> SysSymbols.tbl had a size of 659456. (1.0 is run on a Windows/NT, 1.1 on
> Linux)
>
> So, the 64,000 dollar question is: how is Xindice bootstrapped to find th=
e
> system data? Now it fails to find much data and does this depend on
> information *in* the storage hierarch? Or does it depend on some other
> configuration data that no longer is in a healtthy state?
>
> Really, I have no good clue how to progress on this.
>
> /olle
>
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> The accessible system files
> ______________________
> system_SysConfig
>
> <?xml version=3D"1.0" encoding=3D"UTF-8"?>
> <?xindice-class org.apache.xindice.xml.SymbolTable?>
> <symbols>
> <symbol name=3D"filer" id=3D"6"/>
> <symbol name=3D"compressed" id=3D"5"/>
> <symbol name=3D"xmlobjects" id=3D"2"/>
> <symbol name=3D"pagecount" id=3D"8"/>
> <symbol name=3D"name" id=3D"1"/>
> <symbol name=3D"indexes" id=3D"9"/>
> <symbol name=3D"class" id=3D"7"/>
> <symbol name=3D"collection" id=3D"4"/>
> <symbol name=3D"collections" id=3D"3"/>
> <symbol name=3D"database" id=3D"0"/>
> </symbols>
>
> ______________________
> w3c-local_meta
>
> <?xml version=3D"1.0" encoding=3D"UTF-8"?>
> <?xindice-class org.apache.xindice.xml.SymbolTable?>
> <symbols>
> <symbol name=3D"content" id=3D"2"/>
> <symbol name=3D"static" id=3D"1"/>
> <symbol name=3D"meta" id=3D"0"/>
> </symbols>
>
>
> ______________________
> database.xml
>
> <?xml version=3D"1.0" encoding=3D"UTF-8"?>
> <database name=3D"db">
> <xmlobjects/>
> <collections>
> <collection compressed=3D"true" name=3D"w3c">
> <filer class=3D"org.apache.xindice.core.filer.BTreeFiler" pagecount=3D"1"=
/>
> <indexes/>
> <xmlobjects/>
>
> =A0 =A0 =A0 ... and so on ...
>
> </collection>
> <collection compressed=3D"true" name=3D"w3c-local">
> <filer class=3D"org.apache.xindice.core.filer.BTreeFiler" pagecount=3D"1"=
/>
> <indexes/>
> <xmlobjects/>
>
> =A0 =A0 =A0 .... and so on ....
>
> </collection>
> </collections>
> </database>
>
>
>
>
>
> On 2010-06-10 03:15, Natalia Shilenkova wrote:
>>
>> Hi Olle,
>>
>> Looking at the exception stack trace, I would say that you've got proble=
m
>> with collection symbol table. That would explain your problems accessing=
 all
>> the documents.
>>
>> I was able to reproduce exactly the same stack trace by intentionally
>> modifying symbol table.
>>
>> When Xindice saves a document to the collection, the document is
>> compressed, which means that its element and attribute names are replace=
d
>> with generated identifiers. The information about these identifiers save=
d in
>> the collection symbol table. When retrieving the document, Xindice looks=
 up
>> the identifiers and replaces them with actual names. If it encounters
>> identifier that cannot be found in the table... well, that exception is =
what
>> happens.
>>
>> I would suggest checking the symbol table first - it is just XML documen=
t
>> that is located in /db/system/SysSymbols collection and document name is=
 the
>> same as collection path, but fragments of the path are separated with '_=
'
>> instead of '/'. So, for collection /db/a/b/c it will be a_b_c. =A0Check =
if
>> anything looks out of place, maybe you can see right away that some elem=
ent
>> or attribute names are missing...
>>
>> This is an example of symbol table for SysConfig collection:
>> <?xml version=3D"1.0" encoding=3D"UTF-8"?>
>> <?xindice-class org.apache.xindice.xml.SymbolTable?>
>> <symbols>
>> =A0 <symbol name=3D"filer" id=3D"6"/>
>> =A0 <symbol name=3D"inline-metadata" id=3D"5"/>
>> =A0 <symbol name=3D"compressed" id=3D"4"/>
>> =A0 <symbol name=3D"name" id=3D"1"/>
>> =A0 <symbol name=3D"indexes" id=3D"8"/>
>> =A0 <symbol name=3D"class" id=3D"7"/>
>> =A0 <symbol name=3D"collection" id=3D"2"/>
>> =A0 <symbol name=3D"collections" id=3D"3"/>
>> =A0 <symbol name=3D"database" id=3D"0"/>
>> </symbols>
>>
>> Regards,
>> Natalia
>>
>>
>
>
> --
> ------------------------------------------------------------------
> Olle Olsson =A0 [email protected] =A0 Tel: +46 8 633 15 19 =A0Fax: +46 8 751 =
72 30
> =A0 =A0 =A0 =A0[Svenska W3C-kontoret: [email protected]]
> SICS [Swedish Institute of Computer Science]
> Box 1263
> SE - 164 29 Kista
> Sweden
> ------------------------------------------------------------------
>
>
>