Re: Parsing JSON

Hi Hannes,

It appears to me that the Rhino leakage is always occurring.  It has 
been a long on going struggle trying to pin down where the leakage was 
coming from.

I found that whenever I parse a simple JSON string, many char[] and 
Strings are left hanging around in memory that are never recovered in 
GC.  As a result, the server will run out of memory after about 20K 
parses of JSON strings.  (My memory limit was set to 256MB for catching 
memory leaks.)  Once I replaced the Rhino eval with a javascript routine 
that parses the JSON string, then my memory usage was a stable.

As for code, try a simple loop  on the attached JSON string and monitor 
the memory.  For example, start up Helma with JVM max memory usage of 
256MB and then run the following:

for(var i=1; i<15000; i++) { var o = test.parseJSON(); }

The variable 'test' contains the attached file as a string.  (Note:  I 
tested using Helma Inspector and put the code into the text area for 
running scripts.)  I believe you will notice that memory is used and 
never recovered and the server will run out of memory.

Thanks,
Kris





Hannes Wallnoefer wrote:
> Hi Kris,
>
> On 11/26/07, Kris Leite <[email protected]> wrote:
>   
>> In case anybody is parsing a large number of JSON strings and found
>> memory leakage issues you might consider replacing the Rhino Eval
>> command with another JSON parser.
>>     
>
> Is this something you can reproduce, or does it happen intermittently?
> Because if it is a problem with Rhino, I'd rather have this fixed.
>
>   
>> I found that using the JSON parser in Helma, that uses the Rhino Eval
>> command, to cause unexplained memory lost when using it to parse some
>> 75K+ small JSON strings.  This occurred with a single long running task
>> that using JSON strings for passing data between a Helma server and
>>     
>
> Can you send me some test data or code?
>
>   
>> another computer.  When I replaced the Rhino Eval with a javascript JSON
>> parser, my memory lost disappeared.
>>
>> I also found a couple changes to Property.java to be useful also:
>>
>> 1) Changed the line in 'setStringValue' from:
>>
>> value = str:
>>
>> to:
>>
>> value = (str == null) ? null : new String(str);
>>
>> 2) Changed the line 'Property(String propname, Node node)' from:
>>
>> this.propname = propname;
>>
>> to:
>>
>> this.propname = (propname == null) ? null : new String(propname);
>>     
>
> That would be strange, but it would be possible if these strings are
> somehow used in weak references. I'll look into this.
>
> hannes
>
>   
>> Your Mileage May Vary,
>> Kris
>>
>> _______________________________________________
>> Helma-user mailing list
>> [email protected]
>> http://helma.org/mailman/listinfo/helma-user
>>
>>     
> _______________________________________________
> Helma-user mailing list
> [email protected]
> http://helma.org/mailman/listinfo/helma-user
>
>

_______________________________________________
Helma-user mailing list
[email protected]
http://helma.org/mailman/listinfo/helma-user

{"merchcommission":"","merchcolor":"","merchsearch":"BINOCULARS 8-20 POWER 50MM","merchlistdisc":"","merchprice":"111000","merchwarrantable":"Y","merchchange":"","lastmodified":{},"parent":"8fab6be0-e4be-4183-8912-792c57d6c196","merchtype":"","merchvendordesc":"","name":"12100027","merchstyle":"T1-.5-G.SI2","applyon":"","merchgroupprice":"","merchmsgcode":"SCS","merchsellinglimit":"","applyid":"P56IC590","merchringsize":"","merchcentercwt":"","merchdesc3":"rubber coated zoom by Tasco","merchdesc2":"8-20 Power x 50mm Binoculars","merchserials":"","merchpurity":"","merchdesc1":"Tasco 8-20x50mm Binoculars","merchgroupcode":"","merchalt":"","merchlistprice":"","merchaltacq":"001","merchvendor":"37775","merchsellingprice":"","merchcertificate":"N","merchcerts":"","status":"A","merchsellingtype":"R","merchsubtype":"","merchgroupdisc":"","applyto":"merchandise","merchsellingcode":"S","merchinstock":"Y","merchserial":"N","merchcode10":"","merchsku":"121-0002-7   - -","merchclass":"119","merchplu":"12100027","applydelete":"","merchclarity":"","json":"","merchpricedate":"2003/07/24","merchcode9":"","merchlastpurdate":"","merchcode8":"","merchdept":"","merchforsale":"Y","merchcode7":"","merchcode6":"","merchcode5":"SCS","merchcode4":"","merchcode3":"","merchcode2":"","merchmetal":"14Y","merchcode1":"","merchprompt":"","merchsellingdisc":"","merchtotalcwt":"","merchlastactdate":"2009/07/24","merchreturnable":"Y"}