Re: YAJL

My best guess, based on your description, is that whatever is calling 
the API is using the HTTP 1.0 protocol.  This protocol, which was 
published in 1996, did not offer the "chunked" transfer coding.  That 
would also explain why "Connection: close" is in your header info -- it 
is telling it to keep receiving until the connection is closed.

Starting with HTTP 1.1  (originally published in 1997, but didn't become 
mainstream until further revisions were made in 1999) the chunked 
transfer coding is part of the core concepts of this protocol.  One 
important idea in 1.1 is to keep the connection open for multiple 
transfers, because it reduces the overhead of negotiating a new TCP 
channel (and frequently TLS parameters as well.)   1.1 is absolutely 
ubiquitous, although it's gradually being replaced by HTTP/2, it is 
still by far the most widely used HTTP version.

My guess is that whatever is calling your API is still using 1.0, which 
didn't support chunked coding -- that's weird, I haven't seen a client 
still using this older protocol in well over a decade, but I'm sure they 
are still out there.  But it perfectly fits your description.

So -- if the server allows you to specify the content-length header, 
that should solve the problem for you.   (However, in my experience, 
many HTTP servers don't allow it -- they discard the content-length, 
since they consider it their responsibility to establish this, not the 
application's responsibility.  They also often handle CCSID translation, 
which can change the length of the content.)

If adding content-length is working for you, I guess it's a good 
compromise for now.  I would strongly recommend updating the caller to 
use 1.1 or newer, though.  1.0 is well over 20 years out of date.


On 10/18/2022 3:39 PM, [email protected] wrote:
> I don't know where the chunk data processing problem is but...
>
> I created another version of yajl_writeStdout that included the Content-length: parameter in the header and it is now returning just the JSON data.
>
> No more chunk data.
>
> -----Original Message-----
> From: Scott Klement <[email protected]>
> Sent: Tuesday, October 18, 2022 12:45 PM
> To: Web Enabling the IBM i (AS/400 and iSeries) <[email protected]>; [email protected]
> Subject: Re: [WEB400] YAJL
>
> They aren't "extra" they are part of the HTTP protocol, it is telling the HTTP client how many bytes need to be received.
>
> It should not be included in the data that you process -- they are there just to tell the HTTP program how to receive the data, it should not be including them in the actual downloaded result. Something is wrong with the HTTP client that is calling out to the Apache.
>
> On 10/17/2022 9:25 PM, [email protected] wrote:
>> The program that is receiving the data is expecting pure JSON data to be returned.  Instead is it receiving these extra bytes prior to the first { and is abending.
>>
>>
-- 
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400) mailing list
To post a message email: [email protected]
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/web400
or email: [email protected]
Before posting, please take a moment to review the archives
at https://archive.midrange.com/web400.