Re: random maddening bug - "corrupted" file upload contents
Cyrille Delaunay <[email protected]>
| Newsgroups | gmane.comp.lang.smalltalk.squeak.seaside |
|---|---|
| Message-ID | <CAMEdb_D9freOHuP8VR1gdOAS8vjZW7T28o+1BVN9urqg_OacYQ@mail.gmail.com> |
Hello,
Has anyone been able to reproduce my little scenario ?
As a reminder, suspicion is that something "wrong" has been introduced with
Pharo6
The scenario to reproduce:
"Download Pharo 6.1 on my Mac (Sierra 10.12.6): https://pharo.org/
web/download
Then, iterate over this process till spotting the issue:
=> start the pharo image
=> execute this piece of code in a playground
ZnServer startDefaultOn: 1701.
ZnServer default maximumEntitySize: 80* 1024 * 1024.
'/Users/cdelaunay/myzip.zip' asFileReference writeStreamDo: [ :out |
out binary; nextPutAll: #[80 75 3 4 10 0 0 0 0 0 125 83 67 73 0 0 0 0 0 0].
18202065 timesRepeat: [ out nextPut: 0 ]
].
=> Open a web browser page on: http://localhost:1701/form-test-3
=> Upload the file zip file previously generated ('myzip.zip').
=> If the web page displays: "contents=000000000a00..." (or anything
unexpected), THIS IS THE ISSUE !
=> If the web page displays: "contents=504b03040a00..", the upload worked
fine. You can close the image (without saving)
Mettre à jour <http://37.139.2.203/redmine/issues/906/edit> Saisir temps
<http://37.139.2.203/redmine/issues/906/time_entries/new> Surveiller
<http://37.139.2.203/redmine/watchers/watch?object_id=906&object_type=issue>
Copier <http://37.139.2.203/redmine/projects/general/issues/906/copy>
Supprimer <http://37.139.2.203/redmine/issues/906>
Usually, after starting 3 / 4 times pharo image, I am getting the problem."
2018-01-05 10:23 GMT+01:00 Cyrille Delaunay <[email protected]>:
> I made some experiments which might help to restrict the scope:
>
> I applied my little "manual" scenario with a Pharo 5 image (
> https://files.pharo.org/get-files/50/)
> After 7 tries, the issue never show up.
>
> With a Pharo 6.1 image, the issue shows up after maximum 3 / 4 tries.
>
> My feeling: something changed between these 2 versions.
> This idea would tend to be confirmed by the period we started to
> experiment the issue: We recently migrated some production images from
> Pharo 4 to Pharo 6.
> Is it Zinc or some Pharo inside bricks used by Zinc ? I could not say.
>
>
>
>
>
>
> 2018-01-04 16:48 GMT+01:00 Cyrille Delaunay <[email protected]>:
>
>> I tried several times and the test I wrote always passed.
>>
>> All I have been able to do, is to experiment some failures by playing
>> with the client timeout.
>> My test initially contains the following statement: *client timeout:
>> 120000.*
>> If I comment this statement, sometimes the server received a request with
>> a nil entity.
>> The test then get an unexpected response.
>> But I am not sure it an unexpected behavior, as there is a lot of data
>> being sent.
>>
>>
>>
>>
>>
>> 2018-01-02 20:28 GMT+01:00 Cyrille Delaunay <[email protected]>:
>>
>>>
>>>
>>> 2018-01-02 17:37 GMT+01:00 Sven Van Caekenberghe <[email protected]>:
>>>
>>>>
>>>>
>>>> > On 2 Jan 2018, at 12:53, Cyrille Delaunay <[email protected]>
>>>> wrote:
>>>> >
>>>> > Hi Sven,
>>>> >
>>>> > I am back on this issue :)
>>>> >
>>>> > I have kind of a "clean" scenario that could help to reproduce.
>>>> > It is still random, but at one point, it shows up for me.
>>>> >
>>>> > Here is how I am doing:
>>>> > Download Pharo 6.1 on my Mac (Sierra 10.12.6):
>>>> https://pharo.org/web/download
>>>> > Then, iterate over this process till spotting the issue:
>>>> >
>>>> > => start the pharo image
>>>> > => execute this piece of code in a playground
>>>> >
>>>> > ZnServer startDefaultOn: 1701.
>>>> > ZnServer default maximumEntitySize: 80* 1024 * 1024.
>>>> > '/Users/cdelaunay/myzip.zip' asFileReference writeStreamDo: [ :out |
>>>> > out binary; nextPutAll: #[80 75 3 4 10 0 0 0 0 0 125 83 67 73 0 0 0 0
>>>> 0 0].
>>>> > 18202065 timesRepeat: [ out nextPut: 0 ]
>>>> > ].
>>>> >
>>>> > => Open a web browser page on: http://localhost:1701/form-test-3
>>>> > => Upload the file zip file previously generated ('myzip.zip').
>>>> > => If the web page displays: "contents=000000000a00..." (or anything
>>>> unexpected), THIS IS THE ISSUE !
>>>> > => If the web page displays: "contents=504b03040a00..", the upload
>>>> worked fine. You can close the image (without saving)
>>>> >
>>>> > Mettre à jour Saisir temps Surveiller Copier Supprimer
>>>> >
>>>> >
>>>> > Usually, after starting 3 / 4 times pharo image, I am getting the
>>>> problem.
>>>> > I have been able to reproduce with chrome and firefox.
>>>> >
>>>> >
>>>> > Seing that I was able to reproduce within a fresh pharo image, I
>>>> tried to automate this scenario.
>>>> > I wrote the following test:
>>>> >
>>>> >
>>>> > ZnServerTests >>testFormTest4
>>>> > | inputs client part fileReference fileContents entity |
>>>> > fileReference := 'test-zip.zip' asFileReference.
>>>> > fileReference
>>>> > writeStreamDo: [ :out |
>>>> > out
>>>> > binary;
>>>> > nextPutAll: #[80 75 3 4 10 0 0 0 0 0
>>>> 125 83 67 73 0 0 0 0 0 0].
>>>> > 18202065 timesRepeat: [ out nextPut: 0 ] ].
>>>> > fileContents := fileReference binaryReadStreamDo: [ :in | in
>>>> upToEnd ].
>>>> > 15
>>>> > timesRepeat: [ self
>>>> > withServerDo: [ :server |
>>>> > server maximumEntitySize: 80 *
>>>> 1024 * 1024.
>>>> > server debugMode: true.
>>>> > (client := ZnClient new)
>>>> > url: server localUrl;
>>>> > addPathSegment:
>>>> 'form-test-3'.
>>>> > client timeout: 120000.
>>>> > entity := (ZnByteArrayEntity
>>>> type: (ZnMimeType main: 'application' sub: 'zip') length: fileContents
>>>> size) bytes: fileContents.
>>>> > part := ZnMimePart fieldName:
>>>> 'file' fileName: fileReference basename entity: entity.
>>>> > client
>>>> > resetEntity;
>>>> > addPart: part;
>>>> > post.
>>>> > self assert: client isSuccess.
>>>> > self assert: (client contents
>>>> includesSubstring: '504b03040a') ] ]
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > In order to make this test run correctly I had to modify this method.
>>>> > Without this change, I always got an empty html page as response
>>>> >
>>>> >
>>>> >
>>>> > ZnDefaultServerDelegate >> formTest3: request
>>>> > | contents filename contentType page |
>>>> > contents := filename := contentType := ''.
>>>> > (request hasEntity and: [ request contentType matches:
>>>> ZnMimeType multiPartFormData ])
>>>> > ifTrue: [
>>>> > (request entity partNamed: #file ifNone: [ nil
>>>> ])
>>>> > ifNotNil: [ :part |
>>>> > filename := part fileName.
>>>> > contents := part contents.
>>>> > contentType := part
>>>> contentType.
>>>> > contentType isBinary ifTrue: [
>>>> contents := contents hex ] ] ].
>>>> > page := ZnHtmlOutputStream streamContents: [ :html |
>>>> > html page: 'Form Test 3' do: [
>>>> > html
>>>> > tag: #form
>>>> > attributes: #(action 'form-test-3'
>>>> 'accept-charset' 'utf-8'
>>>> >
>>>> enctype 'multipart/form-data' method POST)
>>>> > do: [
>>>> > html
>>>> > str: 'File'; space;
>>>> > tag: #input attributes:
>>>> #(type file name file); space;
>>>> > tag: #input attributes:
>>>> #(type submit) ];
>>>> > tag: #p do: [ html str: 'filename = ';
>>>> str: filename ];
>>>> > tag: #p do: [ html str: 'content-type =
>>>> '; str: contentType asString ];
>>>> > tag: #p do: [ html str: 'contents = ';
>>>> str: (contents copyFrom: 1 to: 20) asString ] ] ].
>>>> > ^ ZnResponse ok: (ZnEntity html: page)
>>>> >
>>>> >
>>>> >
>>>> > Unfortunately, the tests always passed till now !
>>>>
>>>> So you have to upload 80Mb ?
>>>>
>>>
>>> Sometimes yes.
>>> But here the file I build and use to reproduce is only 18 mb.
>>> (the same size as the original file highlighting the issue on my
>>> machine).
>>>
>>>
>>>> And if you script it in Pharo and repeat it 15 times it keeps on
>>>> working ?
>>>> Really ?
>>>>
>>>
>>> Yes indeed.
>>> But again, it is completely random.
>>> I could not ensure that the problem would never show up with the script.
>>> I will try to make more tries tomorrow.
>>>
>>>
>>>>
>>>> It seems that it only fails when a manually operated browser is
>>>> involved.
>>>>
>>>
>>> Yes, exactly
>>>
>>>
>>>>
>>>> Have you tried doing the upload with curl ?
>>>>
>>>
>>> No.
>>> How would I do that ?
>>>
>>>
>>>>
>>>> > I will try to go on refining the context parameters that could make
>>>> the issue show up or not
>>>> >
>>>> >
>>>> >
>>>> > 2017-12-22 18:05 GMT+01:00 Sven Van Caekenberghe <[email protected]>:
>>>> > Well, I can't repeat it, for now.
>>>> >
>>>> > Try to make a minimal self-contained example that fails (reliably),
>>>> like how I did it, you can even try to do the upload in Pharo too (see
>>>> #testFormTest3).
>>>> >
>>>> > > On 22 Dec 2017, at 18:00, Cyril Ferlicot <[email protected]>
>>>> wrote:
>>>> > >
>>>> > >
>>>> > > On ven. 22 déc. 2017 at 17:50, Sven Van Caekenberghe <[email protected]>
>>>> wrote:
>>>> > > Cyrille,
>>>> > >
>>>> > > I wrote a file like this:
>>>> > >
>>>> > > FileLocator temp / 'myzip.zip' writeStreamDo: [ :out |
>>>> > > out binary; nextPutAll: #[80 75 3 4 10 0 0 0 0 0 125 83 67 73 0 0
>>>> 0 0 0 0] ].
>>>> > >
>>>> > > Which I can read as follows:
>>>> > >
>>>> > > FileLocator temp / 'myzip.zip' binaryReadStreamDo: [ :in | in
>>>> upToEnd ].
>>>> > >
>>>> > > => #[80 75 3 4 10 0 0 0 0 0 125 83 67 73 0 0 0 0 0 0]
>>>> > >
>>>> > > If I upload this file to http://localhost:1701/form-test-3 this
>>>> works perfectly (macOS 10.13.2 Pharo #60402, Safari. Firefox & Chrome),
>>>> first time and second time.
>>>> > >
>>>> > >
>>>> > >
>>>> > > Maybe you should try on a co-workers machine just to make sure it
>>>> is not a local problem on your machine ?
>>>> > >
>>>> > > Hi Sven,
>>>> > >
>>>> > > It happen on Cyrille's machine, my machine, Yann's machine and our
>>>> boss's machine.
>>>> > >
>>>> > > We first saw it on Windows and never on mac/Linux. But now we see
>>>> it also on mac.
>>>> > >
>>>> > > I did not see it yet on Linux but we use it less for now.
>>>> > >
>>>> > > Also, when it happen, it tend to happen a lot afterward.
>>>> > >
>>>> > > Also, by random it means that we can have 1/2 week without problem,
>>>> then 4 failure in 2h.
>>>> > >
>>>> > >
>>>> > >
>>>> > > Sven
>>>> > >
>>>> > > --
>>>> > > Cyril Ferlicot
>>>> > > https://ferlicot.fr
>>>> > >
>>>> > > http://www.synectique.eu
>>>> > > 2 rue Jacques Prévert 01,
>>>> > > 59650 Villeneuve d'ascq France
>>>> > > <Screen Shot 2017-12-22 at 17.48.31.png>_________________
>>>> ______________________________
>>>> > > seaside mailing list
>>>> > > [email protected]
>>>> > > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>>>> >
>>>> > _______________________________________________
>>>> > seaside mailing list
>>>> > [email protected]
>>>> > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Cyrille Delaunay
>>>> > _______________________________________________
>>>> > seaside mailing list
>>>> > [email protected]
>>>> > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>>>>
>>>> _______________________________________________
>>>> seaside mailing list
>>>> [email protected]
>>>> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>>>>
>>>
>>>
>>>
>>> --
>>> Cyrille Delaunay
>>>
>>
>>
>>
>> --
>> Cyrille Delaunay
>>
>
>
>
> --
> Cyrille Delaunay
>
--
Cyrille Delaunay
_______________________________________________
seaside mailing list
[email protected]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside