Re: [Proftpd-user] Directory listing fails

Hmm, I tried upping the timeout in Transmit, and now the file listing 
dies after about 60 seconds with:

	Remote host has closed the connection.

in the transcript.

While I understand a large number of files like this can be problematic, 
I've been able to list directories of similar sizes with Transmit using 
PureFTPd.

Other ideas?

	- John


On 3/3/17 12:04 PM, TJ Saunders wrote:
>
>> Here's the complete session transcript:
>
>> Remote server is running ProFTPD.
>> 220: ProFTPD 1.3.6rc2 Server (ftp10.pointinspace.com) [216.27.95.22]
>
> Here we see the IP address used for the control connection: 216.27.95.22
>
>> Cmd: PORT 192,168,0,34,228,158
>> 200: PORT command successful
>
> Here we see the FTP client requesting an active data transfer, to
> 192.168.0.34.  Curious, as that is an RFC 1918, non-publicly routable
> address, but...
>
>> Cmd: LIST -a
>> 150: Opening ASCII mode data connection for file list
>> 226: Transfer complete
>
> Here we see that the requested data transfer succeeds.  This suggests
> that there's a client-side router/NAT/firewall which can properly handle
> active data transfers from the FTP server.  This is good; it rules out
> the possibility of such network middleboxes causing FTP data transfer
> issues (as they often do).
>
>> Cmd: PORT 192,168,0,34,228,159
>> 200: PORT command successful
>> Cmd: MLSD
>> 150: Opening ASCII mode data connection for MLSD
>> 226: Transfer complete
>
> Another successful active data transfer.  And several more.
>
>> Cmd: PORT 192,168,0,34,228,164
>> 200: PORT command successful
>> Cmd: LIST -a
>> 150: Opening ASCII mode data connection for file list
>> 226: Transfer complete
>
> Here we see an active data transfer using "LIST -a", which succeeds.
> Also a good sign.  Then...
>
>> Cmd: PORT 192,168,0,34,228,167
>> 200: PORT command successful
>> Cmd: LIST -a
>> 150: Opening ASCII mode data connection for file list
>> Could not directory listing data -- timed out.
>
> Here we see the timeout.  I'm assuming the directory listed is the one
> with *many* entries.
>
>>From this transcript, it looks like it's the FTP _client_ which is
> timing out, waiting for the requested transfer of LIST data.  Is there a
> way to change that client-side data transfer timeout?
>
> In general, listing tens to hundreds of thousands of files (all in the
> same directory) will be an issue for any FTP client.  (It's also an
> issue on the filesystem.)  Such "wide" directories should be minimized,
> as much as possible, if performance is a goal.  Strategies such as using
> a "directory hash/sharding" layout, e.g.:
>
>   /dir/f/fi/filename
>
> can help to reduce the number of files in any given directory, at the
> cost of having to navigate a couple of levels deeper into the
> filesystem.
>
> Cheers,
> TJ
>


-- 

-------------------------------------------------------------------
John May : President                   http://www.pointinspace.com/
Point In Space Internet Solutions         800.664.8610 919.338.8198

         Professional FileMaker Pro, MySQL, PHP & Lasso Hosting
           on shared, virtual and hardware dedicated servers


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
ProFTPD Users List   <[email protected]>
Unsubscribe problems?
http://www.proftpd.org/list-unsub.html