Re: [Proftpd-user] Directory listing fails

> Here's the complete session transcript:

> Remote server is running ProFTPD.
> 220: ProFTPD 1.3.6rc2 Server (ftp10.pointinspace.com) [216.27.95.22]

Here we see the IP address used for the control connection: 216.27.95.22

> Cmd: PORT 192,168,0,34,228,158
> 200: PORT command successful

Here we see the FTP client requesting an active data transfer, to
192.168.0.34.  Curious, as that is an RFC 1918, non-publicly routable
address, but...

> Cmd: LIST -a
> 150: Opening ASCII mode data connection for file list
> 226: Transfer complete

Here we see that the requested data transfer succeeds.  This suggests
that there's a client-side router/NAT/firewall which can properly handle
active data transfers from the FTP server.  This is good; it rules out
the possibility of such network middleboxes causing FTP data transfer
issues (as they often do).

> Cmd: PORT 192,168,0,34,228,159
> 200: PORT command successful
> Cmd: MLSD
> 150: Opening ASCII mode data connection for MLSD
> 226: Transfer complete

Another successful active data transfer.  And several more.

> Cmd: PORT 192,168,0,34,228,164
> 200: PORT command successful
> Cmd: LIST -a
> 150: Opening ASCII mode data connection for file list
> 226: Transfer complete

Here we see an active data transfer using "LIST -a", which succeeds. 
Also a good sign.  Then...

> Cmd: PORT 192,168,0,34,228,167
> 200: PORT command successful
> Cmd: LIST -a
> 150: Opening ASCII mode data connection for file list
> Could not directory listing data -- timed out.

Here we see the timeout.  I'm assuming the directory listed is the one
with *many* entries.

>From this transcript, it looks like it's the FTP _client_ which is
timing out, waiting for the requested transfer of LIST data.  Is there a
way to change that client-side data transfer timeout?

In general, listing tens to hundreds of thousands of files (all in the
same directory) will be an issue for any FTP client.  (It's also an
issue on the filesystem.)  Such "wide" directories should be minimized,
as much as possible, if performance is a goal.  Strategies such as using
a "directory hash/sharding" layout, e.g.:

  /dir/f/fi/filename

can help to reduce the number of files in any given directory, at the
cost of having to navigate a couple of levels deeper into the
filesystem.

Cheers,
TJ

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
ProFTPD Users List   <[email protected]>
Unsubscribe problems?
http://www.proftpd.org/list-unsub.html