Re: Possible Bug in lftp mirror using mirror:sort-by option

--===============1617102312294162826==
Content-Type: multipart/alternative; boundary=f46d041827ea599bb30506a1feed

--f46d041827ea599bb30506a1feed
Content-Type: text/plain; charset=UTF-8

lftp have options like --depth-first and --no-recursion I think it will
make sense to have an option like  " --iterative" for iterative file
transfer. This option will fetch the full file list and then fetch them
iteratively.

On Wed, Oct 29, 2014 at 2:29 PM, akshay gupta <[email protected]>
wrote:

> I think the functionality of mirror:sort-by option implies that the order
> is for all the files irrespective of the directory structure. Or may be it
> should support both the options and it would be configuration defined which
> one to use.
>
> On Wed, Oct 29, 2014 at 2:09 PM, Alexander Lukyanov <[email protected]>
> wrote:
>
>> Mirror in lftp works directory by directory. It does not build full list
>> of files first, so it cannot sort the whole list. Maybe the mirror
>> algorithm should be redesigned to build the whole recursive list first.
>> What do you think?
>>
>> 2014-10-29 10:07 GMT+03:00 akshay gupta <[email protected]>:
>>
>>> Hi,
>>>
>>> I am trying to mirror files using mirror:sort-by option set to
>>> 'date-desc'. My  file structure is like this
>>>
>>> $ ls -l -R
>>> total 3191264
>>> drwxr-xr-x  3 vikasgoyal  biadmin         102 Oct 22 12:35 1
>>> -rw-r--r--  1 vikasgoyal  biadmin           0 Aug  1 01:00 tmp.txt
>>> -rw-r--r--  1 vikasgoyal  biadmin           0 Aug  1 05:31 tmp1.txt
>>> -rw-r--r--  1 vikasgoyal  biadmin   263940738 Oct 28 13:51 abc.csv
>>>
>>> ./1:
>>> total 0
>>> drwxr-xr-x  4 vikasgoyal  biadmin  136 Oct 28 14:19 2
>>>
>>> ./1/2:
>>> total 515512
>>> -rw-r--r--  1 vikasgoyal  biadmin          0 Oct 28 14:19 3
>>> -rw-r--r--  1 vikasgoyal  biadmin  263940732 Oct 28 14:03 bc.csv
>>>
>>>
>>> I expect the file synch order should be
>>>
>>> ~/tmp.txt
>>> ~/tmp1.txt
>>> ~/abc.csv
>>> ~/1/2/bc.csv
>>> ~/1/2/3
>>>
>>>
>>> but the synch order is:
>>> ~/tmp.txt
>>> ~/tmp1.txt
>>> ~/1/2/bc.csv
>>> ~/1/2/3
>>> ~/abc.csv
>>>
>>>
>>> I think its the folder modification time that is causing this issue.
>>> Lftp considers folder synch time to decide the order even though files
>>> contained in the folder are newer then the remaining files in the parent
>>> folder. File synch order should be in order of the modification time
>>> irrespective of the folder structure. Please help.
>>>
>>> Thanks,
>>> Akshay
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>>    Alexander.
>>
>
>

--f46d041827ea599bb30506a1feed
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">lftp have options like=C2=A0<span style=3D"color:rgb(0,0,0=
)">--depth-first and=C2=A0</span><span style=3D"color:rgb(0,0,0)">--no-recu=
rsion I think it will make sense to have an option like =C2=A0&quot; --iter=
ative&quot; for iterative file transfer. This option will fetch the full fi=
le list and then fetch them iteratively.=C2=A0</span></div><div class=3D"gm=
ail_extra"><br><div class=3D"gmail_quote">On Wed, Oct 29, 2014 at 2:29 PM, =
akshay gupta <span dir=3D"ltr">&lt;<a href=3D"mailto:[email protected]=
om" target=3D"_blank">[email protected]</a>&gt;</span> wrote:<br><bl=
ockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #=
ccc solid;padding-left:1ex"><div dir=3D"ltr">I think the functionality of m=
irror:sort-by option implies that the order is for all the files irrespecti=
ve of the directory structure. Or may be it should support both the options=
 and it would be configuration defined which one to use.</div><div class=3D=
"HOEnZb"><div class=3D"h5"><div class=3D"gmail_extra"><br><div class=3D"gma=
il_quote">On Wed, Oct 29, 2014 at 2:09 PM, Alexander Lukyanov <span dir=3D"=
ltr">&lt;<a href=3D"mailto:[email protected]" target=3D"_blank">lavv17f@gma=
il.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"=
margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"=
ltr">Mirror in lftp works directory by directory. It does not build full li=
st of files first, so it cannot sort the whole list. Maybe the mirror algor=
ithm should be redesigned to build the whole recursive list first. What do =
you think?<br></div><div class=3D"gmail_extra"><div><div><br><div class=3D"=
gmail_quote">2014-10-29 10:07 GMT+03:00 akshay gupta <span dir=3D"ltr">&lt;=
<a href=3D"mailto:[email protected]" target=3D"_blank">akshaygupta20=
[email protected]</a>&gt;</span>:<br><blockquote class=3D"gmail_quote" style=3D"m=
argin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"l=
tr">Hi,<div><br></div><div>I am trying to mirror files using mirror:sort-by=
 option set to &#39;date-desc&#39;. My =C2=A0file structure is like this=C2=
=A0</div><div><br></div><div><div>$ ls -l -R</div><div>total 3191264</div><=
div>drwxr-xr-x =C2=A03 vikasgoyal =C2=A0biadmin =C2=A0 =C2=A0 =C2=A0 =C2=A0=
 102 Oct 22 12:35 1</div><div>-rw-r--r-- =C2=A01 vikasgoyal =C2=A0biadmin =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0 Aug =C2=A01 01:00 tmp.txt</div><div>-r=
w-r--r-- =C2=A01 vikasgoyal =C2=A0biadmin =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 0 Aug =C2=A01 05:31 tmp1.txt</div><div>-rw-r--r-- =C2=A01 vikasgoyal =
=C2=A0biadmin =C2=A0 263940738 Oct 28 13:51 abc.csv</div><div><br></div><di=
v>./1:</div><div>total 0</div><div>drwxr-xr-x =C2=A04 vikasgoyal =C2=A0biad=
min =C2=A0136 Oct 28 14:19 2</div><div><br></div><div>./1/2:</div><div>tota=
l 515512</div><div>-rw-r--r-- =C2=A01 vikasgoyal =C2=A0biadmin =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A00 Oct 28 14:19 3</div><div>-rw-r--r-- =C2=A01 vikas=
goyal =C2=A0biadmin =C2=A0263940732 Oct 28 14:03 bc.csv</div></div><div><br=
></div><div><br></div><div>I expect the file synch order should be=C2=A0</d=
iv><div><br></div><div>~/tmp.txt</div><div>~/tmp1.txt</div><div>~/abc.csv</=
div><div>~/1/2/bc.csv</div><div>~/1/2/3</div><div><br></div><div><br></div>=
<div>but the synch order is:</div><div><div>~/tmp.txt</div><div>~/tmp1.txt<=
/div></div><div>~/1/2/bc.csv</div><div>~/1/2/3</div><div>~/abc.csv</div><di=
v><br></div><div><br></div><div>I think its the folder modification time th=
at is causing this issue. Lftp considers folder synch time to decide the or=
der even though files contained in the folder are newer then the remaining =
files in the parent folder. File synch order should be in order of the modi=
fication time irrespective of the folder structure. Please help.</div><div>=
<br></div><div>Thanks,</div><div>Akshay</div><div><br></div><div><br></div>=
<div><br></div><div><br></div><div><br></div></div>
</blockquote></div><br><br clear=3D"all"><br></div></div><span><font color=
=3D"#888888">-- <br><div dir=3D"ltr"><div></div>=C2=A0=C2=A0 Alexander.<br>=
</div>
</font></span></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>

--f46d041827ea599bb30506a1feed--

--===============1617102312294162826==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
lftp mailing list
[email protected]
http://univ.uniyar.ac.ru/mailman/listinfo/lftp

--===============1617102312294162826==--