Improving the performance of "mingw-get update"

Guys,

At present, the performance of "mingw-get update" is fairly abysmal; the 
bottleneck is the round trip time to set up individual downloads of 100+ 
small files, many of which will simply duplicate copies already present 
on the user's machine, and are therefore redundant.

To alleviate this, I'm exploring a technique whereby mingw-get, after 
having downloaded the top level package-list.xml.lzma, will be able to 
identify the redundancies in the additional catalogue files which are 
subsequently referenced, without actually downloading the redundant 
files.  This will rely on dynamically capturing the issue stamp of each 
referenced catalogue, within the referring package list, at mingw-dist 
"make" time, so that mingw-get can compare this with the issue stamp of 
any local copy, before downloading a redundant duplicate, and so being 
able to skip such downloads; the mingw-dist part of the implementation 
would be something along the lines of the attached patch.

Do note that this will represent a significant infrastructure update, 
(hence the major version number bump), requiring all of you who have 
existing working copies of mingw-dist to pull the update, run autoconf 
again, in the top source directory of your mingw-dist tree, followed by 
running configure again, in your corresponding top build directory.

Any comments, before I commit?

-- 
Regards,
Keith.

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter

_______________________________________________
MinGW-dvlpr mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr