Re: question why it does work on not changed files here?
Bram Moolenaar <[email protected]> Sat, 03 Dec 2005 14:58:25 +0100
| Newsgroups | gmane.comp.tools.aap.user |
|---|---|
| Message-ID | <[email protected]> |
Calmar wrote: > ---[ recipe_1 ]------------- >=20 > dir =3D /home/calmar > all : $dir/file > $dir/file : ./file > :sys cp $source $dir >=20 > ---------------------------- >=20 > here, when I change ./file, aap executes the :sys call. >=20 >=20 > on recipe_2, I changed two things. The `dir' variable and the I > replaced `cp' with `curl -T' >=20 > ---[ recipe_2 ]--------------------------------------------- >=20 > dir =3D ftp://n4a403:[email protected]/html/test/ > all : $dir/file > $dir/file : ./file > :sys curl -T $source $dir >=20 > ------------------------------------------------------------ >=20 > Here it executes the :sys command always, even when ./file has > not changed.=20 >=20 > I assume aap first computes the source-file signature (located > in the targets AAPDIR folder `sign' file, obviously). Then, if > that is the same as it was, it will not do anything. ok. >=20 > In the recipe_2, why can AAP not do the same? checking the > (local) source files signature, and when it has changed, it > executes the :sys command? When not, nothing shall happen. There > is not really a need for checking the target(remote) file, since > it's only a target, isn 'it? >=20 > I can only see, that aap does not build that AAPDIR dir on the > server (target Directory) with the source-file's sign. (as it > does it on the first example). So AAP is plainly not able to > store the local sign file, and therefore is forced to execute > the :sys command all the time (because there is no reference > (sign file). That's why the publish attribute should be used for > handling such remote things? >=20 > is that correct in some way? >=20 > At least, it's that would make sense to me. There is another reason to update a file. In the first recipe, if you delete $dir/file then the command will be executed too. For the remote file it's like Aap doesn't see it. You can see in the log file that the target is updated because it doesn't exist. Now, I do remember adding some code to check for remote files... Ah, apparently there is an error in figuring out the absolute name of a node. The current directory is prepended and there is no exception for a URL. After that it will no longer be recognized as a URL. Going through the code I found a few more places where a URL was handled as a local path, causing the URL to be changed. I'll fix these. Now the next problem is that the code that obtains a timestamp doesn't really work for ftp://. Thus the file can be found, but it doesn't have a timestamp. Then Aap handles it like it doesn't exist. I'll change this to pretend the file is very old. Now it seems to work for me. Try this patch: diff -ru ./DoBuild.py /home/mool/aap/Aap/Exec/DoBuild.py --- ./DoBuild.py Fri Aug 12 12:14:30 2005 +++ /home/mool/aap/Aap/Exec/DoBuild.py Sat Dec 3 13:24:01 2005 @@ -16,7 +16,7 @@ from RecPos import rpcopy, RecPos from Util import * from Depend import Depend, depend_auto -from Remote import url_time +from Remote import url_time, is_url from VersCont import fetch_nodelist from Cache import local_name from Sign import get_old_sign, get_new_sign, sign_clear, sign_clear_targ= et @@ -355,7 +355,11 @@ done =3D build_done_NOT for src_name in names: # Find the node after removing "./" things ($SRCPATH may cause i= t) - node =3D work.get_node(os.path.normpath(src_name), 1, src_dict) + if is_url(src_name): + norm_name =3D src_name + else: + norm_name =3D os.path.normpath(src_name) + node =3D work.get_node(norm_name, 1, src_dict) =20 # Check if we skip updating this item. if node.attributes.get("update") =3D=3D "no": @@ -416,8 +420,6 @@ else: nlevel =3D None =20 - from Remote import is_url - for src_dict in dictlist: # For an absolute path use the source name literally, otherwise = use # $SRCPATH to locate the source. @@ -497,6 +499,8 @@ node =3D src_dict["_node"] src_name =3D node.get_name() virtual =3D node.attributes.get("virtual") + elif is_url(src_dict["name"]): + src_name =3D src_dict["name"] else: src_name =3D os.path.normpath(src_dict["name"]) =20 diff -ru ./Node.py /home/mool/aap/Aap/Exec/Node.py --- ./Node.py Mon May 30 13:18:36 2005 +++ /home/mool/aap/Aap/Exec/Node.py Sat Dec 3 12:58:00 2005 @@ -110,10 +110,14 @@ # Remember the absolute path for the Node. When it's virtual ab= sname # should not be used! Use get_name() instead. # A URL, "~/" and "~user/" are also absolute. - if os.path.isabs(name) or is_url(name): + if os.path.isabs(name): self.name_relative =3D 0 if absname is None: absname =3D os.path.normpath(name) + elif is_url(name): + self.name_relative =3D 0 + if absname is None: + absname =3D name elif name[0] =3D=3D '~': self.name_relative =3D 0 if absname is None: diff -ru ./Remote.py /home/mool/aap/Aap/Exec/Remote.py --- ./Remote.py Mon May 30 13:24:26 2005 +++ /home/mool/aap/Aap/Exec/Remote.py Sat Dec 3 13:33:49 2005 @@ -89,7 +89,10 @@ up =3D urlopen(name) t =3D get_header_date(up.info()) up.close() + if t =3D=3D 1: + msg_info(recdict, _('"%s" can be found but has no timestamp'= ) % name) except: + msg_info(recdict, _('Could not obtain timestamp for "%s"') % nam= e) t =3D 0 return t =20 @@ -102,7 +105,9 @@ return time.mktime(parsedate(headers["Last-Modified"])) if headers.has_key("Date"): return time.mktime(parsedate(headers["Date"])) - return 0 + # When a file does exist but has no timestamp return 1, so that it's + # different from a file that does not exist. + return 1 =20 =20 def get_progname_rsync(recdict): diff -ru ./Util.py /home/mool/aap/Aap/Exec/Util.py --- ./Util.py Wed Jun 1 17:10:39 2005 +++ /home/mool/aap/Aap/Exec/Util.py Sat Dec 3 13:09:33 2005 @@ -542,6 +542,10 @@ If "dir" is not given, use the current directory. Prefers using "../" when part of "dir" matches. Also handles backslashes on non-posix systems""" + from Remote import is_url + if is_url(name): + return name # can't shorten a URL + adir =3D dir if adir is None: adir =3D os.getcwd() diff -ru ./Work.py /home/mool/aap/Aap/Exec/Work.py --- ./Work.py Mon Jul 11 11:48:35 2005 +++ /home/mool/aap/Aap/Exec/Work.py Sat Dec 3 13:23:26 2005 @@ -230,7 +230,10 @@ # Then check the short name, only for virtual nodes (may have be= en used # in another recipe). if absname is None: - absname =3D os.path.abspath(os.path.expanduser(name)) + if is_url(name): + absname =3D name + else: + absname =3D os.path.abspath(os.path.expanduser(name)) =20 # Might try again with a folded name if fname_fold() may return = a # different file name. This is just to optimize the speed. @@ -274,7 +277,10 @@ A new node is added to the global list if "add" is non-zero. When "dict" is given, check for attributes that apply to the Node.""" - absname =3D os.path.abspath(os.path.expanduser(name)) + if is_url(name): + absname =3D name + else: + absname =3D os.path.abspath(os.path.expanduser(name)) n =3D self.find_node(name, absname, use_alias =3D use_alias) if n is None: n =3D Node(name, absname) --=20 GALAHAD: No look, really, this isn't nescess ... PIGLET: We must examine you. GALAHAD: There's nothing wrong with ... that. "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURE= S LTD /// Bram Moolenaar -- [email protected] -- http://www.Moolenaar.net \= \\ /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ = \\\ \\\ download, build and distribute -- http://www.A-A-P.org = /// \\\ help me help AIDS victims -- http://www.ICCF.nl /= // ------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log fi= les for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_id=3D7637&alloc_id=3D16865&op=3Dclick