| Newsgroups |
gmane.editors.sed.user |
| Message-ID |
<[email protected]> |
With a slight modification in the previous code, we bring out
the "ID" field to the front & then perform the various comparisons
and finally chop it away when the time comes to write it.
the "ID" field is supposed to be the 2nd column in this case.
And the number of fields is immaterial , don't know how you
got the idea that they be 7 for my code :-\
sed -e '
1{
# read in the 2015.txt file
:f2015
s/;/&\n/2;s/;/&\n/1;s/\(.*\)\n\(.*\)\n\(.*\)/\2\1\2\3/
H;1h;N;s/.*\n//
/^\*\**$/!bf2015
d
# now the 2015.txt file is in the hold space
}
s/;/&\n/2;s/;/&\n/1;s/\(.*\)\n\(.*\)\n\(.*\)/\2\1\2\3/
G
# same
/^\([^\n]*\)\n\(.*\n\)\1\n/{
s/^\([^\n]*\)\n\(.*\n\)\1\n/\2/;bholdit
}
/^\([^\n]*\)\n.*\n\1$/{
s/^\([^\n]*\)\n\(.*\)\n\1$/\2/;bholdit
}
/^\([^\n]*\)\n\1\n/{
s/^\([^\n]*\)\n\1\n//;bholdit
}
/^\([^\n]*\)\n\1$/{
s/^\([^\n]*\)\n\1$//;bholdit
}
# modified
/^\("[1-9][0-9]*"\);[^\n]*\n.*\n\1;/{
s/^\(\("[1-9][0-9]*"\);[^\n]*\n.*\n\)\2;[^\n]*\n/\1/;tmodified
s/^\(\("[1-9][0-9]*"\);[^\n]*\n.*\n\)\2;[^\n]*$/\1/;bmodified
}
/^\("[1-9][0-9]*"\);[^\n]*\n\1;/{
s/^\(\("[1-9][0-9]*"\);[^\n]*\n\)\2;[^\n]*\n/\1/;tmodified
s/^\(\("[1-9][0-9]*"\);[^\n]*\n\)\2;[^\n]*$/\1/
:modified
h;s/\n.*//;s/^[^;]*;//;s/./&/w /tmp/modify_from_2015.log
g;s/^[^\n]*\n//
bholdit
}
# new line
s/\n.*//;s/^[^;]*;//;s/./&/w /tmp/new_in_2016.log
$!d
:end
g;:a
s/\n[^;]*;//;ta
s/^[^;]*;//;b
:holdit
h;$bend;d
' 2015.txt 2016.txt > /tmp/removed_from_2015.log
Here's a perl version for the same:
perl -Mvars='%h,$prev,$ID_COL' -wMstrict -F'/"(?:;")?/' -lane '
BEGIN{
open MODIFY, ">", "/tmp/log/modify_from_2015.log1" or die "$!";
open FRESH, ">", "/tmp/log/new_in_2016.log1" or die "$!";
open REMOVE, ">", "/tmp/log/removed_from_2015.log1" or die "$!";
$ID_COL = 2; --$ID_COL;
}
shift @F;
next if @F < 2;
if ( $. == 1 ) {
# 2015 lines here
$prev = $ARGV;
$h{ $F[$ID_COL] } = $_;
} elsif ( $ARGV ne $prev ) {
# 2016 lines here
if ( ! exists $h{ $F[$ID_COL] } ) {
print FRESH $_;
} elsif( $h{ $F[$ID_COL] } ne $_ ) {
my @F_2015 = split /"(?:;")?/, $h{ $F[$ID_COL] };
shift @F_2015;
my $i=0;
for(@F) {
s/(.*)/\e[31m$1\e[0m/ if $F_2015[$i++] ne $_;
}
print MODIFY q{"} . join(q{";"},@F) . q{"};
delete $h{ $F[$ID_COL] };
} else {
delete $h{ $F[$ID_COL] };
}
} else {
# 2015 lines here
$h{ $F[$ID_COL] } = $_;
}
END{
print REMOVE $_ for values %h;
close MODIFY or die "$!";
close FRESH or die "$!";
close REMOVE or die "$!";
}
' 2015.txt 2016.txt;
HTH
-Rakesh
---In [email protected], <rachid.mokrani@...> wrote :
Hi,
Many thanks for this answer. It work great with my file example that I sent.
If I understand your script,
your proposal seems to reflect a fixed number of columns.
In my sample file exemple, there are 7 columns. And the first column "id" has a unique number.
However, the production file has 39 columns. And this is the second column that have the "id" ( id is a unique number).
Is it possible to define in the script (eg a variable) that indicates the number of columns in the file.
And to specify in the script the number of the column "id" (eg a variable) - in my case in column 2.
These amendments would use this script without having to modify the column if the number of changes and not have to specify on which column we compare.
Many thanks again.
Best regards.
-----Message d'origine-----
De : [email protected] mailto:[email protected] [mailto:[email protected] mailto:[email protected]]
Envoyé : vendredi 8 janvier 2016 10:29
À : [email protected] mailto:[email protected]
Objet : RE: compare files
Since this is a "sed" editor forum, a sed-based solution is presented:
Note that you need to place a dummy line of all stars ****** as the last line of the 2015.txt file in order to distinguish the ending of 2015.txt & start of 2016.txt.
Assuming the GNU "sed" being used, for two main reasons, enough memory for hold space & [^\n] regular expression for a nonnewline to be matched.
Ordering of arguments is: 1st arg = 2015.txt 2nd arg = 2016.txt
sed -e '
1{
# read in the 2015.txt file
:f2015
$!N
/\n\*\**$/!bf2015
s///;h;d
# now the 2015.txt file is in the hold space
}
G
# same
/^\([^\n]*\)\n\(.*\n\)\1\n/{
s/^\([^\n]*\)\n\(.*\n\)\1\n/\2/;h;$bend;d
}
/^\([^\n]*\)\n.*\n\1$/{
s/^\([^\n]*\)\n\(.*\)\n\1$/\2/;h;$bend;d
}
/^\([^\n]*\)\n\1\n/{
s/^\([^\n]*\)\n\1\n//;h;$bend;d
}
/^\([^\n]*\)\n\1$/{
s/^\([^\n]*\)\n\1$//;h;$bend;d
}
# modified
/^\("[1-9][0-9]*"\);[^\n]*\n.*\n\1;/{
s/^\(\("[1-9][0-9]*"\);[^\n]*\n.*\n\)\2;[^\n]*\n/\1/;tmodified
s/^\(\("[1-9][0-9]*"\);[^\n]*\n.*\n\)\2;[^\n]*$/\1/
:modified
h;s/\n.*//;s/./&/w /tmp/modify_from_2015.log
g;s/^[^\n]*\n//;h;$bend;d
}
/^\("[1-9][0-9]*"\);[^\n]*\n\1;/{
s/^\(\("[1-9][0-9]*"\);[^\n]*\n\)\2;[^\n]*\n/\1/;tmodified
s/^\(\("[1-9][0-9]*"\);[^\n]*\n\)\2;[^\n]*$/\1/;bmodified
}
# new line
s/\n.*//;s/./&/w /tmp/new_in_2016.log
$!d
:end
g
' 2015.txt 2016.txt > /tmp/remove_from_2015.log
HTH
-Rakesh
[Non-text portions of this message have been removed]
------------------------------------
Posted by: sharma__r@... mailto:sharma__r@...
------------------------------------
--
------------------------------------
Yahoo Groups Links
__________________________
Avant d'imprimer, pensez à l'environnement ! Please consider the environment before printing !
Ce message et toutes ses pièces jointes sont confidentiels et établis à l'intention exclusive de ses destinataires. Toute utilisation non conforme à sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. IFP Energies nouvelles décline toute responsabilité au titre de ce message. This message and any attachments are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited. IFP Energies nouvelles should not be liable for this message.
__________________________
[Non-text portions of this message have been removed]