| Newsgroups |
gmane.editors.sed.user |
| Message-ID |
<[email protected]> |
Hi,
grep works fine with few lines for "Lines removed" and "Lines added"
My example file has only a few lines, while the original file is composed of several thousand lines. (file size is 400 MB)
The grep command seem to use a lot of memory, and the command (grep -v -f ids_2.x 2015.txt) stops after a while without giving result, it return killed (The machine on which I have been running this command has 64GB memory.)
Concerning the modified lines resul, we should not see the 2 lines appear in the resul, but only display the modified line of the new 2016.txt file.
Exp that I need :
Lines modified:
"111222334";"Company Z";"NAME A ";"FIRSTNAME 4";"JOB 1";"Town 1";"PHONE 1"
And it would be perfect if the field that is modified on a line could be highlighted - In my exemple "Company Z" in red for exemple.
Lines modified:
"111222334";"Company Z";"NAME A ";"FIRSTNAME 4";"JOB 1";"Town 1";"PHONE 1"
May be better to do this with sed/awk. I try it, but with no success yet..
Any help/idea.
Best regards.
-----Message d'origine-----
De : [email protected] [mailto:[email protected]]
Envoyé : mercredi 6 janvier 2016 22:52
À : [email protected]
Objet : Re: compare files
I think this does what you are looking for, at least for the test files:
$ cat ./script1.sh
cut -d ';' -f 1 2015.txt > ids_1.x
cut -d ';' -f 1 2016.txt > ids_2.x
echo
echo Lines removed:
echo
grep -v -f ids_2.x 2015.txt
echo
echo Lines added:
echo
grep -v -f ids_1.x 2016.txt
grep -f ids_1.x 2016.txt > recs_2_with_ids_1.x grep -f ids_2.x 2015.txt > recs_1_with_ids_2.x
echo
echo Lines modified:
echo
diff recs_1_with_ids_2.x recs_2_with_ids_1.x
$ ./script1.sh
Lines removed:
"118822333";"Company B";"NAME T ";"FIRSTNAME 1";"JOB 2";"Town 2";"PHONE 65"
Lines added:
"991275333";"Company Y";"NAME M ";"FIRSTNAME 2";"JOB 7";"Town 1";"PHONE 6"
"881275333";"Company Y";"NAME M ";"FIRSTNAME 25";"JOB 7";"Town 1";"PHONE 6"
Lines modified:
3c3
< "111222334";"Company A";"NAME A ";"FIRSTNAME 4";"JOB 1";"Town 1";"PHONE 1"
---
> "111222334";"Company Z";"NAME A ";"FIRSTNAME 4";"JOB 1";"Town 1";"PHONE 1"
Daniel
On 1/6/2016 4:27 AM, MOKRANI Rachid wrote:
> Hi, (and happy new year !)
>
>
> I have to compare 2 files. These 2 files have many lines/columns. Each column may have changed, and each can contain identical information.
>
>
> Only the "id" column is unique. The "Id" number is unique. This number is single one time, there was never duplicates.
>
>
> My file 2015.txt
>
> "id";"Company";"NAME ";"FIRSTNAME";"JOB";"Town";"PHONE"
> "111222333";"Company A";"NAME A ";"FIRSTNAME 1";"JOB 1";"Town 1";"PHONE 1"
> "111222334";"Company A";"NAME A ";"FIRSTNAME 4";"JOB 1";"Town 1";"PHONE 1"
> "118822333";"Company B";"NAME T ";"FIRSTNAME 1";"JOB 2";"Town 2";"PHONE 65"
> "111275333";"Company C";"NAME M ";"FIRSTNAME 2";"JOB 7";"Town 1";"PHONE 6"
>
>
> My new file 2016.txt
>
> "id";"Company";"NAME ";"FIRSTNAME";"JOB";"Town";"PHONE"
> "111222333";"Company A";"NAME A ";"FIRSTNAME 1";"JOB 1";"Town 1";"PHONE 1"
> "111222334";"Company Z";"NAME A ";"FIRSTNAME 4";"JOB 1";"Town 1";"PHONE 1"
> "111275333";"Company C";"NAME M ";"FIRSTNAME 2";"JOB 7";"Town 1";"PHONE 6"
> "991275333";"Company Y";"NAME M ";"FIRSTNAME 2";"JOB 7";"Town 1";"PHONE 6"
> "881275333";"Company Y";"NAME M ";"FIRSTNAME 25";"JOB 7";"Town 1";"PHONE 6"
>
> When I receive the new file 2016.txt I would like to identify all the
> differences with the first file 2015.txt Wich lines are remove,
> modify, and wich lines are new in the file 2016.txt
>
> For exemple I would like to have 3 files.
>
> Compare 2015.txt and 2016.txt to print what lines are remove.
> Remove_from _2015.txt need to have the result:
> "118822333";"Company B";"NAME T ";"FIRSTNAME 1";"JOB 2";"Town 2";"PHONE 65"
>
>
> Compare 2015.txt and 2016.txt to print what is modify.
> Lines_modify_from_2015.txt
> "111222334";"Company Z";"NAME A ";"FIRSTNAME 4";"JOB 1";"Town 1";"PHONE 1"
>
>
> Compare 2015.txt and 2016.txt to print what lines are new.
> New_Lines_2016.txt
> "991275333";"Company Y";"NAME M ";"FIRSTNAME 2";"JOB 7";"Town 1";"PHONE 6"
> "881275333";"Company Y";"NAME M ";"FIRSTNAME 25";"JOB 7";"Town 1";"PHONE 6"
>
>
> PS : I use sed/awk on CentOS.
>
> Any help,
> Many thanks in advance.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> __________________________
>
> Avant d'imprimer, pensez à l'environnement ! Please consider the environment before printing !
>
> Ce message et toutes ses pièces jointes sont confidentiels et établis à l'intention exclusive de ses destinataires. Toute utilisation non conforme à sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. IFP Energies nouvelles décline toute responsabilité au titre de ce message. This message and any attachments are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited. IFP Energies nouvelles should not be liable for this message.
>
> __________________________
>
>
> [Non-text portions of this message have been removed]
>
>
>
>
------------------------------------
Posted by: Daniel Goldman <[email protected]>
------------------------------------
--
------------------------------------
Yahoo Groups Links
__________________________
Avant d'imprimer, pensez à l'environnement ! Please consider the environment before printing !
Ce message et toutes ses pièces jointes sont confidentiels et établis à l'intention exclusive de ses destinataires. Toute utilisation non conforme à sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. IFP Energies nouvelles décline toute responsabilité au titre de ce message. This message and any attachments are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited. IFP Energies nouvelles should not be liable for this message.
__________________________