Re: identify ISSN numbers in an mrc file
[email protected] (Sergio Letuche) Wed, 2 Nov 2016 12:29:27 +0200
| Newsgroups | perl.perl4lib |
|---|---|
| Message-ID | <CANnh_k-qmx8HVDuqcd7dr3+Lza3mnH2Va8QBTSUrhofx7wT7TA@mail.gmail.com> |
--94eb2c1a1a2e66e6ba05404eea11 Content-Type: text/plain; charset=UTF-8 thank you very much 2016-11-02 12:28 GMT+02:00 Ben Soares <[email protected]>: > Hi Sergio, > > Try > > ^\d{4}-\d{3}[\dxX]$ > > if you know that they will always be formatted with a hyphen in the > middle, or > > ^\d{4}-?\d{3}[\dxX]$ > > if you can't be sure of that. > > (and if you're interested in spotting ISSNs in the middle of a field use > \b\d{4}-?\d{3}[\dxX]\b > but beware this also finds year ranges [e.g. 1990-2000]!) > > Ben > > > On Wednesday, 2 November 2016 12:06:15 GMT Sergio Letuche wrote: > > Thank you dear Stefano, > > > > i am aware of this module, it works great. > > > > But my problem is, what clever regex to use, in order to identify if a > > subfield's content, is an ISSN number. Say our mrc has ISSN numbers > thrown > > in any tag you could imagine... > > > > So my approach, would be, to search the whole mrc, but i do non know > which > > regex to use... > > > > 2016-11-02 11:52 GMT+02:00 Stefano Bargioni <[email protected]>: > > > Hi, Sergio: > > > you can try MARCgrep http://en.pusc.it/bib/MARCgrep. > > > Its help is: > > > > > > MARCgrep.pl > > > > > > Extracts MARC records that match a condition on fields. Count > and > > > invert are available. > > > > > > SYNOPSIS > > > > > > MARCgrep.pl [options] [-e condition] file.mrc > > > > > > Options: > > > -h print this help message and exit > > > -c count only > > > -e condition > > > -f comma separated list of fields to print > > > -o output format "marc" | "line" | "INLINE" > > > -s separator string for condition, default "," > > > -v invert match > > > > > > Condition: > > > -e 'tag,indicator1,indicator2,subfield,value' > > > > > > OPTIONS > > > > > > -h Print this message and exit. > > > > > > -c Count and print number of matching records > > > > > > -e The condition to match in the record. > > > > > > For data fields, the syntax is: > > > tag,indicator1,indicator2,subfield,value > > > > > > where tag, indicator1, indicator2, subfield, and value > are > > > > > > regular expressions patterns. > > > > > > Do not put spaces around the separators. > > > > > > For control fields, the syntax is: > > > tag,pos1,pos2,value > > > > > > where tag starts with '00' (use '000' or 'LDR' for > > > > > > leader), pos1 is the starting position, > > > > > > pos2 is the ending position, both 0-based. Value is a > > > > > > regular expression. > > > > > > Default condition (-e not specified) matches any data > > > > > > field. > > > > > > For control fields, only the tag is mandatory. > > > > > > Examples: -e '100,,,a,^A' will match records that > contain > > > > > > 100$a starting with 'A' > > > > > > -e '008,35,37,(ita|eng)' will match records > with > > > > > > language ita or eng in 008 > > > > > > -e '(1|7)(0|1)(0|1),,2' will match > > > > > > 100,110,111,700,710,711 with ind2=2 > > > > > > -f Comma separated list of fields (tags) to print if output > > > > > > format > > > > > > is "line" or "inline". Default is any field. > > > > > > Note that if a tag is preceded by '#' sign (like in > > > > > > '#nnn'), a > > > > > > count of occurrences will be printed instead. > > > > > > Examples: -f '100,245' will print field 100 and 245 > > > > > > -f '400,#400' will print all occurrences of > 400 > > > > > > field as well as the number of its occurrences > > > > > > -o Output format: "marc" for ISO2709, "line" for each > subfield > > > > > > in > > > > > > a line, "inline" (default) for each field in a line. > > > > > > -s Specify a string separator for condition. Default is > ','. > > > > > > -v Invert the sense of matching, to select non-matching > > > > > > records. > > > > > > -V Print the version and exit. > > > > > > file.mrc > > > > > > The mandatory ISO2709 file to read. Can be STDIN, '-'. > > > > > > DESCRIPTION > > > > > > Like grep, the famous Unix utility, MARCgrep.pl allows to filter > > > > > > MARC > > > > > > bibliographic > > > > > > records based on conditions on tag, indicators, and field > value. > > > > > > Conditions can be applied to data fields, control fields or the > > > > > > leader. > > > > > > In case of data fields, the condition can specify tag, > indicators, > > > subfield and value using regular > > > > > > expressions. In case of control fields, the condition must > contain > > > > > > the > > > > > > tag name, the starting > > > > > > and ending position (both 0-based), and a regular expressions > for > > > > > > the > > > > > > value. > > > > > > Options -c and -v allow respectively to count matching records > and > > > > > > to > > > > > > invert the match. > > > > > > If option -c is not specified, the output format can be "line" > or > > > "inline" (both human readable), > > > > > > or "marc" for MARC binary (ISO2709). For formats "line" or > > > > > > "inline", > > > > > > the -f option allows to specify > > > > > > fields to print. > > > > > > You can chain more conditions using > > > > > > ./MARCGgrep.pl -o marc -e condition1 file.mrc | ./MARCGgrep.pl > -e > > > condition2 - > > > > > > KNOWN ISSUES > > > > > > Performance. > > > > > > Accepts and returns only UTF-8. > > > > > > Checks are case sensitive. > > > > > > AUTHOR > > > > > > Pontificia Universita' della Santa Croce < > http://www.pusc.it/bib/> > > > > > > Stefano Bargioni <[email protected]> > > > > > > SEE ALSO > > > > > > marktriggs / marcgrep at <https://github.com/ > marktriggs/marcgrep> > > > > > > for > > > > > > filtering large data sets > > > > > > > > On 02 nov 2016, at 09:57, Sergio Letuche <[email protected]> > > > > > > wrote: > > > > Hello community, > > > > > > > > how would you treat the following? > > > > > > > > I need a way to identify all tags - subfields, that have stored an > ISSN > > > > > > number in them. > > > > > > > What would you suggest as a clever approach for this? > > > > > > > > Thank you > > > > -- > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > > --94eb2c1a1a2e66e6ba05404eea11 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">thank you very much</div><div class=3D"gmail_extra"><br><d= iv class=3D"gmail_quote">2016-11-02 12:28 GMT+02:00 Ben Soares <span dir=3D= "ltr"><<a href=3D"mailto:[email protected]" target=3D"_blank">ben.soar= [email protected]</a>></span>:<br><blockquote class=3D"gmail_quote" style=3D"m= argin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Sergio,<br= > <br> Try<br> <br> ^\d{4}-\d{3}[\dxX]$<br> <br> if you know that they will always be formatted with a hyphen in the middle,= or<br> <br> ^\d{4}-?\d{3}[\dxX]$<br> <br> if you can't be sure of that.<br> <br> (and if you're interested in spotting ISSNs in the middle of a field us= e<br> \b\d{4}-?\d{3}[\dxX]\b<br> but beware this also finds year ranges [e.g. 1990-2000]!)<br> <br> Ben<br> <div class=3D"HOEnZb"><div class=3D"h5"><br> <br> On Wednesday, 2 November 2016 12:06:15 GMT Sergio Letuche wrote:<br> > Thank you dear Stefano,<br> ><br> > i am aware of this module, it works great.<br> ><br> > But my problem is, what clever regex to use, in order to identify if a= <br> > subfield's content, is an ISSN number. Say our mrc has ISSN number= s thrown<br> > in any tag you could imagine...<br> ><br> > So my approach, would be, to search the whole mrc, but i do non know w= hich<br> > regex to use...<br> ><br> > 2016-11-02 11:52 GMT+02:00 Stefano Bargioni <<a href=3D"mailto:barg= [email protected]">[email protected]</a>>:<br> > > Hi, Sergio:<br> > > you can try MARCgrep <a href=3D"http://en.pusc.it/bib/MARCgrep" r= el=3D"noreferrer" target=3D"_blank">http://en.pusc.it/bib/MARCgrep</a><wbr>= .<br> > > Its help is:<br> > ><br> > > MARCgrep.pl<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 Extracts MARC records that match a con= dition on fields. Count and<br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 invert are available.<br> > ><br> > > SYNOPSIS<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 MARCgrep.pl [options] [-e condition] f= ile.mrc<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Options:<br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-h=C2=A0 =C2=A0print this= help message and exit<br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-c=C2=A0 =C2=A0count only= <br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-e=C2=A0 =C2=A0condition<= br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-f=C2=A0 =C2=A0comma sepa= rated list of fields to print<br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-o=C2=A0 =C2=A0output for= mat "marc" | "line" | "INLINE"<br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-s=C2=A0 =C2=A0separator = string for condition, default ","<br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-v=C2=A0 =C2=A0invert mat= ch<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Condition:<br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-e=C2=A0 'tag,indicat= or1,indicator2,<wbr>subfield,value'<br> > ><br> > > OPTIONS<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 -h=C2=A0 =C2=A0 =C2=A0 Print this mess= age and exit.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 -c=C2=A0 =C2=A0 =C2=A0 Count and print= number of matching records<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 -e=C2=A0 =C2=A0 =C2=A0 The condition t= o match in the record.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0For = data fields, the syntax is:<br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0tag,indicator1,indicator2,<wbr>subfield,value<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0wher= e tag, indicator1, indicator2, subfield, and value are<br> > ><br> > > regular expressions patterns.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Do n= ot put spaces around the separators.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0For = control fields, the syntax is:<br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0tag,pos1,pos2,value<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0wher= e tag starts with '00' (use '000' or 'LDR' for<br> > ><br> > > leader), pos1 is the starting position,<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pos2= is the ending position, both 0-based. Value is a<br> > ><br> > > regular expression.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Defa= ult condition (-e not specified) matches any data<br> > ><br> > > field.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0For = control fields, only the tag is mandatory.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Exam= ples: -e '100,,,a,^A' will match records that contain<br> > ><br> > > 100$a starting with 'A'<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-e '008,35,37,(ita|eng)' will match = records with<br> > ><br> > > language ita or eng in 008<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-e '(1|7)(0|1)(0|1),,2' will match<b= r> > ><br> > > 100,110,111,700,710,711 with ind2=3D2<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 -f=C2=A0 =C2=A0 =C2=A0 Comma separated= list of fields (tags) to print if output<br> > ><br> > > format<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 is "l= ine" or "inline". Default is any field.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Note= that if a tag is preceded by '#' sign (like in<br> > ><br> > > '#nnn'), a<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 count of o= ccurrences will be printed instead.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Exam= ples: -f '100,245' will print field 100 and 245<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-f '400,#400' will print all occurre= nces of 400<br> > ><br> > > field as well as the number of its occurrences<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 -o=C2=A0 =C2=A0 =C2=A0 Output format: = "marc" for ISO2709, "line" for each subfield<br> > ><br> > > in<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 a line, &q= uot;inline" (default) for each field in a line.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 -s=C2=A0 =C2=A0 =C2=A0 Specify a strin= g separator for condition. Default is ','.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 -v=C2=A0 =C2=A0 =C2=A0 Invert the sens= e of matching, to select non-matching<br> > ><br> > > records.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 -V=C2=A0 =C2=A0 =C2=A0 Print the versi= on and exit.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 file.mrc<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 The mandat= ory ISO2709 file to read. Can be STDIN, '-'.<br> > ><br> > > DESCRIPTION<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 Like grep, the famous Unix utility, MA= RCgrep.pl allows to filter<br> > ><br> > > MARC<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 bibliographic<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0records based on conditions on t= ag, indicators, and field value.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 Conditions can be applied to data fiel= ds, control fields or the<br> > ><br> > > leader.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 In case of data fields, the condition = can specify tag, indicators,<br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 subfield and value using regular<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0expressions. In case of control = fields, the condition must contain<br> > ><br> > > the<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 tag name, the starting<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0and ending position (both 0-base= d), and a regular expressions for<br> > ><br> > > the<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 value.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 Options -c and -v allow respectively t= o count matching records and<br> > ><br> > > to<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 invert the match.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 If option -c is not specified, the out= put format can be "line" or<br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 "inline" (both human readabl= e),<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0or "marc" for MARC bin= ary (ISO2709). For formats "line" or<br> > ><br> > > "inline",<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 the -f option allows to specify<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0fields to print.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 You can chain more conditions using<br= > > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 ./MARCGgrep.pl -o marc -e condition1 f= ile.mrc | ./MARCGgrep.pl -e<br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 condition2 -<br> > ><br> > > KNOWN ISSUES<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 Performance.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 Accepts and returns only UTF-8.<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 Checks are case sensitive.<br> > ><br> > > AUTHOR<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 Pontificia Universita' della Santa= Croce <<a href=3D"http://www.pusc.it/bib/" rel=3D"noreferrer" target=3D= "_blank">http://www.pusc.it/bib/</a>><br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 Stefano Bargioni <<a href=3D"mailto= :[email protected]">[email protected]</a>><br> > ><br> > > SEE ALSO<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 marktriggs / marcgrep at <<a href= =3D"https://github.com/marktriggs/marcgrep" rel=3D"noreferrer" target=3D"_b= lank">https://github.com/<wbr>marktriggs/marcgrep</a>><br> > ><br> > > for<br> > ><br> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 filtering large data sets<br> > > ><br> > > > On 02 nov 2016, at 09:57, Sergio Letuche <<a href=3D"mail= to:[email protected]">[email protected]</a>><br> > ><br> > > wrote:<br> > > > Hello community,<br> > > ><br> > > > how would you treat the following?<br> > > ><br> > > > I need a way to identify all tags - subfields, that have sto= red an ISSN<br> > ><br> > > number in them.<br> > ><br> > > > What would you suggest as a clever approach for this?<br> > > ><br> > > > Thank you<br> <br> <br> <br> </div></div><span class=3D"HOEnZb"><font color=3D"#888888">--<br> The University of Edinburgh is a charitable body, registered in<br> Scotland, with registration number SC005336.<br> <br> </font></span></blockquote></div><br></div> --94eb2c1a1a2e66e6ba05404eea11--