[PATCH] Support III "extended" characters in MARC::Charset
[email protected] ("Galen Charlton")
| Newsgroups | perl.perl4lib |
|---|---|
| Message-ID | <[email protected]> |
Hi, The attached patch adds mappings for various non-standard (per MARC-8) characters that III Millennium has been observed to stick into the EACC range. The following characters are now handled - listed are "EACC" codepoint, character name, and the UCS mapping. 21203d - HORIZONTAL ELLIPSIS - U+2026 212040 - LEFT DOUBLE QUOTATION MARK - U+201C 7f2014 - EM DASH - U+2014 7f2019 - RIGHT SINGLE QUOTATION MARK - U+2019 7f2020 - RIGHT DOUBLE QUOTATION MARK - U+201D 7f2122 - TRADE MARK SIGN - U+2122 I suspect there are more of these; if anybody has any additional character mappings to share or can point me to a complete public list of them, I would greatly appreciate it. Regards, Galen -- Galen Charlton Koha Application Developer LibLime [email protected] p: 1-888-564-2457 x709
marc-charset-iii-characters.patch
(application/octet-stream, 5 KB)
diff -u -8 -p -r1.11 Makefile.PL
--- Makefile.PL 27 Dec 2005 22:44:46 -0000 1.11
+++ Makefile.PL 16 Apr 2008 19:45:20 -0000
@@ -27,9 +27,10 @@ WriteMakefile(
sub build_db
{
eval('use lib "lib"; use MARC::Charset::Compiler');
return if $@;
print "compiling marc8/utf8 database, please be patient\n";
my $compiler = MARC::Charset::Compiler->new();
$compiler->compile('etc/codetables.xml');
+ $compiler->compile('etc/additional-iii-characters.xml');
}
diff -N etc/additional-iii-characters.xml
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ etc/additional-iii-characters.xml 16 Apr 2008 19:45:20 -0000
@@ -0,0 +1,56 @@
+<?xml version="1.0"?>
+<codeTables>
+ <codeTable name="East Asian" number="9">
+ <characterSet name="Chinese, Japanese, Korean (EACC)" ISOcode="31">
+ <grouping name="III additional characters" date="Apri 15, 2008" number="9.9">
+ <note>
+ <p>This table contains non-standard character mappings used by the Innovative
+ Interfaces, Inc. (III) ILS to represent such things as smart quotation marks
+ in MARC-8.
+ </p>
+ <p>FIXME: This list is based on comparing MARC-8 records extracted from a III Millennium
+ database with their presentation in the III OPAC. Consequently, this list is
+ incomplete - corrections or a pointer to public documentation of these characters
+ would be gratefully accepted.
+ </p>
+ </note>
+ <code>
+ <marc>21203D</marc>
+ <ucs>2026</ucs>
+ <utf-8>E280A6</utf-8>
+ <name>HORIZONTAL ELLIPSIS</name>
+ </code>
+ <code>
+ <marc>212040</marc>
+ <ucs>201C</ucs>
+ <utf-8>E2809C</utf-8>
+ <name>LEFT DOUBLE QUOTATION MARK</name>
+ </code>
+ <code>
+ <marc>7F2014</marc>
+ <ucs>2014</ucs>
+ <utf-8>E28094</utf-8>
+ <name>EM DASH</name>
+ </code>
+ <code>
+ <marc>7F2019</marc>
+ <ucs>2019</ucs>
+ <utf-8>E28099</utf-8>
+ <name>RIGHT SINGLE QUOTATION MARK</name>
+ </code>
+ <code>
+ <marc>7F2020</marc>
+ <ucs>201D</ucs>
+ <utf-8>E2809D</utf-8>
+ <name>RIGHT DOUBLE QUOTATION MARK</name>
+ </code>
+ <code>
+ <marc>7F2122</marc>
+ <ucs>2122</ucs>
+ <utf-8>E284A2</utf-8>
+ <name>TRADE MARK SIGN</name>
+ </code>
+ </grouping>
+ </characterSet>
+ </codeTable>
+</codeTables>
diff -N t/iii.t
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ t/iii.t 16 Apr 2008 19:45:20 -0000
@@ -0,0 +1,30 @@
+use strict;
+use warnings;
+
+use Test::More tests => 1;
+use MARC::Charset qw(marc8_to_utf8);
+use MARC::Charset::Constants qw(:all);
+
+my $marc8 =
+ 'a ' .
+ ESCAPE . MULTI_G0_A . CJK . # escape to CJK for G0
+ chr(0x21) . chr(0x20) . chr(0x3d) . # horizontal ellipsis
+ chr(0x21) . chr(0x20) . chr(0x40) . # left double quotation mark
+ chr(0x7f) . chr(0x20) . chr(0x14) . # em dash
+ chr(0x7f) . chr(0x20) . chr(0x19) . # right single quotation mark
+ chr(0x7f) . chr(0x20) . chr(0x20) . # right double quotation mark
+ chr(0x7f) . chr(0x21) . chr(0x22) . # trade mark sign
+ ESCAPE . SINGLE_G0_A . BASIC_LATIN . # back to latin
+ ' z';
+
+my $expected = 'a '.
+ chr(0x2026) .
+ chr(0x201c) .
+ chr(0x2014) .
+ chr(0x2019) .
+ chr(0x201d) .
+ chr(0x2122) .
+ ' z';
+is($expected, marc8_to_utf8($marc8), 'III non-standard');
+
+