What is wadokujt for? and: Some Scripts for Chinese and Japanese learning and Noxon Audio
Marc Waeckerlin <[email protected]>
| Newsgroups | gmane.linux.suse.m17n |
|---|---|
| Organization | Siemens Schweiz AG |
| Message-ID | <[email protected]> |
In SuSE 9.3, there's a package wadokujt with a Japanese German dictionary. But it comes with no Program, only data is contained in the package. What programs in the SuSE distribution are thought to be used with wadokujt? KWordQuiz, KVTML: I use it together with KWordQuiz, but for this, I need to convert the data to kvtml. That's why I wrote wadokujt2kvtml.pl. There is another very similar dictionary file, named CEDICT for Chinese English translation. Unfortunately that does not come with SuSE. That file can be converted to kvtml using cedict2kvtml.pl. Noxon and Twonkyvision: All my CDs are stored in MP3 files, served via UPnP using Twonkyvision and played by a Noxon audio device. This works fine, but UTF-8 is not handled correctly. All German Umlauts and all Chinese music files are unreadable. Fortunately, on the unicode.org page, there's a file Unihan.txt, which defines the Chinese Unicode mapping, including the Mandarin PinYin translation. My script create-mapping.sh downloads this file and extracts the PinYin mapping I am interested in. The output from there is then used by utf8-to-ascii.pl, which converts all Chinese characters, German umlauts, French, Spanish, Italian accents to plain 7bit ASCII. I use this script in create-mp3-ascii-dir.sh (which is only an example and works only on my system), to create links to all my MP3 filenames, which are then 7bit ASCII and display well on Noxon. Another possible application: If you also consider the PinYin tones in utf8-to-ascii.pl (as they are considered in cedict2kvtml.pl), then you could easily build a Chinese text to speech synthesizer! - Or semi-automated translation: Translate the individual charaters to English using Unihan.txt. Is this also useful to others? Shall I make a webpage containing this information and scripts? Regards Marc -- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
cedict2kvtml.pl
(application/x-perl, 2.2 KB)
#! /usr/bin/perl
## @file
##
## $Id: cedict2kvtml.pl,v 1.1 2005/05/18 12:34:52 marc Exp $
##
## $Date: 2005/05/18 12:34:52 $
## $Author: marc $
##
## @copy © Marc Wäckerlin
## @license LGPL, see file <a href="license.html">COPYING</a>
##
## This program reads an UTF-8 CEDICT file from
## http://www.mandarintools.com/cedict.html on STDIN and writes a
## KVTML XML file for use e.g. in Linux KDE KWordQuiz to STDOUT.
##
## $Log: cedict2kvtml.pl,v $
## Revision 1.1 2005/05/18 12:34:52 marc
## initial release
##
##
## 1 2 3 4 5 6 7 8
## 45678901234567890123456789012345678901234567890123456789012345678901234567890
print '<?xml version="1.0"?>
<!DOCTYPE kvtml SYSTEM "kvoctrain.dtd">
<kvtml
encoding="UTF-8"
generator="xedict2kvtml.pl"
title="中英辞典 CEDICT"
author="http://www.mandarintools.com/cedict.html">
<lesson>
<desc no="1">中英辞典 CEDICT</desc>
</lesson>
';
%pymap = (
'a1' => 'ā',
'a2' => 'á',
'a3' => 'ǎ',
'a4' => 'à',
'a5' => 'a',
'e1' => 'ē',
'e2' => 'é',
'e3' => 'ě',
'e4' => 'è',
'e5' => 'e',
'i1' => 'ī',
'i2' => 'í',
'i3' => 'ǐ',
'i4' => 'ì',
'i5' => 'i',
'o1' => 'ō',
'o2' => 'ó',
'o3' => 'ǒ',
'o4' => 'ò',
'o5' => 'o',
'u1' => 'ū',
'u2' => 'ú',
'u3' => 'ǔ',
'u4' => 'ù',
'u5' => 'u',
'ü1' => 'ǖ',
'ü2' => 'ǘ',
'ü3' => 'ǚ',
'ü4' => 'ǜ',
'ü5' => 'ü',
);
while (<>) {
s/&/&/g;
s/</</g;
s/>/>/g;
if (/^\s*(\S+) (\S+) \[(\w*)\] \/(.+)\/\s*$/) {
my ($traditional, $simplified, $pinyin, @english) =
($1, $2, $3, split(/\//, $4));
if ($pinyin =~ m/([^aeiouü]*)([aeiouü])(\D*)(\d)/) {
$pinyin = $1.$pymap{$2.$4}.$3;
}
print '
<e m="1" s="1">
<o l="中文" p="'.$pinyin.'">'.$simplified.($simplified ne $traditional ? " ($traditional)":"").'</o>
<t l="English" p="'.$pinyin.'">['.$pinyin.'] '.join('; ', @english).'</t>
</e>
';
}
}
print ''.$pinyin.'
</kvtml>
';
wadokujt2kvtml.pl
(application/x-perl, 1.3 KB)
#! /usr/bin/perl
## @file
##
## $Id: cedict2kvtml.pl,v 1.1 2005/05/18 12:34:52 marc Exp $
##
## $Date: 2005/05/18 12:34:52 $
## $Author: marc $
##
## @copy © Marc Wäckerlin
## @license LGPL, see file <a href="license.html">COPYING</a>
##
## This program reads an UTF-8 WADOKUJT file from
## http://www.mandarintools.com/cedict.html on STDIN and writes a
## KVTML XML file for use e.g. in Linux KDE KWordQuiz to STDOUT.
##
## $Log: cedict2kvtml.pl,v $
## Revision 1.1 2005/05/18 12:34:52 marc
## initial release
##
##
## 1 2 3 4 5 6 7 8
## 45678901234567890123456789012345678901234567890123456789012345678901234567890
print '<?xml version="1.0"?>
<!DOCTYPE kvtml SYSTEM "kvoctrain.dtd">
<kvtml
encoding="UTF-8"
generator="wadokujt2kvtml.pl"
title="日本語"
author="。。。">
<lesson>
<desc no="1">日本語</desc>
</lesson>
';
while (<>) {
s/&/&/g;
s/</</g;
s/>/>/g;
if (/^\s*(\S+)\s+\[(\S*)\]\s+\/(.+)\/\s*$/) {
my ($japanese, $spell, @german) =
($1, $2, split(/\//, $3));
print '
<e m="1" s="1">
<o l="日本語" p="'.$spell.'">'.$japanese."\n".'['.$spell.']</o>
<t l="Deutsch" p="'.$spell.'">'.join('; ', @german).'</t>
</e>
';
}
}
print '
</kvtml>
';
create-mapping.sh
(application/x-shellscript, 335 B)
#! /bin/bash # download unihan.txt from ftp://ftp.unicode.org/Public/UNIDATA/Unihan.zip wget ftp://ftp.unicode.org/Public/UNIDATA/Unihan.zip && \ unzip Unihan.zip || \ ( \ echo "cannot get Unihan.txt from ftp://ftp.unicode.org/Public/UNIDATA/Unihan.zip" 1>&2; \ exit 1 \ ) grep kMandarin Unihan.txt | cut -f1,3 > mapping.txt
create-mp3-ascii-dir.sh
(application/x-shellscript, 485 B)
#! /bin/bash
# clean up and initialize
cd /home/data
sudo rm -rf mp3.ascii
sudo mkdir mp3.ascii
sudo chown marc mp3.ascii
cd mp3
# create all directories
find -mindepth 1 -type d -print0 | ~/pro/chinese-spelling/utf8-to-ascii.pl | xargs -0 -i mkdir ../mp3.ascii/{}
# link the files
IFS="
"
for file in `find -type f`; do
targetname=`echo ../mp3.ascii/$file | ~/pro/chinese-spelling/utf8-to-ascii.pl`
echo "CREATING: "$targetname
ln -s /home/data/mp3/$file$IFS$targetname
done
utf8-to-ascii.pl
(application/x-perl, 1.7 KB)
#! /usr/bin/perl
## @file
##
## $Id: utf8-to-ascii.pl,v 1.1 2005/05/17 22:03:29 marc Exp $
##
## $Date: 2005/05/17 22:03:29 $
## $Author: marc $
##
## @copy © Marc Wäckerlin
## @license LGPL, see file <a href="license.html">COPYING</a>
##
## $Log: utf8-to-ascii.pl,v $
## Revision 1.1 2005/05/17 22:03:29 marc
## initial release
##
##
## 1 2 3 4 5 6 7 8
## 45678901234567890123456789012345678901234567890123456789012345678901234567890
use File::Basename;
use utf8;
binmode STDOUT, ":utf8";
binmode STDIN, ":utf8";
%map = (
# (swiss-) german
"ä" => "ae",
"ö" => "oe",
"ü" => "ue",
"Ä" => "Ae",
"Ö" => "Oe",
"Ü" => "Ue",
"ß" => "s",
# swiss german
"ë" => "e",
"Ë" => "E",
# french, spanish, italian and other
"à" => "a",
"á" => "a",
"â" => "a",
"è" => "e",
"é" => "e",
"ê" => "e",
"ì" => "i",
"í" => "i",
"î" => "i",
"ò" => "o",
"ó" => "o",
"ô" => "o",
"ù" => "u",
"ú" => "u",
"û" => "u",
# french
"ç" => "c",
# spanish
"ñ" => "n"
);
# chinese
open(FILE, dirname($0).'/mapping.txt') or
die 'cannot open '.dirname($0).'/mapping.txt, call create-mapping.sh first!';
while (<FILE>) {
if (/$U\+([0-9A-F]+)\s+[\d\s]*([A-Za-z]+)/) {
$map{chr(hex($1))} = ucfirst(lc($2));
}
}
close(FILE);
while (<>) {
foreach $character (split(//, $_)) {
if (exists($map{$character})) {
print $map{$character};
} else {
print $character;
}
}
}