mdb-export encoding issue
Rob Hills <[email protected]> Sat, 10 Jun 2017 15:08:57 +0800
| Newsgroups | gmane.comp.db.mdb-tools.devel |
|---|---|
| Message-ID | <[email protected]> |
This is a multi-part message in MIME format.
--===============1345802907911637350==
Content-Type: multipart/alternative;
boundary="------------94ED894073A2D11254380A7B"
Content-Language: en-GB
This is a multi-part message in MIME format.
--------------94ED894073A2D11254380A7B
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hi,
I am debugging a project to convert a Forum from WebWiz to phpBB.=20
WebWiz stores its data in an Access DB and our phpBB forum will be on MyS=
QL.
I am using mdb-tools version 0.7.1 on Ubuntu 16.04LTS 64-bit. According
to mdb-ver, the mdb file I am working with is JET4.
The problem I am trying to solve involves forum post text that includes
some characters outside the basic character set. A specific example is
the "half space" character whose UTF-8 representation I believe is the
3-byte sequence E2 80 89. My problem is that when I use mdb-export,
these characters end up being converted to =C3=A2=E2=82=AC=E2=80=B0 (Hex:=
C3 A2 E2 82 AC E2
80 B0).
If I open this database in M$Access and use its export tool, I end up
with the expected UTF-8 representation of these characters in my output
file (E2 80 89).
I've Googled extensively and tried various permutations of the
MDB_JET3_CHARSET and MDBICONV environment variables without any change
to the output.
For example the following command
mdb-export test-forum.mdb tblThread
produces exactly the same output as:
MDB_JET3_CHARSET=3D"UTF-8" mdb-export test-forum.mdb tblThread
other tries include:
MDB_JET3_CHARSET=3DUTF-8 mdb-export test-forum.mdb tblThread
MDB_JET3_CHARSET=3D"utf-8" mdb-export test-forum.mdb tblThread
MDB_JET3_CHARSET=3Dutf-8 mdb-export test-forum.mdb tblThread
MDB_JET3_CHARSET=3D"utf-8" mdb-export test-forum.mdb tblThread
MDB_JET3_CHARSET=3D"CP1252" mdb-export test-forum.mdb tblThread
MDB_JET3_CHARSET=3DCP1252 mdb-export test-forum.mdb tblThread
In each case, the output is the same: normal text is exported correctly,
but the extended characters seem to be double-encoded. As the original
DB is 200MB, I have created a stripped down copy containing just one row
in this table with the "message" field containing text that includes a
number of these special characters.
Is this a bug?
I'm happy to PM a copy of my test DB (377K) if anyone wants to
investigate further.
Cheers,
--=20
Rob Hills
Waikiki, Western Australia
Mobile: +61 (412) 904-357
--------------94ED894073A2D11254380A7B
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable
<html>
<head>
<meta http-equiv=3D"content-type" content=3D"text/html; charset=3Dutf=
-8">
</head>
<body text=3D"#000000" bgcolor=3D"#FFFFFF">
<p>Hi,</p>
<p>I am debugging a project to convert a Forum from WebWiz to
phpBB.=C2=A0 WebWiz stores its data in an Access DB and our phpBB f=
orum
will be on MySQL.</p>
<p>I am using mdb-tools version 0.7.1 on Ubuntu 16.04LTS 64-bit.=C2=A0
According to mdb-ver, the mdb file I am working with is JET4.<br>
</p>
<p>The problem I am trying to solve involves forum post text that
includes some characters outside the basic character set.=C2=A0 A
specific example is the "half space" character whose UTF-8
representation I believe is the 3-byte sequence E2 80 89.=C2=A0 My
problem is that when I use mdb-export, these characters end up
being converted to =C3=A2=E2=82=AC=E2=80=B0 (Hex: C3 A2 E2 82 AC E2=
80 B0).</p>
<p>If I open this database in M$Access and use its export tool, I
end up with the expected UTF-8 representation of these characters
in my output file (E2 80 89).</p>
<p>I've Googled extensively and tried various permutations of the
MDB_JET3_CHARSET and MDBICONV environment variables without any
change to the output.</p>
<p>For example the following command</p>
<blockquote>
<p>mdb-export test-forum.mdb tblThread</p>
</blockquote>
<p>produces exactly the same output as:</p>
<blockquote>
<p>MDB_JET3_CHARSET=3D"UTF-8" mdb-export test-forum.mdb tblThread</=
p>
</blockquote>
<p>other tries include:<br>
</p>
<blockquote>
<p>MDB_JET3_CHARSET=3DUTF-8 mdb-export test-forum.mdb tblThread</p>
<blockquote> </blockquote>
<p>MDB_JET3_CHARSET=3D"utf-8" mdb-export test-forum.mdb tblThread</=
p>
<p>MDB_JET3_CHARSET=3Dutf-8 mdb-export test-forum.mdb tblThread</p>
<p>MDB_JET3_CHARSET=3D"utf-8" mdb-export test-forum.mdb tblThread</=
p>
<p>MDB_JET3_CHARSET=3D"CP1252" mdb-export test-forum.mdb tblThread<=
/p>
<p>MDB_JET3_CHARSET=3DCP1252 mdb-export test-forum.mdb tblThread</p=
>
<blockquote>
</blockquote>
</blockquote>
<p>In each case, the output is the same: normal text is exported
correctly, but the extended characters seem to be double-encoded.=C2=
=A0
As the original DB is 200MB, I have created a stripped down copy
containing just one row in this table with the "message" field
containing text that includes a number of these special
characters.</p>
<p>Is this a bug?</p>
<p>I'm happy to PM=C2=A0 a copy of my test DB (377K) if anyone wants =
to
investigate further.<br>
</p>
<p>Cheers,<br>
</p>
<div class=3D"moz-signature">-- <br>
Rob Hills<br>
Waikiki, Western Australia<br>
Mobile: +61 (412) 904-357
</div>
</body>
</html>
--------------94ED894073A2D11254380A7B--
--===============1345802907911637350==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--===============1345802907911637350==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
mdbtools-dev mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mdbtools-dev
--===============1345802907911637350==--