Re: gpg mangling?

For some time, Fire has supported encrypting and signing messages on 
it's supported IM services, including MSN, ICQ, irc, Yahoo, Jabber, and 
AIM.

In our implementation we use GPG as the encryption engine and key 
repository.  GPG is an open source implementation which is compatible 
with PGP - see http://www.gnupg.org/

Fire treats each message as a separate entity, and attempts to follow 
the normal standards used for Privacy Enhanced Email (as described in 
RFC 1421 - http://www.cis.ohio-state.edu/cgi-bin/rfc/rfc1421.html) 
wherever possible.

There are some nuances about the implementation which differ from how 
an encrypted email message is handled because of the 
limitations/features of the IM protocols:

Character set
-----------
The payload of GPG encrypted messages and keys is a Base64 encoded 
message, which uses characters which lie entirely within the ASCII 
character set.  Therefore, the character set of the message payload 
should be interpreted as ASCII when being read.  In IM clients, there 
are 4 encoding methodologies used today in the wire protocol, and some 
must be handled slightly differently when using encryption.

Group A - No special care necessary:

1) UTF-8 encoding (MSN, Jabber, and -optionally-Yahoo use this scheme)
This method of encoding Unicode 3.0 characters into strings of 1 to 6 
bytes has the desirable property that ASCII characters are sent as-is 
in the data stream, so the raw bytes of an encrypted payload or key 
expressed in BASE-64 can be interpreted directly as ASCII without 
character set transformation for both sending and receiving.

2) ISOLatin1 encoding (used by AIM-TOC and optionally by AIM-Oscar and 
Yahoo)
This encoding scheme, much like UTF-8 express ASCII bytes directly in 
its encoding scheme, and so no transformation to the raw data stream is 
neccessary.

3) Client selected encoding (used by irc and ICQ)
These clients allow the users on either end of a conversation to select 
the encoding scheme they will be using for a particular conversation.  
In this case, the encryption/signing portion of the scheme can simply 
interpret and send the characters directly as ASCII.

Group B - Special handling needed:

4) UTF-16 encoding (Used optionally by AIM-Oscar)
This encoding method expresses Unicode 3.0 characters into strings of 2 
bytes for each character.  By virtue of the layout of the Unicode 
tables, this means that ASCII characters (such as a GPG payload) will 
appear as a NULL byte followed by the ASCII byte in the raw data 
stream.  Upon reception of a UTF-16 message, the bytes must be 
transformed from UTF-16 and into ASCII (by stripping out the NULL 
bytes).  Similarly, prior to transmission, the ASCII payload must be 
converted into UTF-16 encoding.

If the transformation of a received UTF-16 datastream into raw ASCII 
fails, that is an indication that this is not an encrypted message, and 
the encryption and signing phases should be skipped.

Line Breaks
-----------
Some instant messaging clients from MSN and irc do not accept messages 
containing line breaks (CR LF) in them, and based upon the client, will 
have unexpected results when such a message is received.  Also, some 
irc servers will not accept messages containing CR or LF, or will 
ignore everything following the first line break.

As encryption of messages was added to fire, we had two goals we wanted 
to meet.  First, that encryption could be used on any of the services 
supported by Fire.  Second, that any message we sent could be received 
an displayed by any client out there.

The Base64 encoding scheme calls for line breaks to ensure that the 
length of any given line is less than 76 characters. (see RFC 1421)  
This presented us with the unique problem that some irc and MSN 
clients, which did not support decryption would receive our encrypted 
message (in BASE 64) and not be able to even display the BASE64 ASCII 
message - because of the line breaks.

To overcome this limitation, it was decided that prior to transmission, 
Fire would replace all occurrences of line breaks (CR or LF in any 
combination) with "<BR>".  Since '<' and '>' do not appear in a Base64 
encoded message, it is assured that this tag will not occur naturally 
within the message.  Furthermore, many clients interpret the received 
message as a form of HTML, so the <BR> tag will be translated to a line 
break by those non-encrypting clients as it is displayed.

Upon reception of a message, Fire first changes any "<BR>" tags in the 
data stream back into line breaks, prior to decrypting, key acceptance, 
or signing validation.

Message Type Detection
----------------------
IM data streams do not contain a header portion which can be used to 
send information to the remote client about the accompanying message.  
Therefore, it is necessary to embed information in the message body 
itself to indicate the type of message enclosed.  Each of these types 
of message is distinguished by a human readable line at the beginning 
of the message (encoded in ASCII as described above)

There are three message types involved in encrypted IM messages:

1) Public Key
-------------
This message type is used for sending a user's public key to another 
user.  The first line of messages containing a public key is:

-----BEGIN PGP PUBLIC KEY BLOCK-----

2) Signed Message
-----------------
A message whose contents are not encrypted, but are signed so that 
changes to the message while in transit can be detected.  The first 
line of a signed message is:

-----BEGIN PGP SIGNED MESSAGE-----

3) Encrypted Message
--------------------
A message which has been encrypted to obscure the message contents.  
The first line of an encrypted message is:

-----BEGIN PGP MESSAGE-----



Signed & Encrypted Messages
---------------------------
A message can be both signed and encrypted in this case, the ordering 
of operations is as follows:

Sending a message:
user msg ---> encryption ---> signing ---> transmission

Receiving a message:
reception ---> signing validation ---> decryption ---> user msg


Handling Failures
-----------------
In case a key acceptance, decryption or validation fails, the user is 
alerted via the UI, and the original received data payload is displayed 
as the message.

Encoding of Signed Messages
---------------------------
In Fire's implementation of encryption, Signed messages are Base64 
encoded before they are signed with a signature.  This means that a 
signed message will not be readable on a client which does not support 
the encryption.  This facilitates handling of messages which contain 
embedded NULL bytes in the character encoding scheme.

Hopefully this gives you a greater understanding of how Fire handles 
encryption today.  I am working with other IM client developers to 
create a common standard for encryption, signing, and key exchange so 
that we can have an interoperable encryption solution in the future.

Let me know if you have any questions.  (the problem you are seeing is 
likely due to either character set encoding issues, or the line break 
substitution.)

Alan Humpherys

On Dec 7, 2003, at 2:31 PM, Kolbe Kegel wrote:

> I just started playing with the GPG stuff in Fire, and when I send 
> message they show up signed, but also somehow "encrypted", though not 
> in the usual sense. When I run the resulting message through GPG, it 
> gives me back what appears to be an encrypted string, but it wasn't 
> told to encrypt anything at all. I'm sure this is a common question, 
> but I found nothing in the list archives save one person asking a 
> similar question without a response.
>
> What does Fire do to my text before it has GPG sign it?
>
> Kolbe



-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive?  Does it
help you create better code?  SHARE THE LOVE, and help us help
YOU!  Click Here: http://sourceforge.net/donate/
_______________________________________________
Fire-talk mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/fire-talk
Have a question? Please read the Fire Frequently Asked Questions:
http://fire.sourceforge.net/faq.shtml