opf, hash ids, ids, etc

Hi,

Progress is slow these days, but there is something you can crunch your
teeth into on a lazy evening if you like.  At the end of this mail you find
a way to match the opf file with music tracks on minidisc.

By matching the track info with the omg file and the information in the opf
file below, we might be able to find out something new.

The shared files Sony Shared\OpenG\procfile directory has a database of
imported music files.  There is a db file "device.sal" which I can not
decode (my db III file viewer doesn't like it), so I have no info about
that.  This is one thing that could be analyzed.  Then there is a directory
for each byte in hexadecimal (00 - ff), the directories contain .opf files
for each music file that openmg has seen (through Jukebox or SimpleBurner). 
The files are hashed by the last byte of the identifier.  For example, the
file

010F50000004000000D3651B4F702E0FAD08D227.opf

is in directory 27 because of the last two hex digits in the filename (after
removing the extension .opf).

This file is interesting, because the data in it can be matched against file
uploads, and maybe contains the necessary key information etc.  We should
get as much information about it as possible.

The file name is a 20 byte long identifier in hexadecimal form.  It is
formatted in the following way:

010F50000004000000 UUVV 1B4F702E0F WWXXYYZZ.opf

Where UUVV and WWXXYYZZ seem to vary over the whole possible range.
The meaning of all these bytes is unknown.

The format of the file content is ASN.1.  You can use a standard ASN.1
parser (for example dumpasn1) to view it.  Here is an example output of the
above file.  I have added some remarks, but there are also some files which
diff significantly in some parts.

   0 NDEF: SEQUENCE {
   2    2:   INTEGER 1001

REMARK: This seems to be constant. 1001 reminds us of the "20 10 01" as a
track identifier in the NetMD protocol.

   6    4:   OCTET STRING
         :     2C 72 40 DC A6 B0 6D 08 A8 99 77 B0 2B 8A 28 E4
         :     ED DF 89 20

REMARK: Different for each track, but always 20 bytes.  Might be a key or
identifier.

  28    4:   OCTET STRING
         :     5B B1 26 A7 06 27 A3 25 EE 38 94 85 26 88 FC 03
         :     97 A7 66 69

REMARK: Different for each track, but always 20 bytes.  Might be a key or
identifier?

  50 NDEF:   SEQUENCE {
  52    2:     INTEGER 107

REMARK: Seems to be constant.

  55    2:     INTEGER 1043938341

REMARK: Different for each file.  This is a timestamp (number of seconds
since epoch 1/1/1970), but I don't know what type.  Might be creation time.

  61    2:     INTEGER 104
  64    2:     INTEGER 0
  67    2:     INTEGER 7

REMARK: All these seem to be constant.

  70    2:     INTEGER 1043938443

REMARK: Different for each file.  Another timestamp.

  76    2:     INTEGER 100

REMARK: Seems to be constant.

  79    4:     OCTET STRING
         :       01 00 00 00 00 00 00 00 00 00 58 E3 00 00 90 76
         :       00 00 D2 34 00 01 09 05 00 01 52 48 00 01 B4 EF
         :       00 01 F2 8E 00 02 3B C7 00 02 7F 10 00 02 97 4F
         :       00 02 DB 22 00 03 23 52

REMARK: Different for each file, even in length.  Possibly always a multiple
of 4 in length.  The first line can actually have "20" or "1E" as byte 8 in
the first line, like:
         :       01 00 00 00 00 00 00 20 00 00 45 97 00 00 80 84

This indicates that the first bytes are always "01 00 00 00 00 00 00" then
"00", "1E" or "20", then four byte groups with the first two bytes "00 00",
"00 01" or "00 02", and then two bytes which are always different.  We don't
know what they mean, but maybe they are correlated to the length of the
music file, or they are the block keys, or whatever.  Maybe it's even a
title in some strange encoding.


 137    2:     INTEGER 0

REMARK: Nothing known.

 140    4:     OCTET STRING
         :       01 0F 50 00 00 04 00 00 00 D3 65 1B 4F 70 2E 0F
         :       AD 08 D2 27

REMARK: This is the 20 byte identifier that is also used in the file name.

 162    2:     INTEGER 6

REMARK: I think this is not constant, but not sure.  From here, some files
differ significantly.

 165    4:     OCTET STRING 'CDDA ripping'

REMARK: The source of the file.  I took mine from an audio CD.

 179    2:     INTEGER 1
 182    2:     INTEGER 3
 185    2:     INTEGER 8
 188    2:     INTEGER 2
 191    2:     INTEGER 5
 194    2:     INTEGER 4

REMARK: I have also seen 1, 3, 8, 0, 5, 4, and 1, 3, 8, 3, 5, 4.
No idea what it means.  Maybe some status code.

 197    2:     INTEGER 102
 200 NDEF:     SEQUENCE {
 202 NDEF:       SEQUENCE {
 204    4:         OCTET STRING 01 0F 50 00 00 06 00 00 00 00 00 00 00 00 00 01
 222    2:         INTEGER 0

REMARK: If this part is present, the above octet string seems to be
constant.

 225    4:         OCTET STRING 20 93 A8 F3 7A 7A 4E ED
REMARK: Something interesting!  This is the HashID, the hash of the 20 byte
identifier (the one used in the filename???).  The HashID is stored on the
minidisk at checkout, or can be calculated by the minidisc player from the
track properties (unknown).  It is not known how it is caculated, but you
can get it from the player by sending the netmd command:
0018 00 08 00 46 f0 03 01 03 23 ff 10 01 $t $t

Where $t $t is the track number.  The last 8 bytes of the reply are the hash
id.  Note: this command takes a long time, and can thus fial to be received
by Pete's receive routine.  If it fails, try again, because the second time
is faster because less head movement is needed.

By sending this command, you can look up the hash id for a file, and then
you can find it in the database.  This makes it easy to find the opf file
for a file on a minidisc.

         :         }
         :       }
         :     }
 241 NDEF:   SEQUENCE {
         :     }
         :   }

Bonus task:  Somewhere on the disk must be a file with 32 byte long records,
the first 8 bytes being the hashid, the following 20 bytes the full id, and
then 4 unknown (random?) bytes.  If you can find this file, please let me
know.  It might be that the file only exists when the netmd server is
running, or so.

Thanks,
Marcus


-- 
`Rhubarb is no Egyptian god.' GNU      http://www.gnu.org    [email protected]
Marcus Brinkmann              The Hurd http://www.gnu.org/software/hurd/
[email protected]
http://www.marcus-brinkmann.de/