Re: PalmDB requests and bug report

Thanks...

A while ago, I submitted a python module to sourceforge to read and write
both calendar and addressbook. I have used it to create a palm2ics
convertor. I actually didn't write it, but I modified it some. I don't know
if it will be interesting to you or not, but I've attached it here.

On the documentation front, I'd be happy to help you set up and run a
documentation wiki. How much storage/database does sourceforge give us?

Ideally, I'd like to have a set of python modules that would allow us to
sync palm data with XML or a database like MySQL, plugin abilities so that
many people could write their own sync code, and a conduit to make it all
happen when I push my hotsync button!

I'll help you with the code however I can, but I'm sure I can make some
headway on the documentation as we go along.

Is anyone else interested in this? It seems like opensource palm development
is nearly dead.

Oh, on the syntax front, I use scite as my text editor and it shows all
python indentation inconsistencies with a little curly blue line. I myself
always use tabs, and I've set the editor to honor that.



On 9/27/06, Rick Price <[email protected]> wrote:
>
>
>
> On Tue, 26 Sep 2006, "Jeff Mikels" wrote:
>
> > I should have said "unpack" or "crack" instead of decrypt. That was
> silly of
> > me.
>
> No problem, when I reread your email after writing most of my reply, I
> realized what you had meant.
>
> Basically it was a bad day, and I wasn't really paying attention as I read
> it.
>
> >
> > Anyway, about the plugin, the xml produced from the tododb by the basic
> > plugin is perfect except for the payload.
> >
>
> Yea, that makes sense, that is what is supposed to happen.
>
> > The todo db has 3 bytes, then a string for the description, then a
> string
> > for the note. I'm pretty sure those three bytes are for the category
> int,
> > the date, the deleted flag, and the completed flag.
>
> Some of this is already unpacked for you AFAIK. I think the category int,
> and the deleted and so on flags are part of the header we unpack but I
> would have to check.
>
> >
> > It won't be difficult at all to implement, but I'm having trouble
> figuring
> > out the API (yes, it's the latest SVN) and python's unpack doesn't have
> the
> > option for variable length null terminated strings. What do you use?
>
> I believe I just cracked the string on the 0 at the end of the string. I
> believe the function is split(), and you pass zero in for the char
> somehow.
>
> I think the appropriate thing would be something like:
>
> Text,Note=incomingString.split('\0')
>
> You have to be careful when there isn't a note on the end, but I think
> there *should* always be a double zero in that case, in which case the
> above code should work.
>
> >
> >
> >
> > Help me out though, do I need anything more than this?
> >
> > class ToDoRecordPlugin(PalmDB.Plugins.BasePlugin.DataRecord):
> >    def getPDBCreatorID(self):
> >        return 'todo'
> >    def getPalmApplicationName(self):
> >        return 'ToDo'
>
> This part seems fine.
>
> >
> >    def _crackPayload(self,dstr):
> >        [code here]
> >
>
> This part is where you should have overridden the call to create the
> object that is for each Palm record, that object implements _crackPayload.
> I'm hoping to have the time to do it quickly to show you how. It should
> only take 15 minutes to half an hour for me to write the skeleton for the
> plugin for you.
>
> Then you can implement _crackPayload and be done with it.
>
>
> Once the dust settles, would you have time to write a quick howto?
>
> Since you didn't design things, you should be in a better position to
> start the document, I can expand on things for you as required.
>
> >
> >
> > Oh, one more thing, your whitespace is quite inconsistent in your code.
> You
> > have spaces and tabs all over the place.
>
> Yes, I would dearly love to find a good Python reformatter program.
>
> Rick
>
>
> >
> > On 9/26/06, Rick Price <[email protected]> wrote:
> >>
> >>
> >>
> >> On Tue, 26 Sep 2006, "Jeff Mikels" wrote:
> >>
> >>> I'm having a problem with the crackPalmDate function
> >>>
> >>> (I'm using svn code)
> >>>
> >>> PILOT_TIME_DELTA = 2082844800L
> >>> def crackPalmDate(variable):
> >>>        if variable == 0:
> >>>            return None
> >>>        else:
> >>>            return datetime.datetime.fromtimestamp
> >> (variable-PILOT_TIME_DELTA)
> >>>
> >>> I often get a ValueError because variable-PILOT_TIME_DELTA is
> sometimes
> >> a
> >>> negative number. I've changed the code to be...
> >>>
> >>> PILOT_TIME_DELTA = 2082844800L
> >>> def crackPalmDate(variable):
> >>>        if variable == 0 or variable < PILOT_TIME_DELTA:
> >>>            return None
> >>>        else:
> >>>            return datetime.datetime.fromtimestamp
> >>> (variable-PILOT_TIME_DELTA)
> >>>
> >>> Is this a problem?
> >>
> >> No, I don't think so, except that if the variable is before the
> >> PILOT_TIME_DELTA surprises could possibly happen in the Palm...
> >>
> >> The reasoning behind my code (and it could be flawed), is that we want
> to
> >> know if there really wasn't a value there, like in SQL. So since I
> didn't
> >> think that the value would ever be zero (or below zero), using zero for
> >> NULL seemed like a perfect solution.
> >>
> >> If that is not true, then we may need to rethink how things work.
> >>
> >> Can you send me a patch with your changes so I can incorporate it?
> >>
> >> Do make sure you have the latest code from SVN, I haven't changed it
> for a
> >> few weeks, but it's best to make sure you have the latest.
> >>
> >>>
> >>>
> >>> Additionally, I'd like to create a "plugin" for the todo database, but
> >> I'm
> >>> not sure how to go about doing it. The Base Plugin class is working
> >>> perfectly but the payload needs to be decrypted properly. How can I
> >> create a
> >>> plugin that only overloads the payload functions and have it loaded
> when
> >>> necessary?
> >>>
> >>
> >> Well, I like plugins that are loaded automagically and that can be
> >> distributed separately, but in this case, I decided to keep it simple
> and
> >> have you declare the plugin to the code.
> >>
> >> I also figured that we would just roll the plugins (not created by me)
> >> into the release, since just about anything someone makes will be
> useful
> >> to others.
> >>
> >> [ If you use the library for something proprietary, you would just call
> >> the plugin notification code, and keep your code separate. ]
> >>
> >> In the file PluginManager.py, you will find the functions
> >> registerPDBPlugin and deRegisterPDBPlugin. These are the functions you
> use
> >> to register your plugin with the framework.
> >>
> >> At the bottom of the file, I use the standard functions to register my
> >> Progect plugin.
> >>
> >>
> >> Now, I'm not really sure what you mean by decrypted.
> >>
> >> If you mean you need to translate the XML that *would* have been
> generated
> >> into something more useful, the code knows how to apply XSLT
> stylesheets
> >> for you; both coming and going. It can also do some simple
> transformations
> >> like gzip.
> >>
> >> If you need to output to say a binary or text format, I think it can be
> >> done, but I would have to look at the code again to explain how.
> >>
> >>
> >> I really think I misdesigned the plugin somehow as well, but I can't
> >> really describe what I mean except to say this (because I haven't
> thought
> >> it through yet):
> >> * Some classes do more than one thing - very bad
> >> * The symptom is that the plugin is difficult to use.
> >> * I expect that writing/reading formats other than XML will
> >> be more difficult than it should be (as in binary or text).
> >>
> >>
> >> If you can describe what you are doing to me, then maybe I can give you
> >> some suggestions.
> >>
> >> I'm going to have to redesign the code _again_ because of not getting
> the
> >> plugin code right, but I will try and remain compatible, or at least
> make
> >> it so much easier to use that it's not hard for you to update your
> code.
> >>
> >>
> >> If you mean how do I crack the Palm task format, I can certainly help
> you
> >> with that too. Basically the plugin you create returns an object that
> >> knows what to do with the raw data as passed in by the framework.
> >>
> >> You will need to find documentation on the Palm task format, I am
> pretty
> >> sure it can be found easily with Google.
> >>
> >> In this case, you would probably just subclass the default plugin so
> that
> >> it returns a special Task object that knows how to crack the payload
> for
> >> tasks, and puts the values into a dictionary.
> >>
> >> I believe the class ProgectRecord in plugins/ProgectPlugin.py is the
> >> closest thing to what you want. And the important method to create is
> >> _crackPayload.
> >>
> >> The problem is, that the Progect format is quite complex, and we are
> >> forcing a tree into rows. So the Progect class is probably not a good
> >> example of how to do something simple (or not a good example at
> all...).
> >>
> >> For something like the task database, you should have almost _no_ work
> to
> >> do, and so don't let the extreme complexity of the Progect plugin scare
> >> you off.
> >>
> >> I can't predict how much work cracking a task would be, but I would
> hope
> >> it would be under a hundred lines of Python code all told, and simple
> code
> >> at that.
> >>
> >> Generating and reading the XML would happen automatically if I remember
> >> correctly, so the big problem is being able to crack and pack the task
> >> format. (that is, the payload, because we crack the record header for
> >> you).
> >>
> >> I would be more than happy to give you a hand with this task, while I
> am
> >> overloaded at work for the next day or so, perhaps we can talk via IM
> or
> >> on the phone. Even email would be fine, if you can help me understand
> your
> >> problem a little better.
> >>
> >> I can then explain how I think it can be accomplished easily with the
> >> framework.
> >>
> >> I will certainly try to do this for anyone who wants to use the
> framework,
> >> but having the task format converted to and from XML is an important
> >> feature for the framework and is therefore a priority for me.
> >>
> >>
> >> Rick
> >>
> >>>
> >>>
> >>>
> >>> On 9/7/06, Rick Price <[email protected]> wrote:
> >>>>
> >>>> Okay, I just managed to get the Python PalmDB library to go from a
> >> Progect
> >>>> file to XML and back again without dropping anything really obvious.
> >>>>
> >>>> Anyone who wants to check out the Subversion code is welcome to do
> so.
> >>>>
> >>>> I think I need to do a major rewrite at some point in the next little
> >> bit,
> >>>> but first I will setup some conversions to popular programs that
> >>>> read/write XML.
> >>>>
> >>>> First on the list is going to be treeline since I use it right now
> and
> >> the
> >>>>
> >>>> format seems to be basically sane.
> >>>>
> >>>> I will then probably try to do something with Omni Outliner for the
> >> Mac.
> >>>>
> >>>> That will probably require fixing a bug in the library that seems to
> >> show
> >>>> up on the Mac.
> >>>>
> >>>> After that, I will see what I can do based on popularity.
> >>>>
> >>>>
> >>>> Once I have treeline support working, and it's not too ugly to run, I
> >> will
> >>>> make a release of Python PalmDB so people can start to use it.
> >>>>
> >>>> I will release again I would guess after I get Omni Outliner support
> >>>> working.
> >>>>
> >>>> Any questions, just email me.
> >>>>
> >>>> Rick
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> Remember to visit Polls, Files and Bookmarks sections at
> >>>> http://groups.yahoo.com/group/progect. You are welcome to vote,
> upload
> >>>> your files and submit bookmarks.
> >>>> Yahoo! Groups Links
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> Jeff Mikels
> >>> leading people one step closer to Jesus
> >>> http://jeff.mikels.cc
> >>> http://thesouthsidechurch.org
> >>>
> >>>
> >>> --
> >>> Jeff Mikels
> >>> leading people one step closer to Jesus
> >>> http://jeff.mikels.cc
> >>> http://thesouthsidechurch.org
> >>>
> >>>
> >>> [Non-text portions of this message have been removed]
> >>>
> >>>
> >>
> >>
> >>
> >> Remember to visit Polls, Files and Bookmarks sections at
> >> http://groups.yahoo.com/group/progect. You are welcome to vote, upload
> >> your files and submit bookmarks.
> >> Yahoo! Groups Links
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >
> > --
> > Jeff Mikels
> > leading people one step closer to Jesus
> > http://jeff.mikels.cc
> > http://thesouthsidechurch.org
> >
> >
> > [Non-text portions of this message have been removed]
> >
> >
>
>
>
> Remember to visit Polls, Files and Bookmarks sections at
> http://groups.yahoo.com/group/progect. You are welcome to vote, upload
> your files and submit bookmarks.
> Yahoo! Groups Links
>
>
>
>
>
>
>
>
>
>
>


-- 
Jeff Mikels
leading people one step closer to Jesus
http://jeff.mikels.cc
http://thesouthsidechurch.org

  ----------

"""
Python module that parses palm files

The files are parsed into list/dictionary data structure that mirrors
the structure of the palm file.

Thanks to Scott Leighton who provided the Palm File format at
http://www.geocities.com/Heartland/Acres/3216/palmrecs.htm

Usage

import PalmFile.py
fileStruct = PalmFile.readPalmFile(<fileName>)

the structure you get back mirrors the file structure. I use
import pprint
pprint.pprint(fileStruct)
to make sense of the data I get back
see printAWeekWorthOfCalendar and printAllNames for examples

to write this information back to a file, use
PalmFile.writePalmFile(<fileName>,<fileType>,fileStruct)
"""

"""
Author: [email protected]
	(since Version 0.4)

Original Author: atotic-/[email protected]


Version 0.5
Date: 04/08/2004
Added:
	Major additions to enable the writePalmFile() function.
	writePalmFile(fileName, fileData)
		writes a palm desktop .dat format file containing fileData.
		fileType is determined from fileData[0]['versionTag']
		fileData must be an appropriately formed list / dictionary
		of the same format returned by readPalmFile

Version 0.4
Date: 03/28/2004
Added:
	getNextRepeatedEvent(event)
		computes the next occurrence of a repeated event and returns it as an event dictionary
	printAWeekWorthOfCalendar(calendar,traceRepeats=0)
		prints to stdout a week's worth of upcoming events.
		if traceRepeats is 1, then this function will also look for upcoming
		occurrences of repeated events
	getEvents(calendar)
		returns a dictionary of events when given a palm datebook fileStruct as calendar
	getUpcomingEvents(calendar, daysAhead, traceRepeats=0)
		returns a list of events from calendar (palm datebook fileStruct) from the present moment
		up to daysAhead number of days into the future. If traceRepeats is 1, then function will
		also look for upcoming occurrences of repeated events.


Version 0.3
Date: 12/12/03
Patch for readRepeatEvent, from John Lim (Rainlendar guy)

Date: Apr 24, 2003
Version: 0.2: fixed dateExceptionCount parsing

Date: 11/21/2002
Version: 0.1, my first Python code

It works on my machine, Python 2.2, I have not tried it on any others

Terminology used in coding:
record - a list of items in a known predefined format, not composed of fields
		each record read has a HEADERDEF defined in this file that describes it
frecord - a special record, where each item is a *field*
field - fields have type & data. The labels are implicit in their position
		and are defined in this file as a list
items - basic units we know how to read: byte/short/long/, etc
"""

# Globals
readDebug = False
writeDebug = False



# Address book defines

"""
HEADERDEF lists format
HEADERDEF are data that represent the file format.
I've tried to make all parsing data driven, and HEADERDEF structs
describe the grammar. When data to be read go beyound what HEADERDEF
defines, 3rd column provides the name Python to be executed.

headerDef tuple columns format
col 1 property name
col 2 type short, long, palm cstring, record, frecord
3 additional argument:
	if type in col 2 is record, the name of the entry that defines the struct count
	if type in col 2 is frecord, the python code to execute
"""

"""See HEADERDEF lists format above"""
addressHeaderDef = (
	("versionTag", "long"),
	("fileName", "cstring"),
	("tableString", "cstring"),
	("nextFree", "long"),
	("categoryCount","long"),
	("categoryList", "record", "addressCategoryEntryDef", "categoryCount"),
	("resourceID", "long"),
	("fieldsPerRow", "long"),
	("recIDPos", "long"),
	("recStatus", "long"),
	("placementPos", "long"),
	("fieldCount", "short"),
	("fieldEntryList", "record", "addressSchemaFieldDef", "fieldCount"),
	("numEntries", "long"),
	("addresses", "frecord","addressEntryFields")
	)

"""See HEADERDEF lists format above"""
addressCategoryEntryDef = (
	("index", "long"),
	("id", "long"),
	("dirtyFlag", "long"),
	("longName", "cstring"),
	("shortName", "cstring")
	)

"""See HEADERDEF lists format above"""
addressSchemaFieldDef = (
	("fieldEntryType", "short"),
	)

"""See HEADERDEF lists format above"""
addressEntryFields = (
		"recordID",
		"status",
		"position",
		"lastName",
		"firstName",
		"title",
		"companyName",
		"phone1LabelID",
		"phone1Text",
		"phone2LabelID",
		"phone2Text",
		"phone3LabelID",
		"phone3Text",
		"phone4LabelID",
		"phone4Text",
		"phone5LabelID",
		"phone5Text",
		"address",
		"city",
		"state",
		"zip",
		"country",
		"note",
		"private",
		"category",
		"custom1Text",
		"custom2Text",
		"custom3Text",
		"custom4Text",
		"displayPhone"
)




# Calendar defines

"""See HEADERDEF lists format above"""
calendarHeaderDef = (
	("versionTag", "long"),
	("fileName", "cstring"),
	("tableString", "cstring"),
	("nextFree", "long"),
	("categoryCount","long"),
	("categoryList", "record", "addressCategoryEntryDef", "categoryCount"),
	("resourceID", "long"),
	("fieldsPerRow", "long"),
	("recIDPos", "long"),
	("recStatus", "long"),
	("placementPos", "long"),
	("fieldCount", "short"),
	("fieldEntry", "record", "addressSchemaFieldDef", "fieldCount"),
	("numEntries", "long"),
	("datebookList", "frecord","calendarEntryFields")
)

calendarEntryFields = (
	"recordID",
	"status",
	"position",
	"startTime",
	"endTime",
	"text",
	"duration",
	"note",
	"untimed",
	"private",
	"category",
	"alarmSet",
	"alarmAdvUnits",
	"alarmAdvType",
	"repeatEvent"
)



###
# Generic reading and writing routines
# (not to be accessed by user)
###

import struct

def readCString(f):
	"""Read in a Palm-format string."""
	
	"""
	String docs off the net:
	Strings less than 255 bytes are stored with the length specified in the first byte followed by the actual string. 
	Zero length strings are stored with a 0x00 byte.
	Strings 255 bytes or longer are stored with a flag byte set to 0xFF
	followed by a short (2*Byte) that specifies the length of the string, followed by the actual string. 
	"""
	retVal = None
	(firstByte, ) = struct.unpack("B", f.read(1))
	if firstByte == 0 :
		retVal = "";
	elif firstByte == 0xFF:
		(length, ) = struct.unpack("H", f.read(2))
		retVal = f.read(length)
	else: # length was in first byte
		retVal = f.read(firstByte)
	return retVal

def writeCString(f,s):
	"""Writes string to Palm .dat file"""
	if writeDebug:
		print '---------------------------------------'
		print 'WRITE CSTRING'
		print s
		print '---------------------------------------'
	length = len(s)
	if length >= 255:
		format = "BH" + str(length) + "s"
		f.write(struct.pack(format,0xFF,int(length),s))
	else:
		format = "B" + str(length) + "s"
		f.write(struct.pack("B" + str(length) + "s",int(length),s))

def readShort(f):
	"""Read unsigned 2 byte value from a file f."""
	(retVal,) = struct.unpack("H", f.read(2))
	return retVal
	
def writeShort(f,n):
	if writeDebug:
		print '---------------------------------------'
		print 'WRITE SHORT'
		print n
		print '---------------------------------------'
	f.write(struct.pack("H",n))

def readLong(f):
	"""Read unsigned 4 byte value from a file f."""
	(retVal,) = struct.unpack("L", f.read(4))
	return retVal

def writeLong(f,n):
	if writeDebug:
		print '---------------------------------------'
		print 'WRITE LONG'
		print n
		print '---------------------------------------'
	f.write(struct.pack("L",n))
	
def readFloat(f):
	"""Read float (4 byte?) from a file f."""
	(retVal,) = struct.unpack("f", f.read(4))
	return retVal

def writeFloat(f,n):
	if writeDebug:
		print '---------------------------------------'
		print 'WRITE FLOAT'
		print n
		print '---------------------------------------'
	f.write(struct.pack("f",n))
	
def readRepeatEvent(f):
	"""Read RepeatEvent, a hacky palm data structure

	must be read programatically due to randomness of the data structure
	"""
	event = {};
	event['dateExceptionCount'] = readShort(f)
	dateExceptions = [];
	for i in range(event['dateExceptionCount']):
		dateExceptions.append(readLong(f))
	if len(dateExceptions) > 0:
		event['dateExceptions'] = dateExceptions
	event['repeatEventFlag']= readShort(f)
	if event['repeatEventFlag'] == 0: return event
	if event['repeatEventFlag'] == 0xFFFF:
		classRecord = {}
		classRecord['constant'] = readShort(f)
		classRecord['nameLength'] = readShort(f)
		classRecord['name'] = f.read(classRecord['nameLength'])
		event['classRecord'] = classRecord
	
	event['brand'] = readLong(f)
	event['interval'] = readLong(f)
	event['endDate'] = readLong(f)
	event['firstDayOfWeek'] = readLong(f)
	if event['brand'] in (1L,2L,3L):
		event['brandDayIndex'] = readLong(f)
	if event['brand'] == 2L:
		event['brandDaysMask'] = f.read(1)
	if event['brand'] == 3L:
		event['brandWeekIndex'] = readLong(f)
	if event['brand'] in (4L, 5L):
		event['brandDayNumber'] = readLong(f)
	if event['brand'] == 5L:
		event['brandMonthIndex'] = readLong(f)
	return event

def writeRepeatEvent(f,repeatEventDetails):
	"""Write RepeatEvent.
	"""
	if writeDebug:
		print '---------------------------------------'
		print 'WRITING REPEAT EVENT'
		import pprint
		pprint.pprint(repeatEventDetails)
		print '---------------------------------------'
	writeShort(f,repeatEventDetails['dateExceptionCount'])
	if repeatEventDetails['dateExceptionCount'] != 0:
		for dateException in repeatEventDetails['dateExceptions']:
			writeLong(f,dateException)
	writeShort(f,repeatEventDetails['repeatEventFlag'])
	if repeatEventDetails['repeatEventFlag'] == 0: return
	if repeatEventDetails['repeatEventFlag'] == 0xFFFF:
		classRecord = repeatEventDetails['classRecord']
		writeShort(f,classRecord['constant'])
		writeShort(f,classRecord['nameLength'])
		f.write(classRecord['name'])

	writeLong(f,repeatEventDetails['brand'])
	writeLong(f,repeatEventDetails['interval'])
	writeLong(f,repeatEventDetails['endDate'])
	writeLong(f,repeatEventDetails['firstDayOfWeek'])
	if repeatEventDetails['brand'] in (1L,2L,3L):
		writeLong(f,repeatEventDetails['brandDayIndex'])
	if repeatEventDetails['brand'] == 2L:
		f.write(repeatEventDetails['brandDaysMask'])
	if repeatEventDetails['brand'] == 3L:
		writeLong(f,repeatEventDetails['brandWeekIndex'])
	if repeatEventDetails['brand'] in (4L, 5L):
		writeLong(f,repeatEventDetails['brandDayNumber'])
	if repeatEventDetails['brand'] == 5L:
		writeLong(f,repeatEventDetails['brandMonthIndex'])
	return
	
def readField(f, fieldType):
	"""Read palm record from a file f.
	
	fieldType -- integer specifying the palm field type
	"""
	if readDebug:
		print '-----------------------------------'
		print 'READING FIELD'
		print 'file:',f
		print 'fieldType:',fieldType
	retVal = None
	if fieldType ==0: # none
		retVal = None
	elif fieldType == 1: # integer
		retVal = readLong(f)
	elif fieldType == 2: # float
		retVal = readFloat(f)
	elif fieldType == 3: # date
		retVal = readLong(f)
	elif fieldType == 4: # alpha
		raise NotImplementedError
	elif fieldType == 5: # cstring
		readLong(f) # padding
		retVal = readCString(f)
	elif fieldType == 6: # boolean
		retVal = readLong(f) != 0
	elif fieldType == 7: # bit flag
		retVal = readLong(f)
	elif fieldType == 8: # repeat event, bad hack, bad
		retVal = readRepeatEvent(f)
	else:
		raise ValueError
	if readDebug:
		import pprint
		pprint.pprint(retVal)
	
	return retVal

def writeField(f, fieldType, s):
	"""Write palm field to a file.
	
	fieldType -- integer specifying the palm field type
	"""
	if writeDebug:
		print '---------------------------------------'
		print 'WRITING FIELD'
		print 'fieldType:', fieldType
		print 'value:',s
		print '---------------------------------------'
	if fieldType ==0: # none
		pass
	elif fieldType == 1: # integer
		writeLong(f,s)
	elif fieldType == 2: # float
		writeFloat(f,s)
	elif fieldType == 3: # date
		writeLong(f,s)
	elif fieldType == 4: # alpha
		raise NotImplementedError
	elif fieldType == 5: # cstring
		writeLong(f,0) # padding
		writeCString(f,s)
	elif fieldType == 6: # boolean
		if s:
			s = 1
		else:
			s = 0
		writeLong(f,s)
	elif fieldType == 7: # bit flag
		writeLong(f,s)
	elif fieldType == 8: # repeat event, bad hack, bad
		writeRepeatEvent(f,s)
	else:
		raise ValueError

def readFRecords(f, fileSoFar, labels):
	"""reads a list of frecords from file f
	
	returns -- a list of records
	fileSoFar -- dictionary of data read so far, used to get the number
				of records to read
	labels -- a list of labels for the fields
	"""
	if readDebug:
		print '---------------------------------'
		print 'READING FRECORDS'
		print 'fileSoFar'
		import pprint
		pprint.pprint(fileSoFar)
	fieldsPerRecord = fileSoFar['fieldCount']
	# make sure that declared 
	if fieldsPerRecord != len(labels):
		raise ValueError
	numberOfRecords = fileSoFar['numEntries'] / fieldsPerRecord;
#	print "reading", str(numberOfRecords), "records"
	entries = []
	for i in range(numberOfRecords):
#		print "reading record", str(i)
		newEntry = {}
		for j in labels:
			fieldType = readLong(f)
			newEntry[j] = readField(f, fieldType)
		entries.append(newEntry)
#	print "done with", str(numberOfRecords), "records"
	return entries

def writeFRecords(f, fieldEntryList, labels, list):
	"""writes a list of frecords to file f

	fieldEntryList -- a list describing the order of field types in each record
	list -- a list of records to write
	labels -- a list of labels for the fields
	"""
	if writeDebug:
		print '---------------------------------------'
		print 'WRITING FRECORDS'
		import pprint
		print '\nfieldEntryList'
		pprint.pprint(fieldEntryList)
		print '\nlabels'
		pprint.pprint(labels)
	if len(fieldEntryList) != len(labels):
		raise ValueError
	for item in list:
		if writeDebug:
			print '\nitem to write'
			import pprint
			pprint.pprint(item)
		for i in range(len(labels)):
			fieldType = fieldEntryList[i]['fieldEntryType']
			if writeDebug:
				print 'in frecords, attempting to write field:'
				print 'f:',f
				print 'fieldType:',fieldType
				print 'label:',labels[i]
				import pprint
				pprint.pprint(item[labels[i]])
			writeLong(f,fieldType)
			writeField(f,fieldType,item[labels[i]])

def readRecords(f, fileFormat, howMany=1):
	"""reads a list of objects from a file f
	
	fileFormat -- HEADERDEF of what format looks like
	howMany -- how many records to read
	returns a list of howMany dictionaries: [ {d1}, .... {dN}]
	"""
	retVal = []
	for i in range(howMany):
		if readDebug:
			print '------------------------------'
			print "entering", str(fileFormat[0]);
			print 'READING RECORDS'
			print 'retVal:'
			import pprint
			pprint.pprint(retVal)
		entry = {}
		for fieldDef in fileFormat:
			if fieldDef[1] == "long":
				entry[fieldDef[0]] = readLong(f)
			elif fieldDef[1] == "short":
				entry[fieldDef[0]] = readShort(f)
			elif fieldDef[1] == "cstring":
				entry[fieldDef[0]] = readCString(f)
			elif fieldDef[1] == "record":
				entry[fieldDef[0]] = readRecords(f, eval(fieldDef[2]), entry[fieldDef[3]])
			elif fieldDef[1] == "frecord":
				entry[fieldDef[0]] = readFRecords(f, entry, eval(fieldDef[2]))
			else:
				raise AssertionError
		retVal.append(entry);
	#	print "returning", str(fileFormat[0])
	return retVal

def writeRecords(f, fileFormat, list):
	"""reads a list of objects from a file f
	
	fileFormat -- HEADERDEF of what format looks like
	list -- a list of dictionaries to write as palm records
	"""
	if writeDebug:
		print '\n----------------------------\nWRITING RECORDS'
		import pprint
		print 'fileFormat:'
		pprint.pprint(fileFormat)
		print '\nlist:'
		pprint.pprint(list)
	#for i in range(len(list)):
	for item in list:
		for fieldDef in fileFormat:
			if writeDebug:
				print '\nfieldDef:'
				print fieldDef
			if fieldDef[1] == "long":
				writeLong(f, item[fieldDef[0]])
			elif fieldDef[1] == "short":
				writeShort(f, item[fieldDef[0]])
			elif fieldDef[1] == "cstring":
				writeCString(f, item[fieldDef[0]])
			elif fieldDef[1] == "record":
				if writeDebug:
					print 'WILL WRITE RECORDS'
				writeRecords(f=f, fileFormat=eval(fieldDef[2]),list=item[fieldDef[0]])
				if writeDebug:
					print 'BACK FROM WRITING RECORDS'
			elif fieldDef[1] == "frecord":
				if writeDebug:
					print 'WILL WRITE FRECORDS'
				writeFRecords(f=f, fieldEntryList=item['fieldEntry'],labels=eval(fieldDef[2]), list=item[fieldDef[0]])
				if writeDebug:
					print 'BACK FROM WRITING FRECORDS'
			else:
				raise AssertionError


######################
# MAIN FUNCTIONS
######################

def readPalmFile(fileName):
	""" Read in a Palm fileName with a specified format
	
	The type of the file is determined automatically by reading
	the first four bytes
	fileFormat -- different files have different formats (address book, calendar...)
				[abHeaderDef | calHeaderDef]
	"""
	retVal = None
	try:
		PalmFile = open(fileName, "rb")
	except IOError:
		print "Palm file", fileName, "cannot be opene\n\n"
		raise IOError
	try:
		sig = PalmFile.read(4)
		PalmFile.seek(0)
		if sig == "\x00\x01BA": # address book
			fileFormat = addressHeaderDef 
		elif sig == "\x00\x01BD": # datebook (calendar)
			fileFormat = calendarHeaderDef
		else:
			print "Unknown file format ", sig
			raise ValueError		
		retVal = readRecords(PalmFile, fileFormat, 1)
	except IOError:
		print "Unexpected error while reading Palm file"
		raise IOError
	if PalmFile : PalmFile.close()
	return retVal

def writePalmFile(fileName, fileData):
	'''Writes a palm desktop file
	'''
	if writeDebug:
		print '---------------------------------------'
		print 'ATTEMPTING TO WRITE PALM FILE\nFILE:',fileName
		print 'fileType:',fileType
		print 'fileData:\n'
		import pprint
		pprint.pprint(fileData)
	""" Write a Palm fileName with a specified format
	
	The type of the file is determined automatically by reading
	the first four bytes
	fileFormat -- different files have different formats (address book, calendar...)
				[abHeaderDef | calHeaderDef]
	"""
	fileType = fileData[0]['versionTag']
	sig = struct.pack("L",fileType)
	if not (sig == '\x00\x01BA' or sig == '\x00\x01BD'):
		print "Unknown file format ", sig
		raise ValueError		

	try:
		PalmFile = open(fileName, "wb")
	except IOError:
		print "Palm file", fileName, "cannot be opened\n\n"
		raise IOError
	try:
		#PalmFile.write(sig)
		if sig == "\x00\x01BA": # address book
			fileFormat = addressHeaderDef 
		elif sig == "\x00\x01BD": # datebook (calendar)
			fileFormat = calendarHeaderDef
		else:
			print "Unknown file format ", sig
			raise ValueError		
		writeRecords(PalmFile, fileFormat, fileData)
	except IOError:
		print "Unexpected error while writing Palm file"
		raise IOError
	if PalmFile : PalmFile.close()

def printAllNames(adBook):
	"""print all names in the address book

	demo of walking the address book structure
	"""
	addressDict = adBook[0]
	addresses = addressDict["addresses"]
	for address in addresses:
		print address["firstName"],address["lastName"]

def printAWeekWorthOfCalendar(calendar,traceRepeats=0):
	"""demo of walking the calendar data structure

	prints all events a week from now	
	"""
	import time 
	print "Your schedule next week:"
	events = getUpcomingEvents(calendar,7,traceRepeats)
	for event in events:
		if event['untimed']:
			print time.strftime("%x ***", time.localtime(event['startTime'])), event['text']
		else :
			print time.strftime("%c-", time.localtime(event['startTime'])), time.strftime("%X", time.localtime(event['endTime'])),event['text']
		if event['note']: print event['note']

def getEvents(calendar):
	return calendar[0]['datebookList']

def getUpcomingEvents(calendar, daysAhead, traceRepeats=0):
	#returns a list of event dictionaries from now until daysAhead days from now
	import time
	calendarDict = calendar[0]
	dateList = calendarDict['datebookList'] #we don't want to change the original calendar so we create a copy
	startTime = time.time();
	endTime = startTime + (daysAhead * 60 * 60 * 24) # converts daysAhead to seconds
	retVal = []
	#print 'Checking for events between',time.localtime(startTime),'and',time.localtime(endTime)
	for event in dateList:
		newEvent = event.copy()
		if newEvent['startTime'] > startTime and newEvent['startTime'] < endTime:
			#print 'Adding [', newEvent['text'], '] [', time.localtime(newEvent['startTime']), ']'
			#pprint.pprint(event)
			#print '\n\n'
			retVal.append(newEvent.copy())
		if traceRepeats and newEvent['repeatEvent']['repeatEventFlag'] and newEvent['repeatEvent']['endDate'] > startTime:
			newEvent = getNextRepeatedEvent(newEvent)
			#print 'Looking [',newEvent['text'],'] [',time.localtime(newEvent['startTime']),']'
			while newEvent['startTime'] < startTime:
				newEvent = getNextRepeatedEvent(newEvent)
				#print 'Looking [',newEvent['text'],'] [',time.localtime(newEvent['startTime']),']'
			while newEvent['startTime'] <= endTime and newEvent['startTime'] <= newEvent['repeatEvent']['endDate']:
				#print 'Adding [', newEvent['text'], '] [', time.localtime(newEvent['startTime']), ']'
				retVal.append(newEvent.copy())
				newEvent = getNextRepeatedEvent(newEvent)
	return retVal

def getNextRepeatedEvent(event):
	import time
	'''A sample from datebookList:
	'repeatEvent': {'brand': 5L,		Daily, Weekly, Monthly Date, Monthly Day, YEARLY
		'brandDayNumber': 4L,		4th
		'brandMonthIndex': 7L,		August
		'dateExceptionCount': 0,
		'endDate': 1028437200L,
		'firstDayOfWeek': 0L,
		'interval': 1L,
		'repeatEventFlag': 65535},
	'''
	repeatDetails = event['repeatEvent']
	repeatStartDST = time.localtime(event['startTime'])[8]
	if repeatDetails['brand'] == 1:	#daily repeat
		event['startTime'] = event['startTime'] + 60*60*24*repeatDetails['interval']
	elif repeatDetails['brand'] == 2:	# repeat weekly on specific days
		targetDaysMask = ord(repeatDetails['brandDaysMask']) #convert character read from file to ascii code
		#print 'targetDaysMask:',targetDaysMask
		repeatStartLocal=time.localtime(event['startTime'])
		if repeatStartLocal[6] == 5: # event's current start time is a Saturday
			'''add one day to get to Sunday, then add whole weeks
			according to the specified interval minus one, because we
			just went from Saturday to Sunday with the first addition.
			'''
			event['startTime'] = event['startTime'] + (60*60*24) + (60*60*24*7) * (repeatDetails['interval'] - 1)	
			repeatStartLocal=time.localtime(event['startTime'])
		else:
			event['startTime'] = event['startTime'] + (60*60*24)
			repeatStartLocal=time.localtime(event['startTime'])
		
		'''Python stores Wdays in element 6 of a time_struct,
		but stores them with Monday as 0 and Sun as 6.
		This line converts from the python time_struct to a WDay value with Sunday as 0
		'''
		rsWDay = (repeatStartLocal[6] + 1) % 7
		#print time.localtime(event['startTime']),'\n\trsWDay:',rsWDay,'\n\t',2**rsWDay,'\n\t',not ((2**rsWDay) & targetDaysMask)
		while not ((2**rsWDay) & targetDaysMask): # targetDaysMask is sum of the following: 1 = Sunday, 2 = Monday, 4 = Tues, . . . , 64 = Saturday
			event['startTime'] = event['startTime'] + (60*60*24)
			repeatStartLocal=time.localtime(event['startTime'])
			rsWDay = (repeatStartLocal[6] + 1) % 7
	elif repeatDetails['brand'] == 3:	#repeat monthly based on day
		from calendar import monthrange
		targetDay = repeatDetails['brandDayIndex'] #returns wday according to Python standards with Monday = 0
		targetWeek = repeatDetails['brandWeekIndex'] #returns week with first week = 0, fourth week = 3, and last week = 4
		for interval in range(1,repeatDetails['interval'] + 1):
			if targetWeek == 4: #event repeats on last ?day of the month.
				#event['startTime'] = event['startTime'] + (60*60*24*7) #add one week to current event
				newTime = event['startTime'] + (60*60*24*7*4) #add four weeks to current event
				newLocalTime = time.localtime(newTime)
				monthBegins, monthLength = monthrange(newLocalTime[0],newLocalTime[1])
				if (monthLength - newLocalTime[3]) >= 7: newTime = newTime + (60*60*24*7)
				event['startTime'] = newTime
			else: #repeats on 1st - 4th ?day of month
				newTime = event['startTime']
				newLocalTime = time.localtime(newTime)
				oldStartTimeMonth = newLocalTime[1]
				#advance event until it's the first occurrence of that weekday in the next month
				while newLocalTime[1] == oldStartTimeMonth:
					newTime = newTime + (60*60*24*7) #add one week to current event
					newLocalTime = time.localtime(newTime)
				event['startTime'] = newTime + (60*60*24*7*(targetWeek))
	elif repeatDetails['brand'] == 4:
		#repeat monthly based on date
		repeatStartLocal = time.localtime(event['startTime'])
		event['startTime'] = time.mktime((repeatStartLocal[0],repeatStartLocal[1]+repeatDetails['interval'],repeatStartLocal[2],repeatStartLocal[3],repeatStartLocal[4],repeatStartLocal[5],0,0,0))
	elif repeatDetails['brand'] == 5:
		#repeat yearly based on date
		repeatStartLocal = time.localtime(event['startTime'])
		event['startTime'] = time.mktime((repeatStartLocal[0]+repeatDetails['interval'],repeatStartLocal[1],repeatStartLocal[2],repeatStartLocal[3],repeatStartLocal[4],repeatStartLocal[5],0,0,0))
	
	newEventDST = time.localtime(event['startTime'])[8]
	correctDST = repeatStartDST - newEventDST
	event['startTime'] = event['startTime'] + (60*60*correctDST)
	return event

def getAppointment(event):
	return event

def createAppointment(event):
	return event

def changeAppointment(event):
	return


if __name__ == "__main__":
	import sys
	import getopt
	try:
		options, args = getopt.getopt(sys.argv[1:], 'v')
		fileToRead = args[0]
		verbose = len(options) > 0 and options[0][0] == "-v"
	except:
		print "Usage: python PalmFile.py [-v] <fileName>"
		print "-v for verbose, print all the data in the file"
		print "Prints out some sample data from a palm file"
		sys.exit(-1)
	palmData = readPalmFile(fileToRead)
	if verbose:	
		import pprint
		pprint.pprint(palmData)
	if palmData[0]["versionTag"] == 1094844672L: # addresses
		printAllNames(palmData)
	elif palmData[0]["versionTag"] == 1145176320L: # datebook
		printAWeekWorthOfCalendar(palmData)

  ----------

#IMPORTS

import palmFile
#import mozCal
import time



# FUNCTION DECLARATIONS
def iCalTimeFmt (timeval):
	_tmpVar = time.localtime(timeval)
	return str(_tmpVar[0])+str(_tmpVar[1]).zfill(2)+str(_tmpVar[2]).zfill(2)+'T'+str(_tmpVar[3]).zfill(2)+str(_tmpVar[4]).zfill(2)+str(_tmpVar[5]).zfill(2)

def getWeekDayString (targetDaysMask):
	weekdayList = ['SU','MO','TU','WE','TH','FR','SA']
	weekday = ''
	for i in range(7):
		if (2**i & targetDaysMask):
			weekday = weekday+weekdayList[i]+','
	return weekday


#BEGIN MAIN SCRIPT
import sys
for arg in sys.argv[1:]:
	userDir = 'c:\\program files\\palm\\'+arg
	palmDataFile = 'c:\\program files\\palm\\'+arg+'\\datebook\\datebook.dat'
	icsOutputPath = 'fromPalm.ics'
	fileStruct = palmFile.readPalmFile(palmDataFile)
	
	#fout = file('C:\\Documents and Settings\\Jeff\\Application Data\\Mozilla\\Profiles\\default\\mak3vke6.slt\\Calendar\\CalendarDataFile.ics','w')
	fout = file(icsOutputPath,'w')
	fout.write('BEGIN:VCALENDAR\nVERSION\n :2.0\nPRODID\n :-//Mozilla.org/NONSGML Mozilla Calendar V1.0//EN\n')
	
	eventsList = palmFile.getEvents(fileStruct)
	
	for event in eventsList:
		#set up the vevent information to write to file
		veventString = 'BEGIN:VEVENT\nSUMMARY\n :'+event['text'].replace('\n','\n ').replace('\r','') +'\nDESCRIPTION\n :'+event['note'].replace('\n','\n ').replace('\r','')+'\n'
		if event['private']:
			icalclass = 'private'
		else:
			icalclass = 'public'
		veventString = veventString + 'CLASS\n :'+icalclass+'\n'
		veventString = veventString + 'DTSTART\n :'+iCalTimeFmt(event['startTime'])+'\n'
		veventString = veventString + 'DTEND\n :'+iCalTimeFmt(event['endTime'])+'\n'
	
		recurString = ''
		if event['repeatEvent']['repeatEventFlag']:
			recurString = 'RRULE\n :'
			rDetails = event['repeatEvent']
			recurString = recurString + 'INTERVAL=' + str(int(rDetails['interval'])) + ';'
			if rDetails['endDate']:
				recurString = recurString + 'UNTIL=' + iCalTimeFmt(rDetails['endDate']) + ';'
			recurString = recurString + 'WKST=SU;' #specifies Sunday as the first day in a Palm Desktop work week
					
			if rDetails['brand'] == 1: #repeat daily
				recurString = recurString + 'FREQ=DAILY;'
			elif rDetails['brand'] == 2:	#repeat on specific days each week
				wdayString = getWeekDayString(ord(rDetails['brandDaysMask']))
				recurString = recurString + 'FREQ=WEEKLY;BYDAY='+wdayString+';'
			elif rDetails['brand'] == 3: #repeat monthly based on day
				targetDay = rDetails['brandDayIndex'] #returns wday according to Python standards with Monday = 0
				targetDay = (targetDay + 1) % 7 #convert to Sunday being indexed as 0
				weekdayList = ['SU','MO','TU','WE','TH','FR','SA']
				targetDay = weekdayList[targetDay]
				#print 'repeat monthly--targetday: ',targetDay
				
				targetWeek = rDetails['brandWeekIndex'] #returns week with first week = 0, fourth week = 3, and last week = 4
				targetWeek = targetWeek + 1
				if targetWeek == 5:
					targetWeek = -1
				recurString = recurString + 'FREQ=MONTHLY;BYDAY=' + str(targetWeek) + targetDay + ';'
			elif rDetails['brand'] == 4: #repeat monthly based on date
				recurString = recurString + 'FREQ=MONTHLY;'
			elif rDetails['brand'] == 5: #repeat yearly
				recurString = recurString + 'FREQ=YEARLY;'
		recurString = recurString[:-1] #remove trailing semicolon
		veventString = veventString + recurString + '\nEND:VEVENT\n'
		#print veventString
		fout.write(veventString)
	fout.write('END:VCALENDAR\n')
	fout.close()

[Non-text portions of this message have been removed]




Remember to visit Polls, Files and Bookmarks sections at http://groups.yahoo.com/group/progect. You are welcome to vote, upload your files and submit bookmarks. 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/progect/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/progect/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:[email protected] 
    mailto:[email protected]

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/