Re: PalmDB requests and bug report

Hi Jeff,

I created a ToDo plugin for you, to help get you started (and I checked it 
into SVN).

It is called PalmToDoPlugin.py.

I have not tried to compile it, and it is incomplete, but it should get 
you started on your way. I put comments in where I expect you to have to 
change things.

If you have to do any more than configure how to crack the Palm task 
header, contact me, because I missed something I don't expect you to have 
to know...

You can change the copyright to anything you want, I just copied the 
Progect plugin and removed things you shouldn't need.

And it includes the abominable tab/spaces :).

Have Fun
Rick


On Wed, 27 Sep 2006, Rick Price wrote:

>
>
> On Tue, 26 Sep 2006, "Jeff Mikels" wrote:
>
>> I should have said "unpack" or "crack" instead of decrypt. That was silly of
>> me.
>
> No problem, when I reread your email after writing most of my reply, I
> realized what you had meant.
>
> Basically it was a bad day, and I wasn't really paying attention as I read
> it.
>
>>
>> Anyway, about the plugin, the xml produced from the tododb by the basic
>> plugin is perfect except for the payload.
>>
>
> Yea, that makes sense, that is what is supposed to happen.
>
>> The todo db has 3 bytes, then a string for the description, then a string
>> for the note. I'm pretty sure those three bytes are for the category int,
>> the date, the deleted flag, and the completed flag.
>
> Some of this is already unpacked for you AFAIK. I think the category int,
> and the deleted and so on flags are part of the header we unpack but I
> would have to check.
>
>>
>> It won't be difficult at all to implement, but I'm having trouble figuring
>> out the API (yes, it's the latest SVN) and python's unpack doesn't have the
>> option for variable length null terminated strings. What do you use?
>
> I believe I just cracked the string on the 0 at the end of the string. I
> believe the function is split(), and you pass zero in for the char
> somehow.
>
> I think the appropriate thing would be something like:
>
> Text,Note=incomingString.split('\0')
>
> You have to be careful when there isn't a note on the end, but I think
> there *should* always be a double zero in that case, in which case the
> above code should work.
>
>>
>>
>>
>> Help me out though, do I need anything more than this?
>>
>> class ToDoRecordPlugin(PalmDB.Plugins.BasePlugin.DataRecord):
>>    def getPDBCreatorID(self):
>>        return 'todo'
>>    def getPalmApplicationName(self):
>>        return 'ToDo'
>
> This part seems fine.
>
>>
>>    def _crackPayload(self,dstr):
>>        [code here]
>>
>
> This part is where you should have overridden the call to create the
> object that is for each Palm record, that object implements _crackPayload.
> I'm hoping to have the time to do it quickly to show you how. It should
> only take 15 minutes to half an hour for me to write the skeleton for the
> plugin for you.
>
> Then you can implement _crackPayload and be done with it.
>
>
> Once the dust settles, would you have time to write a quick howto?
>
> Since you didn't design things, you should be in a better position to
> start the document, I can expand on things for you as required.
>
>>
>>
>> Oh, one more thing, your whitespace is quite inconsistent in your code. You
>> have spaces and tabs all over the place.
>
> Yes, I would dearly love to find a good Python reformatter program.
>
> Rick
>
>
>>
>> On 9/26/06, Rick Price <[email protected]> wrote:
>>>
>>>
>>>
>>> On Tue, 26 Sep 2006, "Jeff Mikels" wrote:
>>>
>>>> I'm having a problem with the crackPalmDate function
>>>>
>>>> (I'm using svn code)
>>>>
>>>> PILOT_TIME_DELTA = 2082844800L
>>>> def crackPalmDate(variable):
>>>>        if variable == 0:
>>>>            return None
>>>>        else:
>>>>            return datetime.datetime.fromtimestamp
>>> (variable-PILOT_TIME_DELTA)
>>>>
>>>> I often get a ValueError because variable-PILOT_TIME_DELTA is sometimes
>>> a
>>>> negative number. I've changed the code to be...
>>>>
>>>> PILOT_TIME_DELTA = 2082844800L
>>>> def crackPalmDate(variable):
>>>>        if variable == 0 or variable < PILOT_TIME_DELTA:
>>>>            return None
>>>>        else:
>>>>            return datetime.datetime.fromtimestamp
>>>> (variable-PILOT_TIME_DELTA)
>>>>
>>>> Is this a problem?
>>>
>>> No, I don't think so, except that if the variable is before the
>>> PILOT_TIME_DELTA surprises could possibly happen in the Palm...
>>>
>>> The reasoning behind my code (and it could be flawed), is that we want to
>>> know if there really wasn't a value there, like in SQL. So since I didn't
>>> think that the value would ever be zero (or below zero), using zero for
>>> NULL seemed like a perfect solution.
>>>
>>> If that is not true, then we may need to rethink how things work.
>>>
>>> Can you send me a patch with your changes so I can incorporate it?
>>>
>>> Do make sure you have the latest code from SVN, I haven't changed it for a
>>> few weeks, but it's best to make sure you have the latest.
>>>
>>>>
>>>>
>>>> Additionally, I'd like to create a "plugin" for the todo database, but
>>> I'm
>>>> not sure how to go about doing it. The Base Plugin class is working
>>>> perfectly but the payload needs to be decrypted properly. How can I
>>> create a
>>>> plugin that only overloads the payload functions and have it loaded when
>>>> necessary?
>>>>
>>>
>>> Well, I like plugins that are loaded automagically and that can be
>>> distributed separately, but in this case, I decided to keep it simple and
>>> have you declare the plugin to the code.
>>>
>>> I also figured that we would just roll the plugins (not created by me)
>>> into the release, since just about anything someone makes will be useful
>>> to others.
>>>
>>> [ If you use the library for something proprietary, you would just call
>>> the plugin notification code, and keep your code separate. ]
>>>
>>> In the file PluginManager.py, you will find the functions
>>> registerPDBPlugin and deRegisterPDBPlugin. These are the functions you use
>>> to register your plugin with the framework.
>>>
>>> At the bottom of the file, I use the standard functions to register my
>>> Progect plugin.
>>>
>>>
>>> Now, I'm not really sure what you mean by decrypted.
>>>
>>> If you mean you need to translate the XML that *would* have been generated
>>> into something more useful, the code knows how to apply XSLT stylesheets
>>> for you; both coming and going. It can also do some simple transformations
>>> like gzip.
>>>
>>> If you need to output to say a binary or text format, I think it can be
>>> done, but I would have to look at the code again to explain how.
>>>
>>>
>>> I really think I misdesigned the plugin somehow as well, but I can't
>>> really describe what I mean except to say this (because I haven't thought
>>> it through yet):
>>> * Some classes do more than one thing - very bad
>>> * The symptom is that the plugin is difficult to use.
>>> * I expect that writing/reading formats other than XML will
>>> be more difficult than it should be (as in binary or text).
>>>
>>>
>>> If you can describe what you are doing to me, then maybe I can give you
>>> some suggestions.
>>>
>>> I'm going to have to redesign the code _again_ because of not getting the
>>> plugin code right, but I will try and remain compatible, or at least make
>>> it so much easier to use that it's not hard for you to update your code.
>>>
>>>
>>> If you mean how do I crack the Palm task format, I can certainly help you
>>> with that too. Basically the plugin you create returns an object that
>>> knows what to do with the raw data as passed in by the framework.
>>>
>>> You will need to find documentation on the Palm task format, I am pretty
>>> sure it can be found easily with Google.
>>>
>>> In this case, you would probably just subclass the default plugin so that
>>> it returns a special Task object that knows how to crack the payload for
>>> tasks, and puts the values into a dictionary.
>>>
>>> I believe the class ProgectRecord in plugins/ProgectPlugin.py is the
>>> closest thing to what you want. And the important method to create is
>>> _crackPayload.
>>>
>>> The problem is, that the Progect format is quite complex, and we are
>>> forcing a tree into rows. So the Progect class is probably not a good
>>> example of how to do something simple (or not a good example at all...).
>>>
>>> For something like the task database, you should have almost _no_ work to
>>> do, and so don't let the extreme complexity of the Progect plugin scare
>>> you off.
>>>
>>> I can't predict how much work cracking a task would be, but I would hope
>>> it would be under a hundred lines of Python code all told, and simple code
>>> at that.
>>>
>>> Generating and reading the XML would happen automatically if I remember
>>> correctly, so the big problem is being able to crack and pack the task
>>> format. (that is, the payload, because we crack the record header for
>>> you).
>>>
>>> I would be more than happy to give you a hand with this task, while I am
>>> overloaded at work for the next day or so, perhaps we can talk via IM or
>>> on the phone. Even email would be fine, if you can help me understand your
>>> problem a little better.
>>>
>>> I can then explain how I think it can be accomplished easily with the
>>> framework.
>>>
>>> I will certainly try to do this for anyone who wants to use the framework,
>>> but having the task format converted to and from XML is an important
>>> feature for the framework and is therefore a priority for me.
>>>
>>>
>>> Rick
>>>
>>>>
>>>>
>>>>
>>>> On 9/7/06, Rick Price <[email protected]> wrote:
>>>>>
>>>>> Okay, I just managed to get the Python PalmDB library to go from a
>>> Progect
>>>>> file to XML and back again without dropping anything really obvious.
>>>>>
>>>>> Anyone who wants to check out the Subversion code is welcome to do so.
>>>>>
>>>>> I think I need to do a major rewrite at some point in the next little
>>> bit,
>>>>> but first I will setup some conversions to popular programs that
>>>>> read/write XML.
>>>>>
>>>>> First on the list is going to be treeline since I use it right now and
>>> the
>>>>>
>>>>> format seems to be basically sane.
>>>>>
>>>>> I will then probably try to do something with Omni Outliner for the
>>> Mac.
>>>>>
>>>>> That will probably require fixing a bug in the library that seems to
>>> show
>>>>> up on the Mac.
>>>>>
>>>>> After that, I will see what I can do based on popularity.
>>>>>
>>>>>
>>>>> Once I have treeline support working, and it's not too ugly to run, I
>>> will
>>>>> make a release of Python PalmDB so people can start to use it.
>>>>>
>>>>> I will release again I would guess after I get Omni Outliner support
>>>>> working.
>>>>>
>>>>> Any questions, just email me.
>>>>>
>>>>> Rick
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Remember to visit Polls, Files and Bookmarks sections at
>>>>> http://groups.yahoo.com/group/progect. You are welcome to vote, upload
>>>>> your files and submit bookmarks.
>>>>> Yahoo! Groups Links
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Jeff Mikels
>>>> leading people one step closer to Jesus
>>>> http://jeff.mikels.cc
>>>> http://thesouthsidechurch.org
>>>>
>>>>
>>>> --
>>>> Jeff Mikels
>>>> leading people one step closer to Jesus
>>>> http://jeff.mikels.cc
>>>> http://thesouthsidechurch.org
>>>>
>>>>
>>>> [Non-text portions of this message have been removed]
>>>>
>>>>
>>>
>>>
>>>
>>> Remember to visit Polls, Files and Bookmarks sections at
>>> http://groups.yahoo.com/group/progect. You are welcome to vote, upload
>>> your files and submit bookmarks.
>>> Yahoo! Groups Links
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Jeff Mikels
>> leading people one step closer to Jesus
>> http://jeff.mikels.cc
>> http://thesouthsidechurch.org
>>
>>
>> [Non-text portions of this message have been removed]
>>
>>
>



Remember to visit Polls, Files and Bookmarks sections at http://groups.yahoo.com/group/progect. You are welcome to vote, upload your files and submit bookmarks. 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/progect/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/progect/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:[email protected] 
    mailto:[email protected]

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/