Re: PalmDB requests and bug report

On Wed, 27 Sep 2006, "Jeff Mikels" wrote:

> Is there a way for me to get SVN upload access? I'd like to submit my
> changes to your indentations and additions to comments/documentation where I
> can.

Sure, I need your SourceForge ID.

If you are going to put in documentation, you should create a new 
directory for it under the main PalmDB directory.

Rick


>
>
>
> On 9/27/06, Rick Price <[email protected]> wrote:
>>
>> Hi Jeff,
>>
>> I created a ToDo plugin for you, to help get you started (and I checked it
>> into SVN).
>>
>> It is called PalmToDoPlugin.py.
>>
>> I have not tried to compile it, and it is incomplete, but it should get
>> you started on your way. I put comments in where I expect you to have to
>> change things.
>>
>> If you have to do any more than configure how to crack the Palm task
>> header, contact me, because I missed something I don't expect you to have
>> to know...
>>
>> You can change the copyright to anything you want, I just copied the
>> Progect plugin and removed things you shouldn't need.
>>
>> And it includes the abominable tab/spaces :).
>>
>> Have Fun
>> Rick
>>
>>
>> On Wed, 27 Sep 2006, Rick Price wrote:
>>
>>>
>>>
>>> On Tue, 26 Sep 2006, "Jeff Mikels" wrote:
>>>
>>>> I should have said "unpack" or "crack" instead of decrypt. That was
>> silly of
>>>> me.
>>>
>>> No problem, when I reread your email after writing most of my reply, I
>>> realized what you had meant.
>>>
>>> Basically it was a bad day, and I wasn't really paying attention as I
>> read
>>> it.
>>>
>>>>
>>>> Anyway, about the plugin, the xml produced from the tododb by the basic
>>>> plugin is perfect except for the payload.
>>>>
>>>
>>> Yea, that makes sense, that is what is supposed to happen.
>>>
>>>> The todo db has 3 bytes, then a string for the description, then a
>> string
>>>> for the note. I'm pretty sure those three bytes are for the category
>> int,
>>>> the date, the deleted flag, and the completed flag.
>>>
>>> Some of this is already unpacked for you AFAIK. I think the category
>> int,
>>> and the deleted and so on flags are part of the header we unpack but I
>>> would have to check.
>>>
>>>>
>>>> It won't be difficult at all to implement, but I'm having trouble
>> figuring
>>>> out the API (yes, it's the latest SVN) and python's unpack doesn't have
>> the
>>>> option for variable length null terminated strings. What do you use?
>>>
>>> I believe I just cracked the string on the 0 at the end of the string. I
>>> believe the function is split(), and you pass zero in for the char
>>> somehow.
>>>
>>> I think the appropriate thing would be something like:
>>>
>>> Text,Note=incomingString.split('\0')
>>>
>>> You have to be careful when there isn't a note on the end, but I think
>>> there *should* always be a double zero in that case, in which case the
>>> above code should work.
>>>
>>>>
>>>>
>>>>
>>>> Help me out though, do I need anything more than this?
>>>>
>>>> class ToDoRecordPlugin(PalmDB.Plugins.BasePlugin.DataRecord):
>>>>    def getPDBCreatorID(self):
>>>>        return 'todo'
>>>>    def getPalmApplicationName(self):
>>>>        return 'ToDo'
>>>
>>> This part seems fine.
>>>
>>>>
>>>>    def _crackPayload(self,dstr):
>>>>        [code here]
>>>>
>>>
>>> This part is where you should have overridden the call to create the
>>> object that is for each Palm record, that object implements
>> _crackPayload.
>>> I'm hoping to have the time to do it quickly to show you how. It should
>>> only take 15 minutes to half an hour for me to write the skeleton for
>> the
>>> plugin for you.
>>>
>>> Then you can implement _crackPayload and be done with it.
>>>
>>>
>>> Once the dust settles, would you have time to write a quick howto?
>>>
>>> Since you didn't design things, you should be in a better position to
>>> start the document, I can expand on things for you as required.
>>>
>>>>
>>>>
>>>> Oh, one more thing, your whitespace is quite inconsistent in your code.
>> You
>>>> have spaces and tabs all over the place.
>>>
>>> Yes, I would dearly love to find a good Python reformatter program.
>>>
>>> Rick
>>>
>>>
>>>>
>>>> On 9/26/06, Rick Price <[email protected]> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Tue, 26 Sep 2006, "Jeff Mikels" wrote:
>>>>>
>>>>>> I'm having a problem with the crackPalmDate function
>>>>>>
>>>>>> (I'm using svn code)
>>>>>>
>>>>>> PILOT_TIME_DELTA = 2082844800L
>>>>>> def crackPalmDate(variable):
>>>>>>        if variable == 0:
>>>>>>            return None
>>>>>>        else:
>>>>>>            return datetime.datetime.fromtimestamp
>>>>> (variable-PILOT_TIME_DELTA)
>>>>>>
>>>>>> I often get a ValueError because variable-PILOT_TIME_DELTA is
>> sometimes
>>>>> a
>>>>>> negative number. I've changed the code to be...
>>>>>>
>>>>>> PILOT_TIME_DELTA = 2082844800L
>>>>>> def crackPalmDate(variable):
>>>>>>        if variable == 0 or variable < PILOT_TIME_DELTA:
>>>>>>            return None
>>>>>>        else:
>>>>>>            return datetime.datetime.fromtimestamp
>>>>>> (variable-PILOT_TIME_DELTA)
>>>>>>
>>>>>> Is this a problem?
>>>>>
>>>>> No, I don't think so, except that if the variable is before the
>>>>> PILOT_TIME_DELTA surprises could possibly happen in the Palm...
>>>>>
>>>>> The reasoning behind my code (and it could be flawed), is that we want
>> to
>>>>> know if there really wasn't a value there, like in SQL. So since I
>> didn't
>>>>> think that the value would ever be zero (or below zero), using zero
>> for
>>>>> NULL seemed like a perfect solution.
>>>>>
>>>>> If that is not true, then we may need to rethink how things work.
>>>>>
>>>>> Can you send me a patch with your changes so I can incorporate it?
>>>>>
>>>>> Do make sure you have the latest code from SVN, I haven't changed it
>> for a
>>>>> few weeks, but it's best to make sure you have the latest.
>>>>>
>>>>>>
>>>>>>
>>>>>> Additionally, I'd like to create a "plugin" for the todo database,
>> but
>>>>> I'm
>>>>>> not sure how to go about doing it. The Base Plugin class is working
>>>>>> perfectly but the payload needs to be decrypted properly. How can I
>>>>> create a
>>>>>> plugin that only overloads the payload functions and have it loaded
>> when
>>>>>> necessary?
>>>>>>
>>>>>
>>>>> Well, I like plugins that are loaded automagically and that can be
>>>>> distributed separately, but in this case, I decided to keep it simple
>> and
>>>>> have you declare the plugin to the code.
>>>>>
>>>>> I also figured that we would just roll the plugins (not created by me)
>>>>> into the release, since just about anything someone makes will be
>> useful
>>>>> to others.
>>>>>
>>>>> [ If you use the library for something proprietary, you would just
>> call
>>>>> the plugin notification code, and keep your code separate. ]
>>>>>
>>>>> In the file PluginManager.py, you will find the functions
>>>>> registerPDBPlugin and deRegisterPDBPlugin. These are the functions you
>> use
>>>>> to register your plugin with the framework.
>>>>>
>>>>> At the bottom of the file, I use the standard functions to register my
>>>>> Progect plugin.
>>>>>
>>>>>
>>>>> Now, I'm not really sure what you mean by decrypted.
>>>>>
>>>>> If you mean you need to translate the XML that *would* have been
>> generated
>>>>> into something more useful, the code knows how to apply XSLT
>> stylesheets
>>>>> for you; both coming and going. It can also do some simple
>> transformations
>>>>> like gzip.
>>>>>
>>>>> If you need to output to say a binary or text format, I think it can
>> be
>>>>> done, but I would have to look at the code again to explain how.
>>>>>
>>>>>
>>>>> I really think I misdesigned the plugin somehow as well, but I can't
>>>>> really describe what I mean except to say this (because I haven't
>> thought
>>>>> it through yet):
>>>>> * Some classes do more than one thing - very bad
>>>>> * The symptom is that the plugin is difficult to use.
>>>>> * I expect that writing/reading formats other than XML will
>>>>> be more difficult than it should be (as in binary or text).
>>>>>
>>>>>
>>>>> If you can describe what you are doing to me, then maybe I can give
>> you
>>>>> some suggestions.
>>>>>
>>>>> I'm going to have to redesign the code _again_ because of not getting
>> the
>>>>> plugin code right, but I will try and remain compatible, or at least
>> make
>>>>> it so much easier to use that it's not hard for you to update your
>> code.
>>>>>
>>>>>
>>>>> If you mean how do I crack the Palm task format, I can certainly help
>> you
>>>>> with that too. Basically the plugin you create returns an object that
>>>>> knows what to do with the raw data as passed in by the framework.
>>>>>
>>>>> You will need to find documentation on the Palm task format, I am
>> pretty
>>>>> sure it can be found easily with Google.
>>>>>
>>>>> In this case, you would probably just subclass the default plugin so
>> that
>>>>> it returns a special Task object that knows how to crack the payload
>> for
>>>>> tasks, and puts the values into a dictionary.
>>>>>
>>>>> I believe the class ProgectRecord in plugins/ProgectPlugin.py is the
>>>>> closest thing to what you want. And the important method to create is
>>>>> _crackPayload.
>>>>>
>>>>> The problem is, that the Progect format is quite complex, and we are
>>>>> forcing a tree into rows. So the Progect class is probably not a good
>>>>> example of how to do something simple (or not a good example at
>> all...).
>>>>>
>>>>> For something like the task database, you should have almost _no_ work
>> to
>>>>> do, and so don't let the extreme complexity of the Progect plugin
>> scare
>>>>> you off.
>>>>>
>>>>> I can't predict how much work cracking a task would be, but I would
>> hope
>>>>> it would be under a hundred lines of Python code all told, and simple
>> code
>>>>> at that.
>>>>>
>>>>> Generating and reading the XML would happen automatically if I
>> remember
>>>>> correctly, so the big problem is being able to crack and pack the task
>>>>> format. (that is, the payload, because we crack the record header for
>>>>> you).
>>>>>
>>>>> I would be more than happy to give you a hand with this task, while I
>> am
>>>>> overloaded at work for the next day or so, perhaps we can talk via IM
>> or
>>>>> on the phone. Even email would be fine, if you can help me understand
>> your
>>>>> problem a little better.
>>>>>
>>>>> I can then explain how I think it can be accomplished easily with the
>>>>> framework.
>>>>>
>>>>> I will certainly try to do this for anyone who wants to use the
>> framework,
>>>>> but having the task format converted to and from XML is an important
>>>>> feature for the framework and is therefore a priority for me.
>>>>>
>>>>>
>>>>> Rick
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 9/7/06, Rick Price <[email protected]> wrote:
>>>>>>>
>>>>>>> Okay, I just managed to get the Python PalmDB library to go from a
>>>>> Progect
>>>>>>> file to XML and back again without dropping anything really obvious.
>>>>>>>
>>>>>>> Anyone who wants to check out the Subversion code is welcome to do
>> so.
>>>>>>>
>>>>>>> I think I need to do a major rewrite at some point in the next
>> little
>>>>> bit,
>>>>>>> but first I will setup some conversions to popular programs that
>>>>>>> read/write XML.
>>>>>>>
>>>>>>> First on the list is going to be treeline since I use it right now
>> and
>>>>> the
>>>>>>>
>>>>>>> format seems to be basically sane.
>>>>>>>
>>>>>>> I will then probably try to do something with Omni Outliner for the
>>>>> Mac.
>>>>>>>
>>>>>>> That will probably require fixing a bug in the library that seems to
>>>>> show
>>>>>>> up on the Mac.
>>>>>>>
>>>>>>> After that, I will see what I can do based on popularity.
>>>>>>>
>>>>>>>
>>>>>>> Once I have treeline support working, and it's not too ugly to run,
>> I
>>>>> will
>>>>>>> make a release of Python PalmDB so people can start to use it.
>>>>>>>
>>>>>>> I will release again I would guess after I get Omni Outliner support
>>>>>>> working.
>>>>>>>
>>>>>>> Any questions, just email me.
>>>>>>>
>>>>>>> Rick
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Remember to visit Polls, Files and Bookmarks sections at
>>>>>>> http://groups.yahoo.com/group/progect. You are welcome to vote,
>> upload
>>>>>>> your files and submit bookmarks.
>>>>>>> Yahoo! Groups Links
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Jeff Mikels
>>>>>> leading people one step closer to Jesus
>>>>>> http://jeff.mikels.cc
>>>>>> http://thesouthsidechurch.org
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Jeff Mikels
>>>>>> leading people one step closer to Jesus
>>>>>> http://jeff.mikels.cc
>>>>>> http://thesouthsidechurch.org
>>>>>>
>>>>>>
>>>>>> [Non-text portions of this message have been removed]
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Remember to visit Polls, Files and Bookmarks sections at
>>>>> http://groups.yahoo.com/group/progect. You are welcome to vote, upload
>>>>> your files and submit bookmarks.
>>>>> Yahoo! Groups Links
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Jeff Mikels
>>>> leading people one step closer to Jesus
>>>> http://jeff.mikels.cc
>>>> http://thesouthsidechurch.org
>>>>
>>>>
>>>> [Non-text portions of this message have been removed]
>>>>
>>>>
>>>
>>
>>
>>
>> Remember to visit Polls, Files and Bookmarks sections at
>> http://groups.yahoo.com/group/progect. You are welcome to vote, upload
>> your files and submit bookmarks.
>> Yahoo! Groups Links
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> -- 
> Jeff Mikels
> leading people one step closer to Jesus
> http://jeff.mikels.cc
> http://thesouthsidechurch.org
>
>
> [Non-text portions of this message have been removed]
>
>



Remember to visit Polls, Files and Bookmarks sections at http://groups.yahoo.com/group/progect. You are welcome to vote, upload your files and submit bookmarks. 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/progect/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/progect/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:[email protected] 
    mailto:[email protected]

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/