Re: importing bank statements

Hi Steve,
i'd love to collaborate as well. I'd be interested to know the following:
1) What functionality does the "fake" product provide
2) Is your code open source/a product you wish to sell?
3) How do you handle security (passwords etc...). Do you have to type
the passwords in manually everytime?
4) Can you program be used as a plugin to moneydance?

What i had in mind was something like the following:
1) have an input file in which the user can specify the relevant fields
for his specific bank site (i.e. login/passwd field, etc...)
Here the problem is security as i would not want to store passwords
(especially to bank accounts) in plain text files. Not sure how to deal
with that yet but i am sure somebody has done something like it before.
2) Then i was going to simulate the browser via the mechanize module in
python.

matt


On 5/19/2011 10:48 AM, Rob Lund wrote:
> Sounds like a great project, Steve!  I'd be happy to contribute, though I'm
> no JS developer.
>
> On Thu, May 19, 2011 at 10:00 AM, <[email protected]>wrote:
>
>>
>> I have been working on a solution off and on (more off than on) for a
>> few years now. There are a few different approaches to take.
>>
>> 1. Automate your browser
>> 2. Write a cURL script
>> 3. Write a program in your favorite computer language
>>
>> Writing a program in your favorite computer language is a boatload of
>> work.
>>
>> Writing a cURL script is often the recommended approach for automating
>> downloading of files from any web site. The usual advice is to use the
>> LiveHTTPHeaders plugin for Firefox to capture the HTTP requests sent
>> from Firefox to your bank and then rebuild each of those requests in
>> cURL. This would work for most web sites, but banks tend to have lots
>> of security on their web sites. Some of those security measures are
>> designed to prevent automated scripts from accessing the bank's web
>> site. For example, I found that using cURL to login to one bank's web
>> site would fail unless I had also used curl to download one tiny little
>> .gif file. Yes, the bank's server would fail any login attempt if that
>> one .gif file had not been downloaded. That can be overcome but then
>> you'll quickly run in to other problems. One such problem is that banks
>> often have the browser execute some JavaScript to calculate a value
>> that then gets sent back to the bank's server. cURL doesn't have a
>> JavaScript engine so you'd need to find some other tool that can
>> execute the JavaScript for you.
>>
>> Automating a browser is the best approach I've found so far. At first I
>> tried using the iMacros plugin for Firefox, but found it very difficult
>> to handle branching on certain conditions. Eventually I gave up on
>> iMacros. After I learned to write JavaScript I realized that
>> JavaScript's very purpose is browser automation. All I needed was a
>> browser that would let me "inject" some JavaScript into the downloaded
>> copy of the bank's web page. Unless you are using a browser debugger
>> most browsers specifically do not allow this for security reasons.
>> Using a debugger I was able to write a complete script to login to one
>> specific bank, parse each page, navigate to the downloads page and
>> download the transactions as an .ofx file.  I became convinced that
>> JavaScript is absolutely the correct approach.
>>
>> So I had decided to write my own special browser app that would allow
>> me to inject JavaScript. Earlier I claimed that writing a special
>> program would be a lot of work, and it would have been. Fortunately
>> before I invested too much time in writing my own program I found a
>> commercial product named "Fake."  Fake is a program for OS X. Fake is a
>> "fake" browser. Fake lets you create a workflow for automating
>> navigation of a web site. One of the things Fake allows you to do is
>> inject JavaScript into a bank's web page. Using Fake I was able to
>> fully automate downloading transactions from one specific bank.
>>
>> What I need to do now is break the JavaScript code into modules so that
>> code specific to one bank is separate from code that is generic to all
>> banks. I'd be happy to collaborate with anyone who wants to help build
>> up a library of code that can automate downloading transactions and
>> statements from any bank's web site. Let me know if you are interested.
>>
>> -SteveL
>>
> _______________________________________________
> moneydance-info mailing list
> [email protected]
> http://moneydance.com/mailman/listinfo/moneydance-info
>