Re: importing bank statements

Sounds like a great project, Steve!  I'd be happy to contribute, though I'm
no JS developer.

On Thu, May 19, 2011 at 10:00 AM, <[email protected]>wrote:

>
>
> I have been working on a solution off and on (more off than on) for a
> few years now. There are a few different approaches to take.
>
> 1. Automate your browser
> 2. Write a cURL script
> 3. Write a program in your favorite computer language
>
> Writing a program in your favorite computer language is a boatload of
> work.
>
> Writing a cURL script is often the recommended approach for automating
> downloading of files from any web site. The usual advice is to use the
> LiveHTTPHeaders plugin for Firefox to capture the HTTP requests sent
> from Firefox to your bank and then rebuild each of those requests in
> cURL. This would work for most web sites, but banks tend to have lots
> of security on their web sites. Some of those security measures are
> designed to prevent automated scripts from accessing the bank's web
> site. For example, I found that using cURL to login to one bank's web
> site would fail unless I had also used curl to download one tiny little
> .gif file. Yes, the bank's server would fail any login attempt if that
> one .gif file had not been downloaded. That can be overcome but then
> you'll quickly run in to other problems. One such problem is that banks
> often have the browser execute some JavaScript to calculate a value
> that then gets sent back to the bank's server. cURL doesn't have a
> JavaScript engine so you'd need to find some other tool that can
> execute the JavaScript for you.
>
> Automating a browser is the best approach I've found so far. At first I
> tried using the iMacros plugin for Firefox, but found it very difficult
> to handle branching on certain conditions. Eventually I gave up on
> iMacros. After I learned to write JavaScript I realized that
> JavaScript's very purpose is browser automation. All I needed was a
> browser that would let me "inject" some JavaScript into the downloaded
> copy of the bank's web page. Unless you are using a browser debugger
> most browsers specifically do not allow this for security reasons.
> Using a debugger I was able to write a complete script to login to one
> specific bank, parse each page, navigate to the downloads page and
> download the transactions as an .ofx file.  I became convinced that
> JavaScript is absolutely the correct approach.
>
> So I had decided to write my own special browser app that would allow
> me to inject JavaScript. Earlier I claimed that writing a special
> program would be a lot of work, and it would have been. Fortunately
> before I invested too much time in writing my own program I found a
> commercial product named "Fake."  Fake is a program for OS X. Fake is a
> "fake" browser. Fake lets you create a workflow for automating
> navigation of a web site. One of the things Fake allows you to do is
> inject JavaScript into a bank's web page. Using Fake I was able to
> fully automate downloading transactions from one specific bank.
>
> What I need to do now is break the JavaScript code into modules so that
> code specific to one bank is separate from code that is generic to all
> banks. I'd be happy to collaborate with anyone who wants to help build
> up a library of code that can automate downloading transactions and
> statements from any bank's web site. Let me know if you are interested.
>
> -SteveL
>