Re: Sudden problems with LWP::UserAgent

Thanks for the suggestion. I use liveHTTP for firefox. I guess I may
have to do more modification to make my script behave exactly like a
browser. I had already tried RobotUA.

Thanks
Uday

On Wed, Oct 27, 2010 at 10:08 PM, Keary Suska <[email protected]> wrote:
> On Oct 27, 2010, at 1:15 AM, Uday Bhaskar wrote:
>
>> I guess this is more a question of how to get around a website
>> blocking LWP::UserAgent. I have been using this code for a while now
>>
>> <snip>
>> This code worked till yesterday. However today I am hitting the 403 error.
>> "
>> error in getting index 403 Forbidden
>> try again...
>> "
>>
>> I used the same UserAgent as that of Mozilla which successfully
>> retrieves the page. I tried it from different IP address to ensure
>> that my IP is not blocked(I run it once a day and am not running a
>> bot) Looking for some help in getting this going again.
>
> Some sites do browser fingerprinting. Most of the time you can fool the site by sending the same headers that the browser sends. You can inspect headers using a plug=in like HttpFox, which is my favorite.
>
> HTH,
>
> Keary Suska
>
>