Using Perl to get data from website
fiazidris <[email protected]>
| Newsgroups | gmane.comp.lang.perl.modules.lwp |
|---|---|
| Organization | http://groups.google.com |
| Message-ID | <f91b7a39-ec3b-4891-8f2f-bd1e3df05ada@s12g2000prg.googlegroups.com> |
Previously, I have written a perl script to access data from this URL:
http://www.bangkokflightservices.com/our_cargo_track.php
Some sample: MAWB - Master Airwaybill Number
724-61972116
724-61976051
724-61238144
But, now there is a change on the website and I couldn't extract
through the same script. One change I noticed is the URL has changed.
How can I programmatically obtain data for a list of MAWBs.
I have reached to a level where the following URL works on a browser:
prefix and serials can be changed:
http://203.151.118.123:8090/showc_track.php?m_prefix=724&m_sn=61972116&h_prefix=HWB&h_sn=&ecy=e076438db64c6190f7b9689a379b7f7093368f1652d14db65fee1ab916713f3f5f4030f53369cb1f669614312c4748899c272f4d976a2b299274a21ad80fc072b1bab2ab1c181d08c670188722e51ec162f9ae337e3f2f132c88d249133815558d241ce8a4e9b3fa75c144268b9e901037c2c7257142ee42ff9b2bf2767f57ed62b94fd938ea4dd2b28c53fea6af74be&ch=%A0%A0%A0%A0
but this URL doesn't return results using perl. When I get it using
this method, the website throws me to the top level page without
showing the results.
Your assistance on this is highly appreciated.
### This is the complete code I used.
#!/usr/bin/perl
use WWW::Mechanize;
my $baseurl =
'http://www.bangkokflightservices.com/our_cargo_track&trace.php';
my $hawb = 'h_prefix=HAWB&h_sn=';
my $M = WWW::Mechanize->new(auto_check => 1);
## Added code for testing Only
my $F = WWW::Mechanize->new(auto_check => 1);
$F->get("http://www.bangkokflightservices.com/
our_cargo_track.php");
my $contentF = $F->content;
#print "$contentF\n";
#$M->add_header("Referer => 'http://
www.bangkokflightservices.com/
our_cargo_track.php'" )
while (<>) {
chomp;
my ($mprefix, $msn) = /(...)-(........)/ or do {
warn "invalid MAWB: '$_'";
next;
};
print "$mprefix $msn\n";
$M->get("$baseurl?m_prefix=$mprefix&m_sn=$msn&$hawb");
$M->follow_link(url_regex => qr/showc_track/);
my $content = $M->content;
print "$content\n"; # for debugging
# process $content as before
#
while ( $content =~ m#(.*)#g ) {
$currline=$1;
if ($currline =~ m#style12#i) {
$currline =~ m#.*>(.*?)<.*#i;
$result = $result . " / " . $1;
}
}
print "***$result\n";
$result = '';
}