Fw: Logging in to form based log in page #5

Nevermind, I got it.  Below is the code.  Thanks to all for everyone's help.
What a GREAT list.  Never belonged to one with people who responded so
quickly.  Thanks again,
Kenny
se strict;
use HTTP::Request::Common;
use HTML::Form;
use LWP::UserAgent;
#use LWP::Debug qw(+);
use HTTP::Cookies;
use FileHandle;
#BEGIN
#{
#    local $^W = 0;
#    *LWP::UserAgent::redirect_ok = sub {1}
#}
my $ua = LWP::UserAgent->new();
$ua->agent("Automation/0.1");
$ua->cookie_jar(
  HTTP::Cookies->new( autosave => 1 ));
my $request = $ua->request( POST
'http://online.wsj.com/login?URI=%2Fdocuments%2Fdatabank.htm');
my $form = HTML::Form->parse( $request->content, $request->base());
#print $form->dump;
$form->value( 'Login',   "kcmria" );
$form->value( 'Password', "money" );
my $response = $ua->request( $form->click() );
$response = $ua->request(GET
'http://online.wsj.com/documents/databank.htm');
my @outputLines = split(/\n/, $response->content);
my $outputFileHandle = new FileHandle;
$outputFileHandle->open(">d:\\etl\\temp\\temp.html");
foreach my $line (@outputLines)
{
   print $outputFileHandle $line;
}
$outputFileHandle->close;
#print $response ->as_string;

----- Original Message -----
From: "Kenny G. Dubuisson, Jr." <kdubuisson@kcmria.com>
To: <libwww@perl.org>
Sent: Tuesday, July 09, 2002 3:01 PM
Subject: Fw: Logging in to form based log in page

> When I manually do this process, after I click the log in button, I get a
> dialog box warning stating that I'm about to transfer to a non-secure
page.
> I have to click "OK" and then the browser (IE) goes on to the desired
page.
> Do you think that the dialog box could be hangin it up?  Do you know of a
> way to clear that dialog box from the code?
>
> Thanks again,
> Kenny
>
> ----- Original Message -----
> From: "Hill, Ronald" <ronald.hill@eds.com>
> To: "'Kenny G. Dubuisson, Jr.'" <kdubuisson@kcmria.com>; <libwww@perl.org>
> Sent: Tuesday, July 09, 2002 2:53 PM
> Subject: RE: Logging in to form based log in page
>
>
> >
> > [snipped]
> > >
> > > Ron:
> > >
> > > Does it have to select the second one...i.e. would the first
> > > (default) one
> > > work?  Thanks,
> > > Kenny
> > I believe it is selecting the first one, and it will just hang. Here is
> the
> > output from the debug
> >
> > LWP::UserAgent::new: ()
> > LWP::UserAgent::request: ()
> > HTTP::Cookies::add_cookie_header: Checking online.wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: Checking .wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: Checking wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: Checking .com for cookies
> > LWP::UserAgent::send_request: POST http://online.wsj.com/login
> > LWP::UserAgent::_need_proxy: Not proxied
> > LWP::Protocol::http::request: ()
> > LWP::Protocol::collect: read 695 bytes
> > LWP::Protocol::collect: read 1896 bytes
> > LWP::Protocol::collect: read 1505 bytes
> > LWP::Protocol::collect: read 1017 bytes
> > LWP::Protocol::collect: read 1207 bytes
> > LWP::Protocol::collect: read 208 bytes
> > HTTP::Cookies::extract_cookies: Set cookie UBID =>
> > Tue%20Jul%20%209%2015:36:37
> > 0EDT%202002%200.610083999
> > LWP::UserAgent::request: Simple response: OK
> >
> > This is the form Dump:
> >
> > POST https://online.wsj.com/wsjra/servlet/WSJAuthReg
> >   URI=                            (hidden)
> >   From=wsjauth                    (hidden)
> >   Login=<UNDEF>
> >   Password=<UNDEF>                (password)
> >   <NONAME>=Log In                 (submit)
> >   <NONAME>=Log In                 (submit)
> >   save_wsjie_login=<UNDEF>        (checkbox) [*<UNDEF>|yes]
> >
> > end Dump
> >
> > LWP::UserAgent::request: ()
> > HTTP::Cookies::add_cookie_header: Checking online.wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: Checking .wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: - checking cookie path=/
> > HTTP::Cookies::add_cookie_header:  - checking cookie
> > UBID=Tue%20Jul%20%209%201
> > 36:37%20EDT%202002%200.610083999
> > HTTP::Cookies::add_cookie_header:    it's a match
> > HTTP::Cookies::add_cookie_header: Checking wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: Checking .com for cookies
> > LWP::UserAgent::send_request: POST
> > https://online.wsj.com/wsjra/servlet/WSJAut
> > eg
> > LWP::UserAgent::_need_proxy: Not proxied
> > LWP::Protocol::http::request: ()
> > HTTP::Cookies::extract_cookies: Set cookie fastlogin =>
> > QZWfEwhiRuuCViHu1ZqiUl
> > MPPcGSxzZPqmHeadfYK9Y%3D
> > HTTP::Cookies::extract_cookies: Set cookie user_type => subscribed
> > HTTP::Cookies::extract_cookies: Set cookie REMOTE_USER => kcmria
> > HTTP::Cookies::extract_cookies: Set cookie wsjperm => 631/51
> > LWP::UserAgent::request: Simple response: Found
> > LWP::UserAgent::request: ()
> > HTTP::Cookies::add_cookie_header: Checking online.wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: Checking .wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: - checking cookie path=/
> > HTTP::Cookies::add_cookie_header:  - checking cookie
> > UBID=Tue%20Jul%20%209%201
> > 36:37%20EDT%202002%200.610083999
> > HTTP::Cookies::add_cookie_header:    it's a match
> > HTTP::Cookies::add_cookie_header:  - checking cookie wsjperm=631/51
> > HTTP::Cookies::add_cookie_header:    it's a match
> > HTTP::Cookies::add_cookie_header:  - checking cookie REMOTE_USER=kcmria
> > HTTP::Cookies::add_cookie_header:    it's a match
> > HTTP::Cookies::add_cookie_header:  - checking cookie
user_type=subscribed
> > HTTP::Cookies::add_cookie_header:    it's a match
> > HTTP::Cookies::add_cookie_header:  - checking cookie
> > fastlogin=QZWfEwhiRuuCViH
> > ZqiUlBsMPPcGSxzZPqmHeadfYK9Y%3D
> > HTTP::Cookies::add_cookie_header:    it's a match
> > HTTP::Cookies::add_cookie_header: Checking wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: Checking .com for cookies
> > LWP::UserAgent::send_request: POST http://online.wsj.com/home
> > LWP::UserAgent::_need_proxy: Not proxied
> > LWP::Protocol::http::request: ()
> > ^C
> > F:\scripts>
> > If I am reading this correctly it is going to https (a secure web page)
> then
> > the last steep
> > is going to http (non secure) So that would tell me the click function
is
> > selecting the
> > secure login button not the standard one.
> > I think now is the time to get Sean's new book!! Like I said, if someone
> > knows a way
> > of selecting the second select button that is not named I would like to
> > know.
> > Anyway I hope this helps!!
> >
> > Ron Hill
0
kdubuisson
7/9/2002 8:05:08 PM
📁 perl.libwww
📃 3335 articles.
⭐ 0 followers.

💬 0 Replies
👁️‍🗨️ 451 Views


Reply: