GET
Hi, I have tried to use libwww with the method GET to emulate a html <form method=get>. I called the HTTP::Request() with one URL and as content the formdata. But I think this way is not the way <form method=get> works because I got a wrong answer from the url. Do you know how to speak to the libwww module to use GET with formdata? Greets Mat
|
0 |
|
mku
1/27/2000 4:17:54 PM
perl.libwww
3335 articles. 
0 followers.
1 Replies
903 Views
Matthias Kaulartz said...
>
>Do you know how to speak to the libwww module to use GET with formdata?
>
Yes, I do. Here is an example. Save this code into a file, call it 'search'.
At the command line type 'perl search perl' or 'perl search oracle' or
'perl search "home cooking"...
I'd strip the comments. It really isn't that hard to do.:)
#####################################################################
#!/usr/bin/perl
use strict;
use HTML::TokeParser;
use LWP::UserAgent;
use HTTP::Request;
use HTML::Form;
use URI;
# I use dumpvar.pl to 'see' the data structures.
# Like the x command in the debugger
require 'dumpvar.pl';
my $found;
# simple command line interface
# could sex it up with use Getopt::Long
#
my $what = $ARGV[0];
$| = 1;
my $base = 'http://www.amazon.com'; # the base URL
my $path = 'exec/obidos/ats-query-page/'; # the advanced search path
# Create a user agent
my $ua = LWP::UserAgent->new();
# Name our new agent
$ua->agent('BookSearch/1.0');
# Create a uri object
my $uri = URI->new_abs($path,$base);
# Create a request object
my $req = HTTP::Request->new(GET => $uri);
# Request the request object
my $res = $ua->request($req);
if ( $res->is_success ) {
# parse all forms in the returned page
my @forms = HTML::Form->parse($res->content,$base);
# Now look for the form that we want
# The Author, Title, Subject form has an action path of '/exec/obidos/ats-query'
my $form;
for (@forms) {
$form = $_ if $_->action =~ /ats-query/;
last if $form;
}
if ($form) {
# Now we fill out the form
$form->value('title-mode','word');
# $what is the word we are looking for in the title
# we got this from @ARGV above
$form->value('title',$what);
# sort the output by publication date
$form->value('sort-type','publication-date');
# the click method returns a new request object
$req = $form->click;
# so now request it!!
$res = $ua->request($req);
# parse_text prints the books found along with the desired
# info. It returns the number of books found.
$found += parse_text($res); # different response object
# Here we look for the continue form just in case there are 'More'
# books to find.
while ( @forms = HTML::Form->parse($res->content,$res->base)) {
for(@forms) {
$form = ($_->{action} =~ /subsequent-query/)?$_:{};
}
if(exists $form->{action}) {
$req = $form->click();
print STDERR "Going out again: $found\n";
$res = $ua->request($req);
print "Next bunch\n\n";
$found += parse_text($res); # different response object
}
last if $form->{action} !~ /subsequent-query/;
}
print "Total Books found: $found\n";
}
} else {
print 'The request failed : ',$res->status_line,"\n";
}
# if we can print it to the screen we can stick it in a database
# but I need to see the schema first.
sub parse_text {
# passed in response object
my $resp = shift;
# parse the content
my $p = HTML::TokeParser->new(\$resp->content);
my $found = 0;
my $uri;
my $req;
my $res;
# The following all depends on what the web page looks like
# all this will have to change if Amazon changes the design of the
# page. You have to look at the returned pages yourself and write
# the code to parse a given page.
while($p->get_tag('dt')) {
my $url = $p->get_tag('a');
$url = $url->[1]{href};
$url =~ m{/exec/obidos/ASIN/(.*)/qid=\d+/sr=\d+\-\d+/.*};
my $isbn = $1;
chomp $isbn;
my $title = $p->get_trimmed_text('/a');
$p->get_tag('dd');
my $text = $p->get_trimmed_text('p');
$text =~ s/Read more about this title.*//;
next if $text =~ /Not Yet Published/;
my $data;
@{$data} = split(m{/},$text,3);
unless (@{$data} == 3) {
if(@{$data} == 1) {
for(1..2) {
push(@{$data},'unknown');
}
} else {
unshift(@{$data},'unknown');
}
}
chomp @{$data};
$data->[2] =~ /Our Price:\s*([^ ]+)/g;
$data->[3] = $1;
$data->[0] ||= 'unknown';
$data->[1] ||= 'unknown';
$data->[2] =~ /(Published\s+\d\d\d\d).*/;
$data->[4] = $1;
$data->[2] ||= 'unknown';
# uncomment this if you want to see what $data looks like
# dumpValue(\$data);
print "Title: $title\n";
print "ISBN: $isbn\n";
print "Author: $data->[0]\n" ;
print "Book Type: $data->[1]\n" if $data->[1] ;
print "Published: $data->[4]\n" if $data->[4] ;
print 'Price: ',$data->[3],"\n";
print "\n\n" ;
$found++;
}
return $found;
}
############################################################
--
Tim Allwine
IX Development Laboratories
|
|
0 |
|
tallwine
1/27/2000 5:23:33 PM
Similar Artilces:
POST GET GET GET GET... WTH?
Hi, I'm at a loss to explain this... I have an ASPX page that I can't always navigate to. When I developed the app (it's Starter Kit Portal-based), I did it on my LAN and have had no trouble accessing the page. However, upon deployment, I see the following behavior: 1. POST request goes out looking for MyForm.aspx 2. GET request for MyForm.aspx 3. GET request for MyForm.aspx 4. ... X. GET request Sometimes, the page finally loads. Other times, it just ends up blank. Many times, it doesn't load at ALL the first POST - I just get a "The pa...
Get LastLoginDate BEFORE it gets updated
I am after a way to get the LastLoginDate before it gets updated when a user logs on. The reason is that i want to provide a list of all changes that have been done since they last logged on. When i use OnLoggingIn it appears that the LastLoginDate has already been updated (with the new Login Date). I have tried numerous ways but am a little stumped. Any help will be greatly appriciated. Try this - Dim LastloginDate As Date Protected Sub Login1_LoggingIn(ByVal sender As Object, ByVal e As System.Web.UI.WebControls.LoginCancelEventArgs) Handles Login1.LoggingIn ...
Get All & Get all mail
Is there any difference between these two extensions? Daniel -- _____________________________________ Daniel Royer University of Geneva daniel at royer dot ch _Daniel Royer_ spoke thusly on 11/09/2006 4:31 AM: > Is there any difference between these two extensions? I can't seem to find a "Get All" extension for Thunderbird. The closest I've found is <http://extensionroom.mozdev.org/more-info.php/getmess>, which is not written to the extension manager API, that was implemented long before Thunderbird 1.0. If that's what you were referri...
How do I get rid of this mozL10n.get()?
So I have a few strings that contains arguments that also needs to be translated https://github.com/mozilla-b2g/gaia/blob/42e93d96ea245c778885b1889b3da589f5342fb1/apps/keyboard/locales/keyboard.en-US.properties#L77 And at the time of writing the code I ended up using mozL10n.get(). https://github.com/mozilla-b2g/gaia/blob/42e93d96ea245c778885b1889b3da589f5342fb1/apps/keyboard/js/settings/layout_item_view.js#L263-L265 Do we already have support on this? The answer is probably obvious if there is already one. The status itself is one <span> element which I prefer not to break...
Get get HotSync to work
Hi, I'm having problems synchronizing my custom Ultra-Lite application. After a HotSync the log file looks as follows: HotSync operation started 11/06/01 03:47:36 PM Expense - sync configured to Do Nothing Outlook Calendar - sync configured to Do Nothing Outlook Contacts - sync configured to Do Nothing Outlook Tasks - sync configured to Do Nothing Outlook Notes - sync configured to Do Nothing Unable to open the stream 11/06/01 03:47:40 PM OK PalmTCS (Adaptive Server Anywhere Conduit) with 1 message(s) 11/06/01 03:47:40 PM OK System Yet, the MobilLink Server still says:...
Getting variables from a post or get?
Sorry for the newbie post, but i am from a PHP world. Im now learning ASP.NET.Im having a bit of problem figuring out how to get/post a variable on an input field. All i want to do is, get the value from an input form and display it again on the page. Here's my code.. <form id="form1" runat="server" acceptcharset="utf-8"> Your name: <input id="name" name="name" type="text" /> <asp:Button ID="Button1" runat="server" Text="Button" PostBackUrl="Default.aspx" /> </form> All i want is to get the value of name and display it on the page it self. How can i do that...
get
Name: hai nhi Email: cobe081991atyahoodotcom Product: Minefield Summary: get Comments: hi Browser Details: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a2pre) Gecko/20070111 Minefield/3.0a2pre From URL: http://www.mozilla.org/projects/minefield/ ...
libwww
--------------D259760D43FAF8EA0BF6B364 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Where is the latest version of the lib-www available from? I coud't find it on the url: http://www.sn.no/libwww-perl/ . Thanks!!!! --------------D259760D43FAF8EA0BF6B364-- Today around 5:05pm, Leonardo Jose Silvestre hammered out this masterpiece: : Where is the latest version of the lib-www available from? : I coud't find it on the url: http://www.sn.no/libwww-perl/ . When in doubt, search CPAN. http://search.cpan.org/search?dist=libwww-perl...
I can't get no cache, I can't get no cache. 'Cause I try and I try and I try and, I can't get no, I can't get no cache.
I have fiddled out for days tinkering with the setting in about:config trying to get FireFox 12 to use the disk cache. I have NOT found the trick. This One Trick Pony ain't doing it like it used to up until recently. Pray tell anyone, What information can I share that will point a knowledgeable person to aid me in getting FF to disk.cache? I have also tried restarting FF with add-ons disabled, (There were NOT too many to do this to, so it was quick and easy) Here is my stab at trying to convey the information that MIGHT govern matters Using about:cache Informatio...
Re: I can't get no cache, I can't get no cache. ' Cause I try and I try and I try and, I can't get no, I can't get no cache.
<div>please com e see me about htis<br /> <br /> ------- Original Message f= rom the Global Relay Archive -------<br /> From: Hp <ferd@farkel.net>= <br /> To: "support-firefox@lists.mozilla.org" <support-firefox@lists.mo= zilla.org><br /> Sent: Sun, 27 May 2012 18:30:18 -0700<br /> Subject: I = can't get no cache, I can't get no cache. 'Cause I try and I try and I try = and, I can't get no, I can't get no cache.<br /> <br /></div> <pre class=3D"gr-maex-body-pre&qu...
Getting line number to show during editing, getting values to show during debug, getting MSDN help to know that I have it installed
G'day, Sorry I feel so dumb but these issues just push me over the edge. I've recently had to re-install VS and now none of these features that I once had now don't work. I obviously have to tweak something but what? It doesn't seem too obvious (I hope!)Help please!Thanks,Geoff Line numbers - that can found under Tools > Options > Text Editor > (Language) > (somewhere there, there is is a check box for line numbers) When you are in the debug mode of the IDE, go into the Debug > Windows > (whatever windows you need - locals/autos are what you're l...
How do you get to get to the NAL debug screen?
I remember a debug screen that you could get to in NAL by doing some function key. I can't remember exactly how it work, could anyone assist? ZFD 4.01 IR4 Thanks , > I remember a debug screen that you could get to in NAL by doing some > function key. I can't remember exactly how it work, could anyone assist? > Help, About, Hold down F2, More - Anders Gustafsson, Engineer, CNE6, ASE NSC Volunteer Sysop Pedago, The Aaland Islands (N60 E20) Novell does not monitor these forums officially. Enhancement requests for all Novell products may be made at h...
I keep getting an error, and get kicked out
Name: Edgar Mitchell Product: Firefox Summary: I keep getting an error, and get kicked out Comments: Where can I attach a screen-shot of the error messages? Browser Details: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 From URL: http://hendrix.mozilla.org/ Note to readers: Hendrix gives no expectation of a response to this feedback but if you wish to provide one you must BCC (not CC) the sender for them to see it. ...
Hi, I'm at a loss to explain this... I have an ASPX page that I can't always navigate to. When I developed the app (it's Starter Kit Portal-based), I did it on my LAN and have had no trouble accessing the page. However, upon deployment, I see the following behavior: 1. POST request goes out looking for MyForm.aspx 2. GET request for MyForm.aspx 3. GET request for MyForm.aspx 4. ... X. GET request Sometimes, the page finally loads. Other times, it just ends up blank. Many times, it doesn't load at ALL the first POST - I just get a "The pa...
Get LastLoginDate BEFORE it gets updated
I am after a way to get the LastLoginDate before it gets updated when a user logs on. The reason is that i want to provide a list of all changes that have been done since they last logged on. When i use OnLoggingIn it appears that the LastLoginDate has already been updated (with the new Login Date). I have tried numerous ways but am a little stumped. Any help will be greatly appriciated. Try this - Dim LastloginDate As Date Protected Sub Login1_LoggingIn(ByVal sender As Object, ByVal e As System.Web.UI.WebControls.LoginCancelEventArgs) Handles Login1.LoggingIn ...
Get All & Get all mail
Is there any difference between these two extensions? Daniel -- _____________________________________ Daniel Royer University of Geneva daniel at royer dot ch _Daniel Royer_ spoke thusly on 11/09/2006 4:31 AM: > Is there any difference between these two extensions? I can't seem to find a "Get All" extension for Thunderbird. The closest I've found is <http://extensionroom.mozdev.org/more-info.php/getmess>, which is not written to the extension manager API, that was implemented long before Thunderbird 1.0. If that's what you were referri...
How do I get rid of this mozL10n.get()?
So I have a few strings that contains arguments that also needs to be translated https://github.com/mozilla-b2g/gaia/blob/42e93d96ea245c778885b1889b3da589f5342fb1/apps/keyboard/locales/keyboard.en-US.properties#L77 And at the time of writing the code I ended up using mozL10n.get(). https://github.com/mozilla-b2g/gaia/blob/42e93d96ea245c778885b1889b3da589f5342fb1/apps/keyboard/js/settings/layout_item_view.js#L263-L265 Do we already have support on this? The answer is probably obvious if there is already one. The status itself is one <span> element which I prefer not to break...
Get get HotSync to work
Hi, I'm having problems synchronizing my custom Ultra-Lite application. After a HotSync the log file looks as follows: HotSync operation started 11/06/01 03:47:36 PM Expense - sync configured to Do Nothing Outlook Calendar - sync configured to Do Nothing Outlook Contacts - sync configured to Do Nothing Outlook Tasks - sync configured to Do Nothing Outlook Notes - sync configured to Do Nothing Unable to open the stream 11/06/01 03:47:40 PM OK PalmTCS (Adaptive Server Anywhere Conduit) with 1 message(s) 11/06/01 03:47:40 PM OK System Yet, the MobilLink Server still says:...
Getting variables from a post or get?
Sorry for the newbie post, but i am from a PHP world. Im now learning ASP.NET.Im having a bit of problem figuring out how to get/post a variable on an input field. All i want to do is, get the value from an input form and display it again on the page. Here's my code.. <form id="form1" runat="server" acceptcharset="utf-8"> Your name: <input id="name" name="name" type="text" /> <asp:Button ID="Button1" runat="server" Text="Button" PostBackUrl="Default.aspx" /> </form> All i want is to get the value of name and display it on the page it self. How can i do that...
get
Name: hai nhi Email: cobe081991atyahoodotcom Product: Minefield Summary: get Comments: hi Browser Details: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a2pre) Gecko/20070111 Minefield/3.0a2pre From URL: http://www.mozilla.org/projects/minefield/ ...
libwww
--------------D259760D43FAF8EA0BF6B364 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Where is the latest version of the lib-www available from? I coud't find it on the url: http://www.sn.no/libwww-perl/ . Thanks!!!! --------------D259760D43FAF8EA0BF6B364-- Today around 5:05pm, Leonardo Jose Silvestre hammered out this masterpiece: : Where is the latest version of the lib-www available from? : I coud't find it on the url: http://www.sn.no/libwww-perl/ . When in doubt, search CPAN. http://search.cpan.org/search?dist=libwww-perl...
I can't get no cache, I can't get no cache. 'Cause I try and I try and I try and, I can't get no, I can't get no cache.
I have fiddled out for days tinkering with the setting in about:config trying to get FireFox 12 to use the disk cache. I have NOT found the trick. This One Trick Pony ain't doing it like it used to up until recently. Pray tell anyone, What information can I share that will point a knowledgeable person to aid me in getting FF to disk.cache? I have also tried restarting FF with add-ons disabled, (There were NOT too many to do this to, so it was quick and easy) Here is my stab at trying to convey the information that MIGHT govern matters Using about:cache Informatio...
Re: I can't get no cache, I can't get no cache. ' Cause I try and I try and I try and, I can't get no, I can't get no cache.
<div>please com e see me about htis<br /> <br /> ------- Original Message f= rom the Global Relay Archive -------<br /> From: Hp <ferd@farkel.net>= <br /> To: "support-firefox@lists.mozilla.org" <support-firefox@lists.mo= zilla.org><br /> Sent: Sun, 27 May 2012 18:30:18 -0700<br /> Subject: I = can't get no cache, I can't get no cache. 'Cause I try and I try and I try = and, I can't get no, I can't get no cache.<br /> <br /></div> <pre class=3D"gr-maex-body-pre&qu...
Getting line number to show during editing, getting values to show during debug, getting MSDN help to know that I have it installed
G'day, Sorry I feel so dumb but these issues just push me over the edge. I've recently had to re-install VS and now none of these features that I once had now don't work. I obviously have to tweak something but what? It doesn't seem too obvious (I hope!)Help please!Thanks,Geoff Line numbers - that can found under Tools > Options > Text Editor > (Language) > (somewhere there, there is is a check box for line numbers) When you are in the debug mode of the IDE, go into the Debug > Windows > (whatever windows you need - locals/autos are what you're l...
How do you get to get to the NAL debug screen?
I remember a debug screen that you could get to in NAL by doing some function key. I can't remember exactly how it work, could anyone assist? ZFD 4.01 IR4 Thanks , > I remember a debug screen that you could get to in NAL by doing some > function key. I can't remember exactly how it work, could anyone assist? > Help, About, Hold down F2, More - Anders Gustafsson, Engineer, CNE6, ASE NSC Volunteer Sysop Pedago, The Aaland Islands (N60 E20) Novell does not monitor these forums officially. Enhancement requests for all Novell products may be made at h...
I keep getting an error, and get kicked out
Name: Edgar Mitchell Product: Firefox Summary: I keep getting an error, and get kicked out Comments: Where can I attach a screen-shot of the error messages? Browser Details: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 From URL: http://hendrix.mozilla.org/ Note to readers: Hendrix gives no expectation of a response to this feedback but if you wish to provide one you must BCC (not CC) the sender for them to see it. ...
Web resources about - GET - perl.libwww
Clinton, Sanders Expected to Clash in S.C.CHARLESTON, S.C. – Since October, the Democratic presidential contenders have contrasted their positions with powder-puff niceties during televised ...
Microsoft updates support policy: new CPUs will require Windows 10, a list of preferred devices will ...Ed Bott / ZDNet : Microsoft updates support policy: new CPUs will require Windows 10, a list of preferred devices will support older Windows ...
Leonardo DiCaprio, Tom Hardy celebrate Oscar nominations at ‘The Revenant’s London premiereCast and crew of 'The Revenant' including Leonardo DiCaprio, Tom Hardy, director Alejandro González Iñárritu and others celebrated their 12 Oscar ...
Chipotle Will Close Stores for Food Safety Meetings After OutbreaksAll Chipotle restaurants will open later than usual on Feb. 8 so employees can discuss safety changes, the chain said.
Concord Monitor Warehouse worker takes third of Powerball jackpot Concord Monitor John Robinson, left, brings in the family dog, Abby, as ...
El Chapo Reportedly Had Time for Erectile Disfunction Surgery While on the RunAfter escaping from prison last summer, Mexico’s most powerful drug lord Joaquin “El Chapo” Guzman surely found himself with a busy schedule ...
Al Qaeda Targets Westerners in Burkina Faso: 20 Reported DeadTerror has once again fallen upon the nation of Africa. Only a week after the Islamic State took credit for a series of attacks on popular resorts ...
Meet Jeb Bush’s police state: The NSA in charge of civilian dataGiven the various recent terrorist attacks, it’s no wonder that hackers , cybersecurity, encryption and surveillance are all major topics of ...
'Prison Break' Revival Ordered Straight to SeriesIt’s really official – Prison Break is coming back. While Fox ordered a pilot episode back in the summer, the network just announced that the ...
French drug trial turns disastrous, leaving 1 brain dead and 5 hospitalizedHealth Minister Marisol Touraine, pictured, told reporters that she was overwhelmed by the distress of the harmed volunteers. "Their lives have ...
Resources last updated: 1/16/2016 7:28:37 AM