LWP and "Location: " header

How to avoid automatic redirect?

[myself]

I wrote a crawler that scans some site. The task is to load all its pages. The crawler reads links and selects ones that belong to the server in hand. My module uses LWP::UserAgent.

Some links (even if they point to site pages) leads to CGI-script that prints "Location: [some-other-site...] and LWP in my crawler goes to that other site.

How can I avoid such behavior? In other words: how to tell LWP not to redirect if there is "Location: " field within http-response header?

[Storyteller]

Set

requests_redirectable

property of your LWP::UserAgent to empty arrayref.

$ua->requests_redirectable([ ]);

or on creation:

$ua = new LWP::UserAgent(
 requests_redirectable => [ ],
);

[myself]

Thanks a log for the story told ;-)

[percentages]

Well heck ain't that just a dandy bit of info.....thanks Story....I looked everywhere for that info when I wanted to solved that problem some months back.....now I know how:)