Forum Moderators: coopster

Message Too Old, No Replies

Parsing URL with Banned IP?

Parsing URL with Banned IP?

         

grim

9:16 pm on Feb 21, 2004 (gmt 0)



Here's the problem - I have written a script that reads in a url and parses out certain information from the page. The site it retrieves the information from has a data mining check which I apparently tripped this morning. Now when I run the script from my server I get a 410 error.

Question - Is there a way to go through a proxy server with gzopen()? I know you can 'spoof' the referrer by manually sending the headers, but the ban was made using an IP address and I'm not too interested in spoofing that.

-or-

I was wondering if anyone can think of any workarounds for the problem I'm facing. I've even considered having the user save the url as an html page, then upload it to my server to have it parsed there... but that seems like a lot of wasted effort. I also hoped that maybe there was a way to have their browser load the page in a frame, then parse the contents of that frame with a combination of javascript and php?

Before you flame - I've already emailed the webmaster of said site to try to get the ip ban lifted. His code was meant to trap automated miners (my script runs manually, but I did too many too fast). He says there are better ways to gather the information and to 'be creative' in his faq. One site that does something similar to mine uses multiple IP addresses (they say they have 6) but I don't have the money for a dedicated server or more hosting accounts just for this small script. Basically all my script does is make it easier for a user to update their profile instead of re-entering all the information that is on said site.

jatar_k

5:19 pm on Feb 22, 2004 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



Welcome to WebmasterWorld grim,

Before you flame

hehe, we don't flame here so no worries.

Do you run this from a hosted site? You could also run it from your local machine which would have a different IP.