Pita now is called WebVac

name change

[webvaccrawler]

Send any questions to me,
or the email in the spider.

We crawl every couple of months
and make our crawl available to
researchers across the US and
Europe who are too resource poor
to do their own crawl.
It cuts down on spider traffic
that way.

Sorry, could not figure out how
to respond to Pita name thread,
told me it was too old.

Gary Wesley
Spider Pilot

[jdMorgan]

Gary,

Welcome to WebmasterWorld [webmasterworld.com]!

Thanks for posting this info. I note that your crawler is NOT on my disallowed list, so it must have been well-behaved. Maybe you can join in here when we razz other spider authors for their badly-behaved agents! :)

Jim

[msgraph]

Thanks Gary,

What's the exact UA to look out for? Do you have an information page set up?

[webvaccrawler]

[www-diglib.stanford.edu...]
" HTTP/1.0\r\n"
"Host: %s\r\n"
"User-Agent: WebVac ([e-mail])\r\n"
"From: [e-mail]\r\n\r\n",

Gary

[stechert]

where can we go to see the data?

thanks,
andre

[webvaccrawler]

Instructions on the website above
for building a client (needed mostly
because it is compressed) and what
ports to access data on.
Please let us know what you use it for.

Gary