How does Slurp behave on your site?

only for banned sites and reincluded sites

[helenp]

Actually,
though things is taken very long time,
I kneed to know,....
i would like to know how slurp is behavioured in your site banned/not showed in serps, and in sites that know is reincluded.

Is slurp spidering your site daily for the robots.txt?

Sites reindexed, after apealing did slurp stop spidering your robots.txt? and then sudenly went for other pages of yours?

My reason for this question asked so many times before, is though slurp daily went for robots txt until 22/3 (date for apealing more or less) stopped, but started again the 6/4.

Now again slurp stopped getting my robots.txt the 2/5 until today havenīt been back, and really donīt like the idea of doing an domain only for yahoo........donīt think I should need to, as long as there is an hope:)

[helenp]

Nobody?

Well now 12 days has gone since slurp stopped getting my robots.txt
Crossing my finger, that when come back please ....my homepage.

This are yahoos diferents databases Iīve seen,
must be more:
66.218.71.233
216.109.117.133
216.109.117.108
66.218.71.114
216.109.127.30

[mars9820]

yahoo slurp visits a couple of my sites from time to time but not regular.

Most of the time it just fetch robots.txt and 1 URL that is already in the index. However it is kinda hard to get slurp going to new pages since there is no single hit on the index.htm where I most of the time include a link to new pages.

Google on the other hand deepcrawls everything everday.

[helenp]

I donīt know if relevant, but scooter just came in, (stopped spidering me 14/4)
Got sucessfully robots.txt, and error 404 for an old page, and an error 206 for an existing page, and then left.

[Barleycorn]

My site gets visited all the time by slurp but still no reinclusion?

[twebdonny]

Penalty imposed via Ink, listings from old google database, Slurp never visits us.

[Visit Thailand]

Slurp seems to hit our site daily quite vigorously but we are still not in the SERPS. Can only hope it is the 301 redirect problem that Y says they are working on.

I bet those that are still in the index are just loving the increase in traffic they are experiencing, because 5,000 odd referrals a day has to be going somewhere else.

[helenp]

Ole..........

My homepage just got spidered by ATW.
Didnīt even looked for my robots.txt..
Cross your fingers..

Yahoo-VerticalCrawler-FormerWebCrawler/3.9 crawler at trd dot overture dot com; [alltheweb.com...]

[walkman]

Helen,
I don't think the vertical one has anything to do with Y! results:

message #6
"I would allow the former Fast crawler into your site but not read to much into its behaviour for now. "
[webmasterworld.com...]

I got that false alarm too a while back...

[helenp]

As to what Yahoo Mike said days ago, itīs for yahoos Index
[webmasterworld.com...]

An older thread from april:
[webmasterworld.com...]

Walkmans link is from Mars.

[outland88]

Overture and Inktomi have been crawling me regularly for months and no inclusion. From what I am seeing Overture seems to be the main crawler or a whole lot of sites have been bumped from Yahoo.

Actually what does the Yahoo Slurp look like in logs?

My conclusion lately is even if you correct for what you think is penalizing your site its going to take a manual override by Yahoo to get back in, for the vast majority of people.

[helenp]

Itīs dificult to have hopes after 4 months e-mailing and investigating....

But anyway, that spider ever entered before to my site, so maybe.

Big changes are happening all the time.
Funny just went to yahoo.com, wrote [nameofmydomain.com...] and my spanish homepage came up, instead of the english that is the main.
But when wroting only www.nameofmydomain.com, this time my english main one came up on top.

Backlinks keeps changing, they seems to be jumping with their databases.

[helenp]

I give up

Slurp came back today for my robots.txt,
stopped the 2 of may.

Had some clicks for important keyworlds, even though I ever seen my site in,
and when checking yahoo webrank with the tool in an website, I had PR5 for long time, yesterday didnīt had any PR and know slurp again.......

What does it mean, penalty was lifted and hit again?

[mars9820]

I had a domain that was completely out of alltheweb and inktomi last year when yahoo made the decision to make the switch away from google.

The reason I was out of alltheweb/inktomi last year after redesigning a page. The reason I think was because I used relative URL's on my index page and I think the spider got some problems with the way I coded them.

I ran the webmasterworld spider and that one as well had some problems with following the links.

Changed everything to hard coded absolute URL's and now my new page are finally picked up one by one. However it goes damn slow.

Yahoo slurp picks up 1 page a time (together with robots.txt

As far as I remember no deep crawls yet :( On the other hand google eats all my bandwidth by giving me a daily deepcrawl.