Welcome to WebmasterWorld Guest from 54.159.190.106

Forum Moderators: open

Message Too Old, No Replies

Slurp out on full crawl?

   
2:01 pm on Jan 13, 2003 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Every site I've looked at this morning has been spidered hard by Inktomi the last few hours.

Is this the start of the first big Yahoo db build?

2:04 pm on Jan 13, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I am seeing this to on a few of my sites.

Craig

2:05 pm on Jan 13, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



saw it to

Tor

2:26 pm on Jan 13, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I see it here in Norway also.
2:54 pm on Jan 13, 2003 (gmt 0)

10+ Year Member



I haven't seen this on my sites yet but maybe it just hasn't gotten to them yet. Probably will soon.
2:57 pm on Jan 13, 2003 (gmt 0)

WebmasterWorld Administrator lawman is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



They've hit both of my sites. Sorta keeps you on pins and needles about any potential change to Y! results.

lawman

2:59 pm on Jan 13, 2003 (gmt 0)

10+ Year Member



And here - just to add my voice to the crowd.

R.

3:00 pm on Jan 13, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Same here. It's like a "did you see the UFO over London the other day?" post. ;)
3:03 pm on Jan 13, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Yep, slurp has certainly increased its activities here too. Isn't this what we've all expected to see?
3:11 pm on Jan 13, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Same Here

-s-

3:27 pm on Jan 13, 2003 (gmt 0)

10+ Year Member



What's the IP address of this bot?
3:30 pm on Jan 13, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I've got this one inktomi4-not.server.ntl.com
but it's a few days old.....
4:26 pm on Jan 13, 2003 (gmt 0)

WebmasterWorld Senior Member zeus is a WebmasterWorld Top Contributor of All Time 10+ Year Member



I got a notice a few days ago that I could not see which site has been crawled, but I don think they have spidered any more then the index page, but then spider the whole web now.

zeus

4:28 pm on Jan 13, 2003 (gmt 0)

10+ Year Member



Haven't seen them yet. How do the spider anyway? Do they start at low IP-addresses and then go up?
5:04 pm on Jan 13, 2003 (gmt 0)

10+ Year Member



Slurp came and got my main site yesterday. Crawled the deepest it has in the past couple of months.
5:55 pm on Jan 13, 2003 (gmt 0)

10+ Year Member



Substantial slurp traffic over the weekend. By far the biggest since 7/8 December.
Got individual pages listed for the first time two days before that...
6:42 pm on Jan 13, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Anyone seeing slup spider the unpaid pages on a site that has some PFI pages on it?
8:20 pm on Jan 13, 2003 (gmt 0)

10+ Year Member



mayor, Yes.
8:50 pm on Jan 13, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



When are we likely to see a refresh of the database?
Any clues?
J
9:10 pm on Jan 13, 2003 (gmt 0)

10+ Year Member



Same here. I have PFI but now they are spidering the whole site, including pages for which I donīt PFI... and they spider all the stuff under another domain as the one on PFI...
9:11 pm on Jan 13, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



whopee...how does that Inktomi algo go? :)
9:25 pm on Jan 13, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Methinks we are a few on this board who are gonna try and learn about inking. ;)
10:27 pm on Jan 13, 2003 (gmt 0)

10+ Year Member



Where can I submit to Inktomi?
11:55 pm on Jan 13, 2003 (gmt 0)

10+ Year Member



Inktomi has requested 3070 pages on my main site this month.

It seems to have been crawling me since sometime in December almost constantly. Very slow going compared with googlebot, which "slurped" ;) over 50,000 pages in the January crawl, stopping on the 11th.

-Pete

11:58 pm on Jan 13, 2003 (gmt 0)

WebmasterWorld Senior Member korkus2000 is a WebmasterWorld Top Contributor of All Time 10+ Year Member


Ink submit
http://www.hotbot.lycos.co.uk/submit.html
12:12 am on Jan 14, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Whoohooo ... can't get my logs until around 3AM. U betcha I'll have a fresh pot of coffee ready.
12:15 am on Jan 14, 2003 (gmt 0)

WebmasterWorld Administrator robert_charlton is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



korkus - I tried the hotbot.lycos.co.uk submission form a few days ago. It now submits to Fast, not Inktomi.

For Inktomi submit, this still works:

[submitit.bcentral.com...]

To be considered for inclusion in the Inktomi Search Index, powering the MSN Web Pages section

Note also, though, on the MSN site:

...only one url allowed per site allowed per day.

They really seem to keep track.

3:48 am on Jan 14, 2003 (gmt 0)

10+ Year Member



You guys think we will see free traffic from Inktomi after Yahoo paid 235M or so....I am not that sure....

Yahoo wants Inktomi PFI or even better PPC.

Get ready to pay in order to play.

12:06 pm on Jan 14, 2003 (gmt 0)

10+ Year Member



1300 pages crawled out a total of 2500 in our site requested from December, 1st.

Seems that Inktomi crawls at a very low pace, 15 pages/hour in the "fast moments" ;) and 1 in the low.

4:00 pm on Jan 14, 2003 (gmt 0)

10+ Year Member



As I wrote in another post I also saw a big increase in traffic from Ink the biggest I ever seen. It was some new domains which had got in for free. 5 days later (sunday) the domains were nowere to be found on Ink. Maybe they are doing this to sell more inclusion...You know show webmasters how much traffic you could get from MSN and etc then turn it off...

/esset

This 37 message thread spans 2 pages: 37
 

Featured Threads

Hot Threads This Week

Hot Threads This Month