homepage Welcome to WebmasterWorld Guest from 54.167.173.250
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
Forum Library, Charter, Moderator: open

Paid Inclusion Engines and Topics Forum

This 38 message thread spans 2 pages: 38 ( [1] 2 > >   posting off  
Inktomi spidering non-paying websites
troels nybo nielsen




msg:18159
 11:57 am on Dec 29, 2002 (gmt 0)

It seems that Slurp over the last few days has begun deepcrawling my websites. I have certainly never paid anything for that.

New politics from the new owners? I have been visited before but always with only one hit.

 

Macguru




msg:18160
 12:05 pm on Dec 29, 2002 (gmt 0)

It not unusal to see Slurp in your logfiles. Unfortunatly, this does not mean that your site will be included in the index.

I care for many sites wich are still crawled by Slurp on a regular basis but where never included in the index for free.

HuhuFruFru




msg:18161
 12:34 pm on Dec 29, 2002 (gmt 0)

why should they crawl without indexing a site?

my site has been deepcrawled too (since wednesday) - i didn't subscribe to basic submit, but i have 3 listings in odp.

heini




msg:18162
 12:38 pm on Dec 29, 2002 (gmt 0)

In fact Ink has indexed sites all the time without being paid, how else could they have collected such an enormous db?
I have never paid Ink for any site, yet eventually they all got in.

Macguru




msg:18163
 12:38 pm on Dec 29, 2002 (gmt 0)

>why should they crawl without indexing a site?

Maybe to let you know they are still alive. :)

Joke aside, I believe you have more chances to get a free ride if the site in non English or in niche markets.

HuhuFruFru




msg:18164
 12:45 pm on Dec 29, 2002 (gmt 0)

I agree heini!
yes macguru: the site is in german :)

similarly to troels my site has got 2-5 hits per month since september, but since wednesday 250 hits and 6.5 MB. so I assume I will get indexed soon.

btw: what does basic submit with the url's mean? if I subscribe *one url*, will my whole site get indexed or just *one page*? (would be obviously very expensive!)

Macguru




msg:18165
 12:48 pm on Dec 29, 2002 (gmt 0)

It's 39 $ for the first page and 25 $ therafter for every single page you want crawled.

HuhuFruFru




msg:18166
 12:58 pm on Dec 29, 2002 (gmt 0)

>It's 39 $ for the first page and 25 $ therafter
>for every single page you want crawled.

that's ridiculous, I have currently about 400 sites and I would never pay so much money (10039$?) just for getting indexed! that's thousands of clicks i could get from google-adwords.

i hope msn will soon switch to FAST...

Thanasus




msg:18167
 7:17 pm on Dec 29, 2002 (gmt 0)

I have submitted some sites to Inktomi but most of mine got crawled over time. IMHO its real simple, you pay for quick inclusion and monthly guarenteed visits.

crash




msg:18168
 8:17 pm on Dec 29, 2002 (gmt 0)

I have several sites that I have not paid INK for inclusion, they are all crawled regularly and included in the index - some have great ranking too.

HuhuFruFru




msg:18169
 9:17 pm on Dec 29, 2002 (gmt 0)

>I have several sites that I have not paid INK for
>inclusion, they are all crawled regularly and included in
>the index - some have great ranking too.

how did you submit them and how long did it take them to show up in the results?

BigDave




msg:18170
 7:09 pm on Dec 31, 2002 (gmt 0)

Ink has been hitting me hard for the last week, after only having a few pages in. They aren't hitting me as hard as Google during the main crawl, but they have been averaging over 300 pages a day, and they are going deep.

I wonder if Yahoo told them to index more non-paying sites.

<added>I never paid, and never submitted my site, as I wasn't going to go to the effort of figuring out how to submit without paying</added>

chiyo




msg:18171
 10:19 am on Jan 16, 2003 (gmt 0)

I really do doubt that if Y! seriously wanted to get people coming back to their search that they would limit the ink listings in Y! to paid pages only. That would be the quickest way to send people back to G! and maybe trying out ATW/T/Wise/ etc.

However i think the way they list ink database listings will be complex and clever, maybe providing some sort of advantage to paid sites, without reducing the functionality and power of their present search using Google.

I think its possible that the reason Y! bought Ink was so they could spider the web themselves and produce their own databases and ranking algos rather than relying on Google. Its all about contol, and G! surely would not have provided enough flexibility in the use of their results and their spiders.

I still feel strongly that they bought Ink for their search technology rather than their database or listings. Using the spidering tech would be part of that.

Personally im not worried at all about opting for ink until we know how Y! will merge their new acquisition. But obviously you should be in there as a definitely! - paid or not. Thats about all im worried about.

So the present spidering splurge is not a great issue for me. To me the significance now is just to remember its Y! spidering you now, not ink, which used to spider on behalf of a lot of customers, unless Y! continue to lease/sell their results to second parties like Ink used to.

spaidermen




msg:18172
 11:23 am on Jan 28, 2003 (gmt 0)

The very same question for crash:

how did you submit them and how long did it take them to show up in the results?

>I have several sites that I have not paid INK for
>inclusion, they are all crawled regularly and included in
>the index - some have great ranking too.

Tor




msg:18173
 11:37 am on Jan 28, 2003 (gmt 0)

how did you submit them and how long did it take them to show up in the results?

I, for my part, submitted:

here
[submitit.bcentral.com]

spaidermen




msg:18174
 11:50 am on Jan 28, 2003 (gmt 0)

So Microsoft is not completely EVIL :)
Anyhow, how long does it take to get indexed after submitting there?

Tor




msg:18175
 12:07 pm on Jan 28, 2003 (gmt 0)

It takes some months normally, but it is worth it because we have many sites that we are responsible for. PFI would have cost us big bucks. ;)

spaidermen




msg:18176
 12:17 pm on Jan 28, 2003 (gmt 0)

some means how many? 2,3,4,5,6...?

Tor




msg:18177
 12:18 pm on Jan 28, 2003 (gmt 0)

3-4

spaidermen




msg:18178
 12:35 pm on Jan 28, 2003 (gmt 0)

thank you

troels nybo nielsen




msg:18179
 12:59 pm on Jan 28, 2003 (gmt 0)

I might add that in january Inktomi have been extensively crawling my websites. They are easily #2 before Fast. Anybody seing the same pattern?

Tor




msg:18180
 1:04 pm on Jan 28, 2003 (gmt 0)

Heavy crawling of all our sites from Slurp in January.

spaidermen




msg:18181
 1:26 pm on Jan 28, 2003 (gmt 0)

What's the inclusion time for free addurl in fast?

Tor




msg:18182
 1:42 pm on Jan 28, 2003 (gmt 0)

I would say 3-5 weeks.

Go60Guy




msg:18183
 2:40 pm on Jan 28, 2003 (gmt 0)

Its probably worth mentioning that some experts who post here at WebmasterWorld believe that submitting to Inktomi is a mistake. The theory is that submitting sends you to the bottom of the barrel as far as rankings are concerned.

The better approach for free listings, so the belief goes, is just to let Slurp find you.

spaidermen




msg:18184
 2:47 pm on Jan 28, 2003 (gmt 0)

so, how often does slurp crawl a site?

Go60Guy




msg:18185
 2:51 pm on Jan 28, 2003 (gmt 0)

Well, from all I can see, Slurp is crawling constantly.

olias




msg:18186
 3:01 pm on Jan 28, 2003 (gmt 0)

why should they crawl without indexing a site?

Link pop is definitely part of the Inktomi algo, so it helps if they can crawl lots of pages but they don't necessarily have to index them all.

spaidermen




msg:18187
 3:01 pm on Jan 28, 2003 (gmt 0)

what does it mean? every day? hour? month? second?

lloyd




msg:18188
 8:50 am on Jan 29, 2003 (gmt 0)

Going back to bcentral (MSN) submission to Inktomi (mentioned above).

Can you only submit the Domain Name or can you submit as many pages as you like.

Many thanks

This 38 message thread spans 2 pages: 38 ( [1] 2 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved