Forum Moderators: open
We supplement this free crawl with the paid inclusion program, which allows us to add dynamic database content and other content that we could not otherwise discover and crawl. Currently less than 1% of our index is content that is included via our inclusion programs.
Our paid inclusion programs provide a superior way for sites to interact with us in a clearly structured fashion. These sites get the following
benefits:
1) Control over which URLs they include in the index (subject to content
guidelines)
2) Frequent refresh
3) Clearly defined way to interact with us (for example: clear feed
specifications)
4) Quality review and consultation (for feed customers)
5) Detailed reporting to track and optimize performance
6) Customer service
Participation in Yahoo!'s paid inclusion program does not guarantee rank in the search results. URLs are ranked in search results in the same way as all other URLs from the free crawl. The key drivers of ranking are keyword-specific relevance and site quality, as assessed by our regular search relevance algorithms with input from the quality review process.
(Note that Inktomi Index Connect clients are receiving Yahoo! Search traffic going forward, and Inktomi Search Submit customers are receiving a free trial of Yahoo! traffic until 4/15/04.)
We will also be launching free URL submission during the week of 3/1. This service allows content providers to suggest that the crawler should visit a site during the main crawl cycle. This service is different than paid inclusion in two ways:
1) There is no guarantee that we will crawl and include the URL in the index.
2) As this site potentially could be added to the main crawl, Yahoo! Slurp will follow links from that URL.
This service will reside on the Yahoo.com site and be accessible via a Yahoo! login. This service has been available for the AltaVista and Alltheweb Indices previously and the tradition will continue with Yahoo! Search.
1) PositionTech plays no role in getting URLs indexed by Yahoo after 4/2004.. correct?
2) I have a site with many, many pages and it has taken the old Inktomi (Slurp) a LONG LONG time (almost a Year) to even index a small portion of them.
Will the new (Yahoo! Slurp) crawler be improved to better handle large sites?
Here is a partial line from a log. Look for the
Yahoo! Slurp.
66.196.65.38 - - [26/Feb/2004:07:14:27 -0600] "GET /robots.txt HTTP/1.0" 200 5818 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; [help.yahoo.com...]
What does this mean exactly? Will we be able to track how well our URLs are performing for various search terms instead of using WPG or something. Or is it just like the previous inktomi where it just tells us traffic generated?
You guys should consider incorporating ranking reporting software, where we pick the search terms and it tells us our current ranking. I think I'd pay just for that. And it would reduce the use of WPG or other automated rank checkers most SE's frown upon us using.
Can you clear this up also... What is the proper command to use in a Yahoo search to determine which and how many URLs of a site are indexed in Yahoo?
is it:
1) site:domain.com, or
2) site:www.domain.com, or
3) site:http://www.domain.com
Note: each one of these commands provides different results?
Thanks ahead of time...
I guess this will be through the Site Match system?
See patent filing:
[tarr.uspto.gov...]
Ric
Looks pretty deceptive to me. Like when Yahoo! persuaded all those webmasters to fork out $299 a year and then dumped the listings out of the search results.
We will also be launching free URL submission during the week of 3/1.rather than pounding on Yahoo! for the past policies of the companies they have recently acquired, or the companies which which those acquired companies did business.
I see clear evidence that Yahoo! is making major improvements to those policies already.
Jim
Gotta give it to their Chairman. Terry is going after Google with vengeance. Maybe this is why the IPO was postponed? All I hear on CNBC is Yahoo & MSFT going after Google. It doesn't look good for Google. I still believe that they will be a strong provider, but second or third to MSFT & Yahoo. These companies have the direction now, and Google is in the way. It either they become an acquisition, or the go out of business (highly doubtful), or stay a distant third to the new bug two.
I had stated this all along. Google was not smart in letting Yahoo go. Now, I do not believe they can put up a good fight. I guess only time will tell. I like the feeling that compotition is back in the search engine market. Very very important. I was one of the people that thought Google was pushing its weight around way too much. Every company needs to be placed back it is place every once and a while.
Done with my rant.
CompWorld
Now only if we had an update on the suspected penalized /banned sites review process. Shall we expect a thread on that soon :)?
Sounds like page-rank is not a factor. I wonder if this is a patent issue thing?
If so, we may find that Google remains more relevant than Yahoo and therefore Compworld might be jumping the gun a bit.
I think Google is melllttttinggg....like the wicked witch of the west.
What's so funny to see is Google actually giving the enemy (Yahoo and MSN) the gun and the ammo to take Google down with (Florida update, etc... word filters).
I, for one, am not even going to bother trying to optimize for Google again because it is a total waste of time. I don't believe they are being honest about their filters. I will only optimize for MSN and Yahoo. I will also recommend that users do not use Google since you can't trust their results. Assuming I am not 'special' I would think Google is going to have a very difficult road ahead.
Some info would be great.
The free inclusion thing sounds very nice. While people tend to think of those as placebos, there seems like there is good evidence that the FAST free inclusion did eventually lead to free pages being crawled (although for me is stops at about 50% of the pages).
What I'd like to know is:
1) what is this FAST Enterprise Crawler thing? I love it, even if I don't see any results of its efforts yet.
2) what will Yahoo be feeding MSN? Currently Inktomi, and Inktomi PFI, feed MSN. Will there be a different database that feeds MSN (which seems weird since it obviously would never be as good as the best product Yahoo creates)? Will pre-March 1st PFI pages be unaffected in terms of MSN? Inktomi via its resellers has contracts with lots of folks, like me, who pfi'ed pages to be in MSN. Those pages never had a "right" or contract to be in Yahoo, so the April 15th date is a non-issue, but it would seem to be a clear breach of contract for those listings to die on April 15th so far as MSN is concerned.
What does the Agreement or Terms of Service with your Inktomi PFI program say about protection of your interest?
Inktomi PFI program
This never got you into Yahoo in the past, so its really a bonus that it does until April 15 - where is the breech of contract? - you will still be in the Ink powered SE's
- providing a free means for submission,
- being open to the SEO world about their plans, and
- putting competition back in the search world.
What we need to do now is just better understand the new Yahoo search. Could somebody please answer my earlier question...
"to Tim:
Can you clear this up also... What is the proper command to use at search.yahoo.com to determine which and how many URLs of a site are indexed in Yahoo?
is it:
1) site:domain.com, or
2) site:www.domain.com, or
3) site:http://www.domain.com
Note: each one of these commands provides different results? "
That's a very interesting take. Never thought of it this way. My site is there. However, what amazes me is that this site who's not on Y!, has at least 20X the number of backlinks of other sites who are in. Y! Slurp stops by several times a day, gets the robots.txt and leaves.
I got the most results back on my domain using:
site:mydomain.com
it produced 2 more results then
site:http://www.mydomain.com
and several hundred more then
site:www.mydomain.com
