homepage Welcome to WebmasterWorld Guest from 54.205.242.179
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld

Home / Forums Index / Yahoo / Yahoo Search Engine and Directory
Forum Library, Charter, Moderators: martinibuster

Yahoo Search Engine and Directory Forum

This 423 message thread spans 15 pages: 423 ( [1] 2 3 4 5 6 7 8 9 ... 15 > >     
Yahoo Inclusion Program
Trying to clear up some misunderstanding in this forum
Tim




msg:822988
 10:02 pm on Feb 27, 2004 (gmt 0)

The primary means of generating our index is via our free crawl, using our new Yahoo! Slurp crawler. Yahoo! Slurp discovers pages by following links on the web. We update our index with a daily crawl to gather newly created and fast-changing URLs, as well as our main crawl which updates our index incrementally twice per week.

We supplement this free crawl with the paid inclusion program, which allows us to add dynamic database content and other content that we could not otherwise discover and crawl. Currently less than 1% of our index is content that is included via our inclusion programs.

Our paid inclusion programs provide a superior way for sites to interact with us in a clearly structured fashion. These sites get the following
benefits:
1) Control over which URLs they include in the index (subject to content
guidelines)
2) Frequent refresh
3) Clearly defined way to interact with us (for example: clear feed
specifications)
4) Quality review and consultation (for feed customers)
5) Detailed reporting to track and optimize performance
6) Customer service

Participation in Yahoo!'s paid inclusion program does not guarantee rank in the search results. URLs are ranked in search results in the same way as all other URLs from the free crawl. The key drivers of ranking are keyword-specific relevance and site quality, as assessed by our regular search relevance algorithms with input from the quality review process.

(Note that Inktomi Index Connect clients are receiving Yahoo! Search traffic going forward, and Inktomi Search Submit customers are receiving a free trial of Yahoo! traffic until 4/15/04.)

We will also be launching free URL submission during the week of 3/1. This service allows content providers to suggest that the crawler should visit a site during the main crawl cycle. This service is different than paid inclusion in two ways:

1) There is no guarantee that we will crawl and include the URL in the index.
2) As this site potentially could be added to the main crawl, Yahoo! Slurp will follow links from that URL.

This service will reside on the Yahoo.com site and be accessible via a Yahoo! login. This service has been available for the AltaVista and Alltheweb Indices previously and the tradition will continue with Yahoo! Search.

 

jbgilbert




msg:822989
 10:40 pm on Feb 27, 2004 (gmt 0)

Tim,
If I understand you correctly that is great information! Please try to confirm and/or answer a couple statements/questions to further clarify...

1) PositionTech plays no role in getting URLs indexed by Yahoo after 4/2004.. correct?

2) I have a site with many, many pages and it has taken the old Inktomi (Slurp) a LONG LONG time (almost a Year) to even index a small portion of them.
Will the new (Yahoo! Slurp) crawler be improved to better handle large sites?

Kirby




msg:822990
 11:07 pm on Feb 27, 2004 (gmt 0)

How do I identify Y!Slurp?

mayor




msg:822991
 11:13 pm on Feb 27, 2004 (gmt 0)

The big question now, Tim, is how much will this cost?

jbgilbert




msg:822992
 11:27 pm on Feb 27, 2004 (gmt 0)

Kirby,

Here is a partial line from a log. Look for the
Yahoo! Slurp.

66.196.65.38 - - [26/Feb/2004:07:14:27 -0600] "GET /robots.txt HTTP/1.0" 200 5818 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; [help.yahoo.com...]

born2drv




msg:822993
 11:31 pm on Feb 27, 2004 (gmt 0)

>>Detailed reporting to track and optimize performance

What does this mean exactly? Will we be able to track how well our URLs are performing for various search terms instead of using WPG or something. Or is it just like the previous inktomi where it just tells us traffic generated?

You guys should consider incorporating ranking reporting software, where we pick the search terms and it tells us our current ranking. I think I'd pay just for that. And it would reduce the use of WPG or other automated rank checkers most SE's frown upon us using.

jbgilbert




msg:822994
 11:49 pm on Feb 27, 2004 (gmt 0)

Tim,

Can you clear this up also... What is the proper command to use in a Yahoo search to determine which and how many URLs of a site are indexed in Yahoo?

is it:

1) site:domain.com, or
2) site:www.domain.com, or
3) site:http://www.domain.com

Note: each one of these commands provides different results?

Thanks ahead of time...

pleeker




msg:822995
 12:04 am on Feb 28, 2004 (gmt 0)

Tim -- thanks for sharing all that. Great to see Yahoo (you) taking a proactive approach to sharing information with the webmastering community.

herlaar2303




msg:822996
 12:15 am on Feb 28, 2004 (gmt 0)

5) Detailed reporting to track and optimize performance

I guess this will be through the Site Match system?

See patent filing:
[tarr.uspto.gov...]

Ric

notsosmart




msg:822997
 12:15 am on Feb 28, 2004 (gmt 0)

How about the sites/pages for which I paid to be in Ink within the past 12 months? Do I gotta go pay for them again? What's the deal?

Dynamoo




msg:822998
 12:22 am on Feb 28, 2004 (gmt 0)

Uh let's see.. that looks likes Yahoo! is using the increased traffic as a bait to get people to buy into Inktomi, and then you'll dump the Inktomi listings and ask people to stump up again?

Looks pretty deceptive to me. Like when Yahoo! persuaded all those webmasters to fork out $299 a year and then dumped the listings out of the search results.

Krapulator




msg:822999
 12:29 am on Feb 28, 2004 (gmt 0)

What about the people like myself who have just forked out for 12 months Ink inclusion? Have we just wasted our money or is there still some extra benefit to pay for Ink inclusion?

jdMorgan




msg:823000
 12:48 am on Feb 28, 2004 (gmt 0)

Yahoo has acquired a lot of properties recently, and therfore cannot implement changes instantly. But in all fairness, let's try to be grateful for such things as
We will also be launching free URL submission during the week of 3/1.
rather than pounding on Yahoo! for the past policies of the companies they have recently acquired, or the companies which which those acquired companies did business.

I see clear evidence that Yahoo! is making major improvements to those policies already.

Jim

cindysunc




msg:823001
 1:01 am on Feb 28, 2004 (gmt 0)

Time will tell.

Compworld




msg:823002
 1:36 am on Feb 28, 2004 (gmt 0)

This may be the beginning of the end to the Google dynasty. Loosing Yahoo, was a BIG blow. I would not be surprised if MSFT and AOL become a little bit more cozy with Yahoo instead of Google.

Gotta give it to their Chairman. Terry is going after Google with vengeance. Maybe this is why the IPO was postponed? All I hear on CNBC is Yahoo & MSFT going after Google. It doesn't look good for Google. I still believe that they will be a strong provider, but second or third to MSFT & Yahoo. These companies have the direction now, and Google is in the way. It either they become an acquisition, or the go out of business (highly doubtful), or stay a distant third to the new bug two.

I had stated this all along. Google was not smart in letting Yahoo go. Now, I do not believe they can put up a good fight. I guess only time will tell. I like the feeling that compotition is back in the search engine market. Very very important. I was one of the people that thought Google was pushing its weight around way too much. Every company needs to be placed back it is place every once and a while.

Done with my rant.

CompWorld

walkman




msg:823003
 1:49 am on Feb 28, 2004 (gmt 0)

This sounds great. As long as you don't get penalized for paying to have a few pages included.

Now only if we had an update on the suspected penalized /banned sites review process. Shall we expect a thread on that soon :)?

logiclamp




msg:823004
 1:49 am on Feb 28, 2004 (gmt 0)

According to Tim: "The key drivers of ranking are keyword-specific relevance and site quality, as assessed by our regular search relevance algorithms with input from the quality review process."

Sounds like page-rank is not a factor. I wonder if this is a patent issue thing?

If so, we may find that Google remains more relevant than Yahoo and therefore Compworld might be jumping the gun a bit.

ILuvSrchEngines




msg:823005
 2:12 am on Feb 28, 2004 (gmt 0)

>>All I hear on CNBC is Yahoo & MSFT going after Google. It doesn't look good for Google.

I think Google is melllttttinggg....like the wicked witch of the west.

What's so funny to see is Google actually giving the enemy (Yahoo and MSN) the gun and the ammo to take Google down with (Florida update, etc... word filters).

I, for one, am not even going to bother trying to optimize for Google again because it is a total waste of time. I don't believe they are being honest about their filters. I will only optimize for MSN and Yahoo. I will also recommend that users do not use Google since you can't trust their results. Assuming I am not 'special' I would think Google is going to have a very difficult road ahead.

atadams




msg:823006
 2:15 am on Feb 28, 2004 (gmt 0)

I also just paid for Ink inclusion. What happens now. It looks like I just paid for 12 months and will be getting 6 weeks.

Some info would be great.

TomWaits




msg:823007
 2:32 am on Feb 28, 2004 (gmt 0)

no message, whateva

steveb




msg:823008
 2:39 am on Feb 28, 2004 (gmt 0)

Could we keep the Google rants in the Google Rants forum please.

The free inclusion thing sounds very nice. While people tend to think of those as placebos, there seems like there is good evidence that the FAST free inclusion did eventually lead to free pages being crawled (although for me is stops at about 50% of the pages).

What I'd like to know is:
1) what is this FAST Enterprise Crawler thing? I love it, even if I don't see any results of its efforts yet.

2) what will Yahoo be feeding MSN? Currently Inktomi, and Inktomi PFI, feed MSN. Will there be a different database that feeds MSN (which seems weird since it obviously would never be as good as the best product Yahoo creates)? Will pre-March 1st PFI pages be unaffected in terms of MSN? Inktomi via its resellers has contracts with lots of folks, like me, who pfi'ed pages to be in MSN. Those pages never had a "right" or contract to be in Yahoo, so the April 15th date is a non-issue, but it would seem to be a clear breach of contract for those listings to die on April 15th so far as MSN is concerned.

mayor




msg:823009
 2:49 am on Feb 28, 2004 (gmt 0)

steveb >> would seem to be a clear breach of contract for those listings to die on April 15th

What does the Agreement or Terms of Service with your Inktomi PFI program say about protection of your interest?

eyeinthesky




msg:823010
 3:14 am on Feb 28, 2004 (gmt 0)

Tim, any idea on whether a current Ink penalty will be removed with the new Yahoo free inclusion? Thanks.

cbpayne




msg:823011
 3:55 am on Feb 28, 2004 (gmt 0)

Inktomi PFI program

This never got you into Yahoo in the past, so its really a bonus that it does until April 15 - where is the breech of contract? - you will still be in the Ink powered SE's

cbpayne




msg:823012
 4:00 am on Feb 28, 2004 (gmt 0)

I should have added that Yahoo NEVER said they were replacing Google search results with Inktomi's. They just said they were dropping Google. So I do not see how Yahoo mislead people into the Inktomi 'pay for inclusion' on the undertsanding this would eventually get them into Yahoo - this is not what was contracted for.

jbgilbert




msg:823013
 4:26 am on Feb 28, 2004 (gmt 0)

I agree with the moderator.. Yahoo is doing a good thing and we should stop ranting. Yahoo is:

- providing a free means for submission,
- being open to the SEO world about their plans, and
- putting competition back in the search world.

What we need to do now is just better understand the new Yahoo search. Could somebody please answer my earlier question...

"to Tim:
Can you clear this up also... What is the proper command to use at search.yahoo.com to determine which and how many URLs of a site are indexed in Yahoo?

is it:

1) site:domain.com, or
2) site:www.domain.com, or
3) site:http://www.domain.com

Note: each one of these commands provides different results? "

cbpayne




msg:823014
 4:37 am on Feb 28, 2004 (gmt 0)

I don't think that any of them work that well - I tried all 3 syntaxes on mt site. The first 2, only returned 2 results but gave me the <more from this site> which gave me 100. The third option gave me 60 but no option for more. When I searched for a couple of specific pages that were not in the 2, 60 or 100, they were in the index.

flobaby




msg:823015
 4:45 am on Feb 28, 2004 (gmt 0)

I found the third option gives me site results from Google as the pages it listed aren't yet in the Yahoo/Ink database.

walkman




msg:823016
 4:50 am on Feb 28, 2004 (gmt 0)

"I found the third option gives me site results from Google as the pages it listed aren't yet in the Yahoo/Ink database."

That's a very interesting take. Never thought of it this way. My site is there. However, what amazes me is that this site who's not on Y!, has at least 20X the number of backlinks of other sites who are in. Y! Slurp stops by several times a day, gets the robots.txt and leaves.

Easy_Coder




msg:823017
 5:06 am on Feb 28, 2004 (gmt 0)

jbgilbert -

I got the most results back on my domain using:
site:mydomain.com
it produced 2 more results then
site:http://www.mydomain.com
and several hundred more then
site:www.mydomain.com

This 423 message thread spans 15 pages: 423 ( [1] 2 3 4 5 6 7 8 9 ... 15 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Yahoo / Yahoo Search Engine and Directory
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved