Questions for Yahoo!

Round them up and post them here.

[WebGuerrilla]

Over the past couple of weeks, we’ve been fortunate to have several Yahoo employees spending time answering questions about the new Yahoo Search.

This level of participation in a webmaster forum from a public search company is truly unprecedented, and is something we definitely would like to continue. However, the shear volume of questions and responses we’ve received has created some threads that have become quite difficult to follow.

So, in order to try and make things a bit easier to follow, I think now would be a good time to move to a Q & A similar to the format that we’ve used in the past in the Google forum.

Here’s how it will work:

Questions will be posted in this thread. We will let this thread run for the next few days. Once we have a solid list of quality questions, this thread will be locked. If/when someone from Yahoo Search decides to respond to a question from this thread, we will split off that particular Q&A into its own thread. Any follow-up questions or discussions will take place in the new thread. Doing this will help us keep everything on-topic.

With that in mind, here are some ground rules:

1. Keep the questions focused on Yahoo Search’s products and services. (No asking Tim what his favorite color is)

2. Keep the questions short and sweet. (You will have the opportunity to get long-winded in any follow-up discussions).

3. No specific site information. (We are not the Yahoo help desk or spam police)

4. Keep the tone positive and professional. (All rants will be nuked)

5. Read the other questions before you post yours. (I hate spending time deleting duplicates).

And finally, please keep in mind that just because you ask, doesn’t mean you will get an answer. There are obviously many questions that Yahoo can’t/won’t answer. And we (Mods/Admins) have no input or control regarding which questions, if any, they choose to answer.

WG

[nkakar]

I had all my pages up on yahoo for a whole month from feb 18 - march 18.. if you guys rememeber feb 18 was the date yahoo switched to google, and exactly a month later, my site dissapeared from the serps even though everything is there when u look for the site with the url.... my titles are very unique and even if i type in my titles the same exact way which nobody else does, im still no where to be found on yahoo.. my sales have dropped 50% because of this.. any ideas on whats going on?

[outland88]

Either Yahoo has a mighty big glitch nkakar or you joined the boat of similarly penalized sites. Yeah, a lot of us have been hurting a lot longer.

[fello]

Any comment on the "Sites Disappearing" problem from this thread: [webmasterworld.com...]

[outland88]

How exactly does one know if one is penalized by Yahoo? What is the specific method Yahoo uses to tell if one is penalized? My reseller Positiontech (who claim to be experts) say if you feel you are penalized ask Yahoo. How do I know it's not related to some horrendous bugs in Inktomi and Yahoo? Since my site has never been penalized by anybody and previuosly listed by Yahoo under the same guidelines what am I doing and everybody else doing that's so bad. With the amount of complaints Yahoo should have seen something that is a "red flag" causing all these problems.

There has been some excellent advice by webmasters but it's still pretty much a shaky guess if Yahoo is penalizing sites or not. Is it possible Yahoo has instituted a considerable amount of harshness that renders many sites permanently unusable in either Google or Yahoo. Somethings going on that doesn't make total sense.

Inquiring minds want to know.

[NeedScripts]

hum... everyone seems to be asking good question.. I assume I better ask a stupid one...

Does Yahoo plan to start any revenue sharing program with Publishers? I mean, Overture and Yahoo's PPC clients would for sure love to see their ads on the web sites of thousands of publishers around the world.

[pkropp]

If we signed up for Inktomi Pay for inclusion, what is the process of converting that to Yahoo! Site Match? I signed up through Network Solutions. Is there a page that details this conversion?

[Chndru]

What is the USP for Yahoo Search. What would lure a user from using Google or MSN?

[nkakar]

Mike, is it true that if you do a search for your domain and all the pages are there, yet if u do an exact search for your titles on the domain, you dont show up, you're being penalized 100%?
let me know what you guys think?

[cyberfyber]

<<<<Mike, is it true that if you do a search for your domain and all the pages are there, yet if u do an exact search for your titles on the domain, you dont show up, you're being penalized 100%?
let me know what you guys think?>>>>

That is absolutely the case with three of my own sites.

I myself was told outright by Jon at Yahoo that I have no ranking penalties on my sites. 'And if that's so, then what's going on?

Which then leads to the same question asked by one or more other individuals in these forums. How many different types of Penalisations are there at Yahoo? and how'd you tell which one you're experiencing?

'a question yet to be answered.

[louponne]

Any point in continuing to pay for Yahoo Directory Listing?

[seasalt]

Mike:

1. When I do an exact phrase search on a multiple keyword string found on my site, I do find my site in the results. However, it is found following other results not containing the exact phrase - or even all the words in the keyword string. Is this an algo shortcoming or glitch?

2. If a new site was listed because of using Inktomi and was never "free crawled", what will happen if one discontinues the Inktomi service? The site has many quality links that causes it to be crawled by other engines including a Yahoo directory listing.

3. I filled out a Yahoo! spam report over a week ago and the only thing that has happened is more spam sites have appeared - including additional pages from sites I reported. Can we expect improvement in this area?

4. Related to the above, why are so many sites coming out on top that violate what Yahoo! deems a quality site? You mentioned Yahoo! uses a combination of algo + manual review. It seems very odd that with that combination that there are still quality sites/pages that are missing/penalized when multiple spam sites make it through. Can one expect progressive improvement or is the "new" YST?

seasalt

[helenp]

Hi,
I found an company that has something wrong with an 301 to my site.
I went in to that site, searched for one of my index pages, clicked to view that page, then searched for another index of mine, clicked to view it, and went to log file.
One of the index pages is an 301 and the other is not, so something most be wrong.
Should we report all these things we find?
Could these 301 errors be the reason for banning?
Ore are yahoo working on it.
I already apealed before I found this.
Or should just contact these companys to have them fixed these errors?

This is from my logfile:

result for my sales index:
80.58.2.170 - - [29/Mar/2004:03:02:51 -0500] "GET /sales HTTP/1.0" 301 327 "http://www.domain.com/sub/m/marbella.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
80.58.2.170 - - [29/Mar/2004:03:02:52 -0500] "GET /sales/ HTTP/1.1" 200 10429 "http://www.domain.com/sub/m/marbella.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"

result for main index:
80.58.2.170 - - [29/Mar/2004:03:03:15 -0500] "GET / HTTP/1.1" 304 - "http://www.domain.com/sub/m/marbella.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"

This 301 is repeated every time when searching that index page of mine from that companys search index.

Also there are a lot of this 301 in log, from same company to same page, but don´t seems to be for search results, this is from their cgi:
82.68.144.21 - - [04/Mar/2004:05:25:55 -0500] "GET /sales HTTP/1.0" 301 327 "http://www.domain.com/cgi/visit.cgi" "Mozilla/4.0 (compatible; MSIE 4.01; Windows 98)"
82.68.144.21 - - [04/Mar/2004:05:25:55 -0500] "GET /sales/ HTTP/1.0" 200 10034 "http://www.mydomain.com/sales" "Mozilla/4.0 (compatible; MSIE 4.01; Windows 98)"

Funny, this search company (pr7) is not to be found in All The Web (Inktomi results I think) or Yahoo as if they are banned as well.

[bcc1234]

Are there any special guidelines for sites that run inhouse affiliate programs with thousands of "unique" incoming links that result in 301 or 302 redirects?

[mr_strong]

Slurp has been coming to my site every day for the last week and grabbling the robots.txt and home page and going no further.

My robots.txt file is valid and allows access to slurp to my whole site. My linking structure is also fine (other bots have crawled my whole site without problem)

Is this normal behaviour for slurp? When can I expect slurp to do a deeper crawl?

Many thanks for your time :)

[webmktg]

Yes, This is how Slurp crawls websites. I'm experiencing the same issue for my websites. Also sometimes you will get 404 errors because Slurp crawls random urls.

Just post a message to the forum if you see any changes.

Good Luck

[mikemcs]

Mike:

What can be done for a site that has a penalty in Yahoo, but does not break any rules found on your Site Guidelines page.

Thanks Mike M.

[mmarlor]

First, apologies - this is a duplicate of my previous question here: [webmasterworld.com...] .. so you may wish to remove the old one.

GoogleBot is very fond of indexing my site's forums - absolutely loves it, and it draws in quite a number of new participants through web searches.
Yahoo, however, has started picking up plenty of articles from my site, and it's even indexed the forum's index page - but it just doesn't seem interested in any topics.

I have previously implemented the phpBB sessions optimization for GoogleBot and Slurp, to remove session ids from URLs. Since Yahoo went live with their engine, I have reduced the UserAgent string search down to just 'slurp', which using strstr in php should find the Yahoo crawler quite fine.

Is there anything I've missed? Or does Yahoo have a thing against phpBB forum pages? :-S

Thanks in advance,

Matt

[donstar]

Hi Mike,
Is Inktomi Slurp same as Yahoo Slurp? My webstats show Inktomi Slurp daily crawls my site but I have not seen Yahoo Slurp crawl my site. I get traffic from yahoo & my pages are in the yahoo search results but I get no visits from yahoo slurp. Should I do something about this?

Thanks an advance,
donstar

[EarWig]

Hi Mike & Tim

Yahoo have recently listed a site of ours that has a robots.txt forbidding access and robots exclusion meta tags in every page.

Is this something that will resolve it self over time or can you advise any action I need to take?

Regards
Ray - EW

[seasalt]

1. Assume a 'penalty' was incurred by an algo (no human review).

Can one also assume that once the instigating factors are removed the site would be restored via the algo? Or is it 'once and out'?

2. If a site is in the Y! directory and has incurred some kind of penalty from elsewhere in the Y! system, will the penalty be looked at before the directory is crawled?

seasalt

[edited by: seasalt at 8:00 pm (utc) on Mar. 30, 2004]

[Eljaybe]

I never got a response for my questions dated March 23, 2004. So, I'm posting them again. Maybe I'll get lucky?...

1. I'm looking for a list of search engines that SiteMatch covers. I was using Position Tech for some time before Site Match took over. Does it submit to all the same search engines that Position Tech covered? HotBot? AltaVista? AlltheWeb? Lycos? MSN? Excite? I want to know which ones I lost by switching to Site Match. Thanks.

2. I have another Yahoo question, actually about the Yahoo Directory...
My client's site was included in the Yahoo Directory at least a couple years ago by a different SEO company. The URL of my client's site changed late last year and we placed a redirect on the site. I've been trying to submit the URL change to Yahoo Directory but never have any luck. I don't know the "magic" email address of the original submitter and neither does my client. So, how do I get thru to Yahoo and get them to change our URL in the Directory?

[chinkchink]

Y! DIRECTORY LISTING RELATED
Got hired to "clean up the mess" left behind by another SEO person. He submitted my client's URL in the WRONG directory with an out-dated description. I have no way to recover the email he used to submit this incorrect listing, thus cannot do the normal "change suggestion" at [add.yahoo.com...] (already tried about 2 months ago)

Should I RE-SUBMIT? as if it is a new site with a new affliated email? despite the fact that the URL is already in Y! Directory? would it be mistaken as duplicate submission?

If not, is there another channel to reach Y! and get them to fix this for me?

[helenp]

Hi Tim or Mike,
After apealed to e-mail you posted for apealing, and when site has been reviewed, will ve have any notice of results? and posible time could take for inclusion on all sites owned by yahoo suposing site is accepted?

Or must we just wait and see what happens.

This could be very important for many.

I really would like to know results as sone as their are any, for several reasons, actually one is to contact companys with wrongly made links to me, but prefer not to do anything until process is over.

[Chicken Juggler]

I just realized that there is a good site that added a link to my site. It is just a bunch of guys that are interested in a subject. They site could make money but they don't seem to care it is just fun and they don't want to taint it with profit. (just a side note)

<Back to my point>

Can I add their front page to site match. What are the ramifications of doing this besides the fact that I have to pay money? It is worth it to me to have Yahoo spider it and consider the outgoing links so my site does better.

[IITian]

I have a site that had about 50 'starter' pages that were crawled regularly by inktomi. In the last three months I added about 500 pages and unfortunately almost all of them remain uncrawled by the Yahoo crawler.

It comes regularly to my site but crawls only those 50 legacy pages. (I think, a couple of times it might have grabbed a couple of newer pages.) From my index page, the one that is regularly crawled, I have links to root directories of these new pages but they are left untouched.

The day (March 4?) I submitted my site using addurl (even though there was no need for this) msnbot came and grabbed almost all the pages. A couple of days ago I noticed FAST crawler grabbing pages.

I wonder who is partnering with who and should I be worried about Yahoo not crawling my pages? Thus far I have almost no referrals from Yahoo (and MSN) for these new pages.

[cabos]

Question for Yahoo

Why not auction off Penalized Site Reviews on Ebay?

Figure each one will sell for $1,000 minimum. Times at least 500 site reviews and you don't have to accept any into the search results.

Provide lip service and collect the cash.

[seasalt]

Tim:

My site is in the current Yahoo!, MSN, ATW results and ranks relatively well. However, in the new Alta Vista results that came out today, it is at the bottom - even for search phrases unique to my site - similar to penalties mentioned here. Does AV have its own penalty database? Is something else going on?

seasalt

[Moff]

Sorry if this has been asked/answered before but I couldnt find it.

Pressuming the Search algo looks at a sites Title and description when ranking then does it use the sites official Title or that given to it in the Yahoo Directory when deciding on SERPs placement?

The reason I ask is that all the sites I see in the SERPS appear to have their Directory Titles/Descriptions as opposed to those in their meta Data.

Thanks in advance

Moff

[edited by: Moff at 1:14 pm (utc) on April 2, 2004]

[newwebster]

Why do I have a listing with www.mydomain.com and mydomain.com both showing in the serps? mydomain.com has an old cach and is about a month old version of my index page. www.mydomian.com has no cach, but has the updated title description so I know it is newer. Should mydomain.com not be replaced by www.mydomain.com alltogether?

[soapystar]

if it takes 30-90 days for a review how comes some people round here got straight back in already? This is a fair review system is it?