Welcome to WebmasterWorld Guest from 220.127.116.11
Forum Moderators: martinibuster
This level of participation in a webmaster forum from a public search company is truly unprecedented, and is something we definitely would like to continue. However, the shear volume of questions and responses we’ve received has created some threads that have become quite difficult to follow.
So, in order to try and make things a bit easier to follow, I think now would be a good time to move to a Q & A similar to the format that we’ve used in the past in the Google forum.
Here’s how it will work:
Questions will be posted in this thread. We will let this thread run for the next few days. Once we have a solid list of quality questions, this thread will be locked. If/when someone from Yahoo Search decides to respond to a question from this thread, we will split off that particular Q&A into its own thread. Any follow-up questions or discussions will take place in the new thread. Doing this will help us keep everything on-topic.
With that in mind, here are some ground rules:
1. Keep the questions focused on Yahoo Search’s products and services. (No asking Tim what his favorite color is)
2. Keep the questions short and sweet. (You will have the opportunity to get long-winded in any follow-up discussions).
3. No specific site information. (We are not the Yahoo help desk or spam police)
4. Keep the tone positive and professional. (All rants will be nuked)
5. Read the other questions before you post yours. (I hate spending time deleting duplicates).
And finally, please keep in mind that just because you ask, doesn’t mean you will get an answer. There are obviously many questions that Yahoo can’t/won’t answer. And we (Mods/Admins) have no input or control regarding which questions, if any, they choose to answer.
There has been some excellent advice by webmasters but it's still pretty much a shaky guess if Yahoo is penalizing sites or not. Is it possible Yahoo has instituted a considerable amount of harshness that renders many sites permanently unusable in either Google or Yahoo. Somethings going on that doesn't make total sense.
Inquiring minds want to know.
That is absolutely the case with three of my own sites.
I myself was told outright by Jon at Yahoo that I have no ranking penalties on my sites. 'And if that's so, then what's going on?
Which then leads to the same question asked by one or more other individuals in these forums. How many different types of Penalisations are there at Yahoo? and how'd you tell which one you're experiencing?
'a question yet to be answered.
1. When I do an exact phrase search on a multiple keyword string found on my site, I do find my site in the results. However, it is found following other results not containing the exact phrase - or even all the words in the keyword string. Is this an algo shortcoming or glitch?
2. If a new site was listed because of using Inktomi and was never "free crawled", what will happen if one discontinues the Inktomi service? The site has many quality links that causes it to be crawled by other engines including a Yahoo directory listing.
3. I filled out a Yahoo! spam report over a week ago and the only thing that has happened is more spam sites have appeared - including additional pages from sites I reported. Can we expect improvement in this area?
4. Related to the above, why are so many sites coming out on top that violate what Yahoo! deems a quality site? You mentioned Yahoo! uses a combination of algo + manual review. It seems very odd that with that combination that there are still quality sites/pages that are missing/penalized when multiple spam sites make it through. Can one expect progressive improvement or is the "new" YST?
This is from my logfile:
result for my sales index:
18.104.22.168 - - [29/Mar/2004:03:02:51 -0500] "GET /sales HTTP/1.0" 301 327 "http://www.domain.com/sub/m/marbella.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
22.214.171.124 - - [29/Mar/2004:03:02:52 -0500] "GET /sales/ HTTP/1.1" 200 10429 "http://www.domain.com/sub/m/marbella.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
result for main index:
126.96.36.199 - - [29/Mar/2004:03:03:15 -0500] "GET / HTTP/1.1" 304 - "http://www.domain.com/sub/m/marbella.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
This 301 is repeated every time when searching that index page of mine from that companys search index.
Also there are a lot of this 301 in log, from same company to same page, but don´t seems to be for search results, this is from their cgi:
188.8.131.52 - - [04/Mar/2004:05:25:55 -0500] "GET /sales HTTP/1.0" 301 327 "http://www.domain.com/cgi/visit.cgi" "Mozilla/4.0 (compatible; MSIE 4.01; Windows 98)"
184.108.40.206 - - [04/Mar/2004:05:25:55 -0500] "GET /sales/ HTTP/1.0" 200 10034 "http://www.mydomain.com/sales" "Mozilla/4.0 (compatible; MSIE 4.01; Windows 98)"
Funny, this search company (pr7) is not to be found in All The Web (Inktomi results I think) or Yahoo as if they are banned as well.
My robots.txt file is valid and allows access to slurp to my whole site. My linking structure is also fine (other bots have crawled my whole site without problem)
Is this normal behaviour for slurp? When can I expect slurp to do a deeper crawl?
Many thanks for your time :)
GoogleBot is very fond of indexing my site's forums - absolutely loves it, and it draws in quite a number of new participants through web searches.
Yahoo, however, has started picking up plenty of articles from my site, and it's even indexed the forum's index page - but it just doesn't seem interested in any topics.
I have previously implemented the phpBB sessions optimization for GoogleBot and Slurp, to remove session ids from URLs. Since Yahoo went live with their engine, I have reduced the UserAgent string search down to just 'slurp', which using strstr in php should find the Yahoo crawler quite fine.
Is there anything I've missed? Or does Yahoo have a thing against phpBB forum pages? :-S
Thanks in advance,
Thanks an advance,
Can one also assume that once the instigating factors are removed the site would be restored via the algo? Or is it 'once and out'?
2. If a site is in the Y! directory and has incurred some kind of penalty from elsewhere in the Y! system, will the penalty be looked at before the directory is crawled?
[edited by: seasalt at 8:00 pm (utc) on Mar. 30, 2004]
1. I'm looking for a list of search engines that SiteMatch covers. I was using Position Tech for some time before Site Match took over. Does it submit to all the same search engines that Position Tech covered? HotBot? AltaVista? AlltheWeb? Lycos? MSN? Excite? I want to know which ones I lost by switching to Site Match. Thanks.
2. I have another Yahoo question, actually about the Yahoo Directory...
My client's site was included in the Yahoo Directory at least a couple years ago by a different SEO company. The URL of my client's site changed late last year and we placed a redirect on the site. I've been trying to submit the URL change to Yahoo Directory but never have any luck. I don't know the "magic" email address of the original submitter and neither does my client. So, how do I get thru to Yahoo and get them to change our URL in the Directory?
Should I RE-SUBMIT? as if it is a new site with a new affliated email? despite the fact that the URL is already in Y! Directory? would it be mistaken as duplicate submission?
If not, is there another channel to reach Y! and get them to fix this for me?
Or must we just wait and see what happens.
This could be very important for many.
I really would like to know results as sone as their are any, for several reasons, actually one is to contact companys with wrongly made links to me, but prefer not to do anything until process is over.
<Back to my point>
Can I add their front page to site match. What are the ramifications of doing this besides the fact that I have to pay money? It is worth it to me to have Yahoo spider it and consider the outgoing links so my site does better.
It comes regularly to my site but crawls only those 50 legacy pages. (I think, a couple of times it might have grabbed a couple of newer pages.) From my index page, the one that is regularly crawled, I have links to root directories of these new pages but they are left untouched.
The day (March 4?) I submitted my site using addurl (even though there was no need for this) msnbot came and grabbed almost all the pages. A couple of days ago I noticed FAST crawler grabbing pages.
I wonder who is partnering with who and should I be worried about Yahoo not crawling my pages? Thus far I have almost no referrals from Yahoo (and MSN) for these new pages.
My site is in the current Yahoo!, MSN, ATW results and ranks relatively well. However, in the new Alta Vista results that came out today, it is at the bottom - even for search phrases unique to my site - similar to penalties mentioned here. Does AV have its own penalty database? Is something else going on?
Pressuming the Search algo looks at a sites Title and description when ranking then does it use the sites official Title or that given to it in the Yahoo Directory when deciding on SERPs placement?
The reason I ask is that all the sites I see in the SERPS appear to have their Directory Titles/Descriptions as opposed to those in their meta Data.
Thanks in advance
[edited by: Moff at 1:14 pm (utc) on April 2, 2004]