Welcome to WebmasterWorld Guest from 188.8.131.52
Forum Moderators: martinibuster
We cannot be found by using any keywords, our company name, OR just by typing in our domain.
In the past I would watch the logs and see Slurp go through all of our pages (over 2000 pages) and it does it regularly. In fact, I can't think of many days I have not seen yahoo in our logs! Can anyone think of why we have NEVER been listed? I doubt it is our content as our competitors are listed and we do not do anything we are not supposed to do; we run things pretty clean.
What is even more odd is that I've noticed lately that yahoo isn't going to our pages anymore, it just keeps accessing "robots.txt" - which we have - but it is an empty file. What is odd.... Yahoo literally comes to our site ever 3-5 SECONDS!
Any assistance would be greatly appreciated. I can't understand why we are not listed. I doubt we are blacklisted (thought I don't know how to check with Yahoo) - as Yahoo! Slurp comes every 3-5 seconds!
Is there a way with Yahoo to figure out if our site has been blocked from inclusion? If so, does anyone have any recommendations on how to contact Yahoo regarding inclusion? I've searched Yahoo and can't find a relavent contact for such a situation.
The only thing I can think of is Yahoo is basically telling the majority of us to pay the $299/$600 annually for inclusion.
Perhaps seeing Yahoo! Slurp in my logs every few seconds is Yahoo's way of getting my interest in Yahoo... who knows!
Has anyone ever written yahoo and received a detailed response?...did it help?
I think this is the bot following links from other sites to yours, the bot wouldnt (if this *is* the case) know about any penalty or ban on your site, its just doing its job and trawling the net.
If it donīt know, why does it only gets the robots? even if you donīt have any robots.txt it only search for the robots, then instead you only get an 404 error.
To me sounds like an redirection or algo problem, not exclusion...is your site found if only search for your domain without .com and www?
and if you write your full domain name, does it state ontop did you search for ....similar domain...?
Or does it not show up at all if searching for your full domain? thats the case for sites banned both by yahoo and google.
Yes, I have done all that. My website appears fine when searched for wwWebmasterWorldebsite.com, mywebsite.com and even when I search for: my web site. And I am #9 on yahoo for a simillar keyword with the url wwWebmasterWorldebsite.com/somethingelse.htm
and linkdomain:wwWebmasterWorldebsite.com brings back 13,700 links
I have no excplanation, maybe I was banned just for a specific keyword or something, I dont know, but as I said so far I have been visited 4 times by Slurp TODAY.
The only thing I can think of is Yahoo is basically telling the majority of us to pay the $299/$600 annually for inclusion
no, that wont get you into the serps, just the directory. Problem is people still try to find the logic in what yahoo does by assuming they are trying to index the web.
So, does anyone know what to do? HOW to get indexed? I've used Yahoo's freed submit MANY times over the past 3 years with no luck at all.
It's sort of embarassing as I see a lot of our competitors listed on Yahoo and quite of few of those sites look like they were designed by grade school kids and make some obvious mistakes that Yahoo states they don't like/want in a website.
This is SOOO frustrating! At this point I would be happy if Yahoo would AT LEAST index our homepage.
Here is a follow up. I finally got a response from Yahoo! In a nutshell they said if our site is a "commercial" site then I "must" use the fee-based Yahoo! Express program.
The support person who wrote back saying "According to our records, we have not received a submission request for this URL." - what a joke as I have submitted to yahoo many many times over the past 3 years... and I also see their bot in our logs (looking at robots.txt only though).
It was nice for Yahoo to write back... but I don't think they should hide the fact that you MUST use their fee-based submit for commercial sites (unless I've missed it along the way). They should be straight forward on their submission page and state that paying $299/$600 is the ONLY way to submit a site for indexing. (and if you are a commercial site and never paid to get indexed and listed in results... it sounds like you were lucky and they are enforcing this from now on).
Try putting information in your robots.txt file for a start.
Could you answer these questions please:
1. Roughly how many inbound links to you have to your site (not ones from sites you own yourself)?
2. Has your site changed topic/product since it started 3 years ago?
3. Do you provide information/product/service or do you simply re-direct to other sources?
4. Have you ever paid someone else to build you link popularity/get links for you?
5. Were you ever listed in the Inktomi index?
6. Is your main page dynamic?
7. Do you have a sitemap linked to from your main page?
8. How much of your site is in the Google index? (10%, 50%, 90% etc.)
Hopefully with a clearer picture we'll be able to help as much as possible. You could always sticky me your URL if you like and I'll have a look.
Also, I don't we should be speculating that Yahoo is dropping sites to get people to use Sitematch. My site is highly commercial with hundreds of thousands of product pages and they've never once dropped me. If they were going to do this kind of thing they wouldn't be aiming at the Mom & Pop sites they'd be tackling the big boys but as the big boys are already paying through the nose with Overture and Sitematch they don't need to bother with such underhanded practices.
They should be straight forward on their submission page and state that paying $299/$600 is the ONLY way to submit a site for indexing.
It seems you're still a bit confused about Y's various programs. The Express submssion your referred to is for the *directory* not *search*. It has nothing to do w/ indexing. The programs are as follows:
Yahoo! Directory [dir.yahoo.com]
- simply a listing of sites
- commercial sites pay $299 for the chance to be included on *one* of the pages
- only bearing on search is the directory *may* be one of the sites the crawler begins
To increase your chances of being indexed by Yahoo!, you need to aquire links from pages that are heavily crawled by Slurp.
1) We have approximately 220 inbound links.
2) No, we haven't changed topic or products in the past 3 years. We have added some products in that time frame though.
3) We don't redirect to any sources at all. We have thousands of product pages and many many articles (on topic).
4) No, we haven't paid anyone to get links for us.
5) I thought we were..but I can't say 'yes' for sure as its been a while and I can't go back and check...but seeing how inktomi is now yahoo...I would have to assume our site wasn't listed with inktomi.
6) No, it is not dynamic. We update some information on it about once every two weeks though.
7) Yes, we have a site map that is linked from our main page.
8) roughly 95-100% of our site is in the Google index. 3,350 pages.
Also, nice comment regarding sitematch - I admit it was unneeded speculation.
Regarding the "robots.txt" file. You recommended placing information in it. What should I place in the robots.txt file?
Also, quick note to "your_store" - I am clear about Yahoo's services...I was just relaying what Yahoo stated in their email. I have a feeling that they are very busy and being so... it's easy to send an incorrect response. I wasn't happy with their response so I quickly replied to their email. I will post a follow up when I hear back from Yahoo.
MikeBeverley - I hope the information I provided helps in some way. I'm grateful for any information you can provide!
6. Is your main page dynamic?
How does it affect ranking or possible banning?
Our index page is dynamic, but only because we put a simple search function on this page.
I'm nearly sure that it could be solved in some other way, so that the page remains static.
Could someone confirm the above statement? If so, we will change our index page, because we are banned from the index entirely for a month or so, though we have been in the Yahoo! directory since 2000.
I can't even find our site in the Yahoo! directory search either, even if I use any relevant keyword combination...
I emailed them on this email and they replied the next day; now I'm waiting for another reply (they're checking my sites now)
Don't bother calling them, they're quite stupid on the phone, you'll be on hold for 20+ minutes being charged for the call the whole time, and then when you do talk to someone, they give you the wrong email address.
My site is Slurped regularly, and yet my site can not be found in Yahoo with a keyword search. I have been told by Yahoo they see no reason for it to be banned, and the only thing I can think of is I had an Inktomi PFI page. I have deactivated the Inktomi page, but no improvement. I have several times emailed Inktomi to remove my account from their database but they never give a direct reply.