Successful Yahoo Reinclusion Request?

Seeing Heavy Crawling

[WebFusion]

Hello all,

We recently purchased a large, well established site (about 20,000 pages of original content) that (for various reasons - no balckhat, just spammy stuff like too many affiliate links, etc.) had been banned from the Yahoo Search Index about 3 years ago (according to the former owners).

The site is listed in the yahoo directory, but - despite having 10,000+ inbound links, and ranking very well every where else, could not be found anywhere in Yahoo when searching for it. The front page and only the front page was crawled 1-2 times a day by Yahoo/slurp, but no other pages were crawled.

After refreshing the site and updating it quite a bit (not to mention removing virtually all affiliate links), to include adding RSS syndication to the original content (which got picked up by several RSS hubs), we submitted a verbose reinclusion request, outlining all we thought the site did wrong, and everything we've done to improve it.

Today, we're getting absolutely slammed by the following crawler:

YahooSeeker/CafeKelsa (compatible; Konqueror/3.2; FreeBSD ; [help.yahoo.com...] (KHTML, like Gecko)

My question is this: Is that crawler hitting us due to the new RSS feed we are providing, or are we about to be reinstated into the search index?

[twebdonny]

Once you're banned, you're banned...

Sorry

[WebFusion]

Once you're banned, you're banned...

Sorry

well, I know for a fact that that isn't true, and I've read many a poster here who've been reinstated to Yahoo.

[soapystar]

for the vast majority of TRULY banned sites twebdonny is totally correct.

[martinibuster]

...for the vast majority of TRULY banned sites twebdonny is totally correct.

You're probably right soapy, but that's because the vast majority deserve to be banned, so there's no news there.

But twebdonny's mistaken to assume a ban is permanent in all cases. I know of someone who had their site unbanned last week, three weeks after a reinclusion request, so it does occur if your site has decent content.

[edited by: martinibuster at 8:42 pm (utc) on Aug. 19, 2005]

[Webdetective]

What if a site is still being spidered by Slurp? Does that mean the site is probably not banned?

Where do we submit reinclusion requests to Yahoo?

[martinibuster]

Afaik, if you do a site:domain search on your domain name and it only shows one page, you're banned.

[WebFusion]

I will say this....

In the 90 days prior to submitting the reinclusion request yesterday, not a single page outside of the front (index.html) page got hit by a Yahoo crawler (aside from robots.txt of coruse). Now, about 24 hours after submitting the request, the crawler listed above has hit more than 10,000 pages in the last 8 hours or so.

Seems a bit strange to be just a coincidence (?)

At any rate, the site is doing just fine on all the other engines (google, MSN, etc.), I just wanted to ge that 3rd slice of the pie too ;-)

I'll let you know if we reappear...

[soapystar]

MB
its all about opinions but while you can say in your view they deserve to be banned on the basis of the content or lack of it...i dont think this is the basis of the bans..it becomes the basis of the review however...

i certainly have seen sites with good content that have stayed banned...i know this topic has been done to death and dont want to start yet another bout of ban bashing.(hell im even bored of it myself and i have helped to start many of those threads :))..but i just feel other factors have more influence than the actual content...i simply dont agree with your view that most banned sites deserve to be so....

then again often we are talking at cross purposes as to what a banned site is...

[walkman]

>> Now, about 24 hours after submitting the request, the crawler listed above has hit more than 10,000 pages in the last 8 hours or so.

if I had to bet, I'd bet that is a coincidence--unless they get very few emails. Also, I'm not sure if YahooSeeker feeds the serps.

I have a banned site and until recently had Y! blocked totally via rewrite (403 error). Once I removed that Slurp is asking for about 50+ pages a day. Who knows...maybe they need pages for bragging rights, and they'll index everything, but will bury them in the bottom.

[hawk]

One of our sites that was in top 10 for almost 7 keywords with a competition of nearly 39,300,000 pages is gone from Yahoo. It is not even listed for any keyword. The site seems to have been removed after this update. We never did anything wrong.

Results 12,504 for www.site.com but no index page.

How is that possible and how to request yahoo for it's re-in-inclusion?

[franckey79]

I hadn't been using any spam techniques and I was banned. I wasn't been included into Yahoo's Directory before ban and I'm not now. Do you think if it may help in reindexing my site -- paid inclusion to directory?

[Iguana]

I was certainly penalised for about 2 years in Inktomi when Yahoo used them. They had about 64 of my 3,000+ pages and they would come last even on exact string searches for my page titles.

Then after 2 or 3 months of Yahoo having their own listings (based on Inktomi) I suddenly re-appeared with a vengence. I now have 3,600 pages indexed and they are highly ranked in their field. Funnily enough, they re-appeared about 10 days after I banned Slurp in my robots.txt - but I'm sure that wasn't the reason why.

I assume this was all based on manual review.

[Iguana]

Sorry, WebFusion, I've just re-read your post and thought I'd better give you some real info rather than my reminiscences.

I have a few thousand rss feeds and they are being hit heavily by:
YahooSeeker-Testing/v3.9 (compatible; Mozilla 4.0; MSIE 5.5; [search.yahoo.com...]
YahooFeedSeeker/1.0 (compatible; Mozilla 4.0; MSIE 5.5; [my.yahoo.com...]

I've not seen the particular crawler you mention but I'm pretty sure that rss helps with getting pages into Yahoo. However, that's for a site that is unpenalised. So, allow yourself a little hope but don't order the Mercedes until you see Slurp gobbling up everything in sight (or should I say everything in site).

[Webdetective]

I hope this is on-topic enough for this thread.

If I removed a large number of illegal doorway pages and throughly cleaned up my site, what if Yahoo still has links to those pages in their database? Does Slurp eventually remove the non-existant pages automatically, or must I go to Yahoo and manually deindex them myself? Can a site be penalized for past sins that were removed long ago?

[warthog]

I search link:www.mysite.com in yahoo - 98,000 results
(and every one of those links is on a very specific niche industry category) a mere fraction of yahoo's 800,000

site:mysite.com - 1 result out of several thousand pages with unique content.

My site has been around for several years and was redesigned last year to a directory based on specific business's listed by city and state. My site represents over 5000 individual business entities.

I decided last month to buy a directory listing in the yahoo directory. They take my $300 and add me to the directory - 1 week later they ban my site and remove thousands of pages. And now I am at the bottom of the directory page as least popular. (93,000 unique visitors in the last 30 days from google)

After several emails I get the canned response - your site "may" not comply and the link to the list

- Cloaking
- Massive domain interlinking
- Use of affiliate programs without...
- Use of reciprocal link programs (aka "link farms")
- Hidden text
- Excessive keyword repetition

I do have a 7 page affiliate program, among several thousand pages of original unique content so I do not see how the hell that could be an issue.

It has to be this
- Massive domain interlinking

I do have several sub domains for geographical areas -but the content changes depending on where you are obviously - its meant to show people business listings in their city.

Im not spending another dime on overture - <snip> They use the exact same methodolgy in their web structure - they are complete hypocrites.

[edited by: martinibuster at 6:23 am (utc) on Sep. 12, 2005]
[edit reason] No insults please. Thanks. [/edit]

[soapystar]

yes....think algo boost before thinking content......despite what the guidelines may have you believe.....

[BillyS]

After several emails I get the canned response - your site "may" not comply and the link to the list

- Cloaking
- Massive domain interlinking
- Use of affiliate programs without...
- Use of reciprocal link programs (aka "link farms")
- Hidden text
- Excessive keyword repetition

I do have a 7 page affiliate program, among several thousand pages of original unique content so I do not see how the hell that could be an issue.

I email a reinclusion request. Got the above message back. I double and triple checked everything and wrote a nice note saying it could have been a problem with a 302 pointing to my home page (when a page was not found). Now I have a 404 (not found) or 410 (gone). I thought this might have been causing a duplicate content penalty. I apologized and asked to be reincluded (a second time). I got the same response back to me.

[Webdetective]

I would like to know how they define "Use of affiliate programs." Affiliate programs have nothing to do with spam.

[soapystar]

dont get confused between what will get you banned and what will get you through a re-review.....guidelines are for the review...you only have to take a look at the index to see that....

[Webdetective]

I do own an affiliate site that appears to be banned, but I am tending to put the blame on my use of a doorway page generater I was using. I got rid of all the pages and quit using the program, but it still has a long ways to go before it's ready for a reinclusion request.

[warthog]

i dont follow soapy

"dont get confused between what will get you banned and what will get you through a re-review.....guidelines are for the review...you only have to take a look at the index to see that.... "

[iblaine]

I got banned from yahoo, did paid inclusion and got reincluded. Luckily the site had only minor blackhat characteristics. Had that URL done bad things I'm sure it would still be banned. Funny story actually. An experiment got out of hand and resulted in 50 million pages being indexed, big bandwith bills and banned class Cs. I tried paid inclusion for many sites and only the whitehat ones were reincluded.

[Voxman]

We worked on a site for 10 months. It has about 50,000 pages and Yahoo had indexed 1,200 of them before the last update. Well during that update we are down to one on site:mydomain so I guess we are banned. We had one link back to another site that sold hard goods / books of the digital we sold on the site I'm talking about here. Totally different products but someone could look at them and say they were both widgets. Anyway 2 1/2 months now so I'm pretty sure we are banned. I'm thinking a competitor complained and an inxeperience editor looked at both sites and said ah-ha without having a clue.

This is a year's worth of work down the freaking tubes here. Google has over 40,000 pages listed so no problem there. We even added a "no follow' tag to the link back but nothing. Just like everyone says ....you get a canned answer that doesn't even address the problem. They don't even read your emails I swear.

Yet they have no problem taking thousands of dollars a month in PPC from us.

What to do? My question is ...the first site is already included in the 'shopping/widget' directory so paying for inclusion may only lead to another so called editor getting it wrong again...

Is there any email you can write too that gets a real response? We've done everything we can to prove that this is a needed arrangement between only two sites and we are not spamming the engines with it. Does slurp even pay attention to "no follow" tags on URL's?

[soapystar]

people do get more than a canned response but i can find no logic in the 4 types of responses being given, they are:

1:no response

2:canned response

3:dig response of naughty schoolboy tease type

4:serious reponse with helpful commnents

i dont follow soapy

bans seem to be handed for a variety of reasons none of which are direct quality/content reasons as far as i can tell. Bans seem to be more to do with techniques you may employ to boost your rankings outside of organic/natural rankings. However when your site is reviewed it will then be looked at in terms of the quality/content elements of the guidelines even though they were not the reasons your site was banned. Your site could still remain banned after review even though it may be more in line with guidelines than a huge number of indexed and ranked sites, its simply that your now under the microscope and their not.

[Patkaa28]

I have just been reincluded in Yahoo again i sent an email and they reindexed me overnight but my question is since its been banned once does that mean your site will be tainted and won;t be able to get the old rankings it used to get?

[Webdetective]

Patkaa28,
It's possible search engines might be using a "points" system similar to a person's driving record. They might accumulate X-number of points for each violation, which are eventually exponged after a specific amount of time has expired.

[soapystar]

Patkaa28

I seriously doubt you were banned. Even by their own admission a review will take weeks at best from the first email.

i also doubt a driving licence system. There is no evidence at all for that. I mean you wouldnt even need a review process if that was the case let alone all the other anomolies that would be associated with it. There is no evidence wotsoever for a expiring time factor here.

[Angelis]

I find it really odd that in the 5 years I have been doing SEO I have never had a single site banned from any of the search engines and some of the practices I used to use where... well.. not exactly above board.

Can say I have ever looked at Yahoo directly for traffic although of late I have seen almost a 400% increase in traffic coming from Yahoo.

This thread seems to of opened these tired eyes a little more.

[warthog]

So let me ask this - because I am not clear on this term "Massive domain interlinking"

does this refer to having hundreds of links to your own pages on your internal pages? I have thousands of pages in my site but its a city based directory - each page may have as manyh as 75-100 links to other cities in the same state - is this what i am being punished for?

Could you get this from having links on other sites - I have some links that are run of site and appear on thousands of pages in another directory - all of them link back to my site - this isnt interlinking is it?

#*$! is interlinking anyway - how is it different from linking.