|Does Google Ban or Filter Web Directories?|
I think the subject worth a thread itself. It's a suspision so far. Yet I don't see dmoz, yahoo nor any major web directory were banned/filter nor PRed zero as my web directory did. I tried to check it in Alexa (powered by google) and I see some results from my site. Appearently, Alexa brings old results from Google but something weird is that Alexa itself has PR0 now. But that's another story!
If you run a web directory, feel free to post your experience here.
28th and not doing good in any of those.
How many kept their ODP portion and got reincluded?
hit from 28th and not on any of those servers listed -
Hit the 22nd and not doing well in those dcs, but it is good to know that someone hit on the 22nd is making a comeback.
Hit on 22th, and nothing changed from that day. Still on 5th page for keywords I used to appear in first page.
Lost 90% of google traffic.
My website is a original directory.
No ODP but Amazon web services (excluded in robots.txt to spiders).
Running Adsense with 3 blocks on all pages
>>Have those pages AdSense blocks on?
No Adsense until about an hour ago. I decided to put them on to see if it'll give a shove, and I'll be taking them off today. Google is just refusing to even look at those pages or crawl them. I don't want Adsense or anything "commercial" on the site, just what I put on myself. It's a bonafide information site.
Hit on 22th, and nothing changed from that day. Still on 5th page for keywords I used to appear in first page.
Lost 90% of google traffic.
Sounds just like what happened to one of my older sites on July 16th. All previous #1's are now languishing on page 3-4.
having said that, I've seen this happen a few times over the last 5 years, and it's always come back within 6-8 weeks or so, so no worries ;-)
hit on march and didnt come back. lost 90% of G traffic.
I as hit on the 22cd, show up on pages 4 or 5 of the results from these dc's that I previously held top 3 positions for.
Things that I have noticed are that the PR is still intact, and google bot is still visiting and caching the site. A site command search and mydomain.com search all return normal results, as does the link command.
However, the main problem that I see (apart from the fall in SERPS) is that when I search for mydomainname (without the dot com) I am listed on page 4 results. Normally I would be listed at position 1. This has happened to me before, and agreeing with Webfuson, the site did return after 2 months. There are actually two other large sites that compete for the same keywords as I do with the same problem.
Hopefully this will turn out to be the same.
"having said that, I've seen this happen a few times over the last 5 years, and it's always come back within 6-8 weeks or so, so no worries ;-) "
Hope you are right. Hit also on the 16th. Pagerank the same, keywords from page 1 to page 4. Now removed adsense and had non-www redirect 301 to www. Crawling is as usual daily. Few days ago googlebot crawled all 1888 pages in my sitemap in one go. Never saw such a neat crawling.
>>Hit the 22nd and not doing well in those dcs, but it is good to know that someone hit on the 22nd is making a comeback.<<
I really donīt know whether Its going to be the real comeback for my site, because I donīt know how important those DCs are at present. My Google referrals are still less than 10%. So I guess I need just to wait and see.
Guys, have you read the topic of this discussion before posting?
This about directories that were banned, lost their PR and totally deindexed from google serps and has nothing to do with the 22th of july massacre nor sites that lost their high ranking for certain keywords!
>>Guys, have you read the topic of this discussion before posting?<<
Cīmon be large and show some hospitality ...
We who have been hit on 22nd and 28th July are just harmless refugees have nowhere to post and dicuss our concerns, so we are seeking refuge in your thread ;-)
reseller, you just find a thread that is on the topic, because it turns all into confusion when threads go off on tangents.
I feel for the folks that have gone out of their way to build a useful directory and have gotten hit, Google should really fire a warning shot first, but I do see Google's point generally. Most directories are there for little else than to pitch ads, manipulate SERPS through link exchanges, or even sell links to manipulate PR while providing little in the way of useful content. Most directories are exclusive and based on some form of quid pro quo rather than really trying to go out there and become an authority in the scholastic sense of the term, that invloves a lot of research not to mention listing sites without expecting anything in return.
In many areas, directories became little more than intrusive middlemen with little value added. I'm sure from a revenue standpoint Google likely stood to make more money if users had to pass through adwords-filled directories to find the sites they were looking for, but I'm glad to see Google has taken steps to cut out the middlemen.
I think the first directory that should have been hit is Yahoo's... but I guess Google didn't want to start a war.
Anyway, my 2 cents... and before anyone gets upset, no I wasn't refering to your directory, really.
|so we are seeking refuge in your thread |
reseller, I am a refugee muself :)
|In many areas, directories became little more than intrusive middlemen with little value added. |
Correction: with NO value added, yes when I search for something in a certain engine I don't like to find any directory results nor any other searching engine results. But believe it or not, not all people are just like you and me. I have a tracker installed on my web directory, and it shows that visitors either continue their navigation from the page they landed on from google, or just find a website link (resource) for the information/service/product they are seeking, or finally they just click a relevant adsense ad.
So, believe it or not, web directories are useful for most of the users. Besides.. when you search on google for something you get a list of results with *snippits*, are you forced to click on the listing? or are you too blind to note that the listing is irrelevent to your search?
That said, I dont care nor mind if google removed all unique directories from its serps. I dont care nor mind if it removed all ODP clones or all directories that are basically seeded with ODP data. What I care, and what bugs me the most, is that so far major ODP clones (opera, alexa, etc..) as well as minor ODP clones still exist in google serps.
Sorry for such long post.
Sorry it took so long to post this... My site was banned, and then reincluded in the google index.
I was likely one of the first ones to see that my site was totally banned from google - not just a penalty and pushed way down in the serps, but totally banned...
So, being one of the first to notice the ban, I was likely one of the first to write to google support. I contacted them through the web form on July 28 - not through the email address. Within a couple of hours I received the canned response of "you may be a dummy and your site just might be way down in the SERPS." I responded to the email, as instructed, and politely asked for an explanation as to why the site was penalized. On August 2, I received a response from google. "We understand your concern and have passed your message on to our engineering team for further investigation." On August 6 my PR was back to full strength - traffic started flowing from google once again.
There are still issues, we do not seem to be totally restored to our previous SERP positions (yet), and we're showing only about 1/3 the previous amount of indexed pages... This, in my opinion, is the proper response from google.
In theory, any particular site (directories included) may have some dup content, might have some re-print articles, probably has a recip directory with links that appear on 1,000 other recip directories. DO NOT BAN THE SITE! Just penalize the site, and let the site owner understand that something else needs to be done. In my opinion, a banning is deserved only by those that game the system, that play dirty. People that put up good sites should be rewarded for it, people that put up bad sites should be penalized, and people that try to cheat should not be tolerated. This group seems to be made mostly (a few good and bad apples aside) of the second group - people that should be penalized - NOT BANNED.
Just my 2 cents - and it's worth what you paid for it...
I have noticed a significant improvements now that many useless directories have been dropped in favor of sites containing original content. I think Google is much better for it.
When you search for something, you want to find sites listed that actually have usefull content directly on topic - not just directories listing links to other sites that supposedly have content, or worse, more directories. While some directories are good, it seems that most are total auto-generated spam.
For example, an actual "lodging" website (like a motel, hotel, inn, BnB, etc) should come up at the top of a search, especially when the term is combined with a location ("lodging location"). Google was terrible at this after they emphasized link popularity. All you got was link directory after link directory. Now it seems you get a few real sites listed.
"Googlebot is back on my site. After sending Google an email informing them that they're idiots and they dropped my site mistakenly, I received the "forwarded to engineers" email.
Last night I see the bot was back in pretty much full action. Looks like Google is trying to clean up their mistakes after all."
- Yes well Googlebot was round my site in the early hours of this morning, and my rankings have now got even worse since the initial penalty was imposed!
My site has taken me 6 years to build up. It's largely a portal for the area where I live with pages of info about places to go, things to see, plus links to local sites and ads alongside.
Is this such a crime?
I'm still well ranked on Yahoo and MSN but income is down to about 20-25% of what it used to be.
I've had plenty of emails appreciating my concern and thanking me for my patience whilst the engineers investigate. I thought that this was a good sign when I got the first one, but I now realise that everyone gets them and I wonder if anyone there really cares.
Why don't they publish the rules of this "game"?
I hear what you're saying. The bot is back, but the site is not. So apparently the "engineers" aren't doing all that much to get my site back in either.
I agree that there are lots of directory sites on the net - but there are lots of everything.
Good directories have a place IMO. I often use them as they often list sites that don't rank in the SE's due to no links or no optimisation and can often find interesting sites and products that I might not normally.
Also, they are a starting place for lots of sites - and cheaper than adsense. I get a some good referals to one of my sites from a directory.
And, if you want to talk about a middleman result, what about adsense? Don't they turn every page that they are on into a mini directory?
I agree however, that some directories are rubbish and hope that google is clever enough (for once) to not throw out the baby with the bathwater.
In this case, I am stilling MIA from Google results. The bot seems to be looking at hundreds of pages every now and then, but we are still not appearing in the Google set of results. Things don't look good, and I've been in contact with Google continuously.
Still looking down, stresssssssssssssful.
We are still not back in the index either.
What is the best way to remove a section of your site that you think is the problem? We decided on removing our DMOZ section, but I am worried that we may have made a mistake by setting a redirect (response.redirect in ASP) up in the script pages that redirect to our main page. This was done in best intention in order not to create dead links if somebody was linking to the section (Highly doubtable).
Would a 301 redirect or just a 404 page not found have been better.
The thing that worries me is that I have a pretty good indication that our site was reviewed yesterday and specifically the pages in question were looked at, but our site was not re included in the index.
If you have removed the pages - use a 404. I wouldn't want to redirect an area of a site that may have caused a problem or been one of the reasons for them dropping the site. If you are keeping the pages, I would block it via robots.txt and may even go as far as noindex, nofollow meta tag on each page. I would then email Google and explain the steps you took. I would not ask for reinclusion unless you are willing to take some corrective steps and letting them know exactly what you have done so they can confirm it.
You are right, that is probably the safest approach. Hope they will give me another chance
Got a reply from G-support--
"Thank you for your note. We understand that you're concerned about your site, www.------.com. Unfortunately, we are unable to send personal responses to all of the requests we receive to review individual website content.
Certain actions such as buying or selling links to increase a site's PageRank value or cloaking - writing text in such a way that it can be seen by search engines but not by users - can result in penalization. Please review our quality guidelines at http:// www. google .com/webmasters/ guidelines.html for more information. If you identify problems with your website and make the changes necessary to comply with these guidelines, please do not hesitate to contact us.
We are sorry that we cannot provide individual assistance at this time.
The Google Team"
My site was not a directory. It *was* the recipient of a site wide link on another banned site. Perhaps the filter penalized both the "sellers" of links and the "buyers" of same. In my case, these were two sites I owned each with unrelated, non-duplicate content, so it was natural to link from one site to the other. No links were bought or sold (sic)
Human driven directory sites offers visitors links to sites
- that are not optimized for search engines
- that have low PR because new or not well linked, but with a great value
- that are penalized by algorithms used by bots, that are NOT perfect as they should be
Btw, today my directory has been spidered by googlebot heavily, but no changes yet on G referrals.
When you say spidered heavilily is it just as much as before the ban? Our site gets around 500 Googlebot hits daily after the ban, but it never goes much deeper than the point of entry. Not at all like before the ban.
Yes, some directories are usefull, both to searchers and new sites. Human reviewed (aka DMOZ, of which I am an editor), and paid directory listing directories can generate a lot of visitors. But wouldn't it be nice to get to the original content website in one click off a Google search, instead of two?
The worse directories are those that get indexed high in the serps and are generated link lists or compilations of adwords/overture ads. That's spam to me.
Not that I am against the webmasters who make money at this. If the engines have a weakness, then sure, go after it.
My website has not been deleted, but penalized. (G referrals are reduced to 5%)
I mean that during last days of july googlebot did not spidered my site, while this morning hit almost double of the last days.
Don't know if this is normal or not, is just an additional info.
|Not that I am against the webmasters who make money at this. If the engines have a weakness, then sure, go after it. |
But please don't cry foul if the weak spot gets patched. :-)
I have tried opening the following related subject on another thread, but appearently WW moderators did not approve it.
If you have pages that does not comply with google webmaster guidelines (duplication content, etc..) and it happened that these pages are very usefull for your users, would banning googlebot via robots.txt from crawling/indexing these pages be enough action before asking for an inclusion request?
An example, assume that your site is banned for having an ODP clone, would excluding googlebot from crawling/indexing the ODP section in your site helps removing the ban?