homepage Welcome to WebmasterWorld Guest from 54.167.179.48
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

This 588 message thread spans 20 pages: < < 588 ( 1 2 3 4 5 [6] 7 8 9 10 11 12 13 14 ... 20 > >     
Does Google Ban or Filter Web Directories?
moftary

10+ Year Member



 
Msg#: 30575 posted 1:06 pm on Jul 28, 2005 (gmt 0)

I think the subject worth a thread itself. It's a suspision so far. Yet I don't see dmoz, yahoo nor any major web directory were banned/filter nor PRed zero as my web directory did. I tried to check it in Alexa (powered by google) and I see some results from my site. Appearently, Alexa brings old results from Google but something weird is that Alexa itself has PR0 now. But that's another story!

If you run a web directory, feel free to post your experience here.

 

JuniorOptimizer

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 30575 posted 10:39 am on Jul 30, 2005 (gmt 0)

Google's automated response to me said that sites fall out of the index all the time.

"Our spiders regularly crawl the web to rebuild our index, but keeping tabs on eight billion pages is tough work, and they may miss a few". They go on to say that if your site is well-linked it should be picked back up.

internetheaven

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 30575 posted 10:41 am on Jul 30, 2005 (gmt 0)

I hope this comment doesn't throw us offtrack from the current thread but in response to the many, many comments regarding how quickly the evaluation guys "flip the switch" on sites without checking thoroughly I would strongly disagree that such is the case.

I sent several emails regarding scraper sites that had stolen my content and after several templated responses about contacting the webmaster of the site (as if it were possible!) I eventually got threatened by them (Google) saying that I'd better have a water-tight case and demanded an extensive list of documents and signed waivers before they would even listen to me about removing the offending sites.

I don't know if I can paste the exact wording here but take it from me - they couldn't give a rats a** about scrapers or copyright infringement unless the offending site doesn't have Adsense on it.

runboy

10+ Year Member



 
Msg#: 30575 posted 10:50 am on Jul 30, 2005 (gmt 0)

"Google's automated response to me said that sites fall out of the index all the time.
"Our spiders regularly crawl the web to rebuild our index, but keeping tabs on eight billion pages is tough work, and they may miss a few". They go on to say that if your site is well-linked it should be picked back up.
"

I can believe this to be true with sites that have few incomming links, but I have never seen it happen to any of my sites that are heavily linked.

Dayo_UK

10+ Year Member



 
Msg#: 30575 posted 10:54 am on Jul 30, 2005 (gmt 0)

IH

Well to a degree you can see Googles stand point - yep there should be a water tight case for removing sites at request of us webmasters. It is up to Google whether sites go past the quality control guidelines and if they feel they dont then DMCA etc must be necessary for the removal of the site.

However, Google might have moved the goalpost for the quality control and a lot of sites can then suddenly get effected by a "flip of the switch"

However, I am still open minded about what is happening at the moment - obviously sites that have been totally removed from the serps (No results in site:example.com check) I would be very worried.

But there does seem to be some evidence of a index rebuild going on starting about mid July. So at the same time as these sites being totally removed we have the situation where Google might be revamping the index - whether the removal of these sites is a side effect of this or the scraper fall out I am not sure.

Dayo_UK

10+ Year Member



 
Msg#: 30575 posted 10:59 am on Jul 30, 2005 (gmt 0)

Runboy

Have you got total removal - eg nothing for site:www.example.com check?

Did you ask G if you had a penalty?

Also retracing back to your Adsense point.

What you need to remember about MediaPartners bot is that it visits the page after someone has visited it.

Quick example. If your site is indexed in G you should get relevant ads. If your site is not indexed then the first time someone visits - unrelevant ads - mediapartners gbot visits - the page will then show relevant ads for a period of time (weeks, months? - if no-one visits the page again then unrelevant ads will appear)

So you may find that pages which are not often visited may get unrelevant ads.

IMO - I am not a Adsense guru though.

lammert

WebmasterWorld Senior Member lammert us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 30575 posted 11:08 am on Jul 30, 2005 (gmt 0)

(...) (Google) saying that I'd better have a water-tight case and demanded an extensive list of documents and signed waivers before they would even listen to me about removing the offending sites.

Although not a real friendly response, I think Google's attitude was the right one. If it was so easy to just send an email to Google to get a site removed from the SERPs, then they would be bombed by emails from webmasters accusing other webmasters of copyright violations and asking for immediate removal of those sites from the index. You can always file a DMCA complaint, which Google has to act on, but that requires a document with your signature. That will block at least 95% of all bogus complaints.

My experience until now is, that the only reason for a removal of a site from the SERPs is a clear violation of the webmaster rules. Cloaking sites and those that violated Google copyrights/trademarks were the fastest to disappear.

Bad linking strategies is mentioned in the webmaster rules and maybe they turned a control knob a little higher resulting in sudden removal of several sites.

internetheaven

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 30575 posted 11:09 am on Jul 30, 2005 (gmt 0)

IH
Well to a degree you can see Googles stand point

I wasn't making a complaint, I was simply suggesting that those messages saying how easily they have been removed for "no good reason" may not have substantial interaction with the actual Google complaints process to make such a claim.

I have never, ever had a page/site removed from an index where the reason why wasn't clear to me - so obviously I find it very hard to believe that sites are removed despite having done "nothing wrong". I have lost rankings due to duplicate content issues, 301/302 issues but have never been banned from an index without a cause - even if that cause was accidental like the time I put a black background in <body bg=> but put the white writing code in a styles.css file.

Dayo_UK

10+ Year Member



 
Msg#: 30575 posted 11:16 am on Jul 30, 2005 (gmt 0)

>>I wasn't making a complaint

I know sorry - just trying to explain how a flip of the switch can lead to a load of sites being banned.

>>Bad linking strategies is mentioned in the webmaster rules and maybe they turned a control knob a little higher resulting in sudden removal of several sites.

Of the sites I have seen that is the only thing that I have not really looked at - the linking structure.

I have seen some people mention an auto page generator (which I have never heard of) - but surely certain link programs, page generators leave footprints which Google might find and then lead to tightning of any filter.

runboy

10+ Year Member



 
Msg#: 30575 posted 11:30 am on Jul 30, 2005 (gmt 0)

I have a couple of different sites that are very similar in the way they are set up. Just with different subject matter. The only difference between the ones that got hit (2 sites) and the ones that didn't is that the ones that got hit contained a DMOZ section (Which I have now removed just in case it should help any) and they had significantly more traffic and inbound links.

cicru

10+ Year Member



 
Msg#: 30575 posted 12:13 pm on Jul 30, 2005 (gmt 0)

runboy, your message is very similar to all the rest.

High Traffic + High Inbound = Higher chance to get banned

It is really confusing why the more traffic and the more inbound links the higher the chance of a directory style website to get banned.

This is the inverse of the properties of MOST scraper sites.

Did some one, inverted the
If then BAN statment in algo ? :)

moftary

10+ Year Member



 
Msg#: 30575 posted 12:22 pm on Jul 30, 2005 (gmt 0)

If (site contains DMOZ} then BAN;

But before this algorithm there is another one that exclude big names.

dataguy

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 30575 posted 12:25 pm on Jul 30, 2005 (gmt 0)

High Traffic + High Inbound = Higher chance to get banned

The site of mine that got banned had high inbound links, but it is one of the lowest traffic sites that I have.

Funny how nearly everyone that got hit comments that they have multiple similar sites..... I guess this blows the "satellite site" approach to pieces.

moftary

10+ Year Member



 
Msg#: 30575 posted 12:40 pm on Jul 30, 2005 (gmt 0)

My directory that got banned had the highest inbound links, but from top authority sites. The other small directory that got banned has a few inbound links. The other three small directories that has NOT got banned has a few inbound links as well.

I for one not sure it's about DMOZ as my three unbanned directories use DMOZ. The only semilarity between the two banned directory is the term "directory" in meta keywords tag, but then again it's proved that many other directories - not mine - have this and were not touched.

I wont get myself tired asking why, what or how as even if I knew the answers there is no chance to get anything fixed after googlebot stopped spidering/indexing the banned sites.

The only thing I can do is praying for a google rep reply my email enquiry. Which leads me to a question; where is GG?

Andem

5+ Year Member



 
Msg#: 30575 posted 12:50 pm on Jul 30, 2005 (gmt 0)

I would just like to clear the controversy of having multiple satellite sites and say that I run a sort of Yahoo! which covers many topics which have been written over the years. I do not allow duplicate content on other sites but of course, I've seen my content scraped before.

I just hope things work out, but a site: command still yields NO results. No reply from Google yet, but it's Saturday now so I expect any replies on Monday, if I get any.

moftary

10+ Year Member



 
Msg#: 30575 posted 1:09 pm on Jul 30, 2005 (gmt 0)

When you enquire about your adwords account, you will get a personal reply in hours. When you enquire about your adsense account, you get a personal reply in days. When you enquire about your website, you should pray to get a reply.

dvduval

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 30575 posted 2:24 pm on Jul 30, 2005 (gmt 0)

I've seen a number of directories that got banned are from free directory scripts that require links in the footer. I wonder if anyone else can verify.

JuniorOptimizer

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 30575 posted 2:58 pm on Jul 30, 2005 (gmt 0)

I wrote my own script for the directory and have no footers.

Dayo_UK

10+ Year Member



 
Msg#: 30575 posted 3:19 pm on Jul 30, 2005 (gmt 0)

>>I've seen a number of directories that got banned are from free directory scripts that require links in the footer. I wonder if anyone else can verify.

I have not seen this - but reports of this auto generated thing being on some sites which have been hit fits.

These types of programs like the directory software and certain link programs leave footprints - if the software was being used primarly to build pseudo directories then sites which have these footprints of the software use may get hit. Even if you are a genuine directory.

Although as I posted earlier even genuine directory are limited in the level of unique content they can achieve - when people submit to directories they dont change the title and description of their site that much.

[edited by: Dayo_UK at 3:22 pm (utc) on July 30, 2005]

europeforvisitors



 
Msg#: 30575 posted 3:20 pm on Jul 30, 2005 (gmt 0)

When you enquire about your adwords account, you will get a personal reply in hours. When you enquire about your adsense account, you get a personal reply in days. When you enquire about your website, you should pray to get a reply.

You're comparing apples and oranges and apricots. Even if the AdWords, AdSense, and Google Search teams weren't different departments, wouldn't you expect paying customers to get priority treatment?

shiva777

10+ Year Member



 
Msg#: 30575 posted 3:36 pm on Jul 30, 2005 (gmt 0)

I run two directory sites and they are still listed in Google (whew!). They were made from a customized commercial script.

~shiva

Andem

5+ Year Member



 
Msg#: 30575 posted 3:39 pm on Jul 30, 2005 (gmt 0)

I like your point EFV, it makes perfect sense... in a perfect world.

But onto the real issue at hand: Has anybody managed to get any responses from Google or any status updates on your site's situation?

Dayo_UK

10+ Year Member



 
Msg#: 30575 posted 3:42 pm on Jul 30, 2005 (gmt 0)

>>>I like your point EFV, it makes perfect sense... in a perfect world.

Er - it just makes perfect sense in this not so perfect world too.

Although Google may prioritise the reports very highly - they may not priorotise the reply as highly. After all they are interested in making sure they have the best serps.

Andem

5+ Year Member



 
Msg#: 30575 posted 3:46 pm on Jul 30, 2005 (gmt 0)

They are also very interested in Adsense/Adwords revenue... much more so than search results I would expect. Makes me wonder what their shareholders might see in their next earnings report if this dropped site thing is as big as I think it is.

Andem

5+ Year Member



 
Msg#: 30575 posted 3:52 pm on Jul 30, 2005 (gmt 0)

I forgot to mention the symptoms I am currently feeling on my site:

- The page is _still_ cached!
- The toolbar PR is greyed.
- Site: command yieleds 0 results.
- Googlebot visits are here and there, but nothing special. I can't tell if its the news bot or the web bot, they look the same. It has looked at a few non-news-article pages.

Dayo_UK

10+ Year Member



 
Msg#: 30575 posted 3:59 pm on Jul 30, 2005 (gmt 0)

>>>The page is _still_ cached!

That strange - just checking the sites that I know of and they do still have a cache and not just the homepage. Is this normal for a ban (Looks like recently crawled cache too)

Could be just cache database out of line.

Andem

5+ Year Member



 
Msg#: 30575 posted 4:11 pm on Jul 30, 2005 (gmt 0)

>>>That strange - just checking the sites that I know of and they do still have a cache and not just the homepage. Is this normal for a ban (Looks like recently crawled cache too)

Well, another weird thing about caches is this: I disallowed a certain page from being indexed via robots.txt last year and the page is still cached of the old version, March 2004. Strange.

I'm anxious to see what comes of this. My living is on the line.

europeforvisitors



 
Msg#: 30575 posted 4:13 pm on Jul 30, 2005 (gmt 0)

They are also very interested in Adsense/Adwords revenue... much more so than search results I would expect. Makes me wonder what their shareholders might see in their next earnings report if this dropped site thing is as big as I think it is.

Why would dropped directory or scraper sites have a negative effect on AdSense/AdWords revenues?

If mysite.com drops in the SERPs, yoursite.com or anothersite.com will move up to take its place. And if Google is correct in its assumption that yoursite.com or anothersite.com is more relevant to the user than mysite.com was, then its AdSense clicks will probably convert better and will yield more revenue for Google because of smaller "smart pricing" discounts to advertisers.

Now, it's possible that some of the dropped sites that have been discussed here are high-quality sites that don't have duplicate content, don't use automated page-generation or link-submission scripts, and will be missed by users if they aren't in the SERPs. If that's the case, and if the problem is widespread enough to be noticed by users, then presumably Google will identify the collateral damage and try to fix it (just as it's done with other types of sites in other updates). But the process may take time: Google doesn't move at breakneck speed, and a number of us watched our sites or individual pages go missing for months.

Dayo_UK

10+ Year Member



 
Msg#: 30575 posted 4:18 pm on Jul 30, 2005 (gmt 0)

>>>>I disallowed a certain page from being indexed via robots.txt last year and the page is still cached of the old version, March 2004. Strange.

That is not so strange - happens all the time.

Andem

5+ Year Member



 
Msg#: 30575 posted 4:19 pm on Jul 30, 2005 (gmt 0)

EFV,

I understand what you are saying and agree that removing scraper sites is in the best internet of the web. But I have been in contact with a couple of webmasters and have seen some very high quality sites that have been dropped, mine included.

pocoloco

5+ Year Member



 
Msg#: 30575 posted 4:39 pm on Jul 30, 2005 (gmt 0)

"For the rest of you that had directories hit, do you still rank well on these ips 66.102.9.104/99"
I bet those 2 data centers will efect all the other data centers by next week ,that's why Google keeps them one week now without any change.

Dayo_UK

10+ Year Member



 
Msg#: 30575 posted 4:40 pm on Jul 30, 2005 (gmt 0)

I thought they were just the old data on those dcs.

I am much more intrigued that the sites which no longer appear anywhere in the serps still have upto date cache. Anyone familiar with the banning process know if this is normal.

This 588 message thread spans 20 pages: < < 588 ( 1 2 3 4 5 [6] 7 8 9 10 11 12 13 14 ... 20 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved