Forum Moderators: Robert Charlton & goodroi
My site - original content first registered in 1998 semi informational - travel but not very competitive keywords mostly towns and villages that no one has ever heard of.
Results now shown - autogenerated snippets from my site.
What is happening?
I've excluded them via robots.txt and with noindex,nofollow.
However these files are indexed too and may cause a dupe content penalty. When looking for site:www.widget.com printxyz.php I do see all these print versions as url only links.
In these days I see also Googlebot crawling these mail and print versions of my articles. What's going on there. Is my robots.txt defective? Did I do something wrong with noindex,nofollow?
[Added]
One more thing:
When invoking Google's URL removal console I saw something remarkeable: After submitting my robots.txt to the console I saw only "removing image xyz.php".
The possible cause for that: The URL console does not seem to interpret a robots.txt with
[code]User-agent: *[/url]
properly.
I simply duplicated the exclude list and put a
[code]User-agent: Googlebot[/url]
in front of that.
After resubmitting the robots.txt to the URL console Google shows a "removing file" in the status list.
My theory: Maybe you'll have to add a
[code]User-agent: Googlebot[/url]
explicitely in your robots.txt.
Anyone with duplicate print and mail versions: What's in your robots.txt?
[/Added]
What really sucks is how a cloud is now cast on my domain. Even adding new material won't help.
There's about 50 .ru scrapper sites, that have their scrapped results, and then at the bottom of each page the entire text content of my home page! - I don't think emails will do me much good in this circumstance - I think I'll wait and see how this works out.
You may wish to submit a spam report to Google at once:
[google.com...]
I know...I know...
Many fellow members post that they have reported spam to Google with no results at all.
However, from reading GoogleGuyīs and Mattīs recent posts I sense that "Google Search Quality Team" is paying more attention to spam reports.
You have everything to win and nothing to lose.
I hope this helps.
There is at least one site running an IP delivery script using a DMOZ dump as a data source.
If your site is in DMOZ then you are in danger of having duplicated data in Google's index. This duplication is not excerpts but entire pages.
I have found 3 sites that duplicated our home page using such scripts, there may be many more such sites out there.
This situation has been reported to Google through multiple channels.
Like a lot of folks here adding &filter=0 returns our pages in the SERPS.
This is truly preposterous. I've already found out all the contact info on one of these people, and he as a dedicated server at the same place I do. At least it's my fellow small-time webmaster giving it to me this time.
How stupid. Not only are we competing with our own content but to add insult to injury we are penalized for it! and then not even found when people search for it.
Is there nothing we can do?
I tried to call the number listed in their WHOIS to give them a friendly warning, but of course, it didn't work.
I've still got my fingers crossed that not all the data for this update is folded in, and it'll work out OK.
I've tried contacting YPN directly but can't find a direct contact form or email address. As it still is in Beta, they haven't thought to put up a contact form for those wanting to report TOS violators.
Either way, Google Engineers are too incompetent and too high from smoking $100 bills to develop an algo that can tell the simple difference between those that copy websites and the original.
I'm sure Cutts and the gang will tell you, "no, our duplicate content algo is fine, nothing wrong with it. We easily can tell the difference."
<sarcasm> I'm contemplating copying his blog and see if I can't prove my point that way. One of my 10,000 doorway pages will become #1 for "Matt Cutts Blog" and I'll laugh when his gets slapped with an auto penality.</sarcasm>
What about subdomains? If mysite.com gets filtered will it affect old or new subdomains?
Friends!
It could be the duplicate filter which is causing your problems, but also it could be something else. Have you ever thought about it.
May I suggest the followings:
The four of you exchange your urlīs (by stickies) and take a good look deep in your sites. Try to see whether the four sites of yours have something in common which might trigger a red flag. You donīt need to post your findings on this thread if you donīt wish to do so. We shall understand and respect that.
I thought this duplicate content thing was solved already. My main site was hijacked earlier this year, and now it looks like it is all happening again.
I think I am ready to be done with Google. For me, this means replacing all my AdSense ads with YPN ads or MSN ads when they get a contextual ad setup running, and paying more attention to optimizing for Yahoo and MSN. I think AdSense is the choice of spammers and sitejackers anyway, and Google doesn't seem to mind. In fact, I think that is the crowd they prefer - possibly because they think that is who "produces" for them.
And yes, if you haven't picked up on it yet, I'm extremely frustrated with Google right now!
I have had my site online since 2001, it was all done by hand html coding with wordpad, and I worked and tweak it constantly, and have done so almost every day since 2001 to present.
Amyway, I also ranked #1 for many of the search terms related to my site, for a long time, and google was my #1 source of traffic. Around the end of May of this year (2005) my site disapeared from google, I searched and found a few of my search phrases back around page 50 or worse lol.
Will here it is Sep 24 2005, and I am still the same, yesterday my search engine traffic was:
83% MSN
10% Yahoo
5% ask jeeves
2% Google
When will the site come back to google?
Who knows what google did this for to a site all designed and worked on for years by hand?
&filter=0 brings my site up to where it was before the "update" also.
&filter=0 works like a charm. My site is completely back when applying this parameter.
can confirm "&filter=0" does bring back my site too
If you like, you can repeat the search with the omitted results included.
Am I wrong?
Select some short snippets of text from your best pages, just a few words each.
Google for those, and see what shows up. Hopefully your pages are in there somewhere.
I'm not talking about keywords here, but unique phrases from the body of your text.
See if somebody else has copied (scraped) your content.
It is entirely possible that other sites were given credit for your work.
If OTOH you have been borrowing text from other sites, and only you can judge that,
then you may have gotten 'found out'.
Whenever a site suddenly drops in the SERPS, goes 'supplemental', or vanishes entirely,
the first thing I think of is duplicate content. -Larry
