Forum Moderators: open

Message Too Old, No Replies

Incorrect URLs and Mirror URLs

Causing duplication penalties.

         

crobb305

12:39 am on Nov 25, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Google has indexed numerous incorrect URLs and mirror URLs all pointing to my index page. Subsequently, the original URL (www.mydomain.com) has been suppressed to the bottom of the results for any search (presumably a duplication penalty). This problem was also mentioned in message 11 of the following thread:

[webmasterworld.com...]

The URLs pertaining to my website that all point to my index page take the following form.

www.mydomain.com/?S=AC3%26Document=document
www.mydomain.com/?S=AC3%26Document=document
www.mydomain.com/?SID=xRSUNVW8R9P44HSYQ6UWED&
www.mydomain.com/?S=AC3%26Document=document
www.mydomain.com/default.asp?S=AC3&am
www.some-other-URL.com/go.php?id=aHR0cDovL3d3dy5jcmVkaXRjaGFtcGlvbi5jb20v
www.some-other-URL-2.com/go.php?id=aHR0cDovL3d3dy5jcmVkaXRjaGFtcGlvbi5jb20v
www.some-other-URL-3.com/file/callink.php?linkid=3

I have emailed google, but have received no reply. I am unsure what I can do to A) eliminate the incorrect URL's that appear to originate from my site and B) eliminate the mirror URLs that originate from unrelated websites.

Any help would be greatly appreciated.

Shurik

4:27 am on Jan 9, 2005 (gmt 0)

10+ Year Member



Below are the steps I went thru last night to try to remove a 302 link to my site from the index.

1) configured my web server to return 404 on the request for default page
2) submitted hijacking URL that redirects to my domain to services.google.com:8882/urlconsole/controller
3) selected “anything associated with this URL” option
4) google’s “URL removal bot” came in right away, got a 404 status and reported back that removal request is successfully queued
5) restored server’s default page to what it was before
6) after about 10 hours i received a "request denied" status in urlconsole.

Apparently the above steps failed but I can’t think of any reason why. The purpose of google’s “link removal tool” is to remove dead links and in the step 4 google acknowledged the dead link (404 status). Did anyone else tried something like this? Perhaps I’m too dumb to rationalize google’s thinking. If someone could enlighten me as to why google denied the request later I would appreciate it.

friendlyseo

6:49 am on Jan 9, 2005 (gmt 0)

10+ Year Member



I have the same problem with dupe content and goo.

My site has been popular for a long time and webmasters who have directories and such are linking to my front page and inside pages but are using php syntax redirecting to my content. I also do reviews and they link to them thinking they are doing me a favor.

Not only are they depriving me of pagerank but now they are getting credit for content on my site.

Then when I search for my pages on Goo the other domains show up. Clearly Goo should figure out that xyz.com is not on my site:aaa.com? Why do they list them when I do a site: search on my domain?

www.polyphoniconline.com/links/link.php?id=32

That goes my domain home page... (At this writing)

Also, my home page used to be top 10 for a very popular term and now I can't even find it when I search my own site. The PR is still 6 but the page has no juice... odlly enough the site still has 1400 pages in the index but as you can guess my traffic has dropped to 1/3 of what it used to be.

Spiders rarely visit my home page. Is it possible only the home page is banned? It still has a PR 6 even after the last update.

I wrote emails to some of the fookwahds that use the PHP BS for linking and a few said they'd remove the links. Others no response.

This is tragic. Am I going to have to start over and throw away a successful 6 year old site because I am getting dinged with dupe content penalties from php directory jerks?

Google has an obligation to find the footprint these php directory url's are using and BAN them - not the sites they link to.

Sorry, but this site is too deep to recreate again.

eyezshine

9:34 am on Jan 9, 2005 (gmt 0)

10+ Year Member



One of my sites got completely banned because some other site somewhere linked to it like this?

www.ladyoflorien@mysite.com/

The link goes directly to my home page which is clearly dupe content. But why can't google do something about this?

They are so messed up these days it's rediculous!

Let's all link to www.madwebmasters@google.com

That'll teach'em to mess with us! Yeah right.

These problems are getting worse it seems to me.

brixton

10:43 am on Jan 9, 2005 (gmt 0)



i will strongly advise to all victim webmasters,we all know here that there very good programmers and certain websites that you can find them why dont you pay a good amound of money to hire one of those brains to hack those bast^&rds?

eyezshine

11:01 am on Jan 9, 2005 (gmt 0)

10+ Year Member



I just want google to fix this simple problem that even I could probably fix in a week or 2 myself if they would just let me in the googleplex!

I just think they are too busy (building other new things that will make them more money) to be worrying about the few thousand websites affected by these 302 redirects.

I know there have to be alot of sites affected by these problems but I am sure most of them don't even know anything is going on. So people haven't complained enough yet for them to fix the problem.

Or they are working on it and it's more complicated than we think and that's why it has taken over a year for them to fix it.

I can't believe Yahoo has better serp's than google? But they clearly do. Google was doing so good until about a year ago. Now it's just crap.

I might as well be searching for the keyword "Crap"! But I should do it on Yahoo because the site I want is probably banned from google. It's a joke!

mikeD

12:24 pm on Jan 9, 2005 (gmt 0)

10+ Year Member



yes i agree eyezshine, my site hasn't been effected yet "touch wood" but I know one which has. Not only penalised but pr drop to 3 from 6. Did have 2000+ backlinks. This problem need fixing quick. I have found hundreds of these site linking to me which is very worrying.

Google really needs to concentrate on what made them popular to begin with, the best serps. They cant claim this now and if they dont address this they could have problems.

Tallon

1:21 pm on Jan 9, 2005 (gmt 0)

10+ Year Member



Hi everyone, first post here and I'm still learning about SEO and all this techy stuff, but thought maybe I could offer some info to help get a clearer picture of what's happening lately (if anything). Hopefully I'm understanding your situation correctly.

I have a handful of affiliate product sites that I link to my merchants product links as:

mydomain.com/visit/hop.php?id=#*$!

I don't do this to knock anyone out of the SERPS, but to protect my links from being blocked by ad blockers (re: Norton's), trying to dodge parasites that want to overwrite my cookies as well as track clicks to see what my visitors are responding to. Lots of reasons for me to do this. I would guess that the majority of webmasters linking to outside sites in this manner have the same or similar intentions.

Since reading this thread and looking at my linking practices, I've noticed a few things.

#1. Two sites with a fairly high amount of php redirect/tracking links (over a 100) no longer show the php redirect link when I do:

site:www.mydomain.com in Google

Previously the links would show in the results. No title, no description, just the php redirect script. Although I noticed the links showing in the results, I never thought much of it or paid much attention to it. I don't think the links ever showed up in Yahoo or MSN. They don't now if they did. Sites that only have a handful of php redirect links still show in the results for that query.

#2. This week one site inexplicably wiped right out of SERPS for a few search terms I've been monitoring - except for one search phrase. Now this site never did well in Google to begin with so believe me it's no loss, but now all but one phrase are gone from the results (monitored 4 key phrases). The only thing I've changed on this site in the past few weeks is the background color. With the disappearance of the php scripted links in my site:domain query and the banishment from SERPS with 3 out 4 phrases I monitored, this makes me wonder if I was somehow benefiting in the SERPS (dismal as it was) from using these php redirect links? Was Google giving link credit to my site and now isn't?

#3. My sites didn't get much from the latest PR toolbar update, one moved up a notch, another moved down, others stayed the same. I've done some link building and thought I'd move up, but didn't really. Could it be that the php scripted links were passing some PR within site as well and now aren't? Or it could simply be I didn't do enough to merit a rise in PR.

I wanted to post this just to give you some feedback from someone who does link out this way, the motivation for it and what's been happening with my site:domain results, PR update and maybe even the SERPS. Hopefully it shows that Google is working on this problem and that results are starting to show?

friendlyseo

7:52 pm on Jan 9, 2005 (gmt 0)

10+ Year Member



Many webmasters have good intentions when they link that way, but there still are problems as a result of that link syntax.

Some off the shelf programs even link that way (content management, link exchange directories etc.)

What irks me is those webmasters with directories and link exchanges who knowingly use this system because they know that the links with that syntax will not pass PageRank to the sites they are linking too.

It has gotten to the point that everytime I am proposed a link exchange I have to look at the source code to find out if standard html syntax is being used, or if some sneaky redirect syntax is being used. It's a huge hassle.

Now - it should be up to Google to acknowledge this problem and act upon it. Problem is they're not interested in how webmasters feel - they're worried about how the people that search on Google feel - and it's invisible to searchers whether they get to my page about "free loops" via PHP redirect URL or the actual domain URL. So Google may never know unless they read this post and I doubt they care enough.

By the way, I have spoken to programmers and they've suggested you can still do tracking and such without linking that way.

walkman

8:30 pm on Jan 9, 2005 (gmt 0)



some are saying that the new serps are much worse to the sites already hit by the 302 bug. Google apparently still doesn't care.
[66.102.7.99...]
[216.239.53.104...]
[66.102.7.104...]
[66.102.7.105...]
[66.102.7.147...]

what do we have to do to get their attention and have this fixed?

dazzlindonna

9:08 pm on Jan 9, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



i wish i knew, walkman. i DO know that people have sent examples to the special email that GG requested. and they have received generic "we are investigating" replies. this issue has me in a state of depression. something so out of our control is frustrating.

walkman

9:28 pm on Jan 9, 2005 (gmt 0)



"i wish i knew, walkman. i DO know that people have sent examples to the special email that GG requested. and they have received generic "we are investigating" replies. this issue has me in a state of depression. something so out of our control is frustrating. "

does anybody want to volunteer to light themselves up in front of Google's headquarters? ;) No doubt the media would pick it up and Google would get going on this. We have seen several updates since GG was let know and no results. Actually it has gotten worst.

zeus

9:52 pm on Jan 9, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Crob you can not remove another domain name on the removal tools, even if it is hijacking your site.

crobb305

10:22 pm on Jan 9, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Crob you can not remove another domain name on the removal tools, even if it is hijacking your site.

u can if it no longer exists (i.e, serves a 404 page not found) But it may be a moot point since the removal tool page has been unavailable for 2 days.

c

Tallon

10:48 pm on Jan 9, 2005 (gmt 0)

10+ Year Member



friendlyseo the only other ways that I know of to cloak the links are to use frames or to mask domains. This is not an option because the merchant sites appear unsecure to the shopper I send (no yellow padlock at checkout, no signs of a secure transaction unless that page has a framebreaker in place). I'm going to try a domain redirect using .htaccess which might work. btw for the most part the method needs to be capable of cloaking hundreds/thousands of affiliate links per site efficiently. There are a tonne of options for link cloaking when you're only dealing with a few links, but not feasible when doing hundreds or thousands and dealing with several merchants and affiliate networks at once.

This isn't just php either, I have a directory site linking to me with cgi:

www.directorysite/cgi-bin/links/out.cgi?id=number

That shows up in my site:domain results so it looks like I'm dealing with a mirror issue as well. I'll go through the steps above to have it removed, thanks to everyone for the info.

I'm thinking Google IS making changes with this type of linking. Lately there has been talk of affiliate sites being targetted by Google and disappearing from the SERPS. Since I perceive a good majority of affiliates link in this manner (cloak links in redirect/tracking scripts), I'm wondering if the sudden drop in SERPS is because the redirect links were actually giving some PR value and/or link authority to the domain using them, but no longer do.

With the disappeance of the php links from my query site:domain in google, I really think they are tackling this issue at Google.

Tallon

11:02 pm on Jan 9, 2005 (gmt 0)

10+ Year Member



Also friendlyseo you mention having to look at the source code before doing link exchanges. I also always looked for and provided pure html link exchanges and just right clicked and checked the properties of the links already listed on their site to ensure no scripts were used.

Is that method not safe enough, what else should I be looking for?

crobb305

11:24 pm on Jan 9, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



With the disappeance of the php links from my query site:domain in google, I really think they are tackling this issue at Google.

I agree with you. The last email I sent to Google (using the special keyword in the subject line per GoogleGuy's request) got what I think was a genuine response (not auto-generated) telling me to be patient--they are working on the issue.

In the past few days I have seen the number of tracker2.php urls indexed dwindle on some datacenters. Granted, these data fluctuate but the net effect seems toward a solution. Yesterday, I did a search for "inurl:tracker2.php" and there were 106,000 results. A month ago, that number was 440,000. Yes, you may do that search now and get a much larger number, but the bottom line is that changes are in the works. The site:mysite.com search shows much fewer redirects and tracker2s than before.

I am irritated that it is taking them so long, but maybe the problem will be resolved soon. Maybe in a month or two, we will all have our rankings back. My whole positive outlook may change in the coming weeks if the changes I think I am seeing prove to be a false alarm. Google results have had a tendency lately to go stale.

Fingers still crossed, starting to hurt. :)

C

joeguy

11:35 pm on Jan 9, 2005 (gmt 0)

10+ Year Member



Any help would be greatly appreciated

I'm not sure why google dropped me... but after some searching and researching I found a link in google that re-directs to my website.

This is the link found in google's search results: www.online-dating-services.info/links/551

My site: www.discreetadventures.com

Did google drop me because it thought this was duplicated content?

Any help would be greatly appreciated

leoo24

12:06 am on Jan 10, 2005 (gmt 0)

10+ Year Member



my affected site is being battered even worse on these other datacentres.

I have pages indexed that don't exist,

eg www.domain.com/pagename2.html which redirect to a custom 404

i have many like that, i also have one site linking to me via a translate script:

[domain.com...]

what would be the correct dissalow for that in robots.txt?

and where the hell is that damned removal tool!

joeguy, i dare say that is the problem!

energylevel

12:37 am on Jan 10, 2005 (gmt 0)

10+ Year Member



<<<< what would be the correct dissalow for that in robots.txt? >>>>

You can't disallow an external URL if that's what you mean ...

walkman

12:58 am on Jan 10, 2005 (gmt 0)



Joeguy,
I'd bet on it. I think that is the cause of your problems.

walkman

12:59 am on Jan 10, 2005 (gmt 0)



"telling me to be patient--they are working on the issue. "

I don't think we have a choice either way. When was this email sent? Just curios...

zeus

1:10 am on Jan 10, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I think things will get worse, because the real spammers and hijacker will see how easy it is to manipulate the sites to get a good PR and hurt the competition and it is legal, so as long Google is broke, there is nothing we can do, so concentrate a little on other project if any and focus on MSN.

joeguy

3:05 am on Jan 10, 2005 (gmt 0)

10+ Year Member



This really BITES. I have emailed the webmaster to see if they will remove the link. Is there anything else I can do about this? I sent an email off to google, but I doubt that's going to do any good.

I guess I'll just focus my efforts on recruiting more affiliates to my program

crobb305

9:09 pm on Jan 10, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I have noticed something interesting...

Some sites that I have examined, which have tracker2.php urls pointing to the index page are NOT suffering from penalty. Coincidentally, these sites are running Adsense. I wonder if running Adsense, especially using the url channels tool, helps Google distinguish original urls from the spammy redirects.

Zeus, do you run Adsense?

C

walkman

9:33 pm on Jan 10, 2005 (gmt 0)



"which have tracker2.php urls pointing to the index page are NOT suffering from penalty"

in many cases the URL hasn't been indexed yet, maybe this is one of those cases. Not all 302s are followed and indexed (cached) by Google.

crobb305

9:59 pm on Jan 10, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Walkman, the urls to which I am referring are indeed indexed. Searching inurl:tracker2.php inurl:siteinquestion.com shows the tracker2's.

zeus

10:03 pm on Jan 10, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



crobb I am a member but I never use it, I did try for a day to run adsense, to get googlebot back but it diddent help.

I have noticed the hijackers are using adsense.

zeus

12:08 am on Jan 11, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I have added adsense to 70% of my site, lets see what happens, let me know how long you want the test to stay, because I dont want to keep them, because they pay nothing.

crobb305

1:38 am on Jan 11, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Zeus, I doubt it will help. But I thought that if Google is having a hard time distinguishing original/intended/legitimate urls from spammy redirects (despite an order-of-magnitude more backlinks to the original), then maybe Adsense gives Google an indication of the original's purpose/significance.

zeus

1:57 am on Jan 11, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



thats also what I hope for, but what the he.. we have to try everything out, so I will let it be for 1-2 days,
This 172 message thread spans 6 pages: 172