Welcome to WebmasterWorld Guest from 54.158.214.111

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Lost in Google

     
6:39 pm on Feb 27, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:June 17, 2003
posts:43
votes: 0


My site has been in 1st page serps for many years. Just a few weeks ago, my listing went from a title display and decription to just this:

www.mydomain.com/
Similar pages

In addition, my Yahoo listing disappeared as well. I then did a Yahoo search for pages with my domain included and found most of my interior pages indexed but not my home page.

What happened? Is it possible my site was not ready to be crawled when Googlebot and Slurp robots visied my site - simultaneously?

This is a very "white hat" site - no tricks at all, just good content...

I went and manually requested my site be spidered on both Google and Yahoo, and sent an email to Yahoo requesting any explanation as well.

Is there anything else I can do? Any ideas of why this happened?

7:50 am on Mar 8, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 20, 2003
posts:197
votes: 0


in response to IP based protection. the ip of the website is not the ip you'd be worried about, it would be google's spider IP and delivering different content to them is a no-no.

you could check for referrer and if it matches a hijacking page you could display a different page or nothing at all, but this could be interpreted as cloaking.

8:18 am on Mar 8, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:May 10, 2003
posts:929
votes: 13


I'd say someone is probably aware of the problem, since my new thread about not seeing Gbot since 2/28 was intercepted. Strange things are afoot at the Circle G.
8:33 am on Mar 8, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 20, 2003
posts:197
votes: 0


wait, can you please explain that a little more i'm confused.
8:39 am on Mar 8, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 20, 2003
posts:197
votes: 0


wow you're right, a few of my sites that normally get heavy gbot traffic have not seen gbot for an unnusually long time.
8:48 am on Mar 8, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:May 10, 2003
posts:929
votes: 13


All domains we have, even ones with no missing pgs haven't seen Gbot since 2/28. Oops, just found a single access on 3/3 on a multi-thousand pg domain that usually gets a hundred+ a day. Musta been a fluke :). Maybe they're trying to contain the damage, by not adding any more fuel to the fire.
Also their submit page (with the type-in-the-graphicized text) was not working properly earlier today. Now a new one (copy of the older version) seems to be accepting submissions.
8:51 am on Mar 8, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 20, 2003
posts:197
votes: 0


yeah that sounds like what it is to me, or at least i hope thats what it is.
9:24 am on Mar 8, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Dec 13, 2002
posts:246
votes: 0


the ip of the website is not the ip you'd be worried about, it would be google's spider IP and delivering different content to them is a no-no

Good point, I should've thought about that.
Ok, can you elaborate on checking for referrer?
If we simply tell our web site to NOT RETURN html (or return something different) then couldn't we solve the problem? Would Google even know that we are "cloaking" and would it penalize for this if it is directed at certain IPs but not Google's?

9:26 am on Mar 8, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 20, 2003
posts:197
votes: 0


Good point, I should've thought about that.
Ok, can you elaborate on checking for referrer?
If we simply tell our web site to NOT RETURN html (or return something different) then couldn't we solve the problem? Would Google even know that we are "cloaking" and would it penalize for this if it is directed at certain IPs but not Google's?

i think you're misinterpreting here, forget about ip-based protection, it won't work.

what i'm talking about is having the page check for the referrer of each view, that is the page accessed directly before it. if the referrer is a page that is known to be hijacking your page then your site would not serve results.

problem with this is gbot doesn't provide referrer information. so this wont work either sorry.

10:34 am on Mar 8, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:July 20, 2001
posts:74
votes: 0


If Google is giving the impression that a 302 redirect link from an innocent site is hijacking your page and effectively infringing your copyright, who is responsible?

The redirect in itself isn't a problem, it's the way that it is interpreted by the SE.

11:08 am on Mar 8, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member zeus is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 28, 2002
posts:3444
votes: 1


Stargeek - if you have been hijacked/redirecting problem you will not see googlebot, maybe it steps by for a few pages then its gone again, as soon you see a lot of spidering again, you could see better times until you get hijacked again.

I think the only solution to this is to make a big letter or email, where we tell them about the problem and if nothing is being done, we have to take in to use the hijacker way to create our site, we have to follow the way google wants there pages and if we are enogh webmasters they will listen. I fthis is getting real public how easy it is to hurt other sites, the whole index will be hijackers, we are on a good way already with all those scrapers/supplemental results and we see the omitted results so quickly because there are dublicates somewhere or its broke.

I also think the title in the thread should have a text redirecting probmes on google.

11:27 am on Mar 8, 2005 (gmt 0)

Junior Member

joined:Mar 2, 2005
posts:63
votes: 0


the only reason of all the cr-p is that Google accepts almost anybody to adsence ,at the moment the half of the 8000000000 pages indexed are 100% cra..p pages made only for adsence,just 3 days ago i descover a site with 200.000 pages ranks in 200.000 results for one of the most competitive terms in internet business,if you go to the page you wont find what you look,so probably you will click on the gooooooo....ads.If thats the way the net is gona be,then goodbuy.
12:57 pm on Mar 8, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 12, 2001
posts:1150
votes: 0


Lots of valid sites return 302s when you check the header of one of their links --Yahoo, WebmasterWorld --lots of them.

So, it looks like a simple 302 is *not* the problem. Right?

1:09 pm on Mar 8, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 1, 2004
posts:1987
votes: 0


The one way I see to get this 302 issue fixed is to somehow get it into the news headlines. Public companies hate bad press.

If someone could find several 'name brand' sites that have been hijacked - then if someone here has ability or contacts in the media then it would be great to see a segment on 60 Minutes 'The Hijacking of the Internet'.

How about someone hijacting the irs dot gov site - that would get some attention.

1:21 pm on Mar 8, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Dec 13, 2002
posts:246
votes: 0


rfung your sticky mailbox is full:

The list of hijackers I am tracking has grown to about 50 and is probably well into the hundreds, I just don't have time to check everyone out.

If anyone suspects their site has been hijacked I suggest you perform the following search in Google for your site:

www.yourdomain.com -site:yourdomain.com

If you see quite a few domains with titles that begin with small letters (not capitalized) try clicking on them to see if they all appear similar.

2:00 pm on Mar 8, 2005 (gmt 0)

New User

10+ Year Member

joined:Mar 8, 2005
posts:5
votes: 0


I am having the same problem with my site, the TITLE and DECRIPTION dissapeared. I saw this happen to a competitor once before and a few weeks later he was gone from the results and has never came back. I did the inurl: and saw some questionable results, one looks like it is a translated page but it has page rank so I am worried it got indexed somehow and is considered a duplciate page.

Here is the url of the duplicate page:
[l33t.0pointer.de...]

Is it possible that this could be hurting us?

2:12 pm on Mar 8, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 27, 2005
posts:93
votes: 0


RETALIATION METHOD AVAILABLE AGAINST A SKYSCRAPER SITE

Bobby,

Good question about IP banning.

Stargeek,

Absolutely correct about robots having no referrer. And the IP banning method does not work. But if google implemented that googlebot must carry the referrer if a 302, 301 or meta refresh is involved, then we may have the answer. This is a brilliant suggestion and the best alternative yet.

The referrer can indeed be detected and presented with a 404. It would then be oh so simple by removing the hijack via the URL-CONSOLE BOT. Alas, the url-console will not accept the removal of a 302. I managed to block the IP of the console bot at .htacess, presented it with a 404, making it think the target page does not exist, it accepted the removal request, but google manually then denied the process. YOU CANNOT WIN. I wrote back in anger and they wrote back that they cannot comment about it.

======================

So, I think a top serverside expert is required here to explain exactly the process that takes place upon delivery of a 302 directive to googlebot.

Sure, there are a few ways to ban a referrer like a php script

<?

if(strstr($_SERVER['HTTP_REFERER'], 'badguy.com'))
{
header("Status: 404 Not Found");
header("HTTP/1.0 404 Not Found");
exit();
}

?>

The above php at the very top of your index page only stops a referrer and presents a magnificent 404 PAGE NOT FOUND but unfortunately googlebot has no referrer so we are back to square one.

.htaccess also provides some means to play a few tricks but again, prevention of googlebot accessing your site is not what you want.

In extreme cases where a site is completely damaged and you get ZERO hits a day and you see no hope after exhausting every means to correct the problem. There is a mega retaliation method that you can apply to the offending SKYSCRAPER that blasts out 302 directives.

I will not say where to get, or put up the script here, but a dos script on a refreshing dynamic IP can go to a skyscraper site and actually demand its results from its PAY PER CLICK AFFILIATES, the script is then presented with results for the scripts first random keywords. The script now clicks all pay per clicks results, it simultaneously demands another result for another keyword and again clicks the results presented. The cycle is fast and continues until the SKYSCRAPER sites bandwidth, msql functions are overloaded and depleted. Costing the affiliates thousands of dollars of damage to its customers. The spider script cannot be blocked because it will refresh its dynamic IP. The skyscraper webmaster normally is unavailable, probably in th bahamas enjoying his clicks revenues so the spider is totally free to do its job.

You arm yourself with a friends computer abroad also on a dynamic IP via remote access to his computer, you then point your spiders to the AFFILIATE COMPANIES that where feeding the SKYSCRAPER site with pay per click results and demand they stop providing the skyscraper. Make sure that their entire network is spidered 6 levels deep before you threaten them with 302 redirects. Give them 10 minutes to stop the feed.

Your spiders are not doing anything wrong because you are only looking for information on a mass scale. You are a fast reader and you like visiting the phenomenal skyscraper sites.

You now demand from the affiliate companies the removal of all 302 redirects used by the skyscraper site. You do not wish to communicate with the webmaster because it will be contradictory in terms. It is the affiliates duty to make sure that their feeds are not tampered with. Also spider the companies entire network of html pages and prepare your dead website to pump out 302 directives to all of the pages within their network. Don't forget to submit to search engines the vast numbers of internal pages of the affiliate companies networks, but be shure that it is not their pure html that you are submiting, you submit the deadly directives just like they done to you. Thousands of [ttccdeadsite.com...] ....and so on.

Many software exist that will do this automatically any level deep.

You will be threatened with legal action for causing 302 redirects to them. If you reply saying that they done it to you, they will say that they are a public company and you are not allowed to do a redirect to them.

2:16 pm on Mar 8, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 12, 2001
posts:1150
votes: 0


<<Is it possible that this could be hurting us?>>

Looks like the site you listed is attempting to get Google to index your site's content as gibberish, which would obviously affect your ranking. At any rate, I think something is heading your way.

2:22 pm on Mar 8, 2005 (gmt 0)

New User

10+ Year Member

joined:Mar 8, 2005
posts:5
votes: 0


Is there any other steps I can take to deal with this other than reporting that site to Google and trying to contact the site owner? Its hard to believe that our site which has been up and ranking well for three years can be toppled in a matter of days with a simple script!
3:06 pm on Mar 8, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 27, 2005
posts:93
votes: 0


TO THE SENIOR MEMBERS IN WEBMASTERWORLD

We have managed to highlight a problem that has been discussed many times before by very learned and responsible webmasters in great deapth.

This thread has obviously attracted attention as well and some magnificent points have been highlighted, in particular by "stargeek".

At least a temporary answer may lie in that if googlebot was forced to carry referrer information during a 302, 301 or a meta fresh, it may be possible to block and present a 404. Allowing the removal of the link via Google's URL-CONSOLE.

This is the most elegant and refined suggestion to the problem I have read yet in thousands of comments about the issue. We must not let this possibility slip through our hands without bringing it to the attention of google.

Google must also allow the target site to remove the offending redirect. It can easily be done, very easily indeed and this may help thousands of people who have lost their hard earned cash and websites.

In essence, if I do not want ALEXA pointing to my site with their notoriously efficient and incredulous 302 redirecting synchronization system that is in total violation of google’s guidelines, I simply deny alexa within the text range of the referrer. This will only stop googlebot when it passes through the 302 redirect mechanism of alexa's disreputable scripting methods carrying the venomous details of alexa. But it will allow googlebot under all other conditions.

It is at least a temporary answer. I think this will work and provide greater protection to websites without harming any other site.

3:18 pm on Mar 8, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 10, 2003
posts:79
votes: 0


This duscussion is getting very complex.
My main domain is appearing in google without Title and Description.

i did the search: www.mysite.com -site:www.mysite.com

and i founded this url:
enviroone.com/environmentalsite.php?url=http://www.mysite.com&id=11111

Do you think my site was hijacked?

3:46 pm on Mar 8, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Dec 13, 2002
posts:246
votes: 0


Kubano, it could be, have a look and see if they've stolen your content.

After searching for www.mysite.com -site:www.mysite.com you could try searching for an exact match of a phrase that is in your site.

In my case I have a client who is indexed in Google but does not even appear among the 13 results for an exact phrase with quotation marks regarding a product they produce that nobody else does. This is an example of how the other hijackers have ousted my client, it appears to Google that my client is the one with duplicate content.

This brings me to my next silly question (yeah I know it's tough to beat the first one right?), which is whether or not Google can compare cache date to new content.

Let me give you an example of what I mean.

If I have a page that has not changed for some time then Google could assume that my page is the original as opposed to the hijackers' pages.

The problem is the moment I change my page then the dates are older than the hijackers'.

Is it possible to compare the cached page in google to the hijackers' pages and determine that the gist of the content existed years before the new kids on the block were even in diapers?

That said I'd like to join with japanese in calling for some help from either Senior Members or people knowledgable in the field.

4:02 pm on Mar 8, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:June 4, 2002
posts:1830
votes: 3


If I have a page that has not changed for some time then Google could assume that my page is the original as opposed to the hijackers' pages.

When I'm having a duplicate content problem with a site being copied by a competitor soon as I get a new page online and indexed in Google I make a copy of it with Google's cache (which has a date on it--hopefully an accurate one) with my browser and save it in case i need it. Then you have a 3rd party witness as to who was the original author, i.e., Google itself. And every time I update the page I do the same in case it's copied after the update.

4:16 pm on Mar 8, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Dec 13, 2002
posts:246
votes: 0


Hi Lorel, thanks for your input.

How do you make a copy of it with Google cache?
What kind of problems are you dealing with?

4:27 pm on Mar 8, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Sept 24, 2003
posts:318
votes: 0


This happened to me back in December.

Goolgle replied to my email with the standard "theres nothing anyone can do to hurt you" spewage.

There are posts on webmasterworld about this that are a few YEARS old.

4:31 pm on Mar 8, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 12, 2001
posts:1150
votes: 0


<<This happened to me back in December>>

And what's the status of your site now? Still gone?

4:55 pm on Mar 8, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 10, 2003
posts:79
votes: 0


hi Bobby:

<<After searching for www.mysite.com -site:www.mysite.com you could try searching for an exact match of a phrase that is in your site.>>

i dint find it..

No we founded many hits in our logs, mostly from yahoobot and msnbot with 302.

5:16 pm on Mar 8, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 10, 2003
posts:79
votes: 0


well i founded now a site from Colombia that hijacked my website:

[intelexport.com.co...]

The nice thing: this website was designed by TERRA NETWORKS!
See here: [intelexport.com.co...]

What can I do?

5:41 pm on Mar 8, 2005 (gmt 0)

Administrator from US 

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14664
votes: 99


This will only stop googlebot when it passes through the 302 redirect mechanism of alexa's disreputable scripting methods carrying the venomous details of alexa.

Oh the pain from laughing so hard -

Your anxiety about Alex redirects is misguided as many search engines (like Yahoo, Lycos, Excite and others) and directory sites redirect to your site in order to count the traffic as people click on individual websites. If you block everyone that redirects to your site you'll be one busy person and probably only have Google providing traffic.

5:53 pm on Mar 8, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Oct 18, 2000
posts:256
votes: 0


This situation seems pretty simple when you get right down to it. SEs should never treat an URL that consists of a redirect as having any value. It needs to stay in the database for various reasons (passing PR, for example), but it shouldn't come up in search results or be treated as the original by a dupe filter. The fix is simply to implement this correctly. Complaining about Alexa legitimately using 302 redirects is just silly.

The really nasty thing here is that the scraper sites (please stop calling them "SKYSCRAPER" sites, OK?) aren't actually copying anything, allowing them to get the same results they would have if they'd actually scraped your page, but without having to violate copyright law in order to do it. Your best interim solution is probably to send the host a copyright complaint, as pure legal intimidation often works even when you have no real legal claim. If that doesn't work, you just have to wait for Google to finally fix the problem, and I wouldn't hold my breath given how long this has already been a problem.

As for sending referrers when spidering a page, that doesn't really make sense. If you have 5000 inbound links, do you really want Google to request the same page 5000 times with different referrers? Robots aren't browsers and they generally don't follow links. They just add the linked-to URL to their database (if it's not already there) and it'll get spidered along with all the rest at some future date. The problem is that Google is associating the content returned for the target URL with that URL and all URLs that link to it, rather than simply recording the fact that the redirect URLs are redirects and ignoring them when generating search results.

6:30 pm on Mar 8, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member zeus is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 28, 2002
posts:3444
votes: 1


I said before in this post that I have asked adsense if they want us to sent websites that us the scraper scripts and redirecting, that way hurting our sites, believe it or not even that gave me a standard reply, that means they just dont care PERIOD, sorry folks.
This 206 message thread spans 7 pages: 206
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members