homepage Welcome to WebmasterWorld Guest from 54.227.89.236
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

This 206 message thread spans 7 pages: < < 206 ( 1 2 [3] 4 5 6 7 > >     
Lost in Google
rshandy




msg:758431
 6:39 pm on Feb 27, 2005 (gmt 0)

My site has been in 1st page serps for many years. Just a few weeks ago, my listing went from a title display and decription to just this:

www.mydomain.com/
Similar pages

In addition, my Yahoo listing disappeared as well. I then did a Yahoo search for pages with my domain included and found most of my interior pages indexed but not my home page.

What happened? Is it possible my site was not ready to be crawled when Googlebot and Slurp robots visied my site - simultaneously?

This is a very "white hat" site - no tricks at all, just good content...

I went and manually requested my site be spidered on both Google and Yahoo, and sent an email to Yahoo requesting any explanation as well.

Is there anything else I can do? Any ideas of why this happened?

 

stargeek




msg:758491
 7:50 am on Mar 8, 2005 (gmt 0)

in response to IP based protection. the ip of the website is not the ip you'd be worried about, it would be google's spider IP and delivering different content to them is a no-no.

you could check for referrer and if it matches a hijacking page you could display a different page or nothing at all, but this could be interpreted as cloaking.

MikeNoLastName




msg:758492
 8:18 am on Mar 8, 2005 (gmt 0)

I'd say someone is probably aware of the problem, since my new thread about not seeing Gbot since 2/28 was intercepted. Strange things are afoot at the Circle G.

stargeek




msg:758493
 8:33 am on Mar 8, 2005 (gmt 0)

wait, can you please explain that a little more i'm confused.

stargeek




msg:758494
 8:39 am on Mar 8, 2005 (gmt 0)

wow you're right, a few of my sites that normally get heavy gbot traffic have not seen gbot for an unnusually long time.

MikeNoLastName




msg:758495
 8:48 am on Mar 8, 2005 (gmt 0)

All domains we have, even ones with no missing pgs haven't seen Gbot since 2/28. Oops, just found a single access on 3/3 on a multi-thousand pg domain that usually gets a hundred+ a day. Musta been a fluke :). Maybe they're trying to contain the damage, by not adding any more fuel to the fire.
Also their submit page (with the type-in-the-graphicized text) was not working properly earlier today. Now a new one (copy of the older version) seems to be accepting submissions.

stargeek




msg:758496
 8:51 am on Mar 8, 2005 (gmt 0)

yeah that sounds like what it is to me, or at least i hope thats what it is.

Bobby




msg:758497
 9:24 am on Mar 8, 2005 (gmt 0)

the ip of the website is not the ip you'd be worried about, it would be google's spider IP and delivering different content to them is a no-no

Good point, I should've thought about that.
Ok, can you elaborate on checking for referrer?
If we simply tell our web site to NOT RETURN html (or return something different) then couldn't we solve the problem? Would Google even know that we are "cloaking" and would it penalize for this if it is directed at certain IPs but not Google's?

stargeek




msg:758498
 9:26 am on Mar 8, 2005 (gmt 0)

Good point, I should've thought about that.
Ok, can you elaborate on checking for referrer?
If we simply tell our web site to NOT RETURN html (or return something different) then couldn't we solve the problem? Would Google even know that we are "cloaking" and would it penalize for this if it is directed at certain IPs but not Google's?

i think you're misinterpreting here, forget about ip-based protection, it won't work.

what i'm talking about is having the page check for the referrer of each view, that is the page accessed directly before it. if the referrer is a page that is known to be hijacking your page then your site would not serve results.

problem with this is gbot doesn't provide referrer information. so this wont work either sorry.

acee




msg:758499
 10:34 am on Mar 8, 2005 (gmt 0)

If Google is giving the impression that a 302 redirect link from an innocent site is hijacking your page and effectively infringing your copyright, who is responsible?

The redirect in itself isn't a problem, it's the way that it is interpreted by the SE.

zeus




msg:758500
 11:08 am on Mar 8, 2005 (gmt 0)

Stargeek - if you have been hijacked/redirecting problem you will not see googlebot, maybe it steps by for a few pages then its gone again, as soon you see a lot of spidering again, you could see better times until you get hijacked again.

I think the only solution to this is to make a big letter or email, where we tell them about the problem and if nothing is being done, we have to take in to use the hijacker way to create our site, we have to follow the way google wants there pages and if we are enogh webmasters they will listen. I fthis is getting real public how easy it is to hurt other sites, the whole index will be hijackers, we are on a good way already with all those scrapers/supplemental results and we see the omitted results so quickly because there are dublicates somewhere or its broke.

I also think the title in the thread should have a text redirecting probmes on google.

arras




msg:758501
 11:27 am on Mar 8, 2005 (gmt 0)

the only reason of all the cr-p is that Google accepts almost anybody to adsence ,at the moment the half of the 8000000000 pages indexed are 100% cra..p pages made only for adsence,just 3 days ago i descover a site with 200.000 pages ranks in 200.000 results for one of the most competitive terms in internet business,if you go to the page you wont find what you look,so probably you will click on the gooooooo....ads.If thats the way the net is gona be,then goodbuy.

jk3210




msg:758502
 12:57 pm on Mar 8, 2005 (gmt 0)

Lots of valid sites return 302s when you check the header of one of their links --Yahoo, WebmasterWorld --lots of them.

So, it looks like a simple 302 is *not* the problem. Right?

sailorjwd




msg:758503
 1:09 pm on Mar 8, 2005 (gmt 0)

The one way I see to get this 302 issue fixed is to somehow get it into the news headlines. Public companies hate bad press.

If someone could find several 'name brand' sites that have been hijacked - then if someone here has ability or contacts in the media then it would be great to see a segment on 60 Minutes 'The Hijacking of the Internet'.

How about someone hijacting the irs dot gov site - that would get some attention.

Bobby




msg:758504
 1:21 pm on Mar 8, 2005 (gmt 0)

rfung your sticky mailbox is full:

The list of hijackers I am tracking has grown to about 50 and is probably well into the hundreds, I just don't have time to check everyone out.

If anyone suspects their site has been hijacked I suggest you perform the following search in Google for your site:

www.yourdomain.com -site:yourdomain.com

If you see quite a few domains with titles that begin with small letters (not capitalized) try clicking on them to see if they all appear similar.

jddux




msg:758505
 2:00 pm on Mar 8, 2005 (gmt 0)

I am having the same problem with my site, the TITLE and DECRIPTION dissapeared. I saw this happen to a competitor once before and a few weeks later he was gone from the results and has never came back. I did the inurl: and saw some questionable results, one looks like it is a translated page but it has page rank so I am worried it got indexed somehow and is considered a duplciate page.

Here is the url of the duplicate page:
[l33t.0pointer.de...]

Is it possible that this could be hurting us?

japanese




msg:758506
 2:12 pm on Mar 8, 2005 (gmt 0)

RETALIATION METHOD AVAILABLE AGAINST A SKYSCRAPER SITE

Bobby,

Good question about IP banning.

Stargeek,

Absolutely correct about robots having no referrer. And the IP banning method does not work. But if google implemented that googlebot must carry the referrer if a 302, 301 or meta refresh is involved, then we may have the answer. This is a brilliant suggestion and the best alternative yet.

The referrer can indeed be detected and presented with a 404. It would then be oh so simple by removing the hijack via the URL-CONSOLE BOT. Alas, the url-console will not accept the removal of a 302. I managed to block the IP of the console bot at .htacess, presented it with a 404, making it think the target page does not exist, it accepted the removal request, but google manually then denied the process. YOU CANNOT WIN. I wrote back in anger and they wrote back that they cannot comment about it.

======================

So, I think a top serverside expert is required here to explain exactly the process that takes place upon delivery of a 302 directive to googlebot.

Sure, there are a few ways to ban a referrer like a php script

<?

if(strstr($_SERVER['HTTP_REFERER'], 'badguy.com'))
{
header("Status: 404 Not Found");
header("HTTP/1.0 404 Not Found");
exit();
}

?>

The above php at the very top of your index page only stops a referrer and presents a magnificent 404 PAGE NOT FOUND but unfortunately googlebot has no referrer so we are back to square one.

.htaccess also provides some means to play a few tricks but again, prevention of googlebot accessing your site is not what you want.

In extreme cases where a site is completely damaged and you get ZERO hits a day and you see no hope after exhausting every means to correct the problem. There is a mega retaliation method that you can apply to the offending SKYSCRAPER that blasts out 302 directives.

I will not say where to get, or put up the script here, but a dos script on a refreshing dynamic IP can go to a skyscraper site and actually demand its results from its PAY PER CLICK AFFILIATES, the script is then presented with results for the scripts first random keywords. The script now clicks all pay per clicks results, it simultaneously demands another result for another keyword and again clicks the results presented. The cycle is fast and continues until the SKYSCRAPER sites bandwidth, msql functions are overloaded and depleted. Costing the affiliates thousands of dollars of damage to its customers. The spider script cannot be blocked because it will refresh its dynamic IP. The skyscraper webmaster normally is unavailable, probably in th bahamas enjoying his clicks revenues so the spider is totally free to do its job.

You arm yourself with a friends computer abroad also on a dynamic IP via remote access to his computer, you then point your spiders to the AFFILIATE COMPANIES that where feeding the SKYSCRAPER site with pay per click results and demand they stop providing the skyscraper. Make sure that their entire network is spidered 6 levels deep before you threaten them with 302 redirects. Give them 10 minutes to stop the feed.

Your spiders are not doing anything wrong because you are only looking for information on a mass scale. You are a fast reader and you like visiting the phenomenal skyscraper sites.

You now demand from the affiliate companies the removal of all 302 redirects used by the skyscraper site. You do not wish to communicate with the webmaster because it will be contradictory in terms. It is the affiliates duty to make sure that their feeds are not tampered with. Also spider the companies entire network of html pages and prepare your dead website to pump out 302 directives to all of the pages within their network. Don't forget to submit to search engines the vast numbers of internal pages of the affiliate companies networks, but be shure that it is not their pure html that you are submiting, you submit the deadly directives just like they done to you. Thousands of [ttccdeadsite.com...] ....and so on.

Many software exist that will do this automatically any level deep.

You will be threatened with legal action for causing 302 redirects to them. If you reply saying that they done it to you, they will say that they are a public company and you are not allowed to do a redirect to them.

jk3210




msg:758507
 2:16 pm on Mar 8, 2005 (gmt 0)

<<Is it possible that this could be hurting us?>>

Looks like the site you listed is attempting to get Google to index your site's content as gibberish, which would obviously affect your ranking. At any rate, I think something is heading your way.

jddux




msg:758508
 2:22 pm on Mar 8, 2005 (gmt 0)

Is there any other steps I can take to deal with this other than reporting that site to Google and trying to contact the site owner? Its hard to believe that our site which has been up and ranking well for three years can be toppled in a matter of days with a simple script!

japanese




msg:758509
 3:06 pm on Mar 8, 2005 (gmt 0)

TO THE SENIOR MEMBERS IN WEBMASTERWORLD

We have managed to highlight a problem that has been discussed many times before by very learned and responsible webmasters in great deapth.

This thread has obviously attracted attention as well and some magnificent points have been highlighted, in particular by "stargeek".

At least a temporary answer may lie in that if googlebot was forced to carry referrer information during a 302, 301 or a meta fresh, it may be possible to block and present a 404. Allowing the removal of the link via Google's URL-CONSOLE.

This is the most elegant and refined suggestion to the problem I have read yet in thousands of comments about the issue. We must not let this possibility slip through our hands without bringing it to the attention of google.

Google must also allow the target site to remove the offending redirect. It can easily be done, very easily indeed and this may help thousands of people who have lost their hard earned cash and websites.

In essence, if I do not want ALEXA pointing to my site with their notoriously efficient and incredulous 302 redirecting synchronization system that is in total violation of google’s guidelines, I simply deny alexa within the text range of the referrer. This will only stop googlebot when it passes through the 302 redirect mechanism of alexa's disreputable scripting methods carrying the venomous details of alexa. But it will allow googlebot under all other conditions.

It is at least a temporary answer. I think this will work and provide greater protection to websites without harming any other site.

Kubano




msg:758510
 3:18 pm on Mar 8, 2005 (gmt 0)

This duscussion is getting very complex.
My main domain is appearing in google without Title and Description.

i did the search: www.mysite.com -site:www.mysite.com

and i founded this url:
enviroone.com/environmentalsite.php?url=http://www.mysite.com&id=11111

Do you think my site was hijacked?

Bobby




msg:758511
 3:46 pm on Mar 8, 2005 (gmt 0)

Kubano, it could be, have a look and see if they've stolen your content.

After searching for www.mysite.com -site:www.mysite.com you could try searching for an exact match of a phrase that is in your site.

In my case I have a client who is indexed in Google but does not even appear among the 13 results for an exact phrase with quotation marks regarding a product they produce that nobody else does. This is an example of how the other hijackers have ousted my client, it appears to Google that my client is the one with duplicate content.

This brings me to my next silly question (yeah I know it's tough to beat the first one right?), which is whether or not Google can compare cache date to new content.

Let me give you an example of what I mean.

If I have a page that has not changed for some time then Google could assume that my page is the original as opposed to the hijackers' pages.

The problem is the moment I change my page then the dates are older than the hijackers'.

Is it possible to compare the cached page in google to the hijackers' pages and determine that the gist of the content existed years before the new kids on the block were even in diapers?

That said I'd like to join with japanese in calling for some help from either Senior Members or people knowledgable in the field.

Lorel




msg:758512
 4:02 pm on Mar 8, 2005 (gmt 0)

If I have a page that has not changed for some time then Google could assume that my page is the original as opposed to the hijackers' pages.

When I'm having a duplicate content problem with a site being copied by a competitor soon as I get a new page online and indexed in Google I make a copy of it with Google's cache (which has a date on it--hopefully an accurate one) with my browser and save it in case i need it. Then you have a 3rd party witness as to who was the original author, i.e., Google itself. And every time I update the page I do the same in case it's copied after the update.

Bobby




msg:758513
 4:16 pm on Mar 8, 2005 (gmt 0)

Hi Lorel, thanks for your input.

How do you make a copy of it with Google cache?
What kind of problems are you dealing with?

kwasher




msg:758514
 4:27 pm on Mar 8, 2005 (gmt 0)

This happened to me back in December.

Goolgle replied to my email with the standard "theres nothing anyone can do to hurt you" spewage.

There are posts on webmasterworld about this that are a few YEARS old.

jk3210




msg:758515
 4:31 pm on Mar 8, 2005 (gmt 0)

<<This happened to me back in December>>

And what's the status of your site now? Still gone?

Kubano




msg:758516
 4:55 pm on Mar 8, 2005 (gmt 0)

hi Bobby:

<<After searching for www.mysite.com -site:www.mysite.com you could try searching for an exact match of a phrase that is in your site.>>

i dint find it..

No we founded many hits in our logs, mostly from yahoobot and msnbot with 302.

Kubano




msg:758517
 5:16 pm on Mar 8, 2005 (gmt 0)

well i founded now a site from Colombia that hijacked my website:

[intelexport.com.co...]

The nice thing: this website was designed by TERRA NETWORKS!
See here: [intelexport.com.co...]

What can I do?

incrediBILL




msg:758518
 5:41 pm on Mar 8, 2005 (gmt 0)

This will only stop googlebot when it passes through the 302 redirect mechanism of alexa's disreputable scripting methods carrying the venomous details of alexa.

Oh the pain from laughing so hard -

Your anxiety about Alex redirects is misguided as many search engines (like Yahoo, Lycos, Excite and others) and directory sites redirect to your site in order to count the traffic as people click on individual websites. If you block everyone that redirects to your site you'll be one busy person and probably only have Google providing traffic.

gmiller




msg:758519
 5:53 pm on Mar 8, 2005 (gmt 0)

This situation seems pretty simple when you get right down to it. SEs should never treat an URL that consists of a redirect as having any value. It needs to stay in the database for various reasons (passing PR, for example), but it shouldn't come up in search results or be treated as the original by a dupe filter. The fix is simply to implement this correctly. Complaining about Alexa legitimately using 302 redirects is just silly.

The really nasty thing here is that the scraper sites (please stop calling them "SKYSCRAPER" sites, OK?) aren't actually copying anything, allowing them to get the same results they would have if they'd actually scraped your page, but without having to violate copyright law in order to do it. Your best interim solution is probably to send the host a copyright complaint, as pure legal intimidation often works even when you have no real legal claim. If that doesn't work, you just have to wait for Google to finally fix the problem, and I wouldn't hold my breath given how long this has already been a problem.

As for sending referrers when spidering a page, that doesn't really make sense. If you have 5000 inbound links, do you really want Google to request the same page 5000 times with different referrers? Robots aren't browsers and they generally don't follow links. They just add the linked-to URL to their database (if it's not already there) and it'll get spidered along with all the rest at some future date. The problem is that Google is associating the content returned for the target URL with that URL and all URLs that link to it, rather than simply recording the fact that the redirect URLs are redirects and ignoring them when generating search results.

zeus




msg:758520
 6:30 pm on Mar 8, 2005 (gmt 0)

I said before in this post that I have asked adsense if they want us to sent websites that us the scraper scripts and redirecting, that way hurting our sites, believe it or not even that gave me a standard reply, that means they just dont care PERIOD, sorry folks.

japanese




msg:758521
 7:01 pm on Mar 8, 2005 (gmt 0)

incrediBILL

Let us see how much you will laugh by placing the address of your main website here, in this forum.

This time your pain will be associated with the grief of your site, not from laughter.

You fail to see the wood from the trees. No contribution and zero input into this problem other than a tumultuous burst of vilification.

Please place your site URL here, in this forum. I will place a few links pointing to your site, it should give you an increase in pagerank, right?

Let us see how much you will stick to your belief regarding the comments you made.

This 206 message thread spans 7 pages: < < 206 ( 1 2 [3] 4 5 6 7 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved