homepage Welcome to WebmasterWorld Guest from 54.198.224.121
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe and Support WebmasterWorld
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

    
Google visits unlinked new URLs
jmswe




msg:768482
 8:50 pm on Jul 20, 2005 (gmt 0)


The url was registered in april and has been in development since. I recently noticed that the site had had a visit from googlebot and was since added to googles index. The site is not linked so google seem to have picked it up via domain registers. (.se domain)

I thought incoming links was the only way into the index.

Not sure if this is old news. Hadn't seen it before.

 

Clint




msg:768483
 1:52 pm on Jul 21, 2005 (gmt 0)

I have some pages at my site that are NOT LINKED TO by ANY pages (like test pages for example), and they've shown in the G index.

stuartc1




msg:768484
 3:50 pm on Jul 21, 2005 (gmt 0)

I guess the advanced features of the google toolbar could trigger google to know about your pages!

I know in the case of Alexa this is the case... an example is this: I wrote an admin area, and was running some tests online (I had not set-up any password on the pages yet). Anyway, I came back the next day and all the links in the admin area had been visited... the problem was the links where things like 'Edit' or 'Delete'. Imagine my shook when I came back the next day to find all my content deleted, LOL. A hard lesson learned. (My logs showed alexa had visted those pages!)

I know that the google toolbar does take note of pages visited if the advanced features (i.e. pagerank ect) are enabled.... perhaps this is how they found the pages! Although I can't say with certainty they record all visited pages - but I guess they need to, to try and get the PR of that page!

webdude




msg:768485
 4:36 pm on Jul 21, 2005 (gmt 0)

Hey jmswe,

Welcome to Webmaster World!

moltar




msg:768486
 4:46 pm on Jul 21, 2005 (gmt 0)

I had my local, development pages spidered by Yahoo. I was glad I noticed it in time and added the robots.txt to the local server.

Lorel




msg:768487
 5:20 pm on Jul 21, 2005 (gmt 0)

When designing a website I always use a robots text disallowing all SEs as well as a nofollow/noindex meta tag on each page until it's ready and I don't recall an SE ever listing anything before I wanted it to.

Also when redesigning a site I usually post it on my own domain until its ready to be uploaded to the client site. I use the nofollow/noindex meta tags and robots text on it also until time to upload.

No problems encountered.

darth googol




msg:768488
 4:42 pm on Jul 22, 2005 (gmt 0)

Wow, this is all news to us. We have an entire duplicate site, a test duplicate, for a site dealing with [well...], a test site where amateur-SEO friends read new topics we don't want the world to see yet, due to the fact that we're amateurs ourselves and often under the influence. In other words, no one knows about this test site but us and close friends with too much time on their hands. Or so we thought.

Okay, we just now checked and it seems no one else knows about it, but maybe we should make preparations.

We've always left our test pages "live," never having considered the possibility that anyone could ever find them. We actually started doing this because we trust live pages over our design app's test feature and we're always making modifications.

theBear




msg:768489
 7:54 pm on Jul 22, 2005 (gmt 0)

Production site/server

Never trust a toolbar. It is a security hole.

Never have test files on a live site.

Never have test data in a live database.

Never leave any startup code.

Always block the automatic generation of directory index listings.

Remove all scripts that aren't used (that you might use them doesn't negate this rule)

Test site/server

Never trust a toolbar. It is a security hole.

On all test sites, disallow all robots.

Require a password.

Always block the automatic generation of directory index listings.

And there are more but this is a start.

darth googol




msg:768490
 3:17 am on Jul 23, 2005 (gmt 0)

Good to know; thanks. More rules than high school....

Dude3609




msg:768491
 9:00 am on Jul 24, 2005 (gmt 0)

I know this whole issue with google seems a little weird, but have you considered using robots.txt? lol

window




msg:768492
 10:44 am on Jul 24, 2005 (gmt 0)

I was making a new site from last 2 months...I didn't disallow my site to SE's.
In the latest update...Google crawled my site and gave home page PR - 2... Though internal pages are still 0..
My site has not compleated yet even donot linked properly..
What should I do now? I think now its difficult to improve my PR in future...will it remain 2?

need advise...

Dude3609




msg:768493
 10:54 am on Jul 24, 2005 (gmt 0)

uhh.. did you try contacting google? lol

Marcia




msg:768494
 11:27 am on Jul 24, 2005 (gmt 0)

Very funny, dude. :)

>>What should I do now? I think now its difficult to improve my PR in future...will it remain 2?

What you should do now is keep making your site better for your visitors. That's your job. The PR is Google's job. No, it won't remain PR2 - get a *few* good quality links at a time and it'll start to go up - but don't obsess about PR, just work on the site and give it time.

darth googol




msg:768495
 1:41 pm on Jul 24, 2005 (gmt 0)

Window said:

Google crawled my site and gave home page PR - 2... Though internal pages are still 0..

Totally not joking here ... really ... but IMDb has really good PR, yet all the pages that link to us are really low, even though IMDb is a directory edited by people (as opposed to some place where you go and just add your link).

I was under the impression that PR spread from page to page, and I often notice that many sites have a pretty consistent PR throughout the whole site. I assume, then, that Window's site will even out after a while, as do most, or am I just way off?

Clint




msg:768496
 2:01 pm on Jul 24, 2005 (gmt 0)

FWIW, each of my sites have far different PR's throughout their pages. I think it depends some on how/if you link to each of your pages FROM each of your pages, like a nav menu that has ALL links in it on each page. Now I even do that for most pages and of those pages even they have different PR's. That would probably be indicative of whom and how many are linking to your internal pages.

darth googol




msg:768497
 4:36 pm on Jul 24, 2005 (gmt 0)

Clint said:

each of my sites have far different PR's throughout their pages. I think it depends some on how/if you link to each of your pages FROM each of your pages, like a nav menu that has ALL links in it on each page.

Well, besides being free of social restraints ("make your pictures smaller"; "spend more time on design and less time screen capping every blessed frame of the movie"; "do not have an opinion"...), being unpopular does have another advantage: for a very long time, each and every page of your site will have exactly the same PR! :o)

We're just now learning how to properly utilize the template feature; what you're saying about links in the nav menu will come in mighty handy, so thank you.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved