Welcome to WebmasterWorld Guest from 54.226.110.143

Forum Moderators: open

Message Too Old, No Replies

Odd listings

Using my webhost's URL

     
12:09 am on Jun 18, 2003 (gmt 0)

Full Member

10+ Year Member

joined:July 10, 2002
posts:232
votes: 0


For some searches, pages on my site aren't coming up as

[mydomain.org...]

but as

[secure.mywebhost.net...]

Some pages seem to be listed twice, but in at least one case the real listing has been replaced by the listing under my webhost's URL (perhaps the real one has been seen as a duplicate. I presume that this is hurting my site, since inbound links probably aren't attributed to these "new" pages.

Anyone have any clues what I can do about this? Anyone experienced anything similar?

Googleguy? Any ideas?

3:27 am on June 18, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 11, 2003
posts:442
votes: 0


You could ask your web host to add an entry to their robots.txt file for

[secure.mywebhost.net...]

Have them add something like :

User-agent: *
Disallow: /mydomain.org/

That ought to prevent the secure version from getting spidered again.

3:35 am on June 18, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 31, 2003
posts:1316
votes: 0


I thought that Google wasn't supposed to crawl https pages?
3:38 am on June 18, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 11, 2003
posts:442
votes: 0


mcavic : they certainly do, I've seen them in https logs for some time.

And why not, when you think about it. As long as they can spider it, they can index it.

12:34 pm on June 18, 2003 (gmt 0)

Full Member

10+ Year Member

joined:July 10, 2002
posts:232
votes: 0


Marcs: Thanks for the advice. I'll ask my webhost to do just that.
4:43 pm on June 18, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 31, 2003
posts:1316
votes: 0


And why not, when you think about it. As long as they can spider it, they can index it.

Interesting. Putting a secure page into a non-secure index seems counter-intuitive. Then again, some http sites immediately redirect to https, and I guess they do want to be indexed.