Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

spidering without a homepage

kill the head and the body shall live?

         

eman

3:17 am on Jul 10, 2006 (gmt 0)

10+ Year Member



I killed my homepage on purpose. There is a page there, but it links to nothing, it goes nowhere.

However, everywhere else on the site links amongst itself. Think of my site as a circle of dots, each connected to everything else with the homepage dot in the center of the cirle, and then take away all connections to and from that homepage. That's what I've got now.

Every page on my site is on google as it should be, but now that my homepage isn't there, a spider doesn't have anywhere to go.

My question is, will the other portions of my site continue to get spidered, or will they die off since the homepage doesn't go anywhere anymore?

proper_bo

9:48 am on Jul 10, 2006 (gmt 0)

10+ Year Member



as long as the spiders have referenced the other pages they should continue to index them.
If you have incoming links (from other sites) to the other pages then they will certainly continue be indexed.

tedster

4:05 pm on Jul 10, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



With Google still wrestling with the "canonical issue", more changes are the only sure thing I'd say. It's possible that with no root page to your website, you could confuse Google's internal "canonical root" logic, either now or in the future.

It all depends on what specific logic Google programs in their search for a fix to this long-standing issue, and what assumptions that logic makes. Not too long ago, Amazon was showing no urls on a site: operator search, and I suspect (can't prove) that the home page issue at least played a part in that snafu.

I always recommend that my clients have the domain root resolve directly to some content or other. Not doing this is an experiment and even a gamble, IMO -- and one that I would not take with my own websites.

monkeythumpa

5:04 pm on Jul 10, 2006 (gmt 0)

10+ Year Member



When you type in "http://www.yoursite.com" what comes up? I would say that is your homepage.

eman

7:17 pm on Jul 10, 2006 (gmt 0)

10+ Year Member



@monkeythumpa:
<url edited>, as you can see it purposely goes nowhere despite that it technically is my "homepage".

I know it looks odd, but just accept that I meant to make my homepage this way while other parts of the site (namely the blog at /wpBlog/) are still around and fully functional, and I don't intend and redirecting or linking to the blog from the homepage anytime soon.

Like I was saying, since my traffic comes directly from search engines anyway, I'm just trying to see if anybody had any idea if the other portions of the site would continue to get spidered.

<Sorry, no specifics.
See Forum Charter [webmasterworld.com]>

I guess I'll just have to wait and see. Thanks for your help.

[edited by: tedster at 1:32 am (utc) on July 11, 2006]

monkeythumpa

9:14 pm on Jul 10, 2006 (gmt 0)

10+ Year Member



So you get a file view if you go to yourdomain.com. I would advise against doing it that way. People will type in your homepage and as your traffic increases that will be a significant number. Why not send people to your "about" page?

People get confused when they see file views and they are not expecting it.

Demaestro

9:37 pm on Jul 10, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Without your homepage linking your "inner" pages it is still possible that they will get indexed....

All you need is 1 inbound link from some one eles page to point a spider to one of your "inner" pages which will then point the bot to your other "inner" pages as you have indicated that the "inner" pages all still point to each other.

Not sure what this will do for your ranking though.

I am not sure why you want this to happen but perhaps a better way to go about it would be to set up a sub domain. There would be some advantage to this and you can get the same behavior.

So right now you have a homepage that doesn't link to subpages. but instead of doing that and having all the "inner" pages being called supplimental, you could set up a subdomain, all pages in the sub-domain won't be linked to from your www. domain, but your subdomain could have a home page and that page could link the "inner" pages. That way at least one of your "inner" pages could actually be recognized by Google as the main page.

An example of what I mean is:

You have....

www.example.com (stand alone page points to nowhere)
www.example.com/blog (points to a bunch more www.example.com/blog/example pages)

What you could have is:

www.example.com (stand alone page points to nowhere)
blog.example.com (Which can now be the home page for this back end stuff and can link to "inner" pages from there)

The benefit of this is that Google will call www.example.com/blog a supplimental page but will call blog.example.com as a top level page.

Does that make sense to you? I am not sure if this is practical for what you are doing as I am only guessing what your reasons are.

This would also allow you to keep doing this...

www.example.com
blog.example.com
forum.example.com

And so on...... Each subdomain doesn't have to know of the other subdomains or link to them but you can point G at all three pages like they are top level and G will crawl them seperatly.

This way is more transparent to the user to what you are doing as it is to Google.

[edited by: Demaestro at 9:42 pm (utc) on July 10, 2006]

sandpetra

1:13 am on Jul 11, 2006 (gmt 0)

10+ Year Member



Some pages I've removed from the navigation structure, but still reside on the server, so just isolated islands, have gone supplemental recently.

jcmoon

3:47 pm on Jul 11, 2006 (gmt 0)

10+ Year Member



So you get a file view if you go to yourdomain.com. I would advise against doing it that way.

As would I. A list of what files / directories exist is, in effect, a homepage linking to those files & directories. Therefore, it is a homepage, linking to your other pages ... which isn't what you want. It is also a good way for troublemakers to learn more about your site, and while perhaps that's not an issue for you now, it might be later on.

If you truly want something empty that doesn't link anywhere else in your site, you could always put up a page like

<html>
<head>
<title></title>
</head>
<body>
</body>
</html>
as your index.html or whatever. Or, as was suggested, a very simple "about" page ... good for usability and to avoid confusion.

But I think the more interesting question here is ... why? Why is it that you want a full-bodied site, without a head (to use your metaphor)? Why don't you want a homepage? This is a rare concept, and I'm curious what you've got in mind (if you don't mind sharing with the rest of the class).

Petra Kaiser

8:52 pm on Jul 11, 2006 (gmt 0)

10+ Year Member



For more then one year we use a easy to spell short domain name with a redirect for telephone purposes. No external or internal links to the homepage, but nocache nofollow noindex in the meta tag. To cover the hosting cost we added some separate adsense pages. Until today no indexing problems with Google or MSN. Only Yahoo shows an additional url only listing.

eman

7:57 am on Jul 20, 2006 (gmt 0)

10+ Year Member



I can't believe I went on this long without checking this thread. Sorry guys. But to get back on track...

@jcmoon, others: If you look, those links don't really go anywhere. In fact, it's a message from me. So no, it's not a directory listing, it just looks like one.

The reason why I took the site down was basically because it needed maintence. I left the blog running because...well, it works. The rest of the site is in need of updates and maintence. It'll just be a temporary thing, I'm just worried about the rest of the site dropping off engines as well.

As for another question jcmoon asked...it's difficult to explain, but just trust there are reasons why I took down the homepage and didn't link it anywhere.

@Demaestro: [blog.lookitsmysite.com...] is running, thanks for the tip.