Welcome to WebmasterWorld Guest from 54.167.177.207

Forum Moderators: brotherhood of lan & mack

Why not spidered?

   
7:55 pm on Apr 11, 2006 (gmt 0)

5+ Year Member



Hi, I'm very very new to SEO. I have a site that is not getting spidered, except for the main page. I added a site map, what else can i do? I'm at a loss here.

Sorry to ask such a silly question.

4:02 pm on Apr 12, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



The spiders will sooner or later get to the rest of your web site. Get a few web sites to link to your other web pages on your web site to speed it up the process of spidering the whole web site.
7:45 pm on Apr 12, 2006 (gmt 0)

10+ Year Member



google sitemaps is also pretty usefull, it might jumpstart the process. But yeah, in the beginning sometimes things can move slowly, but google is just trying gauge how serious you are about your site. just keep adding content and things will fall into place. also, if you don't have a robots.txt file, you can add this to your meta tags to let googlebot know you want to be spidered:

<META NAME="ROBOTS" CONTENT="INDEX,FOLLOW">

1:50 am on Apr 13, 2006 (gmt 0)

5+ Year Member



Hello, jtaylor.
I would like to share with you this quote from one of Google's engineers: "One of the classic crawling strategies that Google has used is the amount of PageRank on your pages. So just because your site has been around for a couple years (or that you submit a sitemap), that doesn’t mean that we’ll automatically crawl every page on your site. In general, getting good quality links would probably help us know to crawl your site more deeply." As you can see, according to this, your site may be waiting for two years and still not be spidered if it has no incoming links. As robotsdobetter said, you have better get some links to speed the process.
Good luck.
 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month