Welcome to WebmasterWorld Guest from 188.8.131.52
Forum Moderators: mademetop
Question2: Google comes over but it only read my main index page then goes away. it reads my welcome message where it has site update infomation and have navigation bars and title bars. I add a new page but it doesn't want to read. It's weired because the bot once crawled though all the site, and that was it. And some how it is selective as to which page the bot reads. I have about 9 or 10 concert posted on the site, but google only read about 4 or 5 of them. why is it doing this?
1) Do you have links from other sites pointing to yours? Do you have a listing in DMOZ etc
2) Do you have the Google toolbar and do you know the Pagerank of the pages on your site
3) How old is the site
4) What sort of links do you use to link to your concert pages and are they on the home page or a couple of clicks from the homepage
there are tons of reasons for why the bot will behave one way or another, all depends on the cirumstances mostly to do with the way your site naviagation is set up. It's not easy giving a reason without knowing a bit more about the site googlebot is inspecting.
Lots of info can be found on googlebot using the site search [searchengineworld.com] if you want to find out more in spare time ;).....
If you want Google to crawl your site, you should install the Google Toolbar, and then visit all of your pages. Uninstall it, reinstall it and do it again. Google will think multiple people are visiting your site, and they will crawl it.
right after I put my robots.txt, google started crawling my site (sort of)
Have you validated your robots.txt file to make sure there are no errors and you are not chasing GoogleBot away?
Robots Text Validation [searchengineworld.com]