google and yahoo bots not following internal links

for some reason google and yahoo bots are not following my internal inks.

[DopeR]

I am listing items like this:

<ul>
<li><a href="exampleone.html" class="textcolor">Example One</a></li><br>
<li><p class="simpletext">Here is some content related to example one</p></li>
<li><a href="exampletwo.htm" class="textcolor">Example Two</a></li><br>
<li><p class="simpletext">Content related to example two </p></li>
</ul>

and like this:

<p class="style2">
<a href="exampleone.html" class="textcolor">Example One</a> * <a href="exampletwo.html" class="textcolor">Example Two</a> </p>

Can you see any reason that these exampleone.html and exampletwo.html pages are not being indexed in google or yahoo? Google and Yahoo will even re cache my site and still these pages are not indexed. Let me know if anyone has any input.

[edited by: DopeR at 8:53 pm (utc) on July 17, 2006]

[specter]

Are they on the home page?

[5ybarite]

I had this problem (and am still fighting w/Y!) Have you submitted site maps to G & Y!?

[DopeR]

yes it is on the homepage.

I have a Google sitemaps account but i haven’t uploaded the updated sitemap yet. Would that really cause this problem? Why wouldn’t the bot read the pages naturally when the bot indexes the home page?

Also, I dont have these sitemap programs for MSN and that search engine is reading these links just fine. #*$!?!

[Robert Charlton]

How many links do you have on your homepage, and, to use a crude measurement, what's your homepage PageRank?

[DopeR]

I have about 20 internal links on my homepage. Half of them are using the <ul> tags and the other half is using the other list style. None of the links are being followed. I have revamped this site about 6 months ago but google and yahoo will not follow links to the new site from the homepage. I have a PR of 3 on the homepage.

[Robert Charlton]

DopeR - 20 links is not too many. Occasionally, someone puts up hundreds. That said, for a PR3 site, 20 links is a lot. You can help pages get spidered by getting some good inbound links.

Now, when you say "indexed by Google," are you talking about being in the index (as in using a site:domain search), or are you talking about ranking? There's a big difference, and 6 months in Google is not very long for a site to rank, particularly on competitive terms, and particularly if you don't have many good inbounds.

[DopeR]

When i say indexed by google i mean that i see it using site:domain search. I dont really know what you mean by 'ranking'.

[caveman]

Showing up in the search results for the terms you hope for.

[DopeR]

with my old site i was ranking just fine for terms on all my pages. And for the new site i am ranking for terms that are found on the home page but i am NOT ranking for any terms on the other pages of the new site because google isnt indexing them when it visits the home page.

[Robert Charlton]

...but i am NOT ranking for any terms on the other pages of the new site because google isnt indexing them when it visits the home page.

Again, how are you determining this... by searching for your terms, or by checking the site:yourdomain.com command to see what pages are indexed in Google?

[DopeR]

i used both to confirm

[jimbeetle]

Have you looked at your logs to see exactly what gbot and slurp are asking for -- or not asking for? Have you checked your robots.txt and .htaccess to see if there are forgotten directives in there somewhere?

[canadafred]

Your web site's behaviours, based on so little, are similar to those that have only been recently created. Is this a fairly new web site?

If that be the case then it will take some time to build up enough credibility to merit a deeper crawl. Perhaps this would be a good time to start increasing the web site's importance by crafting and introducing additonal supportive content.

[Bewenched]

You really should be linking like this if they are in the root directory:

<a href="http://www.mydomain.com/exampleone.html" class="textcolor">

this will avoid alot of problems for you in the future.

[caveman]

Let's back up a bit...

I have revamped this site about 6 months ago but google and yahoo will not follow links to the new site from the homepage. I have a PR of 3 on the homepage.

Just so we're clear, what are we talking about, exactly? When you say 'new site' are there two different domains involved?

If not different domains, what is the difference between the new site and the old site? Did all the pages change except the homepage? Did the URL's remain the same but the page contents changed? Or, did all of the URL's change? Did old pages, if they were at different URL's, get redirected to new pages at the new URL's? Etc andsoforth. ;-)