homepage Welcome to WebmasterWorld Guest from 54.167.75.155
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

    
Maximum links in Crawler.html?
Maximum links in Crawler.html
josh_bowes




msg:3055759
 2:45 pm on Aug 22, 2006 (gmt 0)

Currently, our crawler.html file has 140 links. I have read that having more than 100 links on a page will hurt SEO since Google and other search engines views pages with this many links as spam.

I need some clarification regarding this rule and the general recommendation for Crawler.html files with more than 100 links.

Thanks so much.

 

Chris_H




msg:3056049
 6:10 pm on Aug 22, 2006 (gmt 0)

Although Google's guidelines suggest keeping the number of links below 100, according to a very well respected authority in the SEO field, you can have as many links as you want, as long as your page file size is under 100k.

Google will only index the first 100k of your page, thereafter truncating everything after this point.

You may want to stay safe, and consider splitting up your links page into smaller chunks of information.

plasma




msg:3056118
 6:57 pm on Aug 22, 2006 (gmt 0)

What exactly do you mean by "Crawler.html"?
Did I miss something?

lammert




msg:3056133
 7:11 pm on Aug 22, 2006 (gmt 0)

Google will only index the first 100k of your page, thereafter truncating everything after this point.

Google crawls and indexes pages larger than 100k nowadays. Don't know exactly when this started, but I remember that it was around the introduction of the Mozilla bot.

walkman




msg:3056604
 2:37 am on Aug 23, 2006 (gmt 0)

>> What exactly do you mean by "Crawler.html"?

means sitemap. I think Google can index up to 500KB, at least it did so a while back. As far as over 100 links, I doubt it will matter; google will index them (provided everything else is fine of course.)

josh_bowes




msg:3073776
 5:16 pm on Sep 6, 2006 (gmt 0)

Thanks so much for the responses.

I have a few more questions.

I assume all search engines treat these crawler pages the same. Am I right in assuming there is no penalty from the big three (yahoo, msn, google)?

Is it recommended to have a link to the crawler.html file (from, say, the home page)? Currently we link to our site map from our homepage, and our site map follows our site standards in terms of layout, color, font, etc. However, the crawler.html is just a flat html file with no formatting, just links. Ideally, we do not want people to visit this file (as we believe its purpose is more to help search engines dive deeper into the site).

If the crawler.html is not linked to by any other page, then am I right in assuming that it must be manually submitted to search engines?

Any help would be greatly apprecaited. Thanks in advance.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved