Welcome to WebmasterWorld Guest from 34.204.194.190

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Googlebot HTML file size vs real file size

     
9:29 am on Jun 6, 2009 (gmt 0)

Preferred Member

10+ Year Member

joined:Aug 14, 2004
posts:602
votes: 0


I am looking at my logs right now and I am seing Googlebot crawling pages reported around 4kb while in reality the filesize is around 17-30kb (copied and pasted in notepad).

The HTTP code is 200 so no problem with this. But I am wondering why when Googlebot crawls my server shows only like 4kb downloaded?

Have I been hacked (again)? Or is there something I might want to check on my server?

1:47 pm on June 6, 2009 (gmt 0)

Senior Member from CA 

WebmasterWorld Senior Member encyclo is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Aug 31, 2003
posts:9074
votes: 6


Rather than viewing the offline filesize, check the size as delivered to your browser. For example, in Firefox, load the page, right-click and choose View Page Info.

The most likely reason is that you have GZIP compression enabled on the server - Googlebot supports GZIP-compressed HTML delivery.

If not, check the contents of the cached link (assuming you allow caching).

3:58 pm on June 6, 2009 (gmt 0)

Preferred Member

10+ Year Member

joined:Aug 14, 2004
posts:602
votes: 0


You are absolutely correct. We switched server last month and while the older server had no compression the new one has and I didn't know that. Thank you! Saves me a bunch of time and worries.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members