Forum Moderators: open

Message Too Old, No Replies

Should I add my .css file to robots.txt

         

dan_popescu

9:23 pm on Jan 22, 2003 (gmt 0)

10+ Year Member



Hi
I'm building a site using CSS styles. Will Google index my .css file? Should I ban Google from doing so by including the file in my robots.txt? Is this good (isn't Googlebot gonna think I'm hiding sth)?

Thank you
Dan

hakre

9:34 pm on Jan 22, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



hi dan,

you won't need to do this.

dan_popescu

10:02 pm on Jan 22, 2003 (gmt 0)

10+ Year Member



Thank you. Are you saying Google will not index the .css file?
Dan

ibpotter

10:10 pm on Jan 22, 2003 (gmt 0)

10+ Year Member



I donnot se why it should - what would be the point?

dan_popescu

10:16 pm on Jan 22, 2003 (gmt 0)

10+ Year Member



right:)
thanks

Key_Master

10:57 pm on Jan 22, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Some search engine spiders have been caught crawling css files and it's always a good idea to place css and js files into a separate directory and then excluding it via robots.txt . The point being, you want spiders to be focused on the content within the site and not the clutter that puts the site together.

What's Google "hiding"?
[google.com...]

pageoneresults

11:07 pm on Jan 22, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I just read something somewhere that advised against the blocking of your css files. I've always disallowed my /css/ directory. After reading the piece that I did, I went through 25+ sites and removed the disallow for the /css/ directories.

Since some marketers have used css to do some unsavory things, blocking the css file from indexing may be an alert to some automated filter flagging a site for manual review. Who knows? I'm always one to play it safe and my css file are sparkling clean.

It's like that darn no cache tag. When you use it, the first thing many think is that someone is cloaking. I guess that same assumption could be applied to external css files. No need for alarm, I just thought it might be a good idea not to block the spiders from my css anymore.

hakre

11:38 pm on Jan 22, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



mmh, if a bot spideres a as text/css classified file, then the bot wants to spider that. it does not have to be a text based search engine spider. maybe some ses are planning special features about how big a specific text is displayed or audio support is given. i would never block a bot out from a publicly available file
okay, if you protect your images not to archiv by google or so, then, you can think about placing a css in robots.txt... ;)

Robert Charlton

9:19 am on Jan 23, 2003 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



>>Since some marketers have used css to do some unsavory things, blocking the css file from indexing may be an alert to some automated filter flagging a site for manual review. Who knows?<<

Hmmm.... I just blocked the css file on a new site a few days ago. Now you've got me rethinking it. Mine's sparkling clean too, though H1's aren't as big as they would be without it.