homepage Welcome to WebmasterWorld Guest from 54.205.144.54
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Google wildcard
Is this allowed?
maha




msg:1525921
 4:39 am on Dec 6, 2004 (gmt 0)

Wildcard syntax for Robots.txt:

User-agent: googlebot
Disallow: /product*product_info.html

because I have 2000 products. Rather than inputing:

Disallow: /product1product_info.html
Disallow: /product2product_info.html
etc.

I thought mabye the Google wildcard will support it?

 

jdMorgan




msg:1525922
 4:53 am on Dec 6, 2004 (gmt 0)

Looks like it should work -- for googlebot only.

[google.com...]

Jim

maha




msg:1525923
 5:02 am on Dec 6, 2004 (gmt 0)

Thanks for the reply!

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved