Welcome to WebmasterWorld Guest from 54.196.214.35

Forum Moderators: goodroi

Message Too Old, No Replies

Blocking Partial Contents of Folder Without Using Robots Tag

10 files (7 want indexed, 3 want blocked)

     
9:47 am on Mar 23, 2007 (gmt 0)

Junior Member

5+ Year Member

joined:Dec 13, 2006
posts:55
votes: 0



System: The following message was cut out of thread at: http://www.webmasterworld.com/robots_txt/3274413.htm [webmasterworld.com] by goodroi - 6:24 am on Mar. 23, 2007 (utc -5)


Hello Friends,

Can you please help me out. I have 10 files contained in a folder. I want to crawl 7 files and dont want index remaining 3 files.

User-agent: *
Disallow: /folder/
- I guess it will not index all the files contained in this folder.

I only want to do this through robots.txt files not with robots meta tags.

I am looking for your replies friends.

Thanks in advance,
- SandySEO

10:16 am on Mar 27, 2007 (gmt 0)

Junior Member

5+ Year Member

joined:Dec 13, 2006
posts:55
votes: 0


Should I guess that no one is capable to give me the answer of my query?
2:05 pm on Mar 27, 2007 (gmt 0)

Junior Member

10+ Year Member

joined:June 23, 2005
posts:145
votes: 0


SandySEO, with a name like that, why are you asking us? Simple, basic, beginner stuff for an SEO expert.

Block each file by name that you wanted blocked in the robots.txt.

Back to Lurking
WW_Watcher

7:53 pm on Mar 27, 2007 (gmt 0)

Administrator from US 

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:June 21, 2004
posts:3080
votes: 67


I only want to do this through robots.txt files not with robots meta tags.

Robots.txt was built to block entire directories and not individual files. I would suggest you rethink about using robots meta tags.

8:49 pm on Mar 27, 2007 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 27, 2001
posts:1472
votes: 0


User-agent: *
Disallow: /folder/file1.ext
Disallow: /folder/file2.ext
Disallow: /folder/file3.ext

[robotstxt.org...]

[edited by: Key_Master at 8:51 pm (utc) on Mar. 27, 2007]