Welcome to WebmasterWorld Guest from 3.81.28.94

Forum Moderators: goodroi

Message Too Old, No Replies

Robots.txt

Tell me about Robots.txt

     
6:02 am on Jul 29, 2015 (gmt 0)

New User

Top Contributors Of The Month

joined:July 29, 2015
posts: 3
votes: 0


Hello Friends,
I am new here, i have one query from all of you, When we use www.example.com/robots.txt to block some Folder and files. But i am confuse about one thing that
User-agent: *
Allow: /
Disallow: /Admin
Disallow: /Admin/
Disallow: Admin
----------------------------------
I want to know different between Disallow: /Admin, Disallow: /Admin/, Disallow: Admin.
Are there is any different between them or is it same. Or we can use any one.
Please help me, i am so confuse about this.

[edited by: bill at 10:16 pm (utc) on Jul 29, 2015]
[edit reason] use example.com [/edit]

11:11 am on July 30, 2015 (gmt 0)

Administrator from US 

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:June 21, 2004
posts:3490
votes: 378


You might want to visit [robotstxt.org...] to learn the basics and then post questions if you still don't understand something.
4:40 am on Aug 5, 2015 (gmt 0)

New User

Top Contributors Of The Month

joined:July 29, 2015
posts: 3
votes: 0


Dear i know about robots.txt but i am asking different between
Disallow: /Admin,
Disallow: /Admin/,
Disallow: Admin.
5:21 am on Aug 5, 2015 (gmt 0)

Administrator from US 

WebmasterWorld Administrator not2easy is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:4407
votes: 316


Google's specifications can be studied here: [developers.google.com...]

The link supplied above describes the way most other robots read the robots.txt file.
7:08 am on Aug 5, 2015 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15769
votes: 832


/admin is both a file and a directory, depending on what comes next.
/admin/ is a directory.
admin is a filename, but probably has no meaning in the context of robots.txt (because it doesn't begin with a / slash)

:: preparing for acute panic at discovery that robotstxt.org is apparently down for the moment ::
6:58 am on Aug 6, 2015 (gmt 0)

New User

Top Contributors Of The Month

joined:July 29, 2015
posts: 3
votes: 0


Thanks for your detail @Luck24 and @not2easy
Dear Luck24,
You mean if I use /Admin/ it will disallow only Admin directory,
And if i used /Admin then it will disallow admin directory and all files that comes after admin.
6:14 pm on Aug 6, 2015 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15769
votes: 832


/admin/ means the /admin/ directory and all its contents.
/admin means the same thing BUT ALSO any top-level files called /admin.cgi or /administrator.html or /adminify.php or etcetera.
If your names don't happen to work that way, the final / makes no difference. The "vanilla" version of robots.txt-- the directives recognized by everyone, not just google-- are based on the beginning of the path: "match up to this point".
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members