Forum Moderators: goodroi
===================================================
User-agent: *
Disallow: /business/ # business directory
Disallow: /admin/ ? ? # admin directory
Disallow: /search/ ? # search page
Disallow: /stats/ ? # stats directory
===================================================
The syntax above is being displayed in Google Webmasters Tool "Crawler Access" section. But when look at the robots.txt at the root of my website then the code is being displayed as below:
===================================================
User-agent: *
Disallow: /business/ # business directory
Disallow: /admin/ # admin directory
Disallow: /search/ # search page
Disallow: /stats/ # stats directory
===================================================
So I want to know that:
1) why both syntax are different as Webmaster Tool Accesses the same lines which are contained in robots.txt text file.
2) What is the significance and effect of ? character after /business/ ?, /admin/ ?
Waiting for replies...............
If there is a UTF-8 character that the web page (ie google WMT) cannot display in your browser then it will show a ? instead. This can happen with some editors - eg smart-quotes and long dashes do not always show on web pages unless they are spefically UTF-8 enabled.
View your robots file within the same web browser you view WMT (exmple . com/robots.txt).
Some editors allow you to switch on "code" characters, or you could read the file in a binary editor.