Welcome to WebmasterWorld Guest from 22.214.171.124
Forum Moderators: goodroi
Robot.txt also known as the Robots Exclusion Protocol and function as a request that specified robots ignore specified files or directories in their search.
This will allows all robots to visit all files because the wildcard "*" specifies all robots:
This will keeps all robots out.