Welcome to WebmasterWorld Guest from 184.108.40.206
Forum Moderators: goodroi
Robot.txt also known as the Robots Exclusion Protocol and function as a request that specified robots ignore specified files or directories in their search.
This will allows all robots to visit all files because the wildcard "*" specifies all robots:
This will keeps all robots out.