i've been a bit thrown with a dynamically written robots.txt file
i've rewritten robots.txt to robotstxtprocessingfile.php
and it all works except when i view robots.txt in the browser where all the directives are in one long line, however if i view source they are on seperate lines as expected.
Code in the file:
echo "User-agent: *\r\n";
echo "Allow: yadda\r\n";
echo "Disallow: nothere\r\n";
exit;
As it appears in the browser:
User-agent: * Allow: yadda Disallow: nothere
As it appears in view source
User-agent: *
Allow: yadda
Disallow: nothere
any thoughts? i'm stumped? thanks!