Welcome to WebmasterWorld Guest from 184.108.40.206
is there such thing as html markup to tell googlebot what to crawl
no. though I always get a chuckle when i see:
'robots revisit' or 'robots all' in the metas.
and what not to?
robots.txt can help exclude 'good' robots. search this site for robots.txt; you should end up at formum 93.
If you look at Brett's robot.txt about 20 lines down you'll see:
Be very careful with this; Brett has chosen to exclude everything. Read up on it before implementing.