Spiders

attracting

[neon9634]

just wondering if there is a robots.txt file that attracts spiders to add your page rather then excludes?

[Sanenet]

Sadly, the spider can't read robots.txt UNTIL it's visited your page. Ah hem.

[neon9634]

mmm

[Oaf357]

The best way to draw spiders/crawlers to your site is to get links from other sites.

A robots.txt file will never attract a spider to your site.

[Palehorse]

Make a tiny little .gif of a fly, and put it at the very bottom of your root directory. That's sure to attract any self respecting spider. :)

[traffik daddy]

Like stated, get some good inbound links with higher PR. Higher PR sites are spidered more frequently and your bound to get spidered more.

Also, try updating your home page after it visits.

Terry

[brucec]

traffik daddy, you said to update the page after the spider visits. How do you know when? I always wondered.

[traffik daddy]

I always use a unique word in my content and then every now and then I run a search in Google to see if my query shows in the SERPS after an update, this is just one technique.

It also depends on which tracking system you use. Some stats you that Googlebot has visited on a certain day.

Terry