Just wondering how often search engines(yahoo, google and msn) update themselves(links)?
elektra
6:52 pm on Jul 29, 2004 (gmt 0)
I get daily fresh crawls from all of them. Both Google and MSN do monthly deep crawls. I am not sure about Yahoo as my site was only indexed by Yahoo last week. Perhaps other more experienced webmaster can shed light on this.
mthorpe
7:06 pm on Jul 29, 2004 (gmt 0)
do you have a meta tag for robots to come back and crawl on a regular basis? i just have a index,follow meta tag.
bhartzer
7:12 pm on Jul 29, 2004 (gmt 0)
I get crawled/updated every day not because of a meta tag but because I update my site's content every day.
elektra
12:43 am on Jul 30, 2004 (gmt 0)
No I don't. They come back everyday, because my data is updated most days
elektra
12:46 am on Jul 30, 2004 (gmt 0)
I have another website which hasn't been updated in 6 months and the only crwler that comes around most days is MSN. Google only shows up about once a month
