Submitting to search engines monthly?

[DXL]

If I have a particular site up, should I be resubmitting its URL to Yahoo and Google once a month? I've read conflicting comments online; people suggesting you should submit a URL once and leave it be, and people who are submitting a site URL to search engines every month so that it will recrawl the site.

I've also noticed people using coding in the meta tags that ask robots to come back every few weeks, does this really work?

[jbinbpt]

Hi DXL,
I would submit a URL once and let it be. Working on inbound links will get you crawled faster.

However.... If I understand correctly, Google wants you to feed them your URLs' using Google Sitemaps. Check it out.

jb

[Import Export]

These days quality incoming links seem to do the trick...

[DXL]

So as long as I either have reciprocal links coming in from quality sites (or paid links, directory/hub links for my client sites who offer goods and services), the search engines are going to be able to pick up on these changes and index my site better without multiple submissions?

I've been looking at Google's site map info, only catch is I have to look into whether my server uses Python and to figure out some of the little details. Perhaps I read that you shouldn't use a sitemap if you have a message board or anything else password protected on your site.

[caveman]

There is no need whatsoever to resubmit to major SE's. They prefer to find your pages by crawling the Web. If the only way they can find your site is by you telling them where it is, what does that suggest? They don't have any trouble finding quality sites with good inbound links, that's for sure. And those are the sites that they want to feature.

G sitemaps are now encouraged by G and I've so far mainly only heard neutral or good things about it. We've not bothered to do it, as all the pages we want found by G are being found and spidered regularly. But soon we'll probably have a look at it.

If anything, I find it harder to keep pages away from G than to get them spidered at all. Those clever little bots can find just about anything, including pages only linked to from your hard disk. ;-)

[jbinbpt]

Perhaps I read that you shouldn't use a sitemap if you have a message board or anything else password protected on your site.

That's news to me. If a properly behaving spider hits a password, it should go onto the next link.

One thing with Google sitemaps you include what you want.

jb

[DXL]

I take that back, I meant a robots.txt file. I want to use one of those, but in one of the threads on WW someone says you shouldn't use that file if you have password protected areas. I would imagine there isn't any fear of Google, but of someone maliciously crawling your site for some reason? I still don't know how people crawl each other's sites, though.