Does fresh content actually help with freshbot?

how long does it take to get it to crawl the new pages

[bluecorr]

I have been doing some thinking. Is it really worth adding fresh content pages (articles) like twice, three times a week since freshbot doesn't go to them? (although one of the few pages that it does deepfresh have direct links to the fresh pages). Or is it better to add 1-2 pages, wait for the deepcrawl and let googlebot see the new content and tell freshbot to come check it out? The point is how to make freshbot visit me more often and index new pages quickly.

I have one more question. I have the root site /, and two subsites /dir1/, /dir2/ which are all 3 completely different sites. The root / gets hit by freshbot on a few pages (like mentioned above) every couple of days, /dir2/ doesn't get any hits and it's not indexed while /dir1/ gets hit every day by freshbot on ALL the pages.

It's true that /dir1/ has no less than 8 links in dmoz BUT when the directory structure of the site changed the links in dmoz became dead. Other than those links the /dir1/ doesn't have any more backlinks. I've submitted the new urls for dmoz and I'm waiting for the url update (it didn't happen just yet).

Can someone enlighten me? Why does /dir1/ have preferential treatment from freshbot compared to / and /dir2/. Both /dir1/ and /dir2/ have been uploaded at the same time (before the beginning of the december crawl) and are equally accessible from the same page within the root site.

The /dir2/ site has even quality inbound links in pages freshly indexed by Google and both / and /dir2/ have had fresh pages on a regular basis compared to /dir1/.

I just don't get it. Google works in mysterious ways :)

[Duke_of_Url]

Not sure if this answers your question, but I always try to add content regularly - eg add something new say 3 times a week (mines an evenings maintained hobby site) rather than one large update once a week. Eg this week I added a page on Wednesday evening, it was spidered Thurs and is in and ranking well today (Fri). Both the change to my homepage and the new page itself are in the index. Once it picks up that content changes regularly, the freshbot tends to do its thing. Even if you don't have anything new to add in, a few tweaks to an existing page (or preferably several) will keep fresh happy imho.

DoU

[bluecorr]

that's what I do at the moment but the problem is freshbot doesn't go to them so how can it get in the habit of checking for fresh content if it doesn't hite those pages?

[steveb]

The easiest way is to link new pages to a page freshbot hits all the time, like a what's new page. You don't have to leave links there forever, just till the next time you get deepcrawled. Your site may not lend itslef to a what's new page, but the basic point is you have to have links from pages freshbot likes to visit... ones that semi-constantly are fresh.

[Bronte]

I think there's a lot of luck involved.

Have you got several links going to the new pages? I add a couple of pages a week, and make sure there are at least two links on already-indexed pages to each one (plus an extra link on my sitemap page). Everything is in the root directory for simplicity. Freshbot seems to love it and I get a good crawling most days.

Now I'm worried that if I slow down the addition of new content, Freshbot will get bored and go away for good. I get twitchy if I haven't added content for 3 days or more...

[bluecorr]

Yes, all new pages are linked directly from the page freshbot likes to hit most. I add one page every 3 days or so. Yet my logs show they are not hit by freshbot. I didn't do anything dodgy to the pages. Is it just a matter of patience?

[steveb]

I also use the add url form. I have no clue whether the links or the url form bring freshbot, but it comes, so I use both.