Pages not being crawled!

crawling pages

[curly_clare]

Hi All,

A small problem.....

As my clients are not willing to opt fot the new Overture Site Match it is up to me to get the pages crawled for free.

Although some of the pages have been crawled and appear high in the SERPS it is apparent that other pages (on the same directory level) are not.

I have added sitemaps and crosssed linked each page but they still do not get indexed.

Any ideas how to 'persuade' the SE's to crawl the additional pages?

Many thanks in advance, hope you all have a wonderful weekend.

[jcoronella]

Assuming you have enough PR to go around, you can just create a "What's New" link on your home page and link to these sites. Presumably, they are already worked into the site map, and the site is not too large for the amount of PR it has.

There is a "Handy Dandy" guide in the Supporters Forum [webmasterworld.com] that ThomasB posted that gives a great breakdown of how much PR you need to get X pages crawled. It's a rough estimate, but a good approximation for a site with evenly distributed PR.

[webnewton]

do some bloging for these pages you want to be crawled with the sites/forums crawled ever 24-36 hours.

[manwah]

If a page with PR 5 or above offers a link to your client's site's homepage, it helps to fasten the process.

[Expert]

Indexing of your pages won't mean that they will appear in SERPs. Google may index you page - but it is probably that it won't be shown in search results for several weeks...

[curly_clare]

Thanks all,

I identify whether a page is indexed by searching for the url (i.e. www.mydomain.com/specificpage.htm)

Thats right isn't it!

[graywolf]

I have a much easier time getting pages in any search engine when I get multiple inbound links from different sites that are already in that engine index.

[curly_clare]

Cheers graywolf,

I'd love to hear how you are so succesful in getting those links..do you know something we don't?

To me, link building is a very time consuming task and often a very unsucessful one!

[graywolf]

I'm nowhere near the link building casanova martinibuster is, but almost all of my linkbuilding tricks I learned here.

I use the searches mentioned in this thread on Y! and G

[webmasterworld.com...]

In fact I'm so lazy I wrote ASP script where I just put the keywords in and it generates the links to the pages for me. For the sites with form pages I fill those out, for the websites that don't I use methods mentioned here.

[webmasterworld.com...]
[webmasterworld.com...]

Thats should be enough to get you started.