Forum Moderators: open
Working with a content-rich website that generates its pages dynamically using ColdFusion, that despite having more than 40,000 pages indexed on Google for their .com site after moving from .co.uk, only has 100 or so indexed on Yahoo! for either. Looking at the data, it looks like Yahoo! Slurp is coming in probably via inbound links but then only visiting one or two pages. Google also comes frequently but bounces all around the site. A couple of projects are under consideration to address this (including sitemap), but before they proceed I thought it might be a good idea to ask you all as I'm sure some of you have been through this before.
Any ideas on how to improve the number of pages indexed by Yahoo! and MSN for a dynamic ColdFusion site like this?
Kind regards,
Lionzshare
[edited by: WebGuerrilla at 4:56 pm (utc) on Oct. 12, 2004]
[edit reason] Removed specifics [/edit]
Mozilla Slurp!
Yet Yahoo had caches of many of my pages up until a few days ago.
What exactly does the entry look like in the log. Even though I no longer appear in the Yahoo index for www.mycompany.com, I see:
7: 7: Mozilla
3: 3: Mozilla/1
in my logs again today. Is that Yahoo?
Keeping in mind that you don't want to cross link, one great way to get listed in Yahoo! for free is to get links from sites that are already in Yahoo!.
You mentioned Site Match in your post and I thought it might be helpful to provide a link to the various offerings.
[search.yahoo.com...]
Thanks,
Yahoo! Mike
Mozilla/5.0 (compatible; Yahoo! Slurp; [help.yahoo.com...] is how Yahoo!'s spider shows up on our website.
Kind regards,
Lionzshare
That was Sitemap not Site Match.
The site has been listed in the Yahoo! Directory for a long time, but I will look at building links from others in the relevant area of the directory.
Anybody have any ideas on what about the site might be causing the Yahoo! Slurp spider to only visit one or two pages at a go (it usually starts on different individual property pages)?
Kind regards,
Lionzshare
I see you removed my links, but they were a crucial part of my question (I wanted people to see the URL structure in case people saw any problems in the URL formation).
I have reposted them again in this reply with XXX in front of them so they don't show up as links (I'm assuming that is your objection and why they were removed).
So, here are examples of the URL structure in case that influences anybody's view on the Yahoo! Slurp issue that is occuring:
XXXhttp://www.holiday-rentals.com/index.cfm/fa/owners/tgt/adv_brochure
XXXhttp://www.holiday-rentals.com/index.cfm/fa/find.aquery/reg_refno/1/igolf/l
XXXhttp://www.holiday-rentals.com/index.cfm/fa/find.aquery/reg_refno/5/urvs/89,116
Hope this is OK, but it gives people an idea of what the dynamic URLs look like.
Kind regards,
Lionzshare
Well I am definitely puzzled as I have never seen Slurp even mentioned in my logs for either of my sites and yet caches from both sites have appeared in Yahoo!
This is what shows up today:
12: 12: Mozilla
9: 9: Mozilla/1
My situation hasn't changed. I show up for my old site:
[myisp.com...]
but not my new site:
[mycompany.com...]
The strange thing is that I have a "noindex,follow" tag on some of the pages that are showing up
Yahoo_Mike HELP!
www.example.com/index.cfm/fa/owners/tgt/adv_brochure
www.example.com/index.cfm/fa/find.aquery/reg_refno/1/igolf/l
www.example.com/index.cfm/fa/find.aquery/reg_refno/5/urvs/89,116
As for indexing, how long has the site been up?
The .co.uk site has been up since 1996 and has about 80,000 pages indexed in Google, but only 200 in Yahoo! and MSN.
Kind regards,
Lionzshare
What level is this site on? e.g. is it a small shopping site or is it a huge news site.
Also, how many pages total are there? (i.e. Google has 80,000 out of how many?)
