Forum Moderators: Robert Charlton & goodroi
In these 15 months, I've done all I can think of "clean up" my site. We have hundreds of IBL from .edu/.org sites and have always been on page one of G, Y and MSN since 2000 when we began.
Actions over last year or so include:
- pulling all affiliate links
- sorting out www/non-www problem, which did exist but we've used .htaccess on that since about Apr. 05
- reduced page sizes with use of CSS and got rid of all html bloat. Now standards compliant for over a year.
- continued adding new materials and opened whole new section to site too.
- continued gaining IBL, but maybe not as much as we could have: this is one potential problem although the net already has hundreds to us - 95% organic, NOT reciprocal.
- revised structure of each page to be more logical regarding title, description and use of Hx headers.
- joined up to Sitemaps about 6 months ago and have fully verified sitemap uploaded to it.
But I still get this gnawing feeling there's something odd lying somewhere on the site that is holding us back. Late at night, my mind has wandered over things like: use of phpadsnew, use of javascript randomising script to show amazon books relevant to users, the idea that the site has just become "old and tired" as far as G is concerned.
My greatest feeling is one of powerlessness. If I search for mysite.com, G has all our pages (and many we have long since deleted) but there seems almost to be a penalty or dampener on the site...educational based search strings which used to see us in numbers 1-4 as regular as clockwork year after year (and rightfully so...) now see us come in at number 137 with 90% of the sites above us totally irrelevant AND quite a few referring to US anyway!
It's total frustration I feel at the moment and I've even toyed with the idea of just dumping the domain, moving to a new host with a new domain and starting again.
I've often told myself in the last 18 months, "it's OK, G will sort themselves out" but it ain't happened and is just getting worse. Y and MSN are basically as they were before - but don't bring in that much traffic.
G has absolutely NO obligation to send me traffic and that is not my complaint here today, but I would like the chance to rank half decently for very niche subjects which I and maybe a handful of other sites deal with, instead of seeing a plethora of cookie-cutter MFA fly-by-night sites ranking pages and pages ahead of me by simply referring to me.
Can that be right?
www/non-www problem
Have you looked into any OTHER technical issues that might be creating multiple urls for the same content? Things like:
1. "custom" 404 error pages that actually return a 200 status in the http header
2. query strings that allow various combinations of variable to point to the same pages
3. 302 redirects accidentally employed where a 301 is intended
4. https: versions of your urls being indexed
-- there are a lot of ways to get things tangled up.
5. Also, are your newly re-worked titles and meta descriptions unique and page specific?
number 1 is an interesting one. I do have a custom 404 and G sitemaps weren't happy with it when I was trying to verify my site, saying they were getting a 200 header.
I have this in the .htaccess:
ErrorDocument 404 /404.shtml
but, here's an odd thing - I've just used WebmasterWorld header check and tried to load mysite.com/asdfasdfasdf and got:
HTTP/1.1 404 Not Found
so that's baffled me. How come that's a 404 and not a 200 for the custom 404.shtml I have? :-¦
As for number 2, that could be a problem on my vB forum but I've blocked just about all of it except the bot friendly archive using robots.txt (another thing I thought to do in last year or so.)
3. I only have a 301. Here is my .htaccess - I switched from html to shtml, but that was in early 2003. I could probably get rid of the .html to .shtml redirect, but a few IBL's use it.
------------------------
RewriteEngine On
RewriteCond %{HTTP_HOST} ^mysite.com
RewriteRule (.*) [mysite.com...] [R=301,L]
RedirectMatch permanent (.*)\.html$ [mysite.com$1.shtml...]
------------------------
4. I don't use any https on my site at all.
5. I have around 2000 pages and the titles are unique. The descriptions are the same in some batches of 15-20. So, I might have:
title: Reading Activity 1
description: A nice reading activity
title: Reading Activity 2
description: A nice reading activity
could that be a problem?
or may be you are gaining traffic on the KWS which you never targeted but because of your SEO efforts you get them through?
Just to give you an idea, in my educational niche, even for the one word, cover-all KW of "widgets", I was top 10 for three years...now about 120.
For "blue widgets" which is 100% what my site is about, I was number one for three years and even now am still number two or three, but it's just about the only KW I rank well for, bar my site name.
The other thousands of KW's that used to bring in the majority of my traffic (widget care, foreign widgets, give me free widgets) have died a death and I'm on page 2-5 for all of them.
My SEO has generally consisted so far of:
- good descriptive titles using relevant KWs
- honest descriptions and keywords tags
- logical use of Hx, b, i and A tags within main text
- cutting down on html bloat. 18 months ago, every page had hundreds of <FONT> tags on it, a sign of a site made in 2000. That's all gone...all CSS now.
I've always tried to use .htaccess, DOCTYPE declaration and robots.txt correctly, my current one being:
User-agent: Mediapartners-Google*
Disallow:User-agent: msnbot
User-agent: googlebot
User-agent: scooter
User-agent: slurp
User-agent: Teoma
Disallow: /graphics/
Disallow: /cgi-bin/
Disallow: /forum/attachment.php
Disallow: /forum/calendar.php
Disallow: /forum/editpost.php
Disallow: /forum/faq.php
Disallow: /forum/login.php
Disallow: /forum/member.php
Disallow: /forum/memberlist.php
Disallow: /forum/misc.php
Disallow: /forum/newreply.php
Disallow: /forum/newthread.php
I suppose, looking at my SEO list above doesn't make great reading, does it? I've always been too busy on other stuff.
BTW, in all this time, my PR has gone from 7 to 5, which is what got me thinking about G considering my site to be a bit tired: the links have certainly slowed down in last year or two, but I just have a gut feeling there's something else to it as well.
Google cares nothing about this. Check the source code for this WebmasterWorld page and you'll see <font> tags.
RewriteEngine On
RewriteCond %{HTTP_HOST} ^mysite.com
RewriteRule (.*) http://www.example.com/$1 [R=301,L]RedirectMatch permanent (.*)\.html$ http://www.example.com$1.shtml
I am certainly not an htaccess pro, but it appears you are asking page to be redirected to shtml format, is this correct?
I would:
Post this htacess code in the Apache forum and make sure it is correct.
Use xenu link sleuth to prowl your site for broken links and fix accordingly
Make sure all links point to one url, and doublecheck that no http://www.example.com/index.html or http://www.example.com/index.php exists (if that applies)
revised structure of each page to be more logical regarding title, description and use of Hx headers.
Good, make sure the H tags are not overused.
Also do some snooping into the amount of backlinks to the sites that are now ahead of you. Is it possible they have been aggressively hunting links?
Make sure you have your site listed in the more authority directories on the Net.
Also,make sure your html sitemap is small in code size, remove images and longer loading elements from the page.
Get more inner links to pages for other keywords, which also bolsters the root page and it's keywords via popularity.
Hope this helps
<edit reason: use example.com>
[edited by: tedster at 4:24 pm (utc) on Aug. 15, 2006]
The .htaccess code, I actually got from here so I think it's fine...it certainly works fine.
I can't use Xenu on my host's server...they hate it with a passion and it's been blocked at root level. 98% of my internal links are in just a couple of include files, so I'm pretty certain I have no link problems.
All internal links are standardized and point to www.mysite.com/ as the homepage. I have even contacted a few IBL sites to add the www whenever possible: most already are pointing to www and without the index.shtml so that's not a problem either.
I use possibly 3-5 H headers per page, and most pages have 1-3, so no overuse there.
I am being more aggressive in link hunting now as that is probably the one thing I've been lax on over the last couple of years.
