Changing urls has affected my rankings

[JivRey]

Hi All,

About 2 months ago, i decided to change the structure of the URL to access my website from :

http://www.example.com/index.php?pr=2182

to

http://www.example.com/products/shop_by_brand/Solid_State_Logic/XLogic_VHD/

My ranking was really good at this time, but I wanted to get higher. I followed all the Google guidelines, avoiding repetitive page titles.. The reason why we read ; A-Designs EM-Blue Module available for sale in Canada

We also have the same products for rent, so we can read «... for sale ...», «... for rent ...», and so on..

Something happened and all my products are now like outranked from Google.. I suspect everything.

1. The «for sale in Canada» in the page title
2. The underscores in the URLs
3. The length of my keywords (programmatically generated)

Does anybody have any idea what's going on?

THANKS!

[edited by: jatar_k at 4:17 pm (utc) on Aug. 28, 2008]
[edit reason] please use example.com [/edit]

[BradleyT]

301 all old pages to the new pages.
Create an XML sitemap.
Add your site to Google Webmaster Tools.
Wait 3-6 months.

Also, it sound like you could have duplicate content issues.
Is www.example.com/products/shop_by_brand/product the ONLY way to access that product? What if someone isn't shopping by brand but performs a search or is shopping by price?

[JivRey]

Instead of 301, when an old URL structure is typed, my website send a query that « transcode » the old example.com/pr=3125 to example.com/products/shop_by_brand/the_targeted_product/

1. I've created a sitemap for Google

2. I've subscribed to Google Webmaster Tools and I followed all the guidelines in there.

3. If somebody perform a search, the URL comes out as this ;
http://www.example.com/search/?search=my+search+terms
That shows a list of all the matching products, with their own links.

4. And if somebody want to change the URL look like this ;
http://www.example.com/products/shop_by_brand/Apogee/?sortProducts=2

I'm truly blind about what's going on, regarding the fact that all my changes are based on Google guidelines...

Thanks!

[JivRey]

Also, if I do test an URL against my robots.txt, here's the answer of Google

URL :
http://www.example.com/products/shop_by_brand/A-Designs/ATTY2D/

Googlebot :
Allowed
Detected as a directory; specific files may have different restrictions

It is normal if you use mod_rewrite to imitate a directory structure, can it cause problems?

[edited by: JivRey at 6:02 pm (utc) on Aug. 28, 2008]

[Funtick]

Nothing wrong with mod_rewrite, and "directory-like" text/html.
The only problem for Google: for instance, avoid *.jpg suffix in URL for text/html content; they block some URLs based on some suffix types (such as http://www.example.com/my_html_file.exe)

You disappeared from most SERPs probably because of significant changes in Title tags... see another topic.

I wouldn't suggest to completely rewrite guery part of URL... Google is picky only if it sees many different parameters in query and can't see descriptive anchor text (for instance, as an anchor text, and ?page=1 as a query) - in this case it suspects "similar content" but it does not mean "penalty" - all your pages will be indexed over time, and united into single unit "more like this"...

[edited by: tedster at 8:32 pm (utc) on Aug. 28, 2008]
[edit reason] switch to example.com - it can never be owned [/edit]

[JivRey]

More opinions everybody?

Thx a lot!

[BenjArriola]

Instead of 301, when an old URL structure is typed, my website send a query that « transcode » the old example.com/pr=3125 to example.com/products/shop_by_brand/the_targeted_product/

You still need the 301 because:

1. You want to tell search engines the page still exist but has moved to a new location. If the page still resolves the same way as two URL versions, you need only 1 so the 301 redirect is imperative. If you show a 404, more reasons to have the 301.

2. You maintain all link popularity the old pages have and will be transferred to the new URL format.

[tedster]

Here are two relevant threads from the Hot Topics area [webmasterworld.com], which is always pinned to the top of this forum's index page. They are full of ideas, experiences, and warnings for things to pay attention to when you make sitewide changes to URLs on your existing domain.

Site-wide 301 Redirects - how long for Google to index [webmasterworld.com]
Changing URL Structure - take on tasks in a logical order [webmasterworld.com]

[seoexperts]

Well I will agree with "BenjArriola',

you must 301 redirect your old pages, otherwise chances are:
* You will loose the link popularity of those pages (to get old pages PR passed to your new ones is also a good reason to 301 redirect all your changed pages to respective new ones :-) )
* May get supplemental issue if the old content is still live
* If you left them to show 404 error, Google may dislike your site as it always hates sites with lots of 404 pages.

SO 301 redirection is highly advised to you.. ! (if not already done)

And after doing everything okey from your end as most of the thing you already have done,, Please Wait and give ' G ' time to index new pages, associate their old page values to the new ones and re-analyze them for URL and any title changes.. [This may take 2 weeks to 2 months or sometimes more if your site is not a trusted one]

so wait ...wait ...wait..!

[tedster]

I'd add some extra advice that comes from an experience with a successful url change. Unfortunately this comes too late for the opening poster's situation, but someone else reading the thread might think about this approach.

Study the old urls and see which ones are bringing in entry traffic from Google search or that have strong backlinks on other domains. Those urls will do well with a 301. But often trying to 301 redirect "everything" can lead to a longer transitional period.

So except for those strong urls, a url rewrite for a large site can find quicker success by letting the rest of the urls return a 404. Google will find the rest of the new urls naturally, by crawling the new structure. The transition can be speeded up because 404 urls just get dropped from the index, but Google needs to trust-check 301 redirects, and that can take time.

I used this approach on a medium sized site and we only 301 redirected about 70 of the key urls. The only legacy url that didn't change was the domain root itself. That site only suffered wobbly Google traffic for about a week, and it didn't wobble all that much.

[Quadrille]

I can vouch for tedster's approach (or very similar) on experience, and on Star Trek principles; assimilation is much more effective if the Borg can do it in their own way, without forcing them to do it all at once. Similar rules pertain to SEs.

Where possible, I'd add that changing part of the site today, the rest in a few weeks time, also makes for a smoother process.

But when merging sites, if a 301 is to be used, making sure there's no duplicate folder issues is very useful - in other words, minor changes in advance of the major change can smooth the way.

(Ask Captain Janeway!).

[JivRey]

So you guys think I should like generate a 3000 lines (I have approx. 3000 dynamic product pages) .htaccess file with instructions like this?

Redirect 301 /index.php?pr=412 http://www.example.com/products/shop_by_brand/brand/model/

[BenjArriola]

Redirect 301 /index.php?pr=412 http://www.example.com/products/shop_by_brand/brand/model/

I am not an expert .htaccess file hacker and I suck a lot in Regex, but I think you need to escape the question mark character. I may be wrong.

[tedster]

No, I do not think you should create a monster htaccess file. As I mentioned above, pick only the key urls and redirect them. Let the rest return a 404 or 410 status.

[BradleyT]

The problem with letting all of them 404 is you get a very noisey report in GWT.

Even if it's a constant number like 110, the number GWT gives can be deceiving because sometimes it will temporarily add or remove 404's. So if you just look at the GWT and see 110, you can't be 100% confident that it's those 110 "old" 404 pages.

[potentialgeek]

I changed one site recently and only 301'd the pages I knew were getting direct traffic, but you may also want to closely monitor the 404 error file to see if you missed any. If you don't, you can lose ranking based on fewer IBLs. The IBLs that bring less traffic may actually help your ranking more (e.g., old sites with higher TrustRank). It could take a week or longer before they show up in your 404 file.

p/g

[JivRey]

Ok guys, thanks for all your solutions.

After many discussions, my boss and I decided to restore the old structure ;

http://www.example.com/index.php?pr=117

instead of

http://www.example.com/products/shop_by_brand/...etc

My question is, should I do a bulk remove using the «Remove URLs» from the Google Webmaster Tools?

Some experts told me that the chances that Google recognize my old URLs and restore my PageRank are great.

What do you guys think?

Thanks a lot for all the help!