Forum Moderators: open

Message Too Old, No Replies

http://widgets.com vs http://www.widgets.com

lost my serps

         

sanblasena

6:29 pm on Nov 15, 2003 (gmt 0)

10+ Year Member



HI,

Someone recently linked to my site from 4 other sites (without my permission) and used [widgets.com...] as the links. Now, I have gone from the front page to page 3 or 4 on all my keywords and my site is showing up in google as widgets.com (all the other site have www in front). [widgets.com...] shows a pr 0 with no backlinks.
[widgets.com...] show a pr4 with many backlinks. Any help would be appreciated.

Thanks. Patricia

jdMorgan

6:37 am on Nov 25, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



caveman,

On 99% of all servers, that is already taken care of. If you are on the 1% where it's not, you can use

 DirectoryIndex index.html 

to declare the file you want to serve when http:www.yourdomain.com/ is requested.

Jim

Powdork

6:39 am on Nov 25, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



You might want to say again what the problem is.
I have been trying to get it right but I keep getting 500-internal server error, regardless of rather i put it before or after the mod_access module. I tried it both with the symlinks option and without. I did include the space between }!
Also, when I first visit the page everything is ok. I get the error when I refresh. I'm pretty sure my browser was always returning a fresh page as long as i wasn't using the back button. i'm at work now so I can't check it.

jdMorgan

6:46 am on Nov 25, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Powdork,

500-Server Error: Hmmm... Well, flush your browser cache, check your raw error log file, and make sure you have a space before the "!" character (see msg#11 above). Jim

caveman

5:08 pm on Nov 25, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



jdMorgan,

The "DirectoryIndex index.html" - is that added to the .htaccess file, and can it be used in conjunction with the mod_rewrite already there per your suggestion?

I'm asking because I added it and still see no difference. What I mean is: I don't see "index.html" after the .com/ (If I don't see it, I tend to believe that it's not working as I intent, since hard coded links into the site that contain the "index.html" in the string will show that file name in the address bar of the browser, after the ".com/"

I would expect that if the command were working properly, links into .com/ would redirect to show .com/index.html.

Make sense?

I've gone from not worrying about this stuff to being paranoid ;-)

Synthetic

11:20 pm on Nov 25, 2003 (gmt 0)

10+ Year Member



Hi all,

I just wanted to clarify something for my own sake:

By including the rewrite code in my .htaccess file that redirects mydomain.com to www.mydomain.com, search engines will know that www.mydomain.com is what they should list?

Or is it the 301 redirect code that does the above? What happens if I use both (or can this even be done?)

bekyed

11:59 pm on Nov 25, 2003 (gmt 0)

10+ Year Member



Google are familiar and even endorse the 301 redirect so you should not have any problems at all and yes google will see this pointing from [domain.com...] to www.domain.com.

Bek

jdMorgan

12:10 am on Nov 26, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



caveman,

Add the DirectoryIndex directive to .htaccess - anywhere in there. It works fine with the 600 or so RewriteRules *I* use... :)

You *will not* see the address bar change. The effect is purely internal to the server. You request www.yourdomain/ and you should see your home page in your browser. Most commercial sites do not include 'index.html' as part of the URL, and this is what makes that work on Apache. Leaving the 'index,html' off your Web address has another big advantage - It'll save you major hassles if you later want to change your home page to index.shtml, or index.php. If you hard-code 'index.html' into all your internal links, and use it for exteral links, and then later want or need to change to using index.php... well, all your old incoming links will need to be updated, you'll need to edit every page on your site, and that is a major chore. If you use the DirectoryIndex method, you edit one line in your .htaccess file, and you're done.

If you want to prove that DirectoryIndex is or is not working, upload a different test page to your server, and put its filename in the DirectoryIndex directive. Then request www.yourdomain/ and you should be served the new file instead of index.html.

Don't be paranoid, be thorough... Always test your code. :)

Synthetic,

I don't understand your question, but you tell the search engines to use your 'main' domain by telling them that your 'alternative' domain has been moved permanently to the 'main' domain. The code above is used to tell Apache to return a 301-Moved Permanently server response code plus the URL of the 'main' domain in response to any request to any of your site's 'alternate' domains. See RFC2616 [w3.org] - Server Response Codes. Test your redirects here [webmasterworld.com].

Jim

steveb

1:41 am on Nov 26, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Powdork, if you have frontpage server extensions installed you will get a 500 error and can't do a 301 redirect.

caveman

2:25 am on Nov 26, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



jdMorgan, good advice. Raised yet another question though... :-) Glad you're a patient guy!

The question is, does using the simpler .com/ instead of the .com/index.html imply that we should do absolute path names?

We've used relative ones, but when linking back to the homepage, the software we use gives the /index.html result, not just .com/

To have it as clean as possible, I guess this would mean creating absolute paths back to the homepage, just using .com/, so as to avoid the index.html filename

Thoughts? I know absoute is supposed to be better for SEO but it is a bit more of a pain in dev work... at least with our site software. Think I'm answering my own question though.

jdMorgan

2:52 am on Nov 26, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



caveman,

Assuming you're using DirectoryIndex index.html, here's a perfectly-valid "relative" link to your home page:


<a href="/">Widget Master home page</a>

Jim

pageoneresults

2:57 am on Nov 26, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Thoughts? I know absolute is supposed to be better for SEO but it is a bit more of a pain in dev work... at least with our site software. Think I'm answering my own question though.

jdMorgan brought up an excellent point about the underlying technology changing. Years ago I used to provide a full URI (.com/index.htm) as I thought it was the right thing to do. I learned the hard way when asp came into the equation. From that point forward, I've never long linked to any index page. I now practice proper URI structure...

www.example.com/

When providing outbound links, you should do the same. Your link partners technology could change too. It is recommended that you always include the trailing forward slash when linking to index pages.

Keep the URI paths short. Use the same format for linking to sub-directory index pages...

www.example.com/sub/
www.example.com/sub/sub/

caveman

4:10 am on Nov 26, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I love this place. Thanks all - very much appreciated!

Powdork

5:49 am on Nov 26, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Powdork, if you have frontpage server extensions installed you will get a 500 error and can't do a 301 redirect.
Ding Ding Ding, you are correct sir. Maybe. My wife actually did get the front page extensions enabled on this site when she uploaded a sub-directory of a friends' wedding page. She was a bit scared to tell me she had killed my site, which was the outcome for a couple days. Did you just guess this or were you able to tell somehow. My host said they had removed them but I still notice some differences in my control panel. Would my mod_access part work with the FPextensions enabled? This was all ten months ago so the downtime is not a cause of my current problems.

steveb

6:31 am on Nov 26, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



"Did you just guess this or were you able to tell somehow."

A few threads on this during Dominic. Bottom line is, nobody knows of a way to make a 301 work with frontpage extensions installed.

====

Neither mod rewrite or 301 work. Don't know about anything else, but I suspect the answer is no, with any alteration causing a server error.

[edited by: steveb at 8:01 am (utc) on Nov. 26, 2003]

Powdork

6:42 am on Nov 26, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



But is there a way I can tell if they (or any traces) are still there?

<added>

Would my mod_access part work with the FPextensions enabled?
I should have said
Would my mod_access part work if somehow the FPextensions were still enabled?
Sorry, not familiar with frontpage.

wanna_learn

11:36 am on Nov 27, 2003 (gmt 0)

10+ Year Member



I am a newbie to Redirects,

My Site disappeared from Google all of sudden (15 DAYS before Florida update).
BUT Before vanishing completely from Google index, the pages were shown at Google without Title and Description , but could not be found with the KWS..it used to Top.

I just noticed
Searching allinurl:www.site.com gives No results found
Searching allinurl:site.com gives 2 Results , and listed like :: site.com/xyz.htm - 18k - Supplemental Result

Any hint whats this Supplemental Result? and does my problem of Slow Death of site pertaing to redirect or smth.?

steveb

11:40 am on Nov 27, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



wanna learn, that is common for pages that don't get crawled, stuff without links going to it (or at least no links Google knows about).

wanna_learn

12:32 pm on Nov 27, 2003 (gmt 0)

10+ Year Member



steveb
just a note
"me and many like me, are still wondering about the probable reason of Slow Death at Google"

seoArt

3:27 pm on Nov 27, 2003 (gmt 0)

10+ Year Member



steveb

301 redirect works on my site with frontpage extensions. My server guy deleted the frontpage additions in the .htaccess file and put the 301 line in there.

Everything still works fine to the best of my knowledge (forms, and uploading changes through frontpage). He said the only problem might be if I was using frontpage to manage site access which I don't do.

I don't know much about server technology so if you know of any other problems associated with this let me know.

steveb

9:07 pm on Nov 27, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



You can make a 301 work with frontpag extensions... until you publish again with frontpage, then the site will return a 500 error. If you have stagnant site there isn't a problem. If you are in fact uploading stuff on a daily basis, and the site and 301 are both working, then your web guy has figured out something nobody else has been able to.

seoArt

5:03 am on Nov 28, 2003 (gmt 0)

10+ Year Member



Yes, the 301 still works, and I've been updating the site. I've duplicated the 301 on other sites as well.

This is what we did. We have duplicate index files in each directory one is index.htm, the other .html (we previously had only one but copied it for the purpose of the redirect)

Web guy deleted all frontpage entries from the .htaccess file and added this redirect:

redirect 301 /index.html [widgets.net...]

The "www" url was not the one we got all our links on yet there are a few sites out there linking to us with that, and it shows duplicate content because it points to the same files as our plain url. Now whenever a browser requests www.widgets.net it takes them to widgets.net/index.htm, hence all PR should transfer and spiders will go to that url.

Now, I know that this doesn't redirect for the rest of the site's pages, but I'm not having problems with those not showing up in SERP's. I'm hoping that eventually any internal pages indexed with the "www" will get dropped, and the spider will get everything from the plain URL (and the site gets spidered frequently so this should not be a problem).

Do you think there will be any problem with this? It seems to be working great so far.

Hissingsid

9:58 am on Nov 28, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hi,

My host for one site has the server set up so that only www.widgets.com is served. If I try to navigate to [widgets.com...] I get a server not found message.

I'm not clear from this thread if this is a good or a bad thing.

Should I ask admin to add the sans www version to httpd.conf and redirect it to www.widgets.com or just leave the server as it is?

I would very much appreciate an answer.

Best wishes

Sid

steveb

11:16 am on Nov 28, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



seoArt what does your htaccess file show now? Did the frontpage information get regenerated, and is now living nicely with the 301 line, or what?

seoArt

5:20 pm on Nov 28, 2003 (gmt 0)

10+ Year Member



Hissingsid

What you describe is not a problem with the search engines. The problem we are discussing is the SE's seeing the "www" and non "www" content as duplicate content and banning or penalizing a site for it.

You may want to do a redirect to assist with user accessibility, but otherwise your ok.

Steveb, only the 301 line is in the .htaccess file (as it's written in my earlier post). Frontpage did not add it's info back to the file upon re-publishing. I use Frontpage XP.

Craig_F

6:24 pm on Nov 28, 2003 (gmt 0)

10+ Year Member



I was showing both the www version and non www version of my site in Google so, following this thread I added this to my htaccess file:

RewriteEngine on
RewriteCond %{HTTP_HOST}!^www\.domain\.com
rewriterule (.*) [domain.com...] [R=permanent,L]

I also tried this version:

Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST}!^www\.domain\.com
rewriterule (.*) [domain.com...] [R=permanent,L]

Both look good using the server header check, but I get a 404 if I try to surf to [domain.com...]

Any ideas what's wrong?

Bowdii

6:36 pm on Nov 28, 2003 (gmt 0)

10+ Year Member



This is what I do for an apache server - in the /usr/local/etc/apache/httpd.conf file

RewriteCond %{SERVER_NAME} [widgets.com...] [NC]
RewriteCond %{REQUEST_URI} ^/(.*)$
RewriteRule ^/(.*)$ [widgets.com...] [R]

jdMorgan

7:20 pm on Nov 28, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Craig_F,

The trailing slash problem is normally handled automatically by Apache mod_dir [httpd.apache.org]. Check with your host to make sure it is enabled, and ask why it's not working.

If you cannot get mod_dir to function, there is a discussion of this problem in the Apache URL Rewriting Guide [httpd.apache.org], where a couple of solutions are presented. However, one may not work for you, and the other is inefficient.

Also, you could use this code in .htaccess to fix a missing slash on the top-level directory only:


RewriteCond %{REQUEST_URI} ^$
RewriteRule .* / [R=301,L]

A more general, but still restricted solution in .htaccess might be:


RewriteCond %{REQUEST_URI} !\.
RewriteCond %{REQUEST_URI} !/$
RewriteRule ^(.*)$ /$1 [R=301,L]

This only works if you do not use directory names containing spaces (doing so is legal under *nix). It basically says, "If the requested URI does not contain a period and does not end with a slash, then assume that it is a directory request, and append a slash." Because directory names including periods are legal under *nix, this solution is not universal.

In both examples above, an external redirect is required. Otherwise, you'll have problems with any pages using relative links to include images, scripts, CSS, etc.

I strongly recommend asking your host why mod_dir is not working properly on your server.

I've never had the problem you describe so I can't test this code, but it might help.

Jim

Craig_F

7:56 pm on Nov 28, 2003 (gmt 0)

10+ Year Member



The trailing slash problem is normally handled automatically by Apache mod_dir. Check with your host to make sure it is enabled, and ask why it's not working.

Thanks for the help Jim, but I'm still new to all this server side stuff so I don't understand what you mean by trailing slash problem. I'm having a problem getting the non www version of my domain to redirect to the www version...I'm sure you are telling me what I need to know, but I dont get how they relate.

jdMorgan

10:18 pm on Nov 28, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



> Both look good using the server header check, but I get a 404 if I try to surf to http://domain.com

Technically, http:domain.com is an invalid URL - the trailing slash is missing. Most of the time, a server will fix this for you and issue a 301 redirect to the corrected URL. I thought you were saying that did not work.

Jim

Craig_F

11:01 pm on Nov 28, 2003 (gmt 0)

10+ Year Member



Most of the time, a server will fix this for you and issue a 301 redirect to the corrected URL. I thought you were saying that did not work.

My server does do that. Both [domain.com...] and [domain.com...] give me a 404. So, it's not the trailing slash. I'm saying the redirect on the non "www" domain seems to be causing the problem.

This 60 message thread spans 2 pages: 60