Forum Moderators: phranque

Message Too Old, No Replies

htaccess causing home page not to show in search engines

home page , 404 error

         

anshu

6:11 am on Mar 9, 2008 (gmt 0)

10+ Year Member



Hello,
I am running a site and we are not able to see the home page in the google. I think it may be problem of .htaccess or else in programming
If we see the site in web browser its been easily shown but not by bots.. Please help me what i should do.
I am pasting my .htaccess below

#####################################################

## Can be commented out if causes errors, see notes above.
Options +FollowSymLinks
#
# mod_rewrite in use

RewriteEngine On

# Uncomment following line if your webserver's URL
# is not directly related to physical file paths.
# Update Your Joomla! Directory (just / for root)

# RewriteBase /

########## Begin Standard SEF Section
## ALL (RewriteCond) lines in this section are only required if you actually
## have directories named 'content' or 'component' on your server
## If you do not have directories with these names, comment them out.
#
#RewriteCond %{REQUEST_FILENAME} !-f
#RewriteCond %{REQUEST_FILENAME} !-d
#RewriteCond %{REQUEST_URI} ^(/component/option,com) [NC,OR] ##optional - see notes##
#RewriteCond %{REQUEST_URI} (/¦\.htm¦\.php¦\.html¦/[^.]*)$ [NC]
#RewriteRule ^(content/¦component/) index.php
#
########## End Standard SEF Section

########## Begin 3rd Party or Core SEF Section
#
#RewriteCond %{REQUEST_URI} ^(/component/option,com) [NC,OR] ##optional - see notes##
RewriteCond %{REQUEST_URI} (/¦\.htm¦\.php¦\.html¦/[^.]*)$ [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule (.*) index.php
#
########## End 3rd Party or Core SEF Section

########## Begin - Rewrite rules to block out some common exploits
## If you experience problems on your site block out the operations listed below
## This attempts to block the most common type of exploit `attempts` to Joomla!
#
# Block out any script trying to set a mosConfig value through the URL
RewriteCond %{QUERY_STRING} mosConfig_[a-zA-Z_]{1,21}(=¦\%3D) [OR]
# Block out any script trying to base64_encode crap to send via URL
RewriteCond %{QUERY_STRING} base64_encode.*\(.*\) [OR]
# Block out any script that includes a <script> tag in URL
RewriteCond %{QUERY_STRING} (\<¦%3C).*script.*(\>¦%3E) [NC,OR]
# Block out any script trying to set a PHP GLOBALS variable via URL
RewriteCond %{QUERY_STRING} GLOBALS(=¦\[¦\%[0-9A-Z]{0,2}) [OR]
# Block out any script trying to modify a _REQUEST variable via URL
RewriteCond %{QUERY_STRING} _REQUEST(=¦\[¦\%[0-9A-Z]{0,2})
# Send all blocked request to homepage with 403 Forbidden error!
RewriteRule ^(.*)$ index.php [F,L]
#
########## End - Rewrite rules to block out some common exploits
-------------------------------

[edited by: jatar_k at 12:44 pm (utc) on Mar. 9, 2008]

[edited by: jdMorgan at 8:38 pm (utc) on May 23, 2008]
[edit reason] no urls thanks [/edit]

wilderness

7:12 pm on Mar 9, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Not sure why this was moved over to SSID?

In any event, these entire lines were copied and pasted from another website, under the guise of both requiring path adjustments and implementing a specific task.

Best thing you may do is emliminate these lines completely from you htaccess and then provide an explantion and some scenario examples of what your attempting to accomplish.

Don

gergoe

8:31 pm on Mar 9, 2008 (gmt 0)

10+ Year Member



This is the default .htaccess provided with Joomla, it makes possible to have 'search engine friendly' urls.

However you need to know that a wrongly configured 404sef module in Joomla can cause penalties in search engines, because the directives in this file rewrites anything and everything to index.php, thus whatever a spider tries to load from your website, it will get a content for that, IF the 404sef is not sending a 404 HTTP status code back. Check your website with a header check tool like LiveHTTPHeaders for Mozilla Firefox, to see what you get back if you request something which does not exists.

This is just a theory, why your website does not not appear in search engines, did you tried searching for your domain name by the way?

anshu

10:10 pm on Mar 9, 2008 (gmt 0)

10+ Year Member



<deleted>

Please do not post URLs. See Terms of Service [webmasterworld.com].

Thanks,
Jim

[edited by: jdMorgan at 12:45 am (utc) on Mar. 10, 2008]

jason1989

8:20 pm on May 23, 2008 (gmt 0)

10+ Year Member



Dear Gergoe,

It is in regards to the .htaccess file for joomla, I just don't have a grip on joomla's rewrite rule:
...
RewriteRule ^(.*)$ index.php [F,L]

It seems that this rule ALWAYS rewrites to index.php and does not add parameters, for example, index.php?url=weblinks.html

If there were parameters added, example index.php?url=weblinks.html, I would understand how joomla knows what URL it needs to display. Receptional_Andy tried explaining in a previous message but I am still having problems and still confused why it is not working.

My #1 question: Can you please tell me how joomla knows what page you want with a simple call to index.php with no parameters?

Thank you.

P.S. I am using IIS and implemented the rule above and I ALWAYS go back to the home page index.php

Please note that GET (not post) is used when navigating pages.

jdMorgan

8:42 pm on May 23, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



The Joomla code can read various server parameters, including the originally-requested URL and the query string attached to that URL.

Be aware that IIS and Apache are different servers. Apache uses mod_rewrite, while IIS uses ISAPI Rewrite. These are subtly different, and the code above may need to be re-written for use on IIS.

Jim

jason1989

11:05 pm on May 23, 2008 (gmt 0)

10+ Year Member



Jim:

Thank you for your response. So if I'm at my joomla homepage:
[127.0.0.1...]

And I click on any one of my joomla links such as:
[127.0.0.1...]

Behind the scenes, the following is rewritten:
[127.0.0.1...]

...yet joomla knows what link I clicked on (...weblinks.html). It then does the applicable lookups and returns the appropriate page?

Is this accurate?

Also, I experimented to see if the client browser's URL at the address bar (not the rewrite URL) was stored in the HTTP server variables and did not find it in there.

Is there a set of client variables or how does joomla know what the URL in the browser's address bar is?

Thanks Again

jason1989

11:25 pm on May 23, 2008 (gmt 0)

10+ Year Member



Jim:

When I test for REQUEST_URI on IIS, it returns nothing. Then I found this info here (I don't think were supposed to use URLs on this forum so I snipped from another webpage):

Ever since moving NeoSmart Technologies to a Windows server, we've had one headache after the other dealing with various PHP scripts that included references to $_SERVER["REQUEST_URI"], and unfortunately, none of the "workarounds" or hacks provided on the internet work. Some of them are for Apache on Windows, and others work most of the time, but break the minute you introduce url rewriting or launch scripts as the default files in a directory.

I'm starting to think that this may be the cause of my nightmares. I'm guessing that joomla URL rewriting relies on REQUEST_URI and since I'm on IIS, I get nothing but a blank. Do you think this is what is happening in my case?

Best Regards,

Jason

jdMorgan

11:43 pm on May 23, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I really don't know. I'm not either a Joomla or an IIS user...

I think you'd probably do better with questions pertaining to this problem on a Joomla-focused forum (much as I'm proud of this one...) :)

It may even be the case that you can't use Joomla with the SEF plug-in on IIS -- But I really don't know.

Jim

jason1989

6:56 pm on May 24, 2008 (gmt 0)

10+ Year Member



Jim,

Thanks for all your help and would like to point out a couple of things just as an FYI that cost me sweat and tears. I've done a bit of research and testing, and am now totally convinced that REQUEST_URI behaves totally different under Apache and IIS.

If an application relies on REQUEST_URI and you implement URL rewriting like I did, you very well may end up with a problem like mine.

There are workarounds on the net, but after reviewing them, they seem bogus. I did find a workaround provided by Helicon ISAPI_Rewrite but it involves dependence on both ISAPI Rewrite and the following mod:

<?php
if (isset($_SERVER['HTTP_X_REWRITE_URL']))
{
$_SERVER['REQUEST_URI'] = $_SERVER['HTTP_X_REWRITE_URL'];
}
?>

From the looks of this, their filter creates a server var (identical to Apache's REQUEST_URI), the value then being reassigned to REQUEST_URI.

Best Regards,

Jason

gergoe

11:16 pm on May 24, 2008 (gmt 0)

10+ Year Member



Joomla indeed use the php's $_SERVER['REQUEST_URI'] variable, and if that's not set under IIS (which seems to be the case - with version 5 at least), then you either do not use 404 sef extension, or find a way around.

If solutions provided by search result does not lead to any solution, you can try to find a way yourself, and as a start add the

var_dump($_SERVER);
line to the top of the joomla's index.php file to see what variables are set with which value, using other elements of the $_SERVER array might do the trick as well. But as I don't have IIS and PHP running together, I can only give you hints about this.

Besides running Apache and IIS next to each other is not difficult either, you might want to consider this as an option too (if you're running your own system).

jason1989

2:50 am on May 25, 2008 (gmt 0)

10+ Year Member



Hi Gergoe,

Not only with version 5 but with IIS 6 too. This is apparently by design and not a bug. I sure wish that feature was available...

Anyway, I very well may end up subscribing to a shared Unix hosting plan. I'm totally Unix ignorant, but I guess something with a control panel will make my joomla installation easy I suppose.

There are different options such as Fedora, CentOS, Debian and FreeBSD. Should I focus on a specific Unix flavor?

g1smd

6:05 pm on May 25, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Anything that can run Apache will probably be better than using any flavour of IIS.