Welcome to WebmasterWorld Guest from 54.196.232.162

Forum Moderators: goodroi

Message Too Old, No Replies

Blocking parts of forum

but not others

     
1:18 pm on Mar 22, 2008 (gmt 0)

Junior Member

10+ Year Member

joined:May 13, 2003
posts: 151
votes: 0


Hi,

I have a well-indexed forum, but have noticed that G is picking up all the links to "report this thread", which are useless and also lead to some 404 errors.

I want to block all urls that start
/foros/?func=report

But allow any other pages that are under
/foros/

If I get this wrong, my forums will drop out of G's index, so I need to make sure I do this carefully! Could anyone advise me on the correct disallow statement that would SPECIFICALLY block the "func=report" urls ONLY?

Many thanks!

5:50 pm on May 12, 2008 (gmt 0)

Full Member

5+ Year Member

joined:Sept 11, 2007
posts: 303
votes: 0


one more solution that you can have that to redirect the 404 urls which is in your point of view are unnecessary urls to your favorite urls. Because stoping 404 error by using robots.txt file is not a good idea.

I advice you to please use 301 redirect on all 404 errors urls..

thanks,

bilal

1:00 pm on May 13, 2008 (gmt 0)

Administrator from US 

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:June 21, 2004
posts:3080
votes: 67


To block googlebot from accessing those pages add this line to the robots.txt file:
Disallow: /*func=report$

bilalseo does make a good point. it is better to identify 404 errors and if possible 301 redirect them to the new file. if there is no new file location and the information does not exist on your site you should serve a 404 error page with helpful links for the users. this will help make it clear to the users what is going on and what options are available to them.

5:40 pm on May 16, 2008 (gmt 0)

Full Member

5+ Year Member

joined:Sept 11, 2007
posts: 303
votes: 0


thanks goodroi:)
5:43 pm on May 16, 2008 (gmt 0)

Full Member

5+ Year Member

joined:Sept 11, 2007
posts:303
votes: 0


Goodroi I want to ask one thing, that if the 404 pages remains cached and got maturity then what is the right method to remove them from cache and what steps should I take to overcome. I read before in google webmaster area about removal or permanent removal of directory, pages and urls from the google. But that wasn't entertained me as I could be.

thanks,

bilal

5:52 pm on May 16, 2008 (gmt 0)

Moderator from US 

WebmasterWorld Administrator martinibuster is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 13, 2002
posts:13972
votes: 123


You can also find where that link is defined within the forum software and add the nofollow to it. I added the nofollow to member messaging links (PM Member). Who needs bots indexing those, right?
5:59 pm on May 16, 2008 (gmt 0)

Senior Member

joined:Jan 27, 2003
posts:2534
votes: 0


Disallow: /*func=report$

Looking at the OP, the request was to block "urls that start /foros/?func=report" so I assume we need to drop the $ at least. And doesn't Google allow question marks in disallow? So, the simplest would be:

Disallow: /foros/?func=report

Sorry if I'm being excessively picky ;)

6:29 pm on May 16, 2008 (gmt 0)

Junior Member

10+ Year Member

joined:May 13, 2003
posts:151
votes: 0


Ah martinibuster, that's the answer I was looking for! Thanks :-)
7:12 pm on May 16, 2008 (gmt 0)

Full Member

5+ Year Member

joined:Sept 11, 2007
posts:303
votes: 0


yes nofollow is another solutions that you might take. but in case of 404, there must be 301... I suggest :)
7:13 pm on May 16, 2008 (gmt 0)

Full Member

5+ Year Member

joined:Sept 11, 2007
posts:303
votes: 0


until you get 200 :)
8:24 pm on July 28, 2008 (gmt 0)

Preferred Member

5+ Year Member

joined:June 8, 2006
posts:556
votes: 0


Disallow: /*func=report

That would be the correct way. You don't need the $ at the end...

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members