Welcome to WebmasterWorld Guest from 54.158.81.119

Forum Moderators: rogerd & travelin cat

Message Too Old, No Replies

HELP! How To Block A Page From SE's & Public View

blog pages, block download pages,

     
9:41 pm on Dec 7, 2012 (gmt 0)

Junior Member

5+ Year Member

joined:July 6, 2009
posts: 99
votes: 0


Hello,

I need to know how to block a page on a Wordpress site where I have a download link from search engine indexing and access to the public?

This is a .htm page created in Cpanel that is only meant to be seen by subscribers to my list because it has the download link to a bonus for sign up.

I know about the "no index no follow head tag", but I have seen this ignored by SE's and other crawlers at times.

The url of this page will be sent in auto responders to subscribers and so they should be able to access it, BUT no one else should have access.

How can I do this? (keep in mind my tech knowledge is limited)

Thanks
3:19 am on Dec 8, 2012 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:13256
votes: 360


Preliminary questions:

Do you care if other random people (including search-engine robots) know that the page exists, so long as they can't physically get there?

Have you ever set foot in your htaccess file? (If the answer is not unambiguously yes, then the answer is no.)
4:31 am on Dec 8, 2012 (gmt 0)

Administrator from US 

WebmasterWorld Administrator not2easy is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:2930
votes: 93


If you are not familiar with editing your htaccess file, use the WP settings. To have access to all the settings, it is easier if you're using a decent SEO plugin that lets you tell WP not to add a page to the sitemap and lets you set a robots meta tag per page

Then the fastest, easiest way to do that would be to put the information from that .htm page into a WP page so that you can check the box to make it password only, then send the password with the auto response - or manually. Set the page to noindex, nofollow and keep it out of your sitemap. Get rid of the .htm page created in CPanel. You may want to change the password frequently so it does not get shared.
5:10 am on Dec 8, 2012 (gmt 0)

Junior Member

5+ Year Member

joined:July 6, 2009
posts: 99
votes: 0


Preliminary questions:

Do you care if other random people (including search-engine robots) know that the page exists, so long as they can't physically get there?

Have you ever set foot in your htaccess file? (If the answer is not unambiguously yes, then the answer is no.)


Thank you

1. Yes I do care, I would rather no one could see it unless they receive the link to it in my auto responder.

2. Yes, have edited htaccess a lot BUT only with coding given to me by a plugin or block bots codes given by coders.
5:12 pm on Dec 17, 2012 (gmt 0)

New User

joined:Dec 17, 2012
posts:6
votes: 0


have you added it to your robots.txt?
5:13 pm on Dec 17, 2012 (gmt 0)

New User

joined:Dec 17, 2012
posts:6
votes: 0


if it is set to "noindex" then search engine bots won't crawl it or index it
5:46 pm on Dec 17, 2012 (gmt 0)

Preferred Member

5+ Year Member

joined:June 14, 2010
posts: 585
votes: 0


If all your efforts can be done away with by one person sharing the link in a forum somewhere, it's going to be harder to manage.

Have you considered adding a plugin, or using a third party service to manage the download? How about your auto-responder, do they offer that option? MailChimp and others can handle an encrypted link within their own servers.

MH
5:50 pm on Dec 17, 2012 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:13256
votes: 360


if it is set to "noindex" then search engine bots won't crawl it or index it

Half right. The "noindex" label means don't index. But they can only see the "noindex" (or equivalent header directive) if they do crawl.

That's why I started out asking whether it matters if people know the page exists. Knowing it's there and being able to get there are different things.
6:33 pm on Dec 17, 2012 (gmt 0)

Junior Member

5+ Year Member

joined:July 6, 2009
posts: 99
votes: 0


Thank you everyone, I have solved this issue
11:07 pm on Dec 17, 2012 (gmt 0)

Moderator from GB 

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month

joined:Apr 30, 2008
posts:2628
votes: 191


Thank you everyone, I have solved this issue

It would be nice to know what did you do, so that the others who have the same or the similar issue can consider your solution.