Welcome to WebmasterWorld Guest from 54.226.113.250

Forum Moderators: Robert Charlton & goodroi

“No information is available for this page.”

     
8:46 pm on Dec 6, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14321
votes: 562


We all know “A description of this page is not available because of this site’s robots.txt”. Has anyone else met the new version?

In place of the snippet we get “No information is available for this page.” On the following line is a “Learn why” link, leading in turn to two categories of action: one for “I don’t own this page” and a second for “I own this page”. Did market research finally disclose that the average human searcher hasn’t the remotest idea what robots.txt is, so the former wording wasn’t especially useful?

I most recently met it while searching for {specific name} {type-of-business} -- a type of search where there’s a clear #1 result, whether or not the search engine has seen the page, because there are bound to be links to it. Further snooping reveals that
(a) the site’s robots.txt* is, in fact, a comprehensive
User-Agent: *
Disallow: /
(Why, for god's sake? Doesn’t that defeat the point of even having a website for your business? But I digress...)
(b) That Other Search Engine mysteriously manages to say {brief and extremely generic description of business}, which in turn is identical to the wording offered by two minor search engines I tried at random.

Once again, I feel as if I have missed a chapter.


* I'm going to assume they are not clever enough to serve up a different robots.txt to every visitor. It’s not that big a business.
9:46 pm on Dec 6, 2017 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator robert_charlton is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2000
posts:11963
votes: 309


lucy24, yes, I just saw this notice for the first time I can remember yesterday.

I didn't look into it deeply... and also can't talk about specifics... but briefly I can say that it was for a search for a domain name that I didn't trust enough to click on, and that it was possibly an alias or mirror site for another domain name that was apparently well known.

Possibly it was the site owner's misguided way of blocking dupe content or getting rid of the reference to the alias... I can't be sure and it wasn't important at the time for me to know.

But with regard to the notice, this might be Google's new modular way of telling us that there's a robots.txt.

12:22 am on Dec 7, 2017 (gmt 0)

Administrator from US 

WebmasterWorld Administrator not2easy is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:3466
votes: 192


They forgot to pay their dev? They found a snippet somewhere and used that without understanding it? We've seen people do that with htaccess, why not robots.txt? :(
12:39 am on Dec 7, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14321
votes: 562


They forgot to pay their dev
Oh, how I love that explanation :)

Incidentally: 1 hour plus 1 minute after posting, I remembered a search that was guaranteed to bring up The Other Search Engine's result-unavailable page. They now say “We would like to show you a description here but the site won’t allow us.”
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members