| 1:08 pm on Aug 16, 2006 (gmt 0)|
Widget is competitive keyword. I cant see any result set where there is a site with no title or description.
It may happen for less competitive keyword where title is just a heading and which is the keyword. It acts like a site ranking well for its own domain name.
| 1:15 pm on Aug 16, 2006 (gmt 0)|
I put widget instead of real keyword becouse of TOS of WW.
And it's strange becouse he block all spiders in robots.txt but he is on SERP
| 1:30 pm on Aug 16, 2006 (gmt 0)|
in bound links with the keyword as the anchor?
I think there might have been a post on this on matt cutts blog?
| 7:15 pm on Aug 16, 2006 (gmt 0)|
Suppose eBay or NY Times completely forbade search engines from crawling. For a query such as [ebay] or [ny times], the best result would still be the url reference (i.e. the uncrawled url without title and snippet) because those are navigational queries. That was the situation with both those sites 4-5 years ago, by the way. :)
When we can't crawl a site, we still try to find e.g. descriptions from the ODP to make the snippet prettier instead of completely empty. And if we think that's the best result, then we can show that url reference even if we haven't crawled the site.
| 8:03 pm on Aug 16, 2006 (gmt 0)|
Thanks for that GG, I was just making that point to someone earlier today.
| 9:14 pm on Aug 16, 2006 (gmt 0)|
3 months ago I blocked MSN in robots.txt couse he eats my bandwidth.
I ranked first for a keyword contained in my domain name in the same manner.It took 3 weeks then my site was delisted.
Then I unblocked him and after 1 month my site apeared in SERP again.
Thank you for explanation.