Welcome to WebmasterWorld Guest from 18.104.22.168
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
|Is it possible to block file names beginning with a text string?
||2||Gemini23||8:07 am Nov 11, 2013|
|Can't stop googlebot crawling specific forum
||10||persons||4:44 am Nov 10, 2013|
|wordpress and sitemaps
generating sitemap for non-wordpress pages
|1||deborahbaker||2:46 pm Oct 2, 2013|
|Is my robots.txt file ok?
||6||Bubzeebub||11:28 pm Sep 28, 2013|
|Google does not honor robots.txt
||11||newbies||3:40 pm Sep 12, 2013|
|Is 301 Redirect info stored in the cached page?
||3||flanok||4:14 pm Aug 31, 2013|
|Search Engines Displaying wrong meta-description
Wordpress login error as description on non-WP pages
|2||jamesMP||7:53 am Aug 29, 2013|
|Inadvertently used 301 redirected URL's in XML Sitemap
||1||dflayfield||7:38 pm Aug 28, 2013|
|Link rel to robots.txt
what's the link tag to pull robots.txt
|8||0verdose||1:13 pm Aug 22, 2013|
|Help needed with XML Sitemaps
||3||shaunm||7:42 am Aug 22, 2013|
|I am confused about my sitemaps. What is best to do?
||6||virtualreality||12:13 pm Jul 26, 2013|
|What to do with deleted pages?
||6||rlopes||12:04 pm Jul 26, 2013|
|Microdata - which to use for a "training course"?
It's for sale, so is it a "Product" or a "CreativeWork"?
|3||bigtoga||1:19 pm Jul 15, 2013|
|Pages with Partially Syndicated Content
No indexing of page elements
|2||jexx||4:59 pm Jun 22, 2013|
|For all sitemap lines (URLs), can you just cut out the http://domain.
||4||2kegx||5:29 pm Jun 10, 2013|
|What will happen to Yahoo urllist.txt now that their SE = Bing?
urllist sitemap.xml yahoo bing webmaster tools
|2||2kegx||8:16 pm Jun 2, 2013|
"y_key" still needed?
|2||Broadway||8:14 pm Jun 2, 2013|
|XML sitemaps encoded characters and GWT
GWT reports 1 URL indexed from all sitemaps
|3||Fiver||9:13 pm May 29, 2013|
|xml sitemap parameters. are they required ?
||9||pkKumar||8:22 am May 23, 2013|
|Page Indexed but not Cached in Google
||5||Akshat_Rastogi||11:53 am May 22, 2013|
|Robots.txt strategy: allow only good, or disallow individual bad?
Should my robots file say "Only google, yahoo, bing" or All are OK except.
|4||bigtoga||4:04 pm May 13, 2013|
|If I changed my site description, could I ruin my spot at the top?
#1 search result
|2||littlecubpanda||4:15 pm May 9, 2013|
|to map or not to map
||1||lucy24||10:30 pm Apr 28, 2013|
|XML Sitemap Generator for 1 million page site
looking for a solution
|5||RichardV||4:59 pm Apr 28, 2013|
|Sitemap.xml & Yoast sitemap-index.xml question
A website and Wordpress Blog on same domain
|5||edwardsgs||3:45 pm Apr 27, 2013|