Welcome to WebmasterWorld Guest from 126.96.36.199
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
|I am confused about my sitemaps. What is best to do?
||6||virtualreality||12:13 pm Jul 26, 2013|
|What to do with deleted pages?
||6||rlopes||12:04 pm Jul 26, 2013|
|Microdata - which to use for a "training course"?
It's for sale, so is it a "Product" or a "CreativeWork"?
|3||bigtoga||1:19 pm Jul 15, 2013|
|Pages with Partially Syndicated Content
No indexing of page elements
|2||jexx||4:59 pm Jun 22, 2013|
|For all sitemap lines (URLs), can you just cut out the http://domain.
||4||2kegx||5:29 pm Jun 10, 2013|
|What will happen to Yahoo urllist.txt now that their SE = Bing?
urllist sitemap.xml yahoo bing webmaster tools
|2||2kegx||8:16 pm Jun 2, 2013|
"y_key" still needed?
|2||Broadway||8:14 pm Jun 2, 2013|
|XML sitemaps encoded characters and GWT
GWT reports 1 URL indexed from all sitemaps
|3||Fiver||9:13 pm May 29, 2013|
|xml sitemap parameters. are they required ?
||9||pkKumar||8:22 am May 23, 2013|
|Page Indexed but not Cached in Google
||5||Akshat_Rastogi||11:53 am May 22, 2013|
|Robots.txt strategy: allow only good, or disallow individual bad?
Should my robots file say "Only google, yahoo, bing" or All are OK except.
|4||bigtoga||4:04 pm May 13, 2013|
|If I changed my site description, could I ruin my spot at the top?
#1 search result
|2||littlecubpanda||4:15 pm May 9, 2013|
|to map or not to map
||1||lucy24||10:30 pm Apr 28, 2013|
|XML Sitemap Generator for 1 million page site
looking for a solution
|5||RichardV||4:59 pm Apr 28, 2013|
|Sitemap.xml & Yoast sitemap-index.xml question
A website and Wordpress Blog on same domain
|5||edwardsgs||3:45 pm Apr 27, 2013|
|Penalty for updating/adding sitemaps?
Google is losing track of sitemap entries
|6||rstidx||7:18 pm Apr 11, 2013|
|How do I block TagIDs?
||6||shaunm||6:40 am Apr 11, 2013|
|NOODP tag Question.
|3||2kegx||7:54 pm Apr 9, 2013|
|Robots.txt Disallow Multiple Level Subdomains
Can a parent subdomain's robots.txt disallow spiders in a deeper subdomain?
|3||rmjvol||9:02 pm Apr 4, 2013|
|Wrong Lastmod Tag
||3||shaunm||11:05 am Mar 27, 2013|
|Site Titles not taken after 2 months, Descriptions Updated - help
After robots.txt issues, google refuses to index Page title changes
|3||SilentiSonas||4:25 pm Mar 25, 2013|
|what is the User-Agent?
||3||lucy24||12:58 pm Mar 24, 2013|
|A huge number of URL's monitored on GWT
Need to remove all of them
|5||morpheus83||4:41 am Mar 20, 2013|
||1||lucy24||11:41 pm Mar 4, 2013|
|googlebot not get robots.txt
|5||Luka2013||1:06 am Mar 2, 2013|