Welcome to WebmasterWorld Guest from 54.167.155.147

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Sitemap "Priority" and bot behavior
Will the priority attribute affect how spiders crawl?
3 arieng 6:07 pm Nov 6, 2009
SiteMap and Robots.txt Indexing
What is the right way to avoid it?
5 kitamed 3:49 pm Nov 6, 2009
Google Sitemap Indexed URLs going down
4 themoron1 1:12 pm Nov 6, 2009
Problems crawling my pages for XML Sitemap
Spider simulators (all of them) seem to struggle
1 Munster 12:15 pm Nov 5, 2009
How to stop search engines caching my webpages?
11 gcan 4:31 am Nov 2, 2009
index.html
redirect in htaccess file
3 anyone 3:53 pm Oct 29, 2009
Will URLs blocked in robots.txt remove currently indexed pages?
6 limitup 12:20 am Oct 28, 2009
Noindex and also Changeing 2 million URL's
2 azn_romeo_4u 10:53 am Oct 27, 2009
Blocked Shot - Robots.txt
blocking numerous duplicate pages using robots.txt
6 underglass 10:50 am Oct 27, 2009
Need a Sitemap Generator
2 eljuan 8:52 pm Oct 26, 2009
site map generator program
site map generator
7 Roobik 12:27 pm Oct 26, 2009
Webmaster Tools Issues - Need Help
Webmaster tools showing data that does not reflect in Google
3 ranadeep 7:16 am Oct 22, 2009
meta name="allow search" content="yes"
do i really need this tag now?
4 supafresh 1:52 am Oct 20, 2009
Question concerning Disallow and *
4 augustwind 1:44 am Oct 20, 2009
Which Is The Better Way To Handle Site Title
Which is the better way to handle the site title.
4 Robert 11:40 pm Oct 18, 2009
Webmaster URL Removal Issues
Robots file/URL removal issues
10 MarkWW 3:24 pm Oct 13, 2009
Different content on the same URL
Users language is stored in a cookie - website displays different content o
3 gcan 9:23 pm Oct 12, 2009
Why search engines are showing the wrong meta title & description
Metadata was correct two months ago but now search engine showing wrong.
4 Kash 2:36 pm Oct 12, 2009
Question about robots.txt file
It is blank
2 iGLOBAL_NiCK 2:45 am Oct 12, 2009
allowing google bot only to index URL with parameters
2 ulmudmezig 7:12 pm Oct 8, 2009
Google Webmaster Tools - Domain Not Found
1 kflanagan 9:25 am Oct 6, 2009
Robots.txt and allow and $
Advanced question
3 djingo 12:43 pm Oct 1, 2009
Index pages: site:www.abc.com vs Google Sitemap(Webmastertools)
Why there's a huge difference between the two?
2 Singhvineet 1:28 pm Sep 29, 2009
Google not obeying robots.txt
Disallow: /*? not going through
4 smallcompany 4:48 am Sep 26, 2009
Successful 301 Redirection?
Switching a website to an entirely new technology, URLS, etc
5 supermanu 2:39 am Sep 23, 2009