Hey miles, I just picked up on this one.
The robot tags are there to help a SE robot find it's way around your site: To point it in the right direction and to stop it from indexing a page you don't want indexed.
If you mean the robots.txt file, then that should have a similar effect.
Both are a means of control, rather than a way to get a site banned.
Banning is a very strong word and is usually associated with some specific activity which breaches the terms and conditions of submission and use for the search engine. An example of this includes deliberate spamming of the index with excessive page submissions or falsifying the visual page content vs. what the search engine spider sees when indexing.
You can have a site that has no robots.txt or robot tags and it will be indexed (eventually) if it meets the criterion on the url submission form.
I hope that has helped clarify.
Sorry I missed this I must have been on vacation :) -
I believe miles was probably asking about this article [aim-pro.com].
When AV first stopped showing the "you submitted too many urls" message, one way that you could tell if your were banned was by submitting a page that blocked AV with the robot.txt tag. If you WEREN'T banned AV would read the robot.txt and reject your submission, If you WERE banned AV would ignore the robot.txt and act like it excepted your submission (even though nothing actually happened beside a quick crawl).
To tell you the truth I don't know if this still works, I don't have any banned sites to test it with.....Anybody with a banned site wanna give it a try?
I was wandering after looking at the url host:www.****.com. I found that it had 23 indexed sites in AV, I dont know if that means that I am positioned and just not showing up or if AV just doesnt like the site. The other thing I was wandering about is how does AV rate link popularity.
Thank you for responding Engine and Seth.
If you have pages listed then your not banned, it probably just means you need better optimization (dig through the past threads for themes)
"how does AV rate link popularity"
They seem to concentrate much more on the number of relevant links rather than just the number of links
Thank you Seth and Engine once again.
I looked at his site and he only has 2 links to his page. I was checking the titles and they are fair to good, but do you think it could be the links or lack there of that could have the site so low. Any more input would be greatly appreciated.
Miles, for sure, generate relevant links. It's always best to start that process before submitting to the search engines.
You can do this several ways. One of the best ways it to ensure you have ODP or Yahoo links into your site. In addition, find some friendly, relevant firms and contact them to exchange links. You can find suitable links by trawling those already listed in the ODP/Yahoo category, or in the AV DB.
Engine the page is in dmoz and yahoo so should I take those links and run them back to his page. I have seen some positioners go to AV and take the top 20 results and put those as links for link popularity, does this work or is a waste of time? Any ideas would be great.
"the page is in dmoz and yahoo so should I take those links and run them back to his page"
I would just go to the directory listing, copy the url and then submit it directly to the SE's. This should not only help you on AV, but you should see a boost on Google as well. I wouldn't stop at this though....you can never have enough links (especially if your in a competitive industry)