having trouble getting indexed by search engines

search engine robot jsp

[lilbear17]

I'm not sure if this is the right forum for this but I couldn't find a better one so here it goes... I have a site that's setup with a index.jsp homepage. I've 10+ backlinks on all the searchengines and I can see googlebot among others coming but nobody is indexing even my homepage. Yahoo has something with mydomain.com/%22%3EMake which I have no idea what is and takes me to my error page. Has anyone had trouble with jsp pages being indexed? The content is static so it's not a question of that.

The site is listed on my profile if it helps.

Thanks,

Mischa

[jdMorgan]

You have disallowed all robots from indexing all pages of your site.

Your robots.txt file contains:

User-Agent: *
Disallow: /

If you wish to allow all robots to index all pages, change it to:

User-Agent: *
Disallow:

or leave your robots.txt file blank.

Jim

[lilbear17]

my robots.txt file says:

User-agent: *
Disallow: /music_list.html

The one file is a huge list of songs I have available. Why would it disallow the rest of the site?

thanks,

Mischa

[lilbear17]

oh wait, now I see there is a second robots.txt file at the wrong level... THANKS!

[jdMorgan]

Actually, the second one is the one the robots will see, whether it looks as if it's at the wrong level or not.

Type your domain name plus robots.txt into your browser address bar to see what the robots see:

http://www.yourdomain.com/robots.txt

Also, whenever you modify your robots.txt, validate it [searchengineworld.com], since errors can be disastrous (as you've seen) :o

Jim