Forum Moderators: open

Message Too Old, No Replies

Spidering Problem

hope i'm in the right forum category

         

lty83

4:13 am on Nov 26, 2003 (gmt 0)

10+ Year Member



Hey guys and girls, recently when checkin out my log files i noticed that googlebot made a visit to my site, however it only hit pdf files, and like maybe 4 of them, it hit like 2 more regular html pages, but no index page! whats up with that!

I have like 67 active seperate pages with unique content, been working on specific keyword redefining for each page. I have no robots.txt file, is that bad? I also do not have a robots meta tag after fears of it screwing with msn spider.

Not sure what to do! Any comments/suggestions would be highely appreciated! Thanks!

Birdman

12:19 am on Nov 27, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



As long as your link structure is okay, you should be fine. I don't think the lack of robots.txt makes a diff, it just means you are giving access to crawl, in their(it's?) eyes. I would forget the robot meta altogether. I believe it is a thing of the past for any major engine.

Keep in mind that the above are just opinions, but I'm sure most around here will say the same.

By the way, WELCOME!

lty83

2:13 am on Nov 27, 2003 (gmt 0)

10+ Year Member



Thanks! I know just opinions, but I like to take it all in, learn different views and approaches to help better mold mine!

I took out the robots meta tag, and I think it the problem is the excess Javascript on the page, also flash, nothing major flash wise, small lil animation.

The person who created the page never took seo into consideration so now I have to redesign it, while fighting with the owners over looks because apparently flash and javascript is more important than exposure.. hehe

I'm slowly winning the battle, but with all this stuff I hear about google changing their search engine algorithm it worries me, I was hired under the condition of increasing their searchability, hopefully this doesn't end my job!

At any rate I will continue reading and redesigning and modifying my pages out the wazoo until i figure it out and the day before they change it all again.. hahah

thanks man, this place rocks..