Does Google crawl Zeal categories?

[martin]

Does Google crawl the Zeal directory, it has something in the index but it's just a few pages and I can't find my listing there.

I saw that LookSmart is more indexed, but I still can't find my site there.

Will it help if I submit to Google the category pages?

[maccas]

If googlebot did it wouldnt help much towards link pop.. click and hold the mouse down on one of the links... [zeal.com...]

[martin]

Yes they have dynamically lookings URIs but why shouldn't it help with PR?

Google crawls dynamic site too.

[Macguru]

I went to google and searched for "Zeal directory". Then I clicked under the second link on : [ More results from www.zeal.com ]

That gave me a good idea of what Google indexes from zeal. You could also try having a look at zeal's robots.txt file. Theorically, Google abide by it.

I then tried another aproach : Lets start with looking for a popular category.

"zeal directory bar games" (without quotes) shows other results.

[edited by: Macguru at 11:49 am (utc) on Aug. 25, 2002]

[martin]

It looks like it is indexing only categories that it found links to or they were submitted for addition.

I was amazed to see that it even has search result pages indexed - most adult.

[Macguru]

You see, martin, the first line of zeal robots.txt prohibits exit.jhtml

User-agent: *
Disallow: /exit.jhtml

So everything past /exit.jhtml wont get in Google's and most other robots.txt compliant SE index. (wich means all the majors)

To bad any link "going out" of Zeal starts by /exit.jhtml....

[martin]

Thanks for that insight.

I was too busy looking if the category pages are listed in robots.txt.

[Macguru]

Thanks for your question martin! It made me discover something I did not know before. Really!

[Grumpus]

My site shows up in Google in Zeal's "Newest Links" page this month, though I don' seem to be getting "link to" credit from it. Granted, the pages on that page are "deep-linked" so it's tough to tell.

G.

[martin]

www.looksmart.com/robots.txt
User-agent: *
Disallow:

I think I read some time ago that this meant everything is allowed.

Unfortunately while I am browsing the LookSmart directory it sometimes doesn't show any sites at all.

Anyway I submitted it to Google, hope that helps.

[Macguru]

>>I think I read some time ago that this meant everything is allowed.

Absolutely right! Simply putting a '/' after the Disallow: command would refuse access to all.

I am afraid submitting to Google wont help at all. Did you try DMOZ? Can you afford Yahoo?

Did you browse both of them for vortals, local directories, topical directories or trade directories?

Submitting to them with good PR can help a lot! Some are still free to submit.

[edited by: Macguru at 12:53 pm (utc) on Aug. 25, 2002]

[martin]

Actually I have a couple of listings in the best topical directory, I was hoping for some mainstream like ODP, Yahoo, Zeal.

Anyway it's not a big deal. I only regret that "on" is not a stopword on MSN, otherwise I would be first on a good two word search.