subbing yahoo pages to a.v.

problemss..

[mr_dredd]

i was trying to sub a yahoo catagory to altavista (for linking pop), and it came back saying:

This page has been disallowed by the robots.txt file.
The page, if already indexed, will be removed from the index.

the last time i did this there was no problem..
any ideas??

[chiyo]

maybe Yahoo has banned the AV spider (:-) or all SE spiders from spidering their database.

[Marcia]

I just did a search and submitted the returned page to AV with no problem.

I then went to a category directly from the directory listings, and when submitting, it was disallowed by robots.txt and will be removed.

Then I did a search on the phrase associated with that specific category, and submitted the returned page to AV with no problem.

So there we have it. Pages reached by going through the directory are excluded by robots.txt - but when a page is found by using a search term, it can be submitted.

Now: what will this do to the perception of the value received for doing the $199 Business Express submission? How will this affect the link popularity effect of the Yahoo listing?

Do they think the public won't know, by some chance?

[bulldog]

>Now: what will this do to the perception
>of the value received for doing the $199
>Business Express submission?

I just submitted to Yahoo and I can tell you that getting the linkpop was one of my prime considerations for paying $199. For me, Yahoo does bring in some traffic, but I think it's going to be almost impossible to rank on the 1st or 2nd page of results for my primary keyword phrases due to Yahoo's crazy algo (I can't figure it out). I will not be paying Yahoo $199 for other sites until they lift the ban on spiders.

BTW: Which SEs will be hurt by the ? mark in the Yahoo search results urls that Marcia is talking about submitting in the above post?

Thanks,
BullDog

[Marcia]

Welcome to WmW bulldog, good to have you here.

I would guess that this gambit on Yahoo's part will raise the relative value of an ODP listing even more. We'd better be praying to the search engine gods that ODP stays free.

As long as the search result is "submittable" then I guess we'd be OK with Google and Alta Vista, right? The others I wouldn't even venture a guess about.

[bulldog]

Actually Marcia, I'd love for the ODP to start charging for submissions like Yahoo. Maybe then they could hire editors that didn't have a personal/financial interest what sites get into the database. I'd gladly pay $199 to get an honest, fair review of my site with an accurate title and description and I'm sure there are a lot more people just like me out there who would do the same in a second.
BullDog

[skibum]

When a site I was working on first got listed in YAHOO! I tried submitting the YAHOO! listing to other engines, and had the problem with the robots.txt not permitting the page to be spidered. It's definately a bummer for link pop.

I can't possibly see ODP starting to charge for reviews. Charging a minimal fee, like 5 bucks might cut down on Spam submisions and help to get genuine submissions get listed faster. Determining who would get the cash, and distributing it to editors? would seem to be a huge PITA that would require implementing some system that would be tough to make equitable to editors, and unless editors actually had full time jobs as editors, charging would not be likely to ensure any more "fair reviews" than the present system does.

[Brett_Tabke]

Dredd, Yahoo's robots.txt appears to be cloaked to me - atleast there is something fishy going on there. They may have added alta to the ban list. Try some different agent names and you'll get a few different things back.

[JamesR]

That robots.txt [yahoo.com] is pretty slim... maybe even svelte...

User-agent: *
Disallow: /gnn
Disallow: /msn
Disallow: /pacbell
Disallow: /pb

# Rover is a bad dog <http://www.roverbot.com>
User-agent: Roverbot
Disallow: /

[sugarkane]

I always get a 404, from a variety of IPs and using several agents...

[Brett_Tabke]

I'm getting a 404 now as well SugarKane.

[glauber]

Mee too. This is new, i think even yesterday i was able to get it. Maybe they just deleted the file? :) (extreme cloaking!)

[NFFC]

Try this one [dir.yahoo.com].

[mr_dredd]

ok.
so if the only way to sub yahoo pages for link pop is by subbing the search query i have a question about that:
for google - doesn't google just spider from the index and not take notice of individually submitted pages? If so, then the only way to do this would be to hard code a yahoo search query somewhere on your site.
correct???

[glauber]

Nice, it has a zen-like simplicity that reminds me of koan.

[angiolo]

Hi all!

I have a question regarding increasing link popularity:

How does search engine find your link (your listing) to other search (directory) engines?

It is not an automatic task?
I thought that searchengines can search theirself for that.

For example if I have listings on Looksmart, NBCI, Yahoo, DMOZ etc, should I submit that listings to other search engines like Altavista, Excite, Inktomi based search engines etc. ?

And what exactly should I submit?

For example, searching on Yahoo for "Holiday in Hungary", I got this directory

[dir.yahoo.com...]

Should I submit that directory where I have a site or should I submit the URL that appear when I point the mouse to my URL (something like "http://srd/yahoo.com/drst/26707499/*http://www.mysite.com?"

I apologize for my silly questions: think about me as I were a Civil Servant clerk.

Angiolo