Welcome to WebmasterWorld Guest from

Forum Moderators: martinibuster

Message Too Old, No Replies

Yahoo Site Explorer

SES Big Announcement



9:13 pm on Aug 12, 2005 (gmt 0)

10+ Year Member

Yahoo announced a new product called site explorer in San Jose on Thursday. Lots of great new tools but it looks like it is down - [sitexplorer.search.yahoo.com...] Right now it just redirects to advanced search.

Anyone else get this?


10:59 pm on Aug 14, 2005 (gmt 0)

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

It's not live yet. It is supposed to help webmasters determine the indexing of the site and maybe also backlinks. Sounds promising. I'd watch the Yahoo Blog for more details.

Robert Charlton

6:36 pm on Aug 15, 2005 (gmt 0)

WebmasterWorld Administrator robert_charlton is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Tim's quick overview, with screen shots, looked great. It clearly has its heritage in the site information center that AllTheWeb once had.

In part its purpose, Tim suggested, is to get certain kinds of queries off of Yahoo's main search page. Among other things, we'll be able to download certain info (like backlinks, if I remember correctly) in csv format, which for some purposes will be much more useful than the serps format we're getting now.

I'm very pleased that Yahoo is doing this.


1:01 pm on Aug 19, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

I think your missing an "e" in the URL above, it is still not live, but I wanted to add that.




1:56 pm on Aug 23, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

Looks like we now have a "coming soon" page.


2:19 pm on Aug 24, 2005 (gmt 0)

working now.


12:26 pm on Aug 26, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

What is working now?

All I see is a coming soon page.


8:50 pm on Sep 27, 2005 (gmt 0)

10+ Year Member

Still not working, but Medium Blue [mediumblue.com] reported the following today in their newsletter: (I'd do the full URL for the newsletter, but they have it cleverly tagged with a UID...)

Yahoo Follows Google in Offering a URL Submission Program

Yahoo has released a program very similar to Google Sitemaps. It gives website owners the ability to send Yahoo a list of all of their pages that they want the engine to spider and include in the index. This helps Yahoo conserve resources when maintaining and building its index since it alerts the engine to the locations of web pages. It also makes the spiders' jobs easier by telling them exactly where to go, rather than relying on them to find and index everything. This program will be very beneficial for large websites that have had trouble in the past with getting all of their pages indexed.

I got in touch with Medium Blue, and they confirmed that they're writing about siteexplorer.search.yahoo.com [siteexplorer.search.yahoo.com] . If the news turns out to be accurate, that's great news. (And if MSN Search can do the same, then we'll only have one free XML feed map to worry about for the major SEs...)


9:00 pm on Sep 27, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

Yes they had the program for a month or two now. Its found at the submit URL page at Yahoo!


9:23 pm on Sep 27, 2005 (gmt 0)

10+ Year Member

Do you have a URL for that? I looked at submit.search.yahoo.com/free/request [submit.search.yahoo.com] and at search.yahoo.com/info/submit.html [search.yahoo.com], but wasn't able to find any mention of anything like "Site Map", "XML", or "Site Explorer".


9:46 pm on Sep 27, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

Its on that page, [submit.search.yahoo.com...]

You can also provide the location of a text file containing a list of URLs, one URL per line, say urllist.txt. We also recognize compressed versions of the file, say urllist.gz.


12:38 pm on Sep 28, 2005 (gmt 0)

10+ Year Member

Oh, I was supposed to read those pages instead of just doing quick text searches? [;-)] Thanks RB!

It's too bad that the Submit Your Site page won't accept a doc in Google Site Map form. I'd love to be able to produce a single doc for multiple SEs instead of dealing with varying standards.

Still leaves the issue of Site Explorer itself up in the air for me. Apparently Medium Blue were writing about Site Explorer in their article, but clearly there's already this other, very "low tech" alternative in place at Yahoo!. Did Medium Blue get it wrong, or is Site Explorer going to be a standard or tool that's very different from "urlist.txt"?


12:46 pm on Sep 28, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

SiteExplorer is suppose to be a tool to help Webmasters see the linkage data to a site/page, see a site's health in the Yahoo! index and more.


12:09 am on Sep 30, 2005 (gmt 0)

10+ Year Member

Well....... look like Yahoo site explorer is live!

Examining all it has to offer takes.... oh, about 10 secs --------- useless! worthless!
huge disappointment!

And, later I'll tell you how I really feel about it...


12:38 am on Sep 30, 2005 (gmt 0)

10+ Year Member

I find it interesting that they finally launched the SiteExplorer tool, but are not accurate when it comes to the amount of site pages and BackLinks.
You would think that SiteExplorer would show everything, but for some reason I am missing about 13 million backlinks.
Is anyone else seeing that all of their pages and links are not being displayed?


5:10 am on Sep 30, 2005 (gmt 0)

10+ Year Member

I fail to see how Site Explorer differs from the normal yahoo search.


9:51 am on Sep 30, 2005 (gmt 0)

10+ Year Member

ciscoforum, welcome to WebmasterWorld!

Unlike Yahoo! Search, Site Explorer only accepts URLs as query terms. If you try to search for keywords in Site Explorer, you get the following error message:

Malformed URL

The query entered is not a valid URL

OK, but Yahoo! Search already provides advanced operators that can retrieve info about a URL in its index; there are at least six of these operators that Yahoo! calls search meta words [help.yahoo.com] (see the bottom of the page): "site:", "hostname:", "link:", "url:", "inurl:", and "intitle:".

Then what's the big deal about Site Explorer if you can already get the same info out of Yahoo! Search? From what I can see so far, it's mostly a difference of interface and, for the totals reported, specificity. Here's the way I think it's working; please let me know if I'm off-base on any of this.

Easier to use
Site Explorer makes it easier to do "site:", "link:", and "url:" queries. The following examples show side-by-side Y!Search vs. Site Explorer queries.

* * * * * * * * * *

Yahoo! Search --> [site:example.com] 
Site Explorer --> [http://example.com]

These find all the documents in the domain, including all its subdomains, e.g., "www.example.com", "store.example.com", "foo.example.com". (Note:I'm still a little unclear on the circumstances under which a "hostname:" query would return different results from a "site:" query. Anyone?)

* * * * * * * * * *

Yahoo! Search --> [site:subdomain.example.com] 
Site Explorer --> [http://subdomain.example.com]

These find all the documents only for the subdomain.

* * * * * * * * * *

Yahoo! Search --> [link:http://www.example.com/page.html] 
Site Explorer --> [http://www.example.com/page.html] --> The link total is displayed at top of page as "Inlinks (XXX)"

These find all the documents that have a link to the URL.

* * * * * * * * * *

[Yahoo! Search --> [url:http://www.example.com/page.html] 
Site Explorer --> [http://www.example.com/page.html]

These find the exact URL specified, but only if it's in the index.

* * * * * * * * * *

More accurate totals(?)
A total in Site Explorer tends to be a specific number compared to the equivalent total in Yahoo! Search. It's usually close to the Yahoo! Search total. Compare the following totals:

Yahoo! Search --> [site:cnn.com [search.yahoo.com]] 
Site Explorer --> [http://cnn.com [siteexplorer.search.yahoo.com]]

Yahoo! Search --> [site:www.cnn.com [search.yahoo.com]] 
Site Explorer --> [http://www.cnn.com [siteexplorer.search.yahoo.com]]

Yahoo! Search --> [link:http://www.cnn.com [search.yahoo.com]] 
Site Explorer --> [http://www.cnn.com [siteexplorer.search.yahoo.com]] --> Look at the "Inlinks".

Other Site Explorer benefits
I don't believe there's a way in Yahoo! Search to determine what documents are indexed within a directory. You can definitely do this with Site Explorer. For example, if you want to see which documents Yahoo! knows about that are in http://www.cnn.com/WORLD/, then:

Site Explorer --> [http://www.cnn.com/world [siteexplorer.search.yahoo.com]]

(I haven't tested whether this lets you do stemming with query parameters, e.g., plugging in http://www.example.com/cgi-bin/app?x=123&y= to retrieve any indexed URLs that contain that string.)

Also, for any URL being searched in Site Explorer, the results page lets you click back and forth between the pages in the index that satisfy the query URL and the pages in the index that link to the query URL.

Then there's the API [developer.yahoo.net], which is a truly promising tool I can't even begin to go into.

Yahoo! Search benefit
If you're experienced with the search operators, there are still some queries you can't do in Site Explorer. For example, if you want to see all external pages that are linking into the CNN site, I believe the only way to do that is:

Yahoo! Search --> [link:http://www.cnn.com -inurl:cnn.com [search.yahoo.com]]

Anyway, hopefully now nobody will fail to see how Site Explorer differs from Yahoo! Search. ;)


1:31 pm on Sep 30, 2005 (gmt 0)

10+ Year Member


Nice run down there....

Fact is (from an experienced user's point of view) Yahoo's site explorer offers MUCH less than the standard search, so I fail to find ANY use in it.

And, I still firmly believe that Yahoo should be embarrassed for what they delivered as compared to what they "pumped" at the SEO conference!


5:06 pm on Sep 30, 2005 (gmt 0)

10+ Year Member

Man, what a letdown. I barely ever do a link search without other parameters, such as when I want to find out if a site has my link on it so I can avoid soliciting them for a link if I don't need to. Normally I would use this:

linkdomain:www.mysite.com site:www.othersite.com

Is it just me or can I not do this with Site Explorer?

If not they need to include some filtering options or it will be useless...


5:11 pm on Sep 30, 2005 (gmt 0)

10+ Year Member

Bit of a let down.

Anyone tried submitting a list of urls?

I get this lovely yahoo error. The following resulted when trying to access your document: timeout


Had a look through the logs, didn't even try.

Go yahoo!


5:23 pm on Sep 30, 2005 (gmt 0)

10+ Year Member

Give you an inch and you want the whole yard… yesterday.

Site Explorer is a WIP.

For starters you can see exactly how many links Yahoo! credits to your competitors Blue Widgets page.

Then you can see how many links Yahoo! credits to your Blue Widgets page.

“What a waste…" Not true. If Yahoo! is on the up-n-up, they’ll tell you exactly how many links (Y! doesn’t see as spam) are pointing to your Blue Widgets domain, unlike the other Guy-gle that only shows you however many links they feel like showing you.

And if you have 13-Million links, it's no small wonder that 12,500,227 will not show up due to the spamy nature of the majority of your 13 million links.

This is a great competitive research tool It’s free and better than nothing.

So go ahead and buy some more spamy links, I'll continue to use site explorer to find out what pages Yahoo! likes to see linking to my Blue Widgets for free.

I bet my money goes further.


5:37 pm on Sep 30, 2005 (gmt 0)

10+ Year Member

I'm wasn't at SES San Jose in August, so I didn't catch Tim Mayer's spiel on Site Explorer. While I don't know what expectation Mayer was setting there, when I look at Stephan Spencer's very sober SES San Jose writeup [stephanspencer.com], it appears as though Mayer was promising more-or-less what Yahoo!'s just delivered.

For example, Spencer wrote (my additions in italics):

Yahoo! (Search) offers a much more comprehensive sample of backlinks than Google, but not a complete set of backlinks. New system (Site Explorer) will be reasonably comprehensive, in his (Mayer's) opinion the most comprehensive out there.

Meanwhile, here's what RustyBrick had to write about that same session with Tim Mayer [forums.searchenginewatch.com]:

He showed an example, you basically type in a URL into it (this is also supported via an API, good good), then you hit explore URl and it spits out the number of pages found in Yahoo Index and also shows you the number of inlinks. You can sort pages by "depth" how deep pages are and you can also submit URLs here. You can also quickly export the results to TSV format.

True, Site Explorer doesn't appear currently to have that "sort pages by 'depth'" functionality. Is that what all the disappointment's about? I'm curious what people's expectations really were.


Featured Threads

Hot Threads This Week

Hot Threads This Month