homepage Welcome to WebmasterWorld Guest from 54.204.94.228
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Marketing and Biz Dev / Keyword Discussion
Forum Library, Charter, Moderator: open

Keyword Discussion Forum

    
Search engine - keyword search mechanism
program to search for key words in multiple URLs
wangdy




msg:267270
 12:40 pm on May 16, 2001 (gmt 0)

Hello,
I wasn't sure how to phrase the question that I have since I am not sure that it would be considered a search engine issue...

I need to find out if there is a "search engine" that can go through numerous URLs and search for keywords? Would I have to write a program? Is there a software package? How difficult would it be to create?
Thanks!
Dennis

 

WebGuerrilla




msg:267271
 3:40 pm on May 16, 2001 (gmt 0)


Hi Dennis,

Welcome to WmW.

I'm afraid I don't quite understand the question. Where are the numerous URL's? Do you have a list on your computer that you want to search through, or are you looking to find URL's with specific keywords in a particular search engine database?

wangdy




msg:267272
 5:09 pm on May 16, 2001 (gmt 0)

Thanks for the welcome!

The former... we are looking to search for keywords in URLS or search engine databases. Basically, to find out what other sites on the web are using brand names.

Thanks!

agerhart




msg:267273
 5:16 pm on May 16, 2001 (gmt 0)

I think this may be able to help you. I found this in my log.

http://www.netcraft.com [netcraft.com]

It returns site from its database that have the search word or phrase in the URL.

-A Gerhart

WebGuerrilla




msg:267274
 5:54 pm on May 16, 2001 (gmt 0)

Yes, Netcraft is very handy. You can also perform URL only searches at most indidual search engines. The results aren't presented in as nice of a format as Netcraft, but you will usually find more URL's due to the larger size of the databases.

You can search at AltaVista for URL's containing specific words by typing url:keyword into the box.

At Google, you can use their advanced search page [google.com] to search for phrases contained only in URL's.

wangdy




msg:267275
 8:19 pm on May 20, 2001 (gmt 0)

Thanks!

Another novice question... are search engines easy to write? Or are there software packages out there?

It seems, after thinking about what I needed to do, that this project is very similar to building a search engine.... any suggestions?
Thanks again...

WebGuerrilla




msg:267276
 4:50 pm on May 21, 2001 (gmt 0)


There are several companies that make search software. AltaVista sells an Enterprise version that can be used to run a search engine. It is typically used by ecommerce sites and company intranets, but it has all the bells and whistles of AltaVista's site.

Inktomi also offers similar software.

A great place to visit for info on search software is [url=http//:www.searchtools.com]searchtools.com.[/url]

wangdy




msg:267277
 2:06 pm on Jun 5, 2001 (gmt 0)

OK... on a related topic,
Am I correct to think that a spider does not crawl a site unless it is submitted to a search engine (Yahoo, google, etc)?

Woz




msg:267278
 3:41 am on Jun 6, 2001 (gmt 0)
wangdy,

back to your first question, you might want to consider setting up a Search Engine Monitoring system. There are two companies I know of that can do this for you:- http://www.trackengine.com/ and http://www.trackengine.com/ (the latter of which came to light in these fora) and there may be more.

These are free systems that will monitor specific searches at specific Search Engines for you and report the results via advertisement supported email.

So you could setup these systems to monitors searches at SEs for your particular product/brand name and be notified if anything new comes up in the searche results.

To your second question,

Some search engines will only spider you if submitted, some follow found links. For some of the Engines, Google and Inktomi for instance, it is better if they FIND you rather than be submitted.

Directories on the other hand, such as Yahoo NBCI and the likes, are human operated and so will only look at your site after you submit it. There are exceptions of course but in the makors exceptions are in the minority.

So a course of well prepared and well targetted submissions is best to gain maximum results and avoind unintentional pitfalls. And the best way to get Well Prepared is to read as much of these fora as you can before starting.

Onya
Woz

asuprun




msg:267279
 8:04 am on Jul 7, 2001 (gmt 0)

Hi WoZ,

You mentioned free tools that send emails when any specific SE results page is updated. Can you give any link?

Thanks a lot!

Asuprun.

wangdy




msg:267280
 12:06 pm on Jul 9, 2001 (gmt 0)

Thanks Woz,

That's some good info there... I've also been diving into what some companies call the "Deep Web"... the information that is served up in databases.

I first read about these in a magazine article in the Standard. It's really neat what they can find out there...

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Marketing and Biz Dev / Keyword Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved