Welcome to WebmasterWorld Guest from 34.238.189.171

Forum Moderators: phranque

Message Too Old, No Replies

Utility Wanted

     
11:06 pm on Jan 29, 2001 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38254
votes: 113


Anyone know of a utility that will search a set of directories and extract all http urls found?
1:07 am on Jan 30, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Aug 24, 2000
posts:1748
votes: 5


Brett,

What type of directories are you referring to?

Minnapple

2:40 am on Jan 30, 2001 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38254
votes: 113


eek. Hard drive - files.
3:15 am on Jan 30, 2001 (gmt 0)

Full Member

10+ Year Member

joined:June 14, 2000
posts:247
votes: 0


"The program allows to analyze
files on your local computer and to extract URL, news and email addresses. Also you can apply  the filter and export obtained data..."

This one looks like a program out of a spammer toolbox, but it might do what you need:
[esalesbiz.com...]

3:49 am on Jan 30, 2001 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38254
votes: 113


Bartek - nice to see you still hanging around here (thanks).

Looks like a nice util. Could I see something in freeware? In the perl isle?

Air

5:44 am on Jan 30, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 10, 2000
posts:1253
votes: 0


BT, if you have this module installed it should do it.

[search.cpan.org...]

6:29 am on Jan 30, 2001 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38254
votes: 113


That will only (afaik) work with html and possibly text files. I want to dig through many flat files and junk/binary files.
7:54 am on Jan 30, 2001 (gmt 0)

Full Member

10+ Year Member

joined:June 14, 2000
posts:247
votes: 0


>nice to see you still hanging around here...

Pleasure is all mine, I assure you. Wish I could spend more time here...

Now let's see...Probably overkill again, but try this:
ftp://ftp.mutt.org/pub/mutt/contrib/urlview-0.9.tar.gz

8:15 am on Jan 30, 2001 (gmt 0)

Full Member

10+ Year Member

joined:June 14, 2000
posts:247
votes: 0


Another from your fave Perl isle... [xensei.com]
6:26 pm on Jan 30, 2001 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38254
votes: 113


Thanks. I came up with a solution that is based on just about all of the above.
I'd forgotten that I had a dos version of grep that came with a borland compiler. A little fine tuning of the output is needed, but I just let it walk through all the files filtering out "lines" with http's and then massage it with perl.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members