Forum Moderators: phranque

Message Too Old, No Replies

Online Page Crawler -- Where can I find one

Online Page Crawler -- Where can I find one

         

zuko105

6:48 am on Apr 20, 2003 (gmt 0)

10+ Year Member




Does anyone know of a free online page crawler that would report broken links inside your domain?

I don't need any of the fancy software, just a tool that would tell me if I had any broken links. Starting from my domain, crawl my entire site, and let me know which to fix.

Maybe there are some open source projects that I could compile, or maybe one exists as a tool on a page, that would do it for me through my browser?

I know its a tall order, but I thought if anyone knew of such a tool, I could find it here.

Thanks in advance.

Zuko

Mohamed_E

10:47 am on Apr 20, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



> I know its a tall order, but I thought if anyone knew of such a tool, I could find it here.

Not really such a tall order, a Google search for "free link checker" (without the quotes) returns 240,000 results :)

The one I use is Xenu Link Sleuth, I am happy with it and have often seen it mentioned here. There are, of course, many, many others which should work just as well.

mack

12:06 pm on Apr 20, 2003 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



[searchengineworld.com...]

Not sure if it is intended for dead link checking, more for testing your sites ability to be crawled.

zuko105

1:56 pm on Apr 21, 2003 (gmt 0)

10+ Year Member



Mohammed,
That Xenu is exactly what I needed.

Thanks for that again.

Zuko

rogerd

2:03 pm on Apr 21, 2003 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



Xenu works well for identifying bad links. It's also worth scanning the page titles Xenu lists - I've occasionally spotted pages in which I hadn't altered the title from a template, or had an otherwise poorly written title.