Forum Moderators: phranque

Message Too Old, No Replies

Ideas on how many pages a site has?

Anyone know if there is a way?

         

mikko

8:01 pm on Aug 11, 2004 (gmt 0)

10+ Year Member



I was wondering if anyone knew or heard of any software that when you type in the URL, it goes through the site like a robot and then displays all the links or number of web pages in that site?

I just thought it would be nice to see the size, in terms of pages/links that my competitors were using.

goodroi

8:45 pm on Aug 11, 2004 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



There are many programs that can do this. You can search on google for site mapping tool or site ripper.

diamondgrl

9:16 pm on Aug 11, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



of course, if the site is in google, simply type site:www.example.com and you will get a list of all the pages google has listed. this would certainly be more friendly than doing one of those site ripper programs which can also piss off a site administrator and get you banned altogether. i know i ban anyone trying that nonsense on my site.

mikko

11:11 pm on Aug 11, 2004 (gmt 0)

10+ Year Member



Diamondgrl - never even knew that existed, spot on. I have a very good friend of mine what has a web site who does the same product as myself, although the we could never directly compete with each other and we can help each other out, there are some things that we kept quiet about. This has been a great help, thanks!

deejay

11:22 pm on Aug 11, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Bear in mind that the site: command will only tell you the pages that Google has indexed, and it is only a snapshot of the date those pages were crawled - which may be up to several months ago. If the site has unfriendly URLs, is new and hasn't been fully crawled, is adding content regularly or has low PR and isn't getting a deep enough crawl Google may grossly underestimate the size of a site. Or overestimate if content is being moved and URLs changed or deleted.

diamondgrl

11:40 pm on Aug 11, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



deejay is, of course, correct on all that. but it can usually give you a pretty good indication.