Welcome to WebmasterWorld Guest from

Forum Moderators: Webwork & skibum

Message Too Old, No Replies

Local directories- crawling, buying or building?

What is the best solution for a city based startup

2:19 am on Nov 9, 2007 (gmt 0)

New User

5+ Year Member

joined:Nov 8, 2007
posts: 5
votes: 0

Would it be realistic nowadays to build your own business directory considering the amount of IYPs around?
Is crawling ( phone numbers and addresses only) an infringement of copyrights?
And if buying out a database is the best solution (legally and practically) how flexible can that database be (for example if one wanted to translate it into different languages, that would be categories and titles such as 'hairdresser', 'restaurant', etc)?

I can imagine also the problem of updating a major one..

Since I have read crawling involves a copyright infringement and a risk of getting false information, it seems buying out a directory from a yip is the best solution..

Now does one of you know how flexible can that database be in terms of adding content, UI, or for example translation into different languages?
(I m pretty new to webmastering! )

Thanks, waiting to hear your responses!

[edited by: Webwork at 4:42 pm (utc) on Nov. 9, 2007]
[edit reason] Tidying up [/edit]

3:15 pm on Nov 9, 2007 (gmt 0)

Preferred Member

5+ Year Member

joined:Dec 26, 2006
votes: 0

Consider building it organically, one listing at a time

I am

5:00 pm on Nov 9, 2007 (gmt 0)


WebmasterWorld Administrator buckworks is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Dec 9, 2001
votes: 63

When you say "city based startup" are you envisioning a directory that is tightly focused on one city or spread among multiple cities?

That would make a difference ...

5:07 pm on Nov 9, 2007 (gmt 0)

New User

5+ Year Member

joined:Nov 8, 2007
posts: 5
votes: 0

Hi Buckworks thanks for your answer, I meant multiple cities..
7:49 pm on Nov 14, 2007 (gmt 0)

Moderator from US 

WebmasterWorld Administrator robert_charlton is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2000
votes: 254

Since I have read crawling involves a copyright infringement...

There's a good discussion of that here....

Anyone Try To Crawl Superpages.com?

9:43 am on Nov 26, 2007 (gmt 0)

Preferred Member

10+ Year Member

joined:Mar 22, 2002
posts: 453
votes: 0

I'm slightly late to the party, but not to worry.

You'll find that crawling and extracting data, regardless of how selective you are with the fields you choose, is a breach of most directories' terms of service and, worst case, will see you on the end of legal action; it is, in its simplest form, theft.

You'll also find that most IYPs seed their data with false data, which is used as a basis for identifying crawled data in breach of terms of service.

Where does that leave the individual who wants to create their own IYP? Well, one of two main options really. There's the start from scratch model, or purchasing data from an external source; the latter being potentially costly, but moving you closer to critical mass, at least in terms of numbers of businesses on your site, far sooner than the latter.

The updating issue you mention is something of a potential headache; the ideal is to have each listing on your directory contacted and their details verified at least annually; it's possible to do this via e-mail for at least some of your listings, but what about those that don't have an e-mail address? Can these be verified by telephone? Are they left as-is and the onus put on the end user to inform of any inaccuracies?

Purchasing data from an external source still presents questions in terms of updates; charges they make for updated data may vary, based upon how often you wish for the data to be refreshed and the level of information you'd like to receive.

Lots to think on! First thing I'd be mulling over would be what USP could your directory offer over those already in the marketplace? What features would users want or find useful? Are there any reasons why somebody isn't offering these already?


12:41 am on Nov 27, 2007 (gmt 0)

New User

5+ Year Member

joined:Nov 8, 2007
posts: 5
votes: 0

Hi and thanks for your answer!
In fact, crawling yips sounds not very ethical..What we had in mind was quite different from the common directories and we are still looking into the technologies behind it, (the possibility to adapt a commercial database to fit our requierements..)
It would also be very helpful if one of you had suggestions about the following case scenario:
- in the event of purchasing a commercial database, would there be any possibility for a business itself to alter, add or attach information to his profile? Or are the databases unchangeable?