Alright Perlsters - How would one go about the following?
a) Get from Wikipedia a list of persons categorised in a certain way eg. Movie stars born in Iowa; b) compare that list with an integer taken from their entry in imdb eg. Number of films they have appeared in; c) put them in a table listed by order of that integer eg. movie stars born in Iowa sorted by number of films they appeared in. d) Update the data daily.
Is this a task for mechanize? Someone must have done this already :D
Thanks guys. Wikipedia also has an api - well, more of a fake-browser really - but they haven't yet implemented the ability to draw down lists of articles in various categories :( It's interesting that IMDB has a TOS prohibition on outright spidering, but they give away their entire dataset. It's also quite odd that they don't yet have an api.