Msg#: 3515498 posted 6:35 pm on Dec 3, 2007 (gmt 0)
You can create a database with a unique key on the keyword and then insert your data, any duplicates will not be inserted.
Without sql you could write a quick php script that uses the keywords as the keys of an array, this will stop duplications and you can get the de-duplicated keywords back with array_keys. There are also more specific array functions like array_unique.
There are probably a few pieces of software about for it too, but its a very simple job with a bit of scripting.