|XML is too slow to identify terrorists?|
| 3:42 pm on Sep 2, 2008 (gmt 0)|
A $500 million project called Railhead was intended to improve - and eventually replace - The Identities Datamart Environment (TIDE), a National Counterterrorism Center database that stores information about international terrorists. The database was badly bungled, so someone decided to switch the whole thing over to XML.
|XML has many advantages, but performance is not one of them. The large size of XML data files results in lengthy transmission times. Thatís a problem because the more than 30 networks that interact with the database require rapid search results. A new generation of efficient XML code could ease matters, but it would still require moving TIDE to newer Oracle database management software that supports Semantic Web standards. |
The "new generation of efficient XML code" is probably denormalized data delivered via multiple manageable schemas, generated by a relational database. The mention of Oracle seems to hint at this.
|Costs and delays escalated, and a slew of software-testing failures suggested that the replacement system might have deeper flaws. Concerns came to a head in recent weeks, when the government fired most of the nearly 900 contractors working on the project. |
It's astounding that among 900 (ex)-contractors, there weren't any that could build an efficient database, and none that could discourage building a system that required downloading massive XML files.
XML is a perfectly viable carrier for transmitting complex data, but it's best consumed in small amounts, on demand. Replacing a huge relational database with a pile of flat XML files is ludicrous (I'm reading a little into this, but it sounds like that's what they attempted to do).
Stories of such ineptitude handling sensitive data like TIDE is disquieting, to say the least. The USA is a country filled with Universities and Technology Schools. Every one of them churns out graduates every year who are qualified to architect a database like this one. The number of contractors - 900? - seems to affirm the suspicion that the inept flourish in great numbers amid government beurocracy, while the talented and capable find better lodgings in the corporate sector.
I hope they don't forget to escape their ampersands.
| 3:55 pm on Sep 2, 2008 (gmt 0)|
|I hope they don't forget to escape their ampersands. |
They wanted to, but some government committee decided it wasn't necessary and took it out of the budget.
| 7:03 pm on Sep 2, 2008 (gmt 0)|
|The number of contractors - 900? - seems to affirm the suspicion that the inept flourish in great numbers amid government beurocracy, while the talented and capable find better lodgings in the corporate sector. |
i think it actually affirms that corporate special interests flourish in the opaque environment of counterintelligence.
for example the prime contractor for TIDE was lockheed martin and the prime contractor for railhead is boeing.
one of the major subcontractors is mark logic, which has been around almost 4 whole years, and whose flagship product is a xml content server, which may explain why they ran so far down the xml path with this solution.
| 7:18 pm on Sep 2, 2008 (gmt 0)|
900 contractors and not one got it right.
That seems about on par with my thinking, as I would say about 0.1% of "technical contractors" actually know what they are talking about when it comes to developing a viable solution.
Usually they come in and use words like "enterprise" and "dot net" come up with huge hardware budgets for things they get kickbacks on. Then when it is time to implement their plan they are on to the next project.
| 8:56 pm on Sep 2, 2008 (gmt 0)|
I'm in the wrong business. I'd be much wealthier if I made a career giving bad advice to big gov't agencies, instead of good advice to WebmasterWorld newbs; better drafting partisan contracts for bloated IT projects than building solid, useful apps on a shoestring.
I'm somewhat reminded of the gun registry fiasco here in Canada
| 9:23 pm on Sep 2, 2008 (gmt 0)|
LOL the gun registry. Worst DB ever!
I can't even stop laughing long enough to talk about it... they really messed that one up bad.
I can't remember some of the stats but things like 500,000 people ended up with guns registered to the same serial number (unique field anyone)
500,000 people ended up with the license being issued in the wrong name.
There were so many messed up things with that.