Welcome to WebmasterWorld Guest from 54.147.0.174

Forum Moderators: httpwebwitch

Message Too Old, No Replies

XML is too slow to identify terrorists?

     

httpwebwitch

3:42 pm on Sep 2, 2008 (gmt 0)

WebmasterWorld Administrator httpwebwitch is a WebmasterWorld Top Contributor of All Time 10+ Year Member



A $500 million project called Railhead was intended to improve - and eventually replace - The Identities Datamart Environment (TIDE), a National Counterterrorism Center database that stores information about international terrorists. The database was badly bungled, so someone decided to switch the whole thing over to XML.

XML has many advantages, but performance is not one of them. The large size of XML data files results in lengthy transmission times. Thatís a problem because the more than 30 networks that interact with the database require rapid search results. A new generation of efficient XML code could ease matters, but it would still require moving TIDE to newer Oracle database management software that supports Semantic Web standards.

The "new generation of efficient XML code" is probably denormalized data delivered via multiple manageable schemas, generated by a relational database. The mention of Oracle seems to hint at this.

Costs and delays escalated, and a slew of software-testing failures suggested that the replacement system might have deeper flaws. Concerns came to a head in recent weeks, when the government fired most of the nearly 900 contractors working on the project.

source [gcn.com]

It's astounding that among 900 (ex)-contractors, there weren't any that could build an efficient database, and none that could discourage building a system that required downloading massive XML files.

XML is a perfectly viable carrier for transmitting complex data, but it's best consumed in small amounts, on demand. Replacing a huge relational database with a pile of flat XML files is ludicrous (I'm reading a little into this, but it sounds like that's what they attempted to do).

Stories of such ineptitude handling sensitive data like TIDE is disquieting, to say the least. The USA is a country filled with Universities and Technology Schools. Every one of them churns out graduates every year who are qualified to architect a database like this one. The number of contractors - 900? - seems to affirm the suspicion that the inept flourish in great numbers amid government beurocracy, while the talented and capable find better lodgings in the corporate sector.

I hope they don't forget to escape their ampersands.

LifeinAsia

3:55 pm on Sep 2, 2008 (gmt 0)

WebmasterWorld Administrator lifeinasia is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



I hope they don't forget to escape their ampersands.

They wanted to, but some government committee decided it wasn't necessary and took it out of the budget.

phranque

7:03 pm on Sep 2, 2008 (gmt 0)

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



The number of contractors - 900? - seems to affirm the suspicion that the inept flourish in great numbers amid government beurocracy, while the talented and capable find better lodgings in the corporate sector.

i think it actually affirms that corporate special interests flourish in the opaque environment of counterintelligence.
for example the prime contractor for TIDE was lockheed martin and the prime contractor for railhead is boeing.
one of the major subcontractors is mark logic, which has been around almost 4 whole years, and whose flagship product is a xml content server, which may explain why they ran so far down the xml path with this solution.

source [democrats.science.house.gov]

Demaestro

7:18 pm on Sep 2, 2008 (gmt 0)

WebmasterWorld Senior Member demaestro is a WebmasterWorld Top Contributor of All Time 10+ Year Member



900 contractors and not one got it right.

That seems about on par with my thinking, as I would say about 0.1% of "technical contractors" actually know what they are talking about when it comes to developing a viable solution.

Usually they come in and use words like "enterprise" and "dot net" come up with huge hardware budgets for things they get kickbacks on. Then when it is time to implement their plan they are on to the next project.

httpwebwitch

8:56 pm on Sep 2, 2008 (gmt 0)

WebmasterWorld Administrator httpwebwitch is a WebmasterWorld Top Contributor of All Time 10+ Year Member



I'm in the wrong business. I'd be much wealthier if I made a career giving bad advice to big gov't agencies, instead of good advice to WebmasterWorld newbs; better drafting partisan contracts for bloated IT projects than building solid, useful apps on a shoestring.

(sigh)

I'm somewhat reminded of the gun registry fiasco here in Canada

Demaestro

9:23 pm on Sep 2, 2008 (gmt 0)

WebmasterWorld Senior Member demaestro is a WebmasterWorld Top Contributor of All Time 10+ Year Member



LOL the gun registry. Worst DB ever!

I can't even stop laughing long enough to talk about it... they really messed that one up bad.

I can't remember some of the stats but things like 500,000 people ended up with guns registered to the same serial number (unique field anyone)

500,000 people ended up with the license being issued in the wrong name.

There were so many messed up things with that.

 

Featured Threads

Hot Threads This Week

Hot Threads This Month