Hadoop And The Opposite Of The Not-Invented-Here Syndrome

Microsoft is famous for having a really bad case of ‘not-invented-here’ syndrome. They don’t like to accept any protocol or standard or even take a perfectly working piece of software and include it. It wasn’t invented at Microsoft so it’s automatically crap. They have to “fix it”. Yahoo! appears to have turned that on its head.

Yahoo’s biggest competitor is arguably Google. Google invented an algorithm for data processing called MapReduce. They use it to process the terabytes and petabytes of data they grind through on a regular basis. They piggy back that on top of their storage system called GFS (Google File System). Because Google published papers on all this software, even though they don’t make the software itself available, there was enough description for people to start developing their own versions of the Google tools.

Yahoo has now decided to both use and endorse the toolset Hadoop. Hadoop encompasses implementations of both GFS and MapReduce so arguably Yahoo is now running software that is based on ideas from their direct competitor. They aren’t shy about it either, they aren’t hiding it, rather they are telling the world that the software is good, they like it, and they intend to support it.

Bravo. I’m impressed.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s