Archive for the ‘Web Applications’ Category

Twitter and Open Source

Февраль 19th, 2010

Twitter Open SourceHow do you build one of the busiest websites on the Internet? You wouldn’t guess the right answer to be, “You download some free software and hack  it”…Actually  the question is how do you build one of the world’s busiest websites that will scale affordably? You use open source software.

Twitter showed everyone their cards recently by publishing all the open source projects that they are contributing to. This is the picture of how open source software should work.

Organization has a a big, hard problem to solve. They write some software or update existing software and then publish what they did for others to use.

The people writing the software are writing it to solve a real problem. Not to sell it to people with real problems or to prevent the problems a clever sales guy manufactures. Given Twitter’s massive amount of users and traffic it’s probably one of the most demanding IT environments in the world. If these are the people making the software than you have some idea that it an stand up to scalability issues in other use cases (Failwhale’s not withstanding).

Not only do the Twitter engineers donate their work to the open source community they also share their insights on what happens in their environment on the Twitter Engineering Blog.

As I look at what the Twitter guys are working on it’s pretty impressive and has a fairly substantial impact on high availability computing. Some of the projects that caught my attention:

  • scribe.gem – A Ruby client for the Scribe distributed log server.
  • contributions to memcached.gem – An interface to the libmemcached C client. (memcache is a distributed systems for improving performance across dynamic web apps)
  • contributions to cassandra – A highly scalable second-generation distributed database.
  • contributions to hadoop – A platform for analyzing large data sets.
  • contributions to pig – A platform for analyzing large data sets.

Any how it’s good to see open source users giving back their improvements, I love it when a plan comes together.

Related Articles
Reblog this post [with Zemanta]

PlanetMySQL Voting: Vote UP / Vote DOWN