Data. It’s everywhere today, and it keeps on growing. According to Eric Schmidt, former CEO of Google, every two days we now create as much information as humanity did from the beginning of recorded history until 2003.

Sure, a lot of that data is user generated: pictures, instant messages, tweets, wall posts. Nevertheless, it’s valuable; in fact, it’s valuable because it’s user generated. Just ask Facebook about advertising and data mining.

All that data represents the future of many businesses as they seek to be more competitive in a global market. I’m sure you can name some of them: tech companies like Google and LinkedIn. But because of all that data, the lines are blurring between traditional non-information technology companies and IT companies.

Case in point: retail giant Wal-Mart. The company has been hiring a lot of developers to staff its @Walmartlabs unit. What is @WalmartLabs?

In May 2011, Wal-Mart purchased Kosmix, a social media technology provider that created a platform that enables users to filter and organize content in social networks.

Content in social networks = Big Data

Walmart bought Kosmix to integrate data from social and mobile commerce. @Walmartlabs is building what they call a ”Social Genome,” based on billions of messages from social media, blog posts, YouTube videos, and more, and organizing that content to connect people with the information that matters to them.

The Social Genome is a huge, constantly changing, knowledgebase, with hundreds of millions of entities and relationships between them. @WalmartLabs can use the Social Genome to infer interests from and provide context for data from social media.

The real power comes in when Wal-Mart is able to combine data on purchase history with data from social networks with actual transaction history. As Anand Rajaraman, who runs @WalmartLabs, put it in an interview with Reuters’ Alistair Barr: “It’s a race to see who can use all this data the best. This will change the retail industry, as well as most other industries.” The transaction history shows what customers have bought in the past, while social networking data has the potential to show what they may buy in the future.

via SmartPlanet