Big data is a collection of data sets so large and complex that it becomes difficult to process using traditional database management applications. By this definition it appears that big data only reflects those in business and IT, but that is not the case.
You’ve never seen data presented like this. With the drama and urgency of a sportscaster, statistics guru Hans Rosling debunks myths about the so-called “developing world.”
This is the most unique presentation relating to the topic of Big Data I’ve ever seen. One example used by this presenter is “how would you visualize all those twitter posts relating to someone tweeting about where they just landed in order to get an idea of where people are traveling to?” Words can’t do…
Indeed, “a big data project can’t be treated like a conventional, large IT project, with its defined outcomes, required tasks, and detailed plans for carrying them out … Commissioned to address a problem or opportunity that someone has sensed, such a project frames questions to which data might provide answers, develops hypotheses, and then iteratively experiments to gain knowledge and understanding.”
Me? Twenty years ago, I was a statistical purist, an unabashed, hypothesis-driven fanatic – perhaps as much as the first author. In fact, I probably would have been considered a top-down analytics “planner.” I did, however, come to expand my horizon to a broader data science discipline that combines statistical orthodoxy with large N data, exploratory visualization and machine learning techniques – balancing an aggressive search for predictive relationships with the cross-validating protection of the no-pattern null hypothesis. So I guess I’ve now evolved into more a bottom-up data-driven “searcher.” Statistical science, though still a central tool, is only a part of a larger analytics portfolio.
Dateline: October 4, 2012 – Facebook reaches one billion users! One billion people connected on a single platform; one-seventh of the world’s population. If you assume 40,000 BCE as the start of modern humans, it took the planet 41,804 years to reach a population level of one billion; it took Facebook just 8 years to connect…
… and any fool will mind it”. ~ Henry David Thoreau. The EPA was in my neighborhood several weeks ago testing well water - never a good sign. It was determined that the subdivision just north and upstream of us had once been the site of a farm contaminated by the dumping of fuel, pesticides and…
The topic of “big data” clearly has reached a tipping point in 2012. With plenty of coverage over the past few years in the IT press, we are now starting to see the topic of “big data” covered in mainstream business press, including a cover story in the October 2012 issue of the Harvard Business Review.
To help customers understand the challenges of managing “big data” as well as the opportunities that can be created by leveraging “big data”, Oracle has recently run and published the results of a customer survey, as well as white papers and articles on this topic.
In the October 2010 edition of the Harvard Business Review, Thomas Davenport and D.J. Patil named “data scientist” as the “sexiest job of the 21 century.” As Big Data shifts from big news to big priority within even small businesses, data scientists will be hired to mine data, derive actionable insight from information and help both startups and mature businesses as they look to become more competitive and optimize operations.
If you’re interesting in becoming a data scientist, here are a few points to guide your career and ensure your resume is attractive for hot technology startups.
Are you a Data Hoarder, or a Data User?
Just as there’s gold in them thar hills, there’s insight hidden in that data, insight that can make a difference, and there are some pretty straightforward applications that can have quite an immediate impact.