Who benefits from big data?

The alarm goes off on your smartphone and you swipe it to snooze. But in the minutes between sleeping and showering, you are far from idle. You consult your calendar and confirm a meeting by email. Checking the local weather, you plan your journey on an app and text a colleague. After a quick online search for more details of your meeting and a glance through the news for conversation starters, there’s still time for a post on social media. By the time the alarm beeps again, you’ve generated a huge amount of data. And you haven’t even gotten out of bed.

As the world becomes more connected, we are generating a staggering amount of digital information: more than 2.5 quintillion bytes a day. Each minute we send more than 16 million text messages, 156 million emails, and share over one hundred million photos and videos. Our digital universe is fast approaching 44 zettabytes of data-that’s 44 trillion gigabytes, 90 percent of which was produced in the last two years. As early as 2005, people realized that buried within all this intel were patterns, connections, and insights that could be useful and profitable. Big data was born.

The term refers to extraordinarily huge and complex data sets that are beyond the capacity of conventional computing. Originally defined by the three V’s of greater volume, velocity, and variety, to be useful big data needs one more V: value. Data science has been driven to a whole new level with ever more powerful computers, advanced software, and complicated algorithms trawling through terabytes, petabytes, and exabytes of often unstructured data to find a nugget of valuable and verifiable information. It’s a global market with revenues fast approaching $200 billion-but who benefits from big data?

Cost and complexity meant that it was initially the preserve of big business, but thanks to cloud computing and free-to-use technologies such as Hadoop, the costs have come down exponentially. Big data is now accessible to almost every organization of any size: Small businesses, national governments, local farmers, and international health care organizations are all harnessing its power to achieve their specific goals more quickly and effectively. As it becomes ubiquitous, it’s impacting almost every aspect of our lives.

Retail was one of the first sectors to take advantage of big data, transforming the shopping experience to meet the expectations of tech-savvy consumers. Armed with vast amounts of information, retailers have gained an in-depth understanding of their customers. This helps them to predict a trend, pick a popular product, ensure its availability, price it competitively, and then market it to the right customers-even texting them special offers based on their movements around a store. Seizing the initiative, global retail giant Walmart has recently built a data center capable of managing 2.5 petabytes of store info every hour, while Macy’s credits big data with improving their customer interactions to the tune of a 10 percent boost in in-store sales.

As early as 2012 the Obama Administration announced a $200 million investment in big data projects, and governments around the world are using it to improve efficiencies across the board. One key area is law enforcement. Automated license plate recognition software helps identify, track, and intercept vehicles of interest, while predictive technologies can calculate trouble hotspots and link particular crimes to particular criminals. Similarly, with 20,000 closed-circuit television (CCTV) cameras in Manhattan alone, the ability to analyze vast amounts of footage and to monitor global online chatter might make all the difference in stopping a terror attack.

From field to table, big data is revolutionizing farming with information being drawn from countless sensors monitoring soil, crops, animals, and machinery across the entire agricultural process. This provides useful insights into everything from weather patterns to soil quality and where crops grow best, as well as predicting when particular interventions will be required, such as precise prescriptions of pesticides or additional irrigation. In the United States, the Department of Agriculture has used big data to increase the yield of dairy herds by analyzing the genetic records of bulls and identifying those most likely to breed high-yielding cows.

Arguably one of the areas where big data is doing the most good is in health care. Digitized medical records, research reports, and even wearables provide a mountain of intel that can be mined for information on the health of individuals and whole populations. As well as reducing costs, perhaps saving hundreds of billions of dollars in the U.S. alone, health care analytics can predict the outbreak of epidemics, prevent avoidable deaths, improve quality of life, and help to detect, understand, and treat diseases, including cancer. One cancer patient can generate one terabyte of data, and collectively this information is driving the search for the most effective treatments-it led to the discovery that a widely used antidepressant could potentially be used to treat lung cancer.

Right now, perhaps as little as 0.5 percent of the world’s accessible data is actually being analyzed and used, and with volumes set to double every two years, it’s going to be hard to keep pace. But as the technology driving big data continues to evolve and improve, the promise is of much more to come. The harnessing of this vast intelligence is only just getting started, and while we’re far from realizing its full potential, it’s already bringing benefits that are being felt individually and globally. From predicting mechanical failures to training machine-learning models to real-time actionable insights into almost anything, the future possibilities of big data seem as limitless and as universal as the data itself.

We asked some big questions about living a better life. Discover more about how we can overcome the world’s biggest challenges at natgeo.com/questionsforabetterlife