The world’s capacity to store, broadcast and compute information is growing exponentially. In this digital age, the scale and velocity of data gathering is unprecedented and the amount of data in our society has exploded, thus analyzing large data sets—so-called Big Data—will become a key basis of competition, underpinning new waves of productivity growth, innovation, and consumer surplus. The increasing volume and detail of information captured by enterprises, the rise of multimedia, social media, and the Internet of Things will fuel exponential growth in data for the foreseeable future.
The ability to store, aggregate, and combine data and then perform deep analyses on the result has become even more accessible. The past decade has seen the increasing availability of very large scale data sets. These data sets arose from the rapid growth of transformative technologies such as the Internet and cellular telephones, along with the development of new and powerful computational methods to analyze such datasets. Such methods, developed in the closely related fields of machine learning, data mining, and artificial intelligence, provide a powerful set of tools for intelligent problem-solving and data-driven policy analysis. These methods have the potential to dramatically improve the public welfare by guiding policy decisions and interventions, and their incorporation into intelligent information systems will improve public services in domains ranging from medicine and public health to homeland security.
Digital data is now everywhere—in every sector, every economy, every organization and user of digital technology. There are many ways that big data can be used to create value across sectors of the global economy. While the use of big data will matter across sectors, some sectors are set for greater gains. Several United States industry sectors have the potential to capture value from Big Data with opportunities and challenges vary from sector to sector. The computer and electronic products and information sectors, as well as finance and insurance, and government are poised to gain substantially from the use of Big Data.
Data-driven discovery is revolutionizing scientific exploration and engineering innovations. Many of the Big Data tools and techniques pioneered in the private sector will have a role to play in the public sector. The White House has a formal Big Data Research and Development initiative to accelerate the use of Big Data in science and engineering, strengthen national security, and transform teaching and learning. To launch the initiative, six federal departments and agencies announced more than $200 million in new commitments that, together, promise to greatly improve the tools and techniques needed to access, organize, and glean discoveries from huge volumes of digital data. Properly executed, Big Data analytics can have a real and direct impact on the way policymakers work and citizens interact with governments.
The public and private sectors can boost their productivity significantly through the effective use of Big Data. However, companies and other organizations and policymakers need to address considerable challenges if they are to capture the full potential of Big Data.
A shortage of the analytical and managerial talent necessary to make the most of Big Data is a significant and pressing challenge and one that companies and policy makers can begin to address in the near term. There will be a shortage of talent necessary for organizations to take advantage of Big Data. By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of Big Data to make effective decisions. This could be effectively dealt with by increasing the number of high-skilled immigrants we allow to stay in the country.
Several other policy issues will have to be addressed as well to capture the full potential of Big Data. Policies related to privacy, security, intellectual property, and even liability will need to be addressed in a Big Data world. Organizations need not only to put the right talent and technology in place but structure workflows and incentives to optimize the use of Big Data. In addition, safeguards will need to be in place to address public concerns about Big Data.
TechAmerica continues to engage many of the leading technology companies in the Big Data industry, as well as the insight of experts in academia and to provide a road map for the federal government to better leverage and utilize Big Data and its necessary technologies. TechAmerica has put forward the following specific set of recommendations on steps that the federal government can take to accelerate the deployment of big data technologies:
Following up on the October 2012 release of the TechAmerica Foundation Big Data Commission report “Demystifying Big Data- A Practical Guide to Transforming the Business of Government,” TechAmerica’s Big Data Subcommittee will be working with the Administration to help implement many of the Commission’s recommendations. Congress has begun to take a keen interest in Big Data, holding hearings on how Big Data can reduce inefficiencies within federal agencies and eliminating waste, fraud, and abuse. Congress will also focus on the ‘state of the art’ in Big Data analytics, playing particular attention to R&D investment. TechAmerica has shared best practices with Congress and the Administration particularly in how Big Data relates to health IT, energy and finance. State legislatures around the country are taking up the issue of Big Data as well by beginning to look at ways to protect consumer privacy, and data from misuse.
The TechAmerica Foundation’s Big Data Commission released its findings October 3rd 2012 to help answer many critical questions about the definition and opportunities surrounding Big Data—including how this new technology will transform the way Washington does business.
The report, “Demystifying Big Data: A Practical Guide to Transforming the Business of Government,” provides the government’s senior policy and decision makers with a comprehensive roadmap to using Big Data to better serve the American people.
Vice Chairs: Fred Baradari, Xerox and Art Conroy, Cloudera
This Sub-Committee assists in the implementation of the TechAmerica Big Data Commission recommendations while also addressing industry ‘best practices’ with the USG.