Challenges and Opportunities with Big Data11. IntroductionWe are awash in a flood of data today. In a broad range of application areas, data is beingcollected at unprecedented scale. Decisions that previously were based on guesswork, or onpainstakingly constructed models of reality, can now be made based on the data itself. Such Big Dataanalysis now drives nearly every aspect of our modern society, including mobile services, retail,manufacturing, financial services, life sciences, and physical sciences.Scientific research has been revolutionized by Big Data [CCC2011a]. The Sloan Digital Sky Survey[SDSS2008] has today become a central resource for astronomers the world over. The field ofAstronomy is being transformed from one where taking pictures of the sky was a large part of anastronomer’s job to one where the pictures are all in a database already and the astronomer’s task is tofind interesting objects and phenomena in the database. In the biological sciences, there is now a well-‐established tradition of depositing scientific data into a public repository, and also of creating publicdatabases for use by other scientists. In fact, there is an entire discipline of bioinformatics that is largelydevoted to the curation and analysis of such data. As technology advances, particularly with the adventof Next Generation Sequencing, the size and number of experimental data sets available is increasingexponentially.Big Data has the potential to revolutionize not just research, but also education [CCC2011b]. Arecent detailed quantitative comparison of different approaches taken by 35 charter schools in NYC hasfound that one of the top five policies correlated with measurable academic effectiveness was the use ofdata to guide instruction [DF2011]. Imagine a world in which we have access to a huge database wherewe collect every detailed measure of every student's academic performance. This data could be used todesign the most effective approaches to education, starting from reading, writing, and math, toadvanced, college-‐level, courses. We are far from having access to such data, but there are powerfultrends in this direction. In particular, there is a strong trend for massive Web deployment ofeducational activities, and this will generate an increasingly large amount of detailed data aboutstudents' performance.It is widely believed that the use of information technology can reduce the cost of healthcarewhile improving its quality [CCC2011c], by making care more preventive and personalized and basing iton more extensive (home-‐based) continuous monitoring. McKinsey estimates [McK2011] a savings of300 billion dollars every year in the US alone.In a similar vein, there have been persuasive cases made for the value of Big Data for urbanplanning (through fusion of high-‐fidelity geographical data), intelligent transportation (through analysisand visualization of live and detailed road network data), environmental modeling (through sensornetworks ubiquitously collecting data) [CCC2011d], energy saving (through unveiling patterns of use),smart materials (through the new materials genome initiative [MGI2011]), computational social sciences15[PCAST2010] Designing a Digital Future: Federally Funded Research and Development in Networkingand Information Technology. PCAST Report, Dec. 2010. Available athttp://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-‐nitrd-‐report-‐2010.pdf[SDSS2008] SDSS-‐III: Massive Spectroscopic Surveys of the Distant Universe, the Milky Way Galaxy,and Extra-‐Solar Planetary Systems. Jan. 2008. Available athttp://www.sdss3.org/collaboration/description.pdf16About this DocumentThis white paper was created through a distributed conversation among many prominent researcherslisted below. This conversation lasted a period of approximately three months from Nov. 2011 to Feb.2012. Collaborative writing was supported by a distributed document editor.Divyakant Agrawal, UC Santa BarbaraPhilip Bernstein, MicrosoftElisa Bertino, Purdue Univ.Susan Davidson, Univ. of PennsylvaniaUmeshwar Dayal, HPMichael Franklin, UC BerkeleyJohannes Gehrke, Cornell Univ.Laura Haas, IBMAlon Halevy, GoogleJiawei Han, UIUCH. V. Jagadish, Univ. of Michigan (Coordinator)Alexandros Labrinidis, Univ. of PittsburghSam Madden, MITYannis Papakonstantinou, UC San Diego
Question
Challenges and Opportunities with Big Data11. IntroductionWe are awash in a flood of data today. In a broad range of application areas, data is beingcollected at unprecedented scale. Decisions that previously were based on guesswork, or onpainstakingly constructed models of reality, can now be made based on the data itself. Such Big Dataanalysis now drives nearly every aspect of our modern society, including mobile services, retail,manufacturing, financial services, life sciences, and physical sciences.Scientific research has been revolutionized by Big Data [CCC2011a]. The Sloan Digital Sky Survey[SDSS2008] has today become a central resource for astronomers the world over. The field ofAstronomy is being transformed from one where taking pictures of the sky was a large part of anastronomer’s job to one where the pictures are all in a database already and the astronomer’s task is tofind interesting objects and phenomena in the database. In the biological sciences, there is now a well-‐established tradition of depositing scientific data into a public repository, and also of creating publicdatabases for use by other scientists. In fact, there is an entire discipline of bioinformatics that is largelydevoted to the curation and analysis of such data. As technology advances, particularly with the adventof Next Generation Sequencing, the size and number of experimental data sets available is increasingexponentially.Big Data has the potential to revolutionize not just research, but also education [CCC2011b]. Arecent detailed quantitative comparison of different approaches taken by 35 charter schools in NYC hasfound that one of the top five policies correlated with measurable academic effectiveness was the use ofdata to guide instruction [DF2011]. Imagine a world in which we have access to a huge database wherewe collect every detailed measure of every student's academic performance. This data could be used todesign the most effective approaches to education, starting from reading, writing, and math, toadvanced, college-‐level, courses. We are far from having access to such data, but there are powerfultrends in this direction. In particular, there is a strong trend for massive Web deployment ofeducational activities, and this will generate an increasingly large amount of detailed data aboutstudents' performance.It is widely believed that the use of information technology can reduce the cost of healthcarewhile improving its quality [CCC2011c], by making care more preventive and personalized and basing iton more extensive (home-‐based) continuous monitoring. McKinsey estimates [McK2011] a savings of300 billion dollars every year in the US alone.In a similar vein, there have been persuasive cases made for the value of Big Data for urbanplanning (through fusion of high-‐fidelity geographical data), intelligent transportation (through analysisand visualization of live and detailed road network data), environmental modeling (through sensornetworks ubiquitously collecting data) [CCC2011d], energy saving (through unveiling patterns of use),smart materials (through the new materials genome initiative [MGI2011]), computational social sciences15[PCAST2010] Designing a Digital Future: Federally Funded Research and Development in Networkingand Information Technology. PCAST Report, Dec. 2010. Available athttp://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-‐nitrd-‐report-‐2010.pdf[SDSS2008] SDSS-‐III: Massive Spectroscopic Surveys of the Distant Universe, the Milky Way Galaxy,and Extra-‐Solar Planetary Systems. Jan. 2008. Available athttp://www.sdss3.org/collaboration/description.pdf16About this DocumentThis white paper was created through a distributed conversation among many prominent researcherslisted below. This conversation lasted a period of approximately three months from Nov. 2011 to Feb.2012. Collaborative writing was supported by a distributed document editor.Divyakant Agrawal, UC Santa BarbaraPhilip Bernstein, MicrosoftElisa Bertino, Purdue Univ.Susan Davidson, Univ. of PennsylvaniaUmeshwar Dayal, HPMichael Franklin, UC BerkeleyJohannes Gehrke, Cornell Univ.Laura Haas, IBMAlon Halevy, GoogleJiawei Han, UIUCH. V. Jagadish, Univ. of Michigan (Coordinator)Alexandros Labrinidis, Univ. of PittsburghSam Madden, MITYannis Papakonstantinou, UC San Diego
Solution
Big Data refers to the vast amount of data that is being collected in various fields and industries. This data is being used to drive decision-making processes and revolutionize research, education, healthcare, urban planning, transportation, and more.
In the field of scientific research, Big Data has transformed the way astronomers conduct their work. Instead of taking pictures of the sky, astronomers now have access to a database of images and their task is to find interesting objects and phenomena within this database. Similarly, in the biological sciences, there is a tradition of depositing scientific data into public repositories and creating databases for use by other scientists. The field of bioinformatics is dedicated to the curation and analysis of such data, which is growing exponentially with advancements in technology.
Big Data also has the potential to revolutionize education. By collecting detailed data on students' academic performance, it is possible to design more effective approaches to education, from basic subjects like reading and math to advanced college-level courses. While we are still far from having access to such comprehensive data, the trend of massive web deployment of educational activities is generating a large amount of detailed data on students' performance.
In the healthcare industry, the use of Big Data can potentially reduce costs and improve quality. By leveraging information technology, care can become more preventive and personalized, with extensive home-based continuous monitoring. Estimates suggest that the use of Big Data in healthcare could save billions of dollars annually in the US alone.
Big Data also holds promise for urban planning, intelligent transportation, environmental modeling, energy saving, smart materials, and computational social sciences. By analyzing and visualizing high-fidelity geographical data, live road network data, sensor networks, and patterns of energy use, it is possible to make informed decisions and improve efficiency in these areas.
This white paper was created through a collaborative effort among prominent researchers in the field. Their discussions and contributions spanned over a period of three months, resulting in this comprehensive document on the challenges and opportunities presented by Big Data.
Similar Questions
challenges and limitations of big data
what is FUTURE TRENDS IN BIG DATA?
what Big Data is, some examples of devices or technology that collect data, and some examples of big data at work in a positive way. why data is relevant to AI, and the critiques of data’s ‘objectivity’. the ideas in the required reading,big data is having on how we understand the world. discuss some of the implications that this impact of big data is having
What is Big Data
Big data:A. Uses analytical tools based on artificial intelligence and machine learning to mine vast data troves.B. Is used to find small patterns or correlations that reveal new insights or truths.C. Represents a new frontier in the way in which information is processed and used to inform decision-making.D. Analyses the way that humans make decisions using information.Group of answer choicesA and B, but not C and DB onlyA, B and C, but not DD only
Upgrade your grade with Knowee
Get personalized homework help. Review tough concepts in more detail, or go deeper into your topic by exploring other relevant questions.