Challenges of Big Data analysis
Princeton University · Johns Hopkins University
Abstract
Big Data bring new opportunities to modern society and challenges to data scientists. On one hand, Big Data hold great promises for discovering subtle population patterns and heterogeneities that are not possible with small-scale data. On the other hand, the massive sample size and high dimensionality of Big Data introduce unique computational and statistical challenges, including scalability and storage bottleneck, noise accumulation, spurious correlation, incidental endogeneity, and measurement errors. These challenges are distinguished and require new computational and statistical paradigm. This article gives overviews on the salient features of Big Data and how these features impact on paradigm change on…
Citation impact
- FWCI
- 92.36
- Percentile
- 100%
- References
- 174
Authors
3Topics & keywords
- Big data
- Data science
- Spurious relationship
- Computer science
- Bottleneck
- Endogeneity
- Scalability
- Population