Big data has grown from the expanding number of devices, sensors, and people generating information, but machine learning does not always need massive datasets to be effective. In practice, many researchers and businesses operate under time, cost, and resource constraints that limit how much data they can gather and curate. At the same time, privacy regulation and data minimization norms—exemplified by GDPR’s requirement to collect only adequate and relevant personal data for a defined purpose—create pressure to reduce unnecessary data collection, which can also streamline data governance.
Small data can be valuable in domains where large datasets are difficult or impossible to obtain (such as hard-to-measure physical phenomena or microscopic chemical structures), where computing needs must be reduced, or where ML features on mobile and IoT devices add significant value. Small data also reframes the problem from amassing volume to maintaining context, including correlating patterns at the level of an individual’s behaviors across their personal devices, closely tied to IoT.
However, small datasets introduce serious technical risks, especially for deep learning: high generalization error, overfitting due to too many parameters, and class imbalance that can encode bias (e.g., datasets dominated by white faces). As a result, small data should be a situational strategy rather than an overarching goal. Several approaches can make small-data ML more viable: using lower-complexity classifiers less prone to overfitting, applying ensemble methods like random forests to reduce variance, leveraging pre-trained models via transfer learning to achieve specialized tasks with fewer inputs, training with federated learning to preserve privacy while drawing from diverse sources, and enabling on-device inference through model compression for efficient memory and power usage. Decision-makers are encouraged to monitor maturity, strengthen governance, augment limited datasets, and align data strategy tightly to real use cases.
