Curse of dimensionality
The curse of dimensionality can refer to a number of phenomenon related to highdimensional data in several fields. In terms of machine learning for radiology, it generally refers to the phenomenon that as the number of image features employed to train an algorithm increases there is a geometric increase in the number of training examples required.
On this page:
Background
A feature is a quantity or trait based on which machine learning algorithms make predictions. Supervised machine learning algorithms take a collection of features as their inputs. Ideally, an algorithm uses the minimum number of features that can differentiate between possible answers. In the context of machine learning in clinical medicine, features may take on many different categories of variables; this may include patient demographic data (e.g. age, sex, weight), clinical characteristics (e.g. CRP, heart rate, temperature), or information derived from medical imaging (e.g. greyscale value of each individual pixel)
Intuition
As the number of features increases, the number of data points (or “training examples” in machine learning) required to train the algorithm increases exponentially. The intuition behind this can be visualized by imagining a set of 10 points on a line 10 centimeters long. If we increase the dimension of this line to 2 (i.e. it becomes a square), the number of points required for a similar density is increased to 100 (or 10^{2}). This property of an exponentiallygrowing requirement for input data as the number of features increases is known as the curse of dimensionality.
Importance in radiology
In the context of radiology, the number of input features can grow very large, particularly in pixelbased machine learning algorithms where each pixel (or voxel) of an input image represents a distinct feature. The process of reducing the dimension size of the input into machine learning algorithms to avoid the curse of dimensionality is known as dimensionality reduction. In medical imaging, this typically involves one or more preprocessing steps applied to inputted images aimed at extracting the most salient features of the images.
History and etymology
The term "curse of dimensionality" was originally used by Richard Bellman, a mathematician, in 1957 to describe some mathematical phenomenon associated with additional dimensions in mathematical spaces ^{3}. One implication of this "curse" was exponentially increasing computational heaviness, an extremely important issue before graphical processing units (GPUs) became popular. Over half a century later, the term is now used by many AI practitioners to imply problems with datasets with many features (which almost all radiology datasets are).
Related Radiopaedia articles
Artificial intelligence
 artificial intelligence (AI)
 imaging data sets
 computeraided diagnosis (CAD)
 natural language processing
 machine learning (overview)
 visualizing and understanding neural networks
 common data preparation/preprocessing steps
 DICOM to bitmap conversion
 dimensionality reduction
 scaling
 centering
 normalization
 principal component analysis
 training, testing and validation datasets
 augmentation
 loss function

optimization algorithms
 ADAM
 momentum (Nesterov)
 stochastic gradient descent
 minibatch gradient descent

regularisation
 linear and quadratic
 batch normalization
 ensembling
 rulebased expert systems
 glossary
 activation function
 anomaly detection
 automation bias
 backpropagation
 batch size
 computer vision
 concept drift
 cost function
 confusion matrix
 convolution
 cross validation
 curse of dimensionality
 dice similarity coefficient
 dimensionality reduction
 epoch
 explainable artificial intelligence/XAI
 feature extraction
 federated learning
 gradient descent
 ground truth
 hyperparameters
 image registration
 imputation
 iteration
 jaccard index
 linear algebra
 noise reduction
 normalization
 R (Programming language)
 radiomics quality score (RQS)
 Python (Programming language)
 segmentation
 semisupervised learning
 synthetic and augmented data
 overfitting
 underfitting
 transfer learning