Note

You are reading the documentation for MMClassification 0.x, which will soon be deprecated at the end of 2022. We recommend you upgrade to MMClassification 1.0 to enjoy fruitful new features and better performance brought by OpenMMLab 2.0. Check the installation tutorial, migration tutorial and changelog for more details.

mmcls.datasets¶

The datasets package contains several usual datasets for image classification tasks and some dataset wrappers.

Custom Dataset¶

class mmcls.datasets.CustomDataset(data_prefix: str, pipeline: Sequence = (), classes: Optional[Union[str, Sequence[str]]] = None, ann_file: Optional[str] = None, extensions: Sequence[str] = ('.jpg', '.jpeg', '.png', '.ppm', '.bmp', '.pgm', '.tif'), test_mode: bool = False, file_client_args: Optional[dict] = None)[source]¶

Custom dataset for classification.

The dataset supports two kinds of annotation format.

An annotation file is provided, and each line indicates a sample:

The sample files:

data_prefix/
├── folder_1
│   ├── xxx.png
│   ├── xxy.png
│   └── ...
└── folder_2
    ├── 123.png
    ├── nsdf3.png
    └── ...

The annotation file (the first column is the image path and the second column is the index of category):

folder_1/xxx.png 0
folder_1/xxy.png 1
folder_2/123.png 5
folder_2/nsdf3.png 3
...

Please specify the name of categories by the argument classes.

The samples are arranged in the specific way:

data_prefix/
├── class_x
│   ├── xxx.png
│   ├── xxy.png
│   └── ...
│       └── xxz.png
└── class_y
    ├── 123.png
    ├── nsdf3.png
    ├── ...
    └── asd932_.png

If the ann_file is specified, the dataset will be generated by the first way, otherwise, try the second way.

Parameters

data_prefix (str) – The path of data directory.
pipeline (Sequence[dict]) – A list of dict, where each element represents a operation defined in mmcls.datasets.pipelines. Defaults to an empty tuple.
classes (str | Sequence[str], optional) –
Specify names of classes.
- If is string, it should be a file path, and the every line of the file is a name of a class.
- If is a sequence of string, every item is a name of class.
- If is None, use cls.CLASSES or the names of sub folders (If use the second way to arrange samples).
Defaults to None.
ann_file (str, optional) – The annotation file. If is string, read samples paths from the ann_file. If is None, find samples in data_prefix. Defaults to None.
extensions (Sequence[str]) – A sequence of allowed extensions. Defaults to (‘.jpg’, ‘.jpeg’, ‘.png’, ‘.ppm’, ‘.bmp’, ‘.pgm’, ‘.tif’).
test_mode (bool) – In train mode or test mode. It’s only a mark and won’t be used in this class. Defaults to False.
file_client_args (dict, optional) – Arguments to instantiate a FileClient. See mmcv.fileio.FileClient for details. If None, automatically inference from the specified path. Defaults to None.

ImageNet¶

class mmcls.datasets.ImageNet(data_prefix: str, pipeline: Sequence = (), classes: Optional[Union[str, Sequence[str]]] = None, ann_file: Optional[str] = None, test_mode: bool = False, file_client_args: Optional[dict] = None)[source]¶

ImageNet Dataset.

The dataset supports two kinds of annotation format. More details can be found in CustomDataset.

Parameters

data_prefix (str) – The path of data directory.
pipeline (Sequence[dict]) – A list of dict, where each element represents a operation defined in mmcls.datasets.pipelines. Defaults to an empty tuple.
classes (str | Sequence[str], optional) –
Specify names of classes.
- If is string, it should be a file path, and the every line of the file is a name of a class.
- If is a sequence of string, every item is a name of class.
- If is None, use the default ImageNet-1k classes names.
Defaults to None.
ann_file (str, optional) – The annotation file. If is string, read samples paths from the ann_file. If is None, find samples in data_prefix. Defaults to None.
extensions (Sequence[str]) – A sequence of allowed extensions. Defaults to (‘.jpg’, ‘.jpeg’, ‘.png’, ‘.ppm’, ‘.bmp’, ‘.pgm’, ‘.tif’).
test_mode (bool) – In train mode or test mode. It’s only a mark and won’t be used in this class. Defaults to False.
file_client_args (dict, optional) – Arguments to instantiate a FileClient. See mmcv.fileio.FileClient for details. If None, automatically inference from the specified path. Defaults to None.

class mmcls.datasets.ImageNet21k(data_prefix: str, pipeline: Sequence = (), classes: Optional[Union[str, Sequence[str]]] = None, ann_file: Optional[str] = None, serialize_data: bool = True, multi_label: bool = False, recursion_subdir: bool = True, test_mode=False, file_client_args: Optional[dict] = None)[source]¶

ImageNet21k Dataset.

Since the dataset ImageNet21k is extremely big, cantains 21k+ classes and 1.4B files. This class has improved the following points on the basis of the class ImageNet, in order to save memory, we enable the serialize_data optional by default. With this option, the annotation won’t be stored in the list data_infos, but be serialized as an array.

Parameters

data_prefix (str) – The path of data directory.
pipeline (Sequence[dict]) – A list of dict, where each element represents a operation defined in mmcls.datasets.pipelines. Defaults to an empty tuple.
classes (str | Sequence[str], optional) –
Specify names of classes.
- If is string, it should be a file path, and the every line of the file is a name of a class.
- If is a sequence of string, every item is a name of class.
- If is None, the object won’t have category information. (Not recommended)
Defaults to None.
ann_file (str, optional) – The annotation file. If is string, read samples paths from the ann_file. If is None, find samples in data_prefix. Defaults to None.
serialize_data (bool) – Whether to hold memory using serialized objects, when enabled, data loader workers can use shared RAM from master process instead of making a copy. Defaults to True.
multi_label (bool) – Not implement by now. Use multi label or not. Defaults to False.
recursion_subdir (bool) – Deprecated, and the dataset will recursively get all images now.
test_mode (bool) – In train mode or test mode. It’s only a mark and won’t be used in this class. Defaults to False.
file_client_args (dict, optional) – Arguments to instantiate a FileClient. See mmcv.fileio.FileClient for details. If None, automatically inference from the specified path. Defaults to None.

CIFAR¶

class mmcls.datasets.CIFAR10(data_prefix, pipeline, classes=None, ann_file=None, test_mode=False)[source]¶

CIFAR10 Dataset.

This implementation is modified from https://github.com/pytorch/vision/blob/master/torchvision/datasets/cifar.py

class mmcls.datasets.CIFAR100(data_prefix, pipeline, classes=None, ann_file=None, test_mode=False)[source]¶: CIFAR100 Dataset.

MNIST¶

class mmcls.datasets.MNIST(data_prefix, pipeline, classes=None, ann_file=None, test_mode=False)[source]¶

MNIST Dataset.

This implementation is modified from https://github.com/pytorch/vision/blob/master/torchvision/datasets/mnist.py

class mmcls.datasets.FashionMNIST(data_prefix, pipeline, classes=None, ann_file=None, test_mode=False)[source]¶: Fashion-MNIST Dataset.

VOC¶

class mmcls.datasets.VOC(difficult_as_postive=None, **kwargs)[source]¶

Pascal VOC Dataset.

Parameters

data_prefix (str) – the prefix of data path
pipeline (list) – a list of dict, where each element represents a operation defined in mmcls.datasets.pipelines
ann_file (str | None) – the annotation file. When ann_file is str, the subclass is expected to read from the ann_file. When ann_file is None, the subclass is expected to read according to data_prefix
difficult_as_postive (Optional[bool]) – Whether to map the difficult labels as positive. If it set to True, map difficult examples to positive ones(1), If it set to False, map difficult examples to negative ones(0). Defaults to None, the difficult labels will be set to ‘-1’.

StanfordCars Cars¶

class mmcls.datasets.StanfordCars(data_prefix: str, test_mode: bool, ann_file: Optional[str] = None, **kwargs)[source]¶

Stanford Cars Dataset.

After downloading and decompression, the dataset directory structure is as follows.

Stanford Cars dataset directory:

Stanford Cars
├── cars_train
│   ├── 00001.jpg
│   ├── 00002.jpg
│   └── ...
├── cars_test
│   ├── 00001.jpg
│   ├── 00002.jpg
│   └── ...
└── devkit
    ├── cars_meta.mat
    ├── cars_train_annos.mat
    ├── cars_test_annos.mat
    ├── cars_test_annoswithlabels.mat
    ├── eval_train.m
    └── train_perfect_preds.txt

Parameters

data_prefix (str) – the prefix of data path
test_mode (bool) – test_mode=True means in test phase. It determines to use the training set or test set.
ann_file (str, optional) – The annotation file. If is string, read samples paths from the ann_file. If is None, read samples path from cars_{train|test}_annos.mat file. Defaults to None.

Base classes¶

class mmcls.datasets.BaseDataset(data_prefix, pipeline, classes=None, ann_file=None, test_mode=False)[source]¶

Base dataset.

Parameters

data_prefix (str) – the prefix of data path
pipeline (list) – a list of dict, where each element represents a operation defined in mmcls.datasets.pipelines
ann_file (str | None) – the annotation file. When ann_file is str, the subclass is expected to read from the ann_file. When ann_file is None, the subclass is expected to read according to data_prefix
test_mode (bool) – in train mode or test mode

class mmcls.datasets.MultiLabelDataset(data_prefix, pipeline, classes=None, ann_file=None, test_mode=False)[source]¶: Multi-label Dataset.

Dataset Wrappers¶

class mmcls.datasets.ConcatDataset(datasets, separate_eval=True)[source]¶

A wrapper of concatenated dataset.

Same as torch.utils.data.dataset.ConcatDataset, but add get_cat_ids function.

Parameters

datasets (list[BaseDataset]) – A list of datasets.
separate_eval (bool) – Whether to evaluate the results separately if it is used as validation dataset. Defaults to True.

class mmcls.datasets.RepeatDataset(dataset, times)[source]¶

A wrapper of repeated dataset.

The length of repeated dataset will be times larger than the original dataset. This is useful when the data loading time is long but the dataset is small. Using RepeatDataset can reduce the data loading time between epochs.

Parameters

dataset (BaseDataset) – The dataset to be repeated.
times (int) – Repeat times.

class mmcls.datasets.ClassBalancedDataset(dataset, oversample_thr)[source]¶

A wrapper of repeated dataset with repeat factor.

Suitable for training on class imbalanced datasets like LVIS. Following the sampling strategy in this paper, in each epoch, an image may appear multiple times based on its “repeat factor”.

The repeat factor for an image is a function of the frequency the rarest category labeled in that image. The “frequency of category c” in [0, 1] is defined by the fraction of images in the training set (without repeats) in which category c appears.

The dataset needs to implement self.get_cat_ids() to support ClassBalancedDataset.

The repeat factor is computed as followed.

For each category c, compute the fraction \(f(c)\) of images that contain it.
For each category c, compute the category-level repeat factor.

\[r(c) = \max(1, \sqrt{\frac{t}{f(c)}})\]

where \(t\) is oversample_thr.
For each image I and its labels \(L(I)\), compute the image-level repeat factor.

\[r(I) = \max_{c \in L(I)} r(c)\]

Each image repeats \(\lceil r(I) \rceil\) times.

Parameters

dataset (BaseDataset) – The dataset to be repeated.
oversample_thr (float) – frequency threshold below which data is repeated. For categories with f_c >= oversample_thr, there is no oversampling. For categories with f_c < oversample_thr, the degree of oversampling following the square-root inverse frequency heuristic above.