mmcls.datasets¶

dataset 包中包含了分类任务中常用的数据集，以及一些数据集封装。

mmcls.datasets

Custom Dataset
ImageNet
CIFAR
MNIST
VOC
CUB
Base classes
Dataset Wrappers

Custom Dataset ¶

class mmcls.datasets.CustomDataset(ann_file='', metainfo=None, data_root='', data_prefix='', extensions=('.jpg', '.jpeg', '.png', '.ppm', '.bmp', '.pgm', '.tif'), lazy_init=False, **kwargs)[源代码]¶

Custom dataset for classification.

The dataset supports two kinds of annotation format.

An annotation file is provided, and each line indicates a sample:

The sample files:

data_prefix/
├── folder_1
│   ├── xxx.png
│   ├── xxy.png
│   └── ...
└── folder_2
    ├── 123.png
    ├── nsdf3.png
    └── ...

The annotation file (the first column is the image path and the second column is the index of category):

folder_1/xxx.png 0
folder_1/xxy.png 1
folder_2/123.png 5
folder_2/nsdf3.png 3
...

Please specify the name of categories by the argument classes or metainfo.

The samples are arranged in the specific way:

data_prefix/
├── class_x
│   ├── xxx.png
│   ├── xxy.png
│   └── ...
│       └── xxz.png
└── class_y
    ├── 123.png
    ├── nsdf3.png
    ├── ...
    └── asd932_.png

If the ann_file is specified, the dataset will be generated by the first way, otherwise, try the second way.

参数

ann_file (str) – Annotation file path. Defaults to ‘’.
metainfo (dict, optional) – Meta information for dataset, such as class information. Defaults to None.
data_root (str) – The root directory for data_prefix and ann_file. Defaults to ‘’.
data_prefix (str | dict) – Prefix for the data. Defaults to ‘’.
extensions (Sequence[str]) – A sequence of allowed extensions. Defaults to (‘.jpg’, ‘.jpeg’, ‘.png’, ‘.ppm’, ‘.bmp’, ‘.pgm’, ‘.tif’).
lazy_init (bool) – Whether to load annotation during instantiation. In some cases, such as visualization, only the meta information of the dataset is needed, which is not necessary to load annotation file. Basedataset can skip load annotations to save time by set lazy_init=False. Defaults to False.
**kwargs – Other keyword arguments in BaseDataset.

ImageNet ¶

class mmcls.datasets.ImageNet(ann_file='', metainfo=None, data_root='', data_prefix='', **kwargs)[源代码]¶

ImageNet Dataset.

The dataset supports two kinds of annotation format. More details can be found in CustomDataset.

参数

ann_file (str) – Annotation file path. Defaults to ‘’.
metainfo (dict, optional) – Meta information for dataset, such as class information. Defaults to None.
data_root (str) – The root directory for data_prefix and ann_file. Defaults to ‘’.
data_prefix (str | dict) – Prefix for training data. Defaults to ‘’.
**kwargs – Other keyword arguments in CustomDataset and BaseDataset.

class mmcls.datasets.ImageNet21k(ann_file='', metainfo=None, data_root='', data_prefix='', multi_label=False, **kwargs)[源代码]¶

ImageNet21k Dataset.

Since the dataset ImageNet21k is extremely big, cantains 21k+ classes and 1.4B files. We won’t provide the default categories list. Please specify it from the classes argument.

参数

ann_file (str) – Annotation file path. Defaults to ‘’.
metainfo (dict, optional) – Meta information for dataset, such as class information. Defaults to None.
data_root (str) – The root directory for data_prefix and ann_file. Defaults to ‘’.
data_prefix (str | dict) – Prefix for training data. Defaults to ‘’.
multi_label (bool) – Not implement by now. Use multi label or not. Defaults to False.
**kwargs – Other keyword arguments in CustomDataset and BaseDataset.

CIFAR ¶

class mmcls.datasets.CIFAR10(data_prefix, test_mode, metainfo=None, data_root='', download=True, **kwargs)[源代码]¶

CIFAR10 Dataset.

This implementation is modified from https://github.com/pytorch/vision/blob/master/torchvision/datasets/cifar.py

参数

data_prefix (str) – Prefix for data.
test_mode (bool) – test_mode=True means in test phase. It determines to use the training set or test set.
metainfo (dict, optional) – Meta information for dataset, such as categories information. Defaults to None.
data_root (str) – The root directory for data_prefix. Defaults to ‘’.
download (bool) – Whether to download the dataset if not exists. Defaults to True.
**kwargs – Other keyword arguments in BaseDataset.

class mmcls.datasets.CIFAR100(data_prefix, test_mode, metainfo=None, data_root='', download=True, **kwargs)[源代码]¶

CIFAR100 Dataset.

参数

data_prefix (str) – Prefix for data.
test_mode (bool) – test_mode=True means in test phase. It determines to use the training set or test set.
metainfo (dict, optional) – Meta information for dataset, such as categories information. Defaults to None.
data_root (str) – The root directory for data_prefix. Defaults to ‘’.
download (bool) – Whether to download the dataset if not exists. Defaults to True.
**kwargs – Other keyword arguments in BaseDataset.

MNIST ¶

class mmcls.datasets.MNIST(data_prefix, test_mode, metainfo=None, data_root='', download=True, **kwargs)[源代码]¶

MNIST Dataset.

This implementation is modified from https://github.com/pytorch/vision/blob/master/torchvision/datasets/mnist.py

参数

data_prefix (str) – Prefix for data.
test_mode (bool) – test_mode=True means in test phase. It determines to use the training set or test set.
metainfo (dict, optional) – Meta information for dataset, such as categories information. Defaults to None.
data_root (str) – The root directory for data_prefix. Defaults to ‘’.
download (bool) – Whether to download the dataset if not exists. Defaults to True.
**kwargs – Other keyword arguments in BaseDataset.

class mmcls.datasets.FashionMNIST(data_prefix, test_mode, metainfo=None, data_root='', download=True, **kwargs)[源代码]¶

Fashion-MNIST Dataset.

参数

data_prefix (str) – Prefix for data.
test_mode (bool) – test_mode=True means in test phase. It determines to use the training set or test set.
metainfo (dict, optional) – Meta information for dataset, such as categories information. Defaults to None.
data_root (str) – The root directory for data_prefix. Defaults to ‘’.
download (bool) – Whether to download the dataset if not exists. Defaults to True.
**kwargs – Other keyword arguments in BaseDataset.

VOC ¶

class mmcls.datasets.VOC(data_root, image_set_path, data_prefix={'ann_path': 'Annotations', 'img_path': 'JPEGImages'}, test_mode=False, metainfo=None, **kwargs)[源代码]¶

Pascal VOC Dataset.

After decompression, the dataset directory structure is as follows:

VOC dataset directory:

VOC2007 (data_root)/
├── JPEGImages (data_prefix['img_path'])
│   ├── xxx.jpg
│   ├── xxy.jpg
│   └── ...
├── Annotations (data_prefix['ann_path'])
│   ├── xxx.xml
│   ├── xxy.xml
│   └── ...
└── ImageSets (directory contains various imageset file)

Extra difficult label is in VOC annotations, we will use gt_label_difficult to record the difficult labels in each sample and corresponding evaluation should take care of this field to calculate metrics. Usually, difficult labels are reckoned as negative in defaults.

参数

data_root (str) – The root directory for VOC dataset.
image_set_path (str) – The path of image set, The file which lists image ids of the sub dataset, and this path is relative to data_root.
data_prefix (dict) – Prefix for data and annotation, keyword ‘img_path’ and ‘ann_path’ can be set. Defaults to be dict(img_path='JPEGImages', ann_path='Annotations').
test_mode (bool) – test_mode=True means in test phase. It determines to use the training set or test set.
metainfo (dict, optional) – Meta information for dataset, such as categories information. Defaults to None.
**kwargs – Other keyword arguments in BaseDataset.

CUB ¶

class mmcls.datasets.CUB(data_root, test_mode, ann_file='images.txt', data_prefix='images', image_class_labels_file='image_class_labels.txt', train_test_split_file='train_test_split.txt', **kwargs)[源代码]¶

The CUB-200-2011 Dataset.

Support the CUB-200-2011 Dataset. Comparing with the CUB-200 Dataset, there are much more pictures in CUB-200-2011. After downloading and decompression, the dataset directory structure is as follows.

CUB dataset directory:

CUB-200-2011 (data_root)/
├── images (data_prefix)
│   ├── class_x
│   │   ├── xx1.jpg
│   │   ├── xx2.jpg
│   │   └── ...
│   ├── class_y
│   │   ├── yy1.jpg
│   │   ├── yy2.jpg
│   │   └── ...
│   └── ...
├── images.txt (ann_file)
├── image_class_labels.txt (image_class_labels_file)
├── train_test_split.txt (train_test_split_file)
└── ....

参数

data_root (str) – The root directory for CUB-200-2011 dataset.
test_mode (bool) – test_mode=True means in test phase. It determines to use the training set or test set.
ann_file (str, optional) – Annotation file path, path relative to data_root. Defaults to ‘images.txt’.
data_prefix (str) – Prefix for iamges, path relative to data_root. Defaults to ‘images’.
image_class_labels_file (str, optional) – The label file, path relative to data_root. Defaults to ‘image_class_labels.txt’.
train_test_split_file (str, optional) – The split file to split train and test dataset, path relative to data_root. Defaults to ‘train_test_split_file.txt’.

使用示例

>>> from mmcls.datasets import CUB
>>> cub_train_cfg = dict(data_root='data/CUB_200_2011', test_mode=True)
>>> cub_train = CUB(**cub_train_cfg)
>>> cub_train
Dataset CUB
Number of samples:  5994
Number of categories:       200
Root of dataset:    data/CUB_200_2011
>>> cub_test_cfg = dict(data_root='data/CUB_200_2011', test_mode=True)
>>> cub_test = CUB(**cub_test_cfg)
>>> cub_test
Dataset CUB
Number of samples:  5794
Number of categories:       200
Root of dataset:    data/CUB_200_2011

Base classes ¶

class mmcls.datasets.BaseDataset(ann_file, metainfo=None, data_root='', data_prefix='', filter_cfg=None, indices=None, serialize_data=True, pipeline=(), test_mode=False, lazy_init=False, max_refetch=1000, classes=None)[源代码]¶

Base dataset for image classification task.

This dataset support annotation file in OpenMMLab 2.0 style annotation format.

Comparing with the mmengine.BaseDataset, this class implemented several useful methods.

参数

ann_file (str) – Annotation file path.
metainfo (dict, optional) – Meta information for dataset, such as class information. Defaults to None.
data_root (str) – The root directory for data_prefix and ann_file. Defaults to ‘’.
data_prefix (str | dict) – Prefix for training data. Defaults to ‘’.
filter_cfg (dict, optional) – Config for filter data. Defaults to None.
indices (int or Sequence[int], optional) – Support using first few data in annotation file to facilitate training/testing on a smaller dataset. Defaults to None, which means using all data_infos.
serialize_data (bool) – Whether to hold memory using serialized objects, when enabled, data loader workers can use shared RAM from master process instead of making a copy. Defaults to True.
pipeline (Sequence) – Processing pipeline. Defaults to an empty tuple.
test_mode (bool) – test_mode=True means in test phase. Defaults to False.
lazy_init (bool) – Whether to load annotation during instantiation. In some cases, such as visualization, only the meta information of the dataset is needed, which is not necessary to load annotation file. Basedataset can skip load annotations to save time by set lazy_init=False. Defaults to False.
max_refetch (int) – If Basedataset.prepare_data get a None img. The maximum extra number of cycles to get a valid image. Defaults to 1000.
classes (str | Sequence[str], optional) –
Specify names of classes.
- If is string, it should be a file path, and the every line of the file is a name of a class.
- If is a sequence of string, every item is a name of class.
- If is None, use categories information in metainfo argument, annotation file or the class attribute METAINFO.
Defaults to None.

class mmcls.datasets.MultiLabelDataset(ann_file, metainfo=None, data_root='', data_prefix='', filter_cfg=None, indices=None, serialize_data=True, pipeline=(), test_mode=False, lazy_init=False, max_refetch=1000, classes=None)[源代码]¶

Multi-label Dataset.

This dataset support annotation file in OpenMMLab 2.0 style annotation format.

The annotation format is shown as follows.

{
    "metainfo":
    {
      "classes":['A', 'B', 'C'....]
    },
    "data_list":
    [
      {
        "img_path": "test_img1.jpg",
        'img_label': [0, 1],
      },
      {
        "img_path": "test_img2.jpg",
        'img_label': [2],
      },
    ]
    ....
}

参数

ann_file (str) – Annotation file path.
metainfo (dict, optional) – Meta information for dataset, such as class information. Defaults to None.
data_root (str) – The root directory for data_prefix and ann_file. Defaults to ‘’.
data_prefix (str | dict) – Prefix for training data. Defaults to ‘’.
filter_cfg (dict, optional) – Config for filter data. Defaults to None.
indices (int or Sequence[int], optional) – Support using first few data in annotation file to facilitate training/testing on a smaller dataset. Defaults to None which means using all data_infos.
serialize_data (bool, optional) – Whether to hold memory using serialized objects, when enabled, data loader workers can use shared RAM from master process instead of making a copy. Defaults to True.
pipeline (list, optional) – Processing pipeline. Defaults to [].
test_mode (bool, optional) – test_mode=True means in test phase. Defaults to False.
lazy_init (bool, optional) – Whether to load annotation during instantiation. In some cases, such as visualization, only the meta information of the dataset is needed, which is not necessary to load annotation file. Basedataset can skip load annotations to save time by set lazy_init=False. Defaults to False.
max_refetch (int, optional) – If Basedataset.prepare_data get a None img. The maximum extra number of cycles to get a valid image. Defaults to 1000.
classes (str | Sequence[str], optional) –
Specify names of classes.
- If is string, it should be a file path, and the every line of the file is a name of a class.
- If is a sequence of string, every item is a name of class.
- If is None, use categories information in metainfo argument, annotation file or the class attribute METAINFO.
Defaults to None.

Dataset Wrappers ¶

class mmcls.datasets.KFoldDataset(dataset, fold=0, num_splits=5, test_mode=False, seed=None)[源代码]¶

A wrapper of dataset for K-Fold cross-validation.

K-Fold cross-validation divides all the samples in groups of samples, called folds, of almost equal sizes. And we use k-1 of folds to do training and use the fold left to do validation.

参数

dataset (mmengine.dataset.BaseDataset | dict) – The dataset to be divided
fold (int) – The fold used to do validation. Defaults to 0.
num_splits (int) – The number of all folds. Defaults to 5.
test_mode (bool) – Use the training dataset or validation dataset. Defaults to False.
seed (int, optional) – The seed to shuffle the dataset before splitting. If None, not shuffle the dataset. Defaults to None.

The dataset wrappers in the MMEngine can be directly used in MMClassification.

`ConcatDataset`	A wrapper of concatenated dataset.
`RepeatDataset`	A wrapper of repeated dataset.
`ClassBalancedDataset`	A wrapper of class balanced dataset.