# Data Process¶

In MMClassification, the data process and the dataset is decomposed. The datasets only define how to get samples’ basic information from the file system. These basic information includes the ground-truth label and raw images data / the paths of images.The data process includes data transforms, data preprocessors and batch augmentations.

## Data Transforms¶

To prepare the inputs data, we need to do some transforms on these basic information. These transforms includes loading, preprocessing and formatting. And a series of data transforms makes up a data pipeline. Therefore, you can find the a pipeline argument in the configs of dataset, for example:

train_pipeline = [
dict(type='RandomResizedCrop', scale=224),
dict(type='RandomFlip', prob=0.5, direction='horizontal'),
dict(type='PackClsInputs'),
]

....
dataset=dict(
pipeline=train_pipeline,
....),
....
)


Every item of a pipeline list is one of the following data transforms class. And if you want to add a custom data transformation class, the tutorial Custom Data Pipelines will help you.

### Processing and Augmentation¶

 Albumentations Wrapper to use augmentation from albumentations library. ColorJitter Randomly change the brightness, contrast and saturation of an image. EfficientNetCenterCrop EfficientNet style center crop. EfficientNetRandomCrop EfficientNet style RandomResizedCrop. Lighting Adjust images lighting using AlexNet-style PCA jitter. RandomCrop Crop the given Image at a random location. RandomErasing Randomly selects a rectangle region in an image and erase pixels. RandomResizedCrop Crop the given image to random scale and aspect ratio. ResizeEdge Resize images along the specified edge.

#### Composed Augmentation¶

Composed augmentation is a kind of methods which compose a series of data augmentation transforms, such as AutoAugment and RandAugment.

 AutoAugment Auto augmentation. RandAugment Random augmentation.

To specify the augmentation combination (The policies argument), you can use string to specify from some preset policies.

Preset policy

Use for

Description

“imagenet”

AutoAugment

Policy for ImageNet, come from DeepVoltaire/AutoAugment

“timm_increasing”

RandAugment

The _RAND_INCREASING_TRANSFORMS policy from timm

And you can also configure a group of policies manually by selecting from the below table.

 AutoContrast Auto adjust image contrast. Brightness Adjust images brightness. ColorTransform Adjust images color balance. Contrast Adjust images contrast. Cutout Cutout images. Equalize Equalize the image histogram. Invert Invert images. Posterize Posterize images (reduce the number of bits for each color channel). Rotate Rotate images. Sharpness Adjust images sharpness. Shear Shear images. Solarize Solarize images (invert all pixel values above a threshold). SolarizeAdd SolarizeAdd images (add a certain value to pixels below a threshold). Translate Translate images. BaseAugTransform The base class of augmentation transform for RandAugment.

### Formatting¶

 Collect Collect and only reserve the specified fields. PackClsInputs Pack the inputs data for the classification. ToNumpy Convert object to numpy.ndarray. ToPIL Convert the image from OpenCV format to PIL.Image.Image. Transpose Transpose numpy array.

### MMCV transforms¶

We also provides many transforms in MMCV. You can use them directly in the config files. Here are some frequently used transforms, and the whole transforms list can be found in mmcv.transforms.

 LoadImageFromFile Load an image from file. Resize Resize images & bbox & seg & keypoints. RandomResize Random resize images & bbox & keypoints. RandomFlip Flip the image & bbox & keypoints & segmentation map. RandomGrayscale Randomly convert image to grayscale with a probability. CenterCrop Crop the center of the image, segmentation masks, bounding boxes and key points. If the crop area exceeds the original image and auto_pad is True, the original image will be padded before cropping. Normalize Normalize the image. Compose Compose multiple transforms sequentially.

## Data Preprocessors¶

The data preprocessor is also a component to process the data before feeding data to the neural network. Comparing with the data transforms, the data preprocessor is a module of the classifier, and it takes a batch of data to process, which means it can use GPU and batch to accelebrate the processing.

The default data preprocessor in MMClassification could do the pre-processing like following:

1. Move data to the target device.

2. Pad inputs to the maximum size of current batch.

3. Stack inputs to a batch.

4. Convert inputs from bgr to rgb if the shape of input is (3, H, W).

5. Normalize image with defined std and mean.

6. Do batch augmentations like Mixup and CutMix during training.

You can configure the data preprocessor by the data_preprocessor field or model.data_preprocessor field in the config file. Typical usages are as below:

data_preprocessor = dict(
# RGB format normalization parameters
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True,    # convert image from BGR to RGB
)


Or define in model.data_preprocessor as following:

model = dict(
backbone = ...,
neck = ...,
data_preprocessor = dict(
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True)
train_cfg=...,
)


Note that the model.data_preprocessor has higher priority than data_preprocessor.

 ClsDataPreprocessor Image pre-processor for classification tasks.

### Batch Augmentations¶

The batch augmentation is a component of data preprocessors. It involves multiple samples and mix them in some way, such as Mixup and CutMix.

These augmentations are usually only used during training, therefore, we use the model.train_cfg field to configure them in config files.

model = dict(
backbone=...,
neck=...,
train_cfg=dict(augments=[
dict(type='Mixup', alpha=0.8),
dict(type='CutMix', alpha=1.0),
]),
)


You can also specify the probabilities of every batch augmentation by the probs field.

model = dict(
backbone=...,
neck=...,

 Mixup Mixup batch augmentation. CutMix CutMix batch agumentation. ResizeMix ResizeMix Random Paste layer for a batch of data.