Shortcuts

Note

You are reading the documentation for MMClassification 0.x, which will soon be deprecated at the end of 2022. We recommend you upgrade to MMClassification 1.0 to enjoy fruitful new features and better performance brought by OpenMMLab 2.0. Check the installation tutorial, migration tutorial and changelog for more details.

Source code for mmcls.core.evaluation.multilabel_eval_metrics

# Copyright (c) OpenMMLab. All rights reserved.
import warnings

import numpy as np
import torch


[docs]def average_performance(pred, target, thr=None, k=None): """Calculate CP, CR, CF1, OP, OR, OF1, where C stands for per-class average, O stands for overall average, P stands for precision, R stands for recall and F1 stands for F1-score. Args: pred (torch.Tensor | np.ndarray): The model prediction with shape (N, C), where C is the number of classes. target (torch.Tensor | np.ndarray): The target of each prediction with shape (N, C), where C is the number of classes. 1 stands for positive examples, 0 stands for negative examples and -1 stands for difficult examples. thr (float): The confidence threshold. Defaults to None. k (int): Top-k performance. Note that if thr and k are both given, k will be ignored. Defaults to None. Returns: tuple: (CP, CR, CF1, OP, OR, OF1) """ if isinstance(pred, torch.Tensor) and isinstance(target, torch.Tensor): pred = pred.detach().cpu().numpy() target = target.detach().cpu().numpy() elif not (isinstance(pred, np.ndarray) and isinstance(target, np.ndarray)): raise TypeError('pred and target should both be torch.Tensor or' 'np.ndarray') if thr is None and k is None: thr = 0.5 warnings.warn('Neither thr nor k is given, set thr as 0.5 by ' 'default.') elif thr is not None and k is not None: warnings.warn('Both thr and k are given, use threshold in favor of ' 'top-k.') assert pred.shape == \ target.shape, 'pred and target should be in the same shape.' eps = np.finfo(np.float32).eps target[target == -1] = 0 if thr is not None: # a label is predicted positive if the confidence is no lower than thr pos_inds = pred >= thr else: # top-k labels will be predicted positive for any example sort_inds = np.argsort(-pred, axis=1) sort_inds_ = sort_inds[:, :k] inds = np.indices(sort_inds_.shape) pos_inds = np.zeros_like(pred) pos_inds[inds[0], sort_inds_] = 1 tp = (pos_inds * target) == 1 fp = (pos_inds * (1 - target)) == 1 fn = ((1 - pos_inds) * target) == 1 precision_class = tp.sum(axis=0) / np.maximum( tp.sum(axis=0) + fp.sum(axis=0), eps) recall_class = tp.sum(axis=0) / np.maximum( tp.sum(axis=0) + fn.sum(axis=0), eps) CP = precision_class.mean() * 100.0 CR = recall_class.mean() * 100.0 CF1 = 2 * CP * CR / np.maximum(CP + CR, eps) OP = tp.sum() / np.maximum(tp.sum() + fp.sum(), eps) * 100.0 OR = tp.sum() / np.maximum(tp.sum() + fn.sum(), eps) * 100.0 OF1 = 2 * OP * OR / np.maximum(OP + OR, eps) return CP, CR, CF1, OP, OR, OF1
Read the Docs v: latest
Versions
master
latest
1.x
dev-1.x
Downloads
html
epub
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.