Mean Intersection over Union (mIoU) for 3D Semantic Segmentation

The mean Intersection over Union (mIoU) is a metric which is often used for measuring the goodness of e.g. 2D- and 3D semantic segmentation and object detection systems, by comparing the predictions made by a system to the ground truth labels. In this post, we will show how the mIoU is calculated for a very simple 3D point cloud toy example consisting of 7 points in which we have 2 classes - green and red. The ground truths and predictions for the example looks the following:

Figure 1: The ground truth. 4 points of the point cloud belong to the green class and 3 points to the red class

Figure 2: The predictions. The system has predicted 4 of the points in the point cloud as the green class and 3 of the points as the red class

Example of calculating mIoU

Now, let’s see how the mIoU is calculated for this toy example. First, we will calculate the IoU for each class. The equation for IoU looks the following:

$$IoU_i = \frac{\text{TPs}_i}{\text{TPs}_i + \text{FPs}_i + \text{FNs}_i}$$

Where $\text{TPs}_i$ is the true positive predictions of class i, $\text{FPs}_i$ are the number of predictions where the system falsely predicted class i and $\text{FNs}_i$ are the number of predictions where the system failed to predict class i.

IoU for the green class

Let’s start off by calculating the IoU for the green class. We see from the ground truths that there are 4 green points, the system managed to predict 3 of them correctly (TPs), but missed 1 (FNs) and also mis-predicted one of the red points as a green one (FPs):

$$IoU_{green} = \frac{\text{TPs}_{green}}{\text{TPs}_{green} + \text{FPs}_{green} + \text{FNs}_{green}} = \frac{3}{3 + 1 + 1} = \frac{3}{5} = 0,6 = 60\%$$

Figure 3: 3 out of the 4 predictions for the green class are TPs. They are marked with a dark green circle

IoU for the red class

And the IoU for the red class, we see from the ground truths that there are 3 red points and the system managed to predict 2 of the points correctly, but again missed 1 and mis-predicted 1:

$$IoU_{red} = \frac{\text{TPs}_{red}}{\text{TPs}_{red} + \text{FPs}_{red} + \text{FNs}_{red}} = \frac{2}{2 + 1 + 1} = \frac{2}{4} = 0,5 = 50\%$$

Figure 4: 2 out of the 3 predictions for the red class are TPs. They are marked with a dark red circle

Finally, we take the mean of all the IoUs to get the mIoU score, which is simply taking the mean of the two IoUs of 50% and 60% = 55%.

Thank you for reading! Please upvote and/or leave a comment below if you found this post interesting

Get back to my other posts here