How to get classes and masks from PixelLab ADE20K — Image Segmentation

Hi everyone!! today I gonna show you how to get classes and masks from the PixelLab. If you don't know about this library, let me give an introduction.

If you already know about PixelLab, just skip and go to the answer at the final of this page.

PixelLab

This library borns in June 2020. Created by @ayoolaolafenwa. You can get the official code from GitHub and the official doc from readthedocs. In both, you will find an awesome use for this library. Besides, you can check her official tutorial here.

This library allows to train, val, and test custom models or the most recent one: DeepLabV3+ trained on different backbones (e.g. Xception) using the ADE20K dataset. If you want to know more about DeepLabV3+ and why is so interesting, check this page.

Why is so interesting??

Example using ADE20K_Xception in PixelLab

As I mention above, this library allows to test models very fast and using a low number of lines. In addition, you can test any image (like me) and you will get an amazing result.

My example using ADEK20-PixelLab

Everything can be wonderful … right??

Well, no. As a curious person, I always using new libraries and try to get the best of them. So, after testing this library I got with the issue that …

Ok, this library does everything for me, but if I need some internal data … or some internal information that isn't showed to me?

As the title says, this library presents a “little issue”: Don't return masks, classes names, object ratios of all segmentations. That could be the most nightmare for a high user level library … But for a CS no!!

How I got the masks, ratios, and all … ??

Well, as a curious person I always review the code (is released and free to check) looking for the process and the specific function to understand it.

So, the first thing was looking at the function called “segmentAsAde20k” and then just … magic (you can find it here). Then I made my own step-by-step simulation of the code:

The yellow mark remarks the most important line to get all the information that I want.

I noticed in the step-by-step replication process, we can take the information explicitly. But first, I needed to create two main functions:

https://gist.github.com/fmorenovr/16e05d10217464e71f82074cd2fd3835

Then, I created another one, in this function I got all the important information that I was looking for: masks, ratio masks, classes, class names

Function to get info

Then, after you create these functions, you can create an additional one, and the just called to get all the data:

If you want to modify directly from the code, just make it in line 422:

Just adding:

masks = ade20k_map_color_mask(raw_labels)

Also, you will need to modify outputs (just adding masks on both returns statements). Here I share with you the whole code:

As you can see, I return all information related to the segmentation. Also, I already sort by the % pixel ratio.

List of masks returned by my modification

As I promise, the direct solution (I highly recommend to you which modifications I made to get the answer).

https://gist.github.com/fmorenovr/d048a54ae2f891799648256f2f2667c2

I hope you enjoy this tutorial. I know sometimes is very difficult but always exist a way to do things.