Smartphone Image Denoising Dataset

Abdelrahman Abdelhamed1             Stephen Lin2             Michael S. Brown1

1York University             2Microsoft Research

SIDD Benchmark

The SIDD Benchmark will be hosted as a challenge at the New Trends in Image Restoration and Enhancement (NTIRE 2019) workshop in conjunction with CVPR 2019.

We have two challenges publicly available:
NTIRE 2019 Real Image Denoising Challenge - Track 1: Raw-RGB
NTIRE 2019 Real Image Denoising Challenge - Track 2: sRGB

After the challenge is over, the benchmark will be hosted back here on this website.

The SIDD Benchmark consists of 40 images representing 40 scene instances. These images can be used to benchmark denoising methods.

For each image, the following is provided in one directory:

  1. Noisy Raw-RGB image (.MAT).
  2. Noisy sRGB image (.PNG).
  3. Metadata extracted from the DNG file (.MAT).

The PSNR and SSIM values are calculated only on 32 blocks of size 256 by 256 pixels. The block positions are provided in a file named "BenchmarkBlocks32.mat".

Benchmark Results

The following tables show the benchmark results published in the paper.

Results of denoising in raw-RGB space

Method PSNR SSIM Time
(Applied on/Evaluated on) (Applied on/Evaluated on) (seconds)
Raw/Raw Raw/sRGB Raw/Raw Raw/sRGB Raw
BM3D 45.52 30.95 0.980 0.863 34.3
NLM 44.06 29.39 0.971 0.846 210.7
KSVD 43.26 27.41 0.969 0.832 2243.9
KSVD-DCT 42.70 28.21 0.970 0.784 133.3
KSVD-G 42.50 28.13 0.969 0.781 153.6
LPG-PCA 42.79 30.01 0.974 0.854 438.1
FoE 43.13 27.18 0.969 0.812 6097.2
MLP 43.17 27.52 0.965 0.788 131.2
WNNM 44.85 29.54 0.975 0.888 1975.8
GLIDE 41.87 25.98 0.949 0.816 12440.5
TNRD 42.77 26.99 0.945 0.744 15.2
EPLL 40.73 25.19 0.935 0.842 653.1
DnCNN 43.30 28.24 0.965 0.829 51.7
More to come...

Results of denoising in sRGB space

Method PSNR SSIM Time
(Applied on/Evaluated on) (Applied on/Evaluated on) (seconds)
sRGB/sRGB sRGB/sRGB sRGB
BM3D 25.65 0.685 27.4
NLM 26.75 0.699 621.9
KSVD 26.88 0.842 9881.0
KSVD-DCT 27.51 0.780 96.3
KSVD-G 27.19 0.771 92.2
LPG-PCA 24.49 0.681 2004.3
FoE 25.58 0.792 12166.8
MLP 24.71 0.641 564.8
WNNM 25.78 0.809 8882.2
GLIDE 24.71 0.774 36091.6
TNRD 24.73 0.643 45.1
EPLL 27.11 0.870 1996.4
DnCNN 23.66 0.583 158.9
CBDNet 33.28 0.868 4.48
DeepProxies BM3D 34.34 0.911 6.69
More to come...