Evaluation of output images

Evaluation of output images#

After applying the rconstruction algorithm we obtain an output image. How can we avlaute the accuracy of the reconstructed image? How can we compare the results obtained by two algorithms?

This is a very difficult task and we don’t have a completely satisfying answer.

We have different evaluation metrics, that sometimes are not in agreement each other and esepcially they do not agree with a visual inspction of the image.

In fact the first metric is our eye. It is obviously not an objective evaluation and this is the first problem.

When we create test problems to evaluate the algorithms, we have the ground truth or éexact image* that we use as reference in the metrics.

In the following, we suppose the images reshaped in a vector and we indicate with \(x\) the output image and with \(x^{GT}\) the ground truth image.

Mean Square Error (MSE)

\[MSE=\frac{1}{N} \sum_{i=1}^N (x_i-x^{GT}_i)^2\]

Less is the MSE better is the image. Its values depends on the scale of the images.

  • it is very simple to compute

  • it is low correlated with the visul quality of the image

  • it is sensible to the scale of the two images

Peak Signal to Noise Ratio (PSNR)

\[PSNR=10 log_{10} (\frac{max(x)}{MSE})\]

It is measured in Decibel (Db). The metric measures the ratio between signal and noise in the image. Its value is in the range [0,100] and greater is PSNR, better is the image.

Standard values for good images are about 30-40.

  • is is a quite standard measure in imaging and allows to compare images in different contexts since it does not depend on the scale.

Structural Similarity Index (SSIM)

There is not a simple formula to compute SSIM (Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612)

  • More informative (more correlated with visual inspection)

  • sensible to contrast and image structure

There are other metrics specific for certain applications, sucha the Contrast-to-Noise-Ratio (CNR) in medical imaging or others.