Non-Local Means Filter for FFMPEG

The Non-Local Means noise reduction filter (see original paper [1]) is capable of restoring video sequences with even strong noise. I have implemented this algorithm as a filter for FFMPEG and found it ideal for enhancing the quality of old VHS tapes. Apart from better visual quality, one can also achieve much lower bit-rates (3-5x less) than coding the noisy video (using x264).

Because of the high computational complexity of the algorithm, I have simplified the calculation slightly to omit the weighting in computing the patch distances. This allows to use integral images to efficiently compute the distances. Furthermore, some SSE2 and OpenMP parallelization makes the algorithm feasible to be applied to video. For SD material, you can typically get about 6 fps.

This implementation extends the original algorithm to also include past frames into the candidates range. While this again increases the computation time, I found that this gives very good results for video.

Demos

Select one of the example images below and move the dividing line to compare original image and post-processed result. The parameters used are given below the images (h;range;temporal).

(10;5;4)

(10;11;1)

(8;3;2)

(10;7;5)

(16;9;10)

If you would use a modern browser, you would see the images here...

Usage

Check out the code from github https://github.com/farindk/ffmpeg.

The filter is named nlmeans and has the following parameters:

Typical command line:
ffmpeg -i input.avi -vf nlmeans output.avi

With custom parameters:
ffmpeg -i input.avi -vf nlmeans=h=10:range=5:temporal=3 output.avi

The default values (h=8, range=3, temporal=2) are a good starting point for the restoration of very noisy video (old VHS tapes). You may also try 10;5;3 for really noisy inputs or 6;3;1 for good quality inputs.

It is also interesting to note that one can simply apply the filter on interlaced material. The algorithm itself is robust enough to handle this nicely without artifacts.

Note that there is a patent [2] on the algorithm by the original paper authors. However, the authors kindly gave me permission to release the filter under the GPL.

Handbrake implementation

This filter was ported to Handbrake and enhanced by Bradley Sepos. He also provided a nice comparison with other noise reduction filters here: Handbrake forum.

References

  1. A. Buades, B. Coll, J.M. Morel: "A non-local algorithm for image denoising" in IEEE Computer Vision and Pattern Recognition 2005, Vol 2, pp: 60-65, 2005.
  2. A. Buades, B. Coll, J.M. Morel: "Image data processing method by reducing image noise, and camera integrating means for implementing said method", EP Patent 1,749,278 (Feb. 7), 2007.
.