It’s an exciting scene in crime investigation movies. A critical image, like the one on the left below, is blurred by pixelization. The detective commands the technician, “Sharpen it!” and the technician pushes a key on a computer keyboard. The key activates an algorithm and, magically, the deblurred image on the right appears.

That can’t be done in real life. An image cannot be sharpened using only the information in the image itself. This is proven by a mathy theorem called the data processing inequality. ¹ The mutual information between an image and a corrupted blurred image cannot be increased by further processing. Period.

That’s why the title of a recent news article from Duke University is misleading: “Artificial intelligence makes blurry faces look more than 60 times sharper.” The Duke software generates a sharp face but there is no guarantee that the regenerated face is the original. The article explicitly confesses to the truth only in the fourth paragraph:

The system cannot be used to identify people, the researchers say: It won’t turn an out-of-focus, unrecognizable photo from a security camera into a crystal clear image of a real person. Rather, it is capable of generating new faces that don’t exist, but look plausibly real.

Robin A. Smith, “Artificial intelligence makes blurry faces look more than 60 times sharper” at TechXplore

There are two ways a pixelated image can be restored. The first is by looking at the equivalent of a catalog of faces that, when pixelated, will match the blurred image. This is roughly the approach used by the Duke researchers, who were using artificial neural networks. The best high-resolution face that matches the pixelated image, in some sense, is chosen. No claim is made that the software regenerates the image of the original face. (That said, the Duke technology may be useful in law enforcement if it is applied to restoring license plate numbers from blurred images.)

The second approach to restoration is to gather further versions of the corrupted image. A second blurred image using different pixelation will help the restoration accuracy. Here’s an extreme example: Suppose the world’s worst camera has a single photodetector. The camera is pointed at a scene and takes a single measurement over a small region of the scene. The portion of the scene in this small region is blurred together into a single number. The camera is then randomly moved to another position and another reading is taken. After a number of such random moves, is it possible to combine all these single readings into a composite image even when many of them overlap?

As seen in the image below, the answer is yes. Shown on the upper left is an original image and to its right a pixelated image. The first few shots from the world’s worst camera are shown in the middle left. Ten thousand readings using the world’s worst camera allows generation of the image on in the middle right. Further readings sharpen the image.

It’s more fun to watch the real-time combining process that generates the above images. Take a look at the short video below:

The takeaway here is that there is no free lunch. If you receive a corrupted image, no AI pixel-pushing algorithm can generate more mutual information. You must have an prior idea of what the corrupted image is. Even then, the reconstruction is but a best guess, biased by your assumption as to what the original could be. Alternatively, other versions of the corrupted image can be gathered. In that case, the question is, how are they combined to give a restored image?

References

¹ Cover, Thomas M., and Joy A. Thomas. Elements of information theory. John Wiley & Sons, 2012.

² If you’d like to know the details, you’ll need to look at my book: Robert J. Marks II, The Handbook of Fourier Analysis and Its Applications, Oxford University Press, 2009.

Further reading on the ways AI can help solve longstanding mysteries:

How can AI help with real-life cold case files? AI doesn’t create new ideas in police work; rather, it does the work that police, who must move on to urgent, fresh cases, don’t have time to do.

Can AI help us decipher lost languages? Yes, sometimes. It depends mainly on the reasons we haven’t yet deciphered ancient texts.

Can AI prove that Shakespeare had ghostwriters? An author’s unique style is like a fingerprint. AI can fill it in and give us some patterns to look at.

and

Does AI challenge Biblical archeology? Many surviving documents are so damaged that they cannot be read using traditional methods. Recently, AI deciphered a nearly destroyed Biblical manuscript, showing that it was the earliest surviving copy.