Generation of super-resolution stills from video

Abstract

The real-time super-resolution technique discussed in this paper increases the effective pixel density of an image sensor by combining consecutive image frames from a video. In surveillance, the higher pixel density lowers the Nyquist rate of the sensor which improves the detection, recognition and identification (DRI) task performance of the system. When a sensor lingers on a stationary target or tracks a moving target then the image of the target would with time move around slightly on the focal plane. If one accurately registers the image of the target on the focal plane to some reference then one can increase the effective sensor pixel density by stacking or appropriately combining the registered images. The super-resolution technique operates on the focal plane array after the image has been degraded by the Modulation Transfer Function (MTF) of the lens and atmosphere. Any high frequencies lost due to the atmosphere or lens cannot be recovered. However, if the MTFs of the lens and atmosphere are good enough to cause aliasing on the focal plane then the sharp stack algorithm discussed here can at least double the resolving power of the sensor.