Image / Video Compression

A comparison of memory consumption between low and high resolution image

We live in an unprecedented period of time where we are surrounded by numerous digital devices that enable us to carry out our daily activities—such as working, playing, and caring for others—much more effectively than in the past. Many of these activities include streaming vast amounts of data made up of photos and videos, and the leisure industry is always raising the standard of experience to provide a better user experience, such as HD vs. 4K UHD vs. 8K UHD, SDR vs. HDR. The pandemic phase we are currently experiencing has intensified this even further.

International standards organizations invest a lot of time creating new encoding standards to address these needs, but are still unable to meet all the demands made by the market and the growing number of applications. As a consequence, video quality is constantly being increased at the expense of compression efficiency.

Therefore, a fair balance between quality and compression efficiency must be found amongst a number of factors including bandwidth, service delivery costs, etc. On the other hand, deep-learning techniques are proposed either as end-to-end approaches to provide a significant improvement of the overall quality of the compression standard itself, or to substitute blocks (i.e., intra-coding, downsampling/upsampling blocks) into an existing compression.

AI Based Media Coding

Encoding Decoding pipeline

As a founding member of the MPAI standard, our group is taking part in the exploration phase to see whether it is possible to improve the performance of the Essential Video Coding (EVC) modified by enhancing/replacing existing video coding tools with AI tools keeping complexity increase to an acceptable level.

On Going Activities

To achieve our goals of enhancing traditional video coding tools with AI, our team is currently working on providing a solution for the Super Resolution Problem that is comparable to state-of-the-art models with less memory consumption and better time efficiency.

Low resolution image is converted to high resolution using deep learning

Super Resolution

Super Resolution is the process of recovering a High Resolution (HR) image from a given Low Resolution (LR) image. It is an important class of image processing techniques in computer vision and image processing and enjoys a wide range of real-world applications, such as medical imaging, satellite imaging, surveillance and security, astronomical imaging, amongst others.

With the advancement in deep learning techniques in recent years, deep learning-based SR models have been actively explored and often achieve state-of-the-art performance on various benchmarks of SR. A variety of deep learning methods have been applied to solve SR tasks, ranging from the early Convolutional Neural Networks (CNN) based method to recent cascading residual architecture.

Our team is working on simplification of recent state of art model i.e. Densely residual Laplacian Network. it has been found through our rigorous testing that a simplified version of proposed model not only reduces memory consumption and increase time efficiency but also provides comparable results.

Thus, the goal of super resolution can be achieved with less complex deep learning architecture.