What is Video Processing?

Why is video processing necessary?

Video processing is an essential step in ensuring high-quality video playback for viewers across different devices. By applying various techniques, video processing enhances the viewing experiences and, in most cases, can improve the quality of video streaming content.

_{What are the different video processing techniques?}

A wide array of video processing techniques exist, each offering unique capabilities to enhance and optimize video content for viewers. Below, we have compiled a list of the most prevalent and widely used techniques:

Video compression: reduces the file size of a video by eliminating unnecessary data whilst preserving critical visual elements such as sharpness or colours. When applied correctly, video compression enables seamless playback experiences for end-users as smaller video files enables smoother streaming experiences through faster loading speeds and decreased buffering durations and improved compatibility across different platforms, players and devices. It is worth noting that common compression techniques require the use of codecs such as H.264 or H.265, which directly reduce the file size of video or audio content.

Video encoding: entails the process of reducing video files into smaller, manageable chunks that allow for smoother playback delivery, prevents buffering, plus, requires lower storage space across devices.

Video decoding:  unlike video encoding, this is the process of converting compressed video data into its original playable format.

Transcoding: refers to taking an encoded (or compressed) video file, decoding (or decompressing) it, before altering and de-coding the video file into a form that provides optimal playback experiences across all kinds of devices, such as mobile phones, tablets, smart TVs and computers. A prime example of this is YouTube, which automatically transcodes uploaded videos so to ensure video content is compatible across a range of devices.

Video upscaling/downscaling: involves increasing the number of pixels in a video file to improve its resolution. Video downscaling, on the other hand, involves the reduction of video pixels, usually to make a file more compatible with low-resolution devices. For example, adjusting your YouTube video playback settings from 1080 to 480 pixels would initiate the video downscaling process.

Video stabilisation: works by analysing each frame of video footage before making adjustments to counteract any inconsistent or shaky movements. This allows for smoother video playback that appears more stable and thus less distracting for viewers.

Video frame rate conversion: is the process of changing the number of frames per second (fps) in a video. Removing or adding frames directly determines the speed and smoothness of the video playback. For example, sports channels can be seen removing frame rates during live action recaps, as this allows viewers to experience key moments in slow motion.

What happens after video processing?

Video processing is followed by the delivery phase. Once the video content has been processed and enhanced, it is essential to ensure that it reaches the end users in a manner that enables optimal viewing experiences. Factors like bandwidth optimisation, adaptive streaming, and efficient encoding play a crucial role in delivering the video content smoothly and without interruptions.

Three common methods for delivering processed video files include the following:

Dynamic adaptive streaming over HTTP

Also referred to as DASH, it allows for optimal video streaming experiences by dynamically adjusting the video quality based on the viewer’s network conditions and device capabilities. It achieves this by breaking the video into small segments and offering multiple quality levels of which it can switch to in real-time as a means of minimising buffering interruptions for each viewer. In short, DASH achieves seamless video playback by dynamically adjusting video quality based on the viewer’s network capacity.

Progressive Download

It involves downloading the entire video file to the viewers device before playback begins. The video file is stored locally, allowing viewers to start watching immediately. However, this delivery method does not offer adaptive streaming capabilities like DASH. ‍

HTTP Live Streaming and M3U8

They work together to optimize video playback by adapting to varying network conditions of viewers. Firstly, HTTP Live Streaming (HLS) breaks a video down into chunks and dynamically adjusts the video quality to prevent buffering whilst streaming. M3U8 (a UTF-8 encoded audio playlist file) on the other hand saves and organises each compressed chunk into an organised playlist, which guides the HLS on the order that each video needs to be downloaded and played for viewers. In short, HLS is a streaming protocol, whilst M3U8 is a URL based media file format that creates an organised streaming playlist of which the HLS plays from.