In the previous edition of this series, we explored the basics of encoding and how raw video is transformed into a streamable format. Encoding, however, is just the first step. The actual delivery of video involves multiple layers, including codec selection, streaming protocols, cloud-based encoding workflows, and content protection. Each plays a vital role in ensuring that video streams are fast, secure, and high quality across a variety of devices and networks.
This article takes a closer look at how modern streaming stacks use codecs, protocols, cloud encoders, and DRM systems to serve video at scale.
Understanding Codecs
A codec, short for coder-decoder, is a compression algorithm that reduces the size of video files so they can be streamed efficiently. Codecs are responsible for determining how much data is required to deliver visual quality. The better the codec, the smaller the file size can be without noticeably degrading video playback. The right codec helps optimize bandwidth, reduce storage costs, and ensure compatibility with a wide range of devices.
Choosing a codec involves trade-offs between compression efficiency, playback compatibility, processing requirements, and licensing constraints. Many streaming platforms use a combination of codecs to ensure smooth playback across both new and older devices.
H.264 (AVC)
H.264 is the most widely used video codec in the world. It is supported on virtually all devices and browsers, making it the most compatible option for global reach. It provides decent compression and fast encoding and decoding, which makes it ideal for general-purpose streaming. While newer codecs offer better efficiency, H.264 remains a reliable default.
H.265 (HEVC)
H.265 offers better compression than H.264, particularly for higher resolutions like 4K and HDR. It produces smaller files with comparable visual quality, helping reduce bandwidth usage. However, its adoption has been slowed by licensing costs and inconsistent hardware support. It is commonly used in controlled environments where high efficiency is critical and device compatibility is known.
VP9
VP9 is a royalty-free codec developed by Google. It offers similar compression benefits to H.265 and is widely used in web-based platforms like YouTube. It performs well on Android devices and in the Chrome browser. VP9 avoids licensing fees, making it attractive for cost-sensitive applications, although encoding can be slower than H.264.
AV1
AV1 is a modern, royalty-free codec developed by the Alliance for Open Media. It delivers even greater compression efficiency than VP9 or H.265. While encoding AV1 can be resource-intensive, it is increasingly supported in hardware on newer devices. Large platforms like Netflix, YouTube, and Facebook have begun using AV1 to optimize delivery to supported endpoints and reduce data usage on mobile networks.
Streaming Protocols
Once a video is encoded, it needs to be packaged and transmitted to viewers over the internet. Streaming protocols define how the video is broken into segments, delivered to players, and reassembled for playback. These protocols also enable adaptive bitrate switching, allowing viewers to receive higher or lower quality streams depending on their connection speed.
HLS (HTTP Live Streaming)
HLS is a streaming protocol developed by Apple and widely used in the industry. It breaks the video into small segments and generates a playlist file that guides the player on which segments to fetch and play. HLS supports adaptive bitrate streaming and is natively supported on all Apple devices, including iPhones, iPads, and Macs. It is also compatible with most modern browsers, making it a popular choice for OTT platforms.
DASH (Dynamic Adaptive Streaming over HTTP)
DASH is an open standard that provides many of the same features as HLS, including segmented delivery and adaptive playback. It is widely used on Android devices and web platforms. DASH allows for more flexibility in codec and encryption choices, although it is not supported natively on Apple devices. Many services use both HLS and DASH to ensure compatibility across ecosystems.
RTMP, SRT, and Contribution Protocols
Protocols like RTMP and SRT are typically used for ingesting live streams into an encoding system. RTMP has been around for years and is still used for live broadcasts due to its simplicity and stability. SRT is a newer, more secure option that performs better on unstable networks and is gaining popularity in live workflows. These protocols are used upstream, before the content is encoded and repackaged into HLS or DASH for playback.
Cloud Encoding Platforms
Encoding at scale involves far more than converting one video file. Most video platforms need to process hundreds or thousands of files simultaneously, across different formats and bitrates. Cloud encoding platforms provide scalable infrastructure to handle these needs, often with APIs and automation features that reduce manual effort.
These platforms are typically built on open-source engines like FFmpeg but add advanced functionality such as job queueing, performance monitoring, live encoding, and multi-device packaging.
A few cloud-based encoders include:
Bitmovin
Bitmovin offers high-speed, cloud-native encoding workflows that support the latest codecs, including AV1. It allows parallel processing of encoding ladders, fast turnaround times, and detailed playback analytics. Bitmovin is often used by large OTT providers looking for flexibility and future-ready support.
AWS Elemental MediaConvert
MediaConvert is part of the AWS Media Services suite. It integrates easily with AWS storage, delivery, and security services. It supports a wide variety of formats, resolutions, and encoding presets. Its built-in support for HLS, DASH, captions, and DRM makes it a solid choice for teams working entirely within the AWS cloud.
Harmonic VOS
Harmonic’s VOS platform is a comprehensive video processing and delivery solution used by broadcasters and telecom providers. It supports both live and on-demand workflows and provides robust encoding, packaging, and delivery capabilities. Harmonic is known for reliability and scale, especially in telecom-grade video environments.
Zencoder (Brightcove)
Zencoder is an encoding service designed for high-throughput media operations. It provides reliable transcoding with strong API support, a wide range of format options, and integrations for enterprise publishing and delivery systems.
These platforms reduce the engineering burden of managing encoding pipelines, especially when working with complex delivery requirements and large video libraries.
DRM and Content Protection
With so much valuable content being distributed online, protecting it against piracy and unauthorized access is essential. Digital Rights Management, or DRM, helps secure video streams by encrypting the content and enforcing playback rules.
DRM systems prevent unauthorized copying, restrict playback to approved devices, and control how long a video is accessible. They are a critical part of monetizing content on subscription services, pay-per-view models, and licensed broadcast platforms.
Three of the most common DRM systems are:
Widevine
Used primarily in the Google ecosystem, Widevine protects content on Android devices, Chrome browsers, and many smart TVs. It is widely supported and used for both standard and high-security video workflows.
FairPlay
Developed by Apple, FairPlay is the DRM system required for encrypted HLS playback on iOS and Safari. It integrates tightly with the Apple ecosystem and is essential for securing content on Apple devices.
PlayReady
Microsoft’s DRM system, PlayReady is commonly used on Windows, Xbox, and Smart TVs. It is widely adopted by pay-TV providers and supports advanced content usage rules, such as rental expiration and concurrent stream limits.
Cloud encoding platforms often include DRM packaging features as part of the workflow. During encoding, the video is encrypted and packaged for delivery. License servers then manage key exchange at playback. This ensures content remains protected without impacting the viewer experience.
Streaming Profiles and Adaptation
From codecs that reduce video size to protocols that manage delivery and cloud platforms that automate processing, every layer of the streaming stack plays a vital role in shaping the viewer experience. DRM ensures content is protected and reaches only the intended audience.
A concept that sits at the core of adaptive streaming is the encoding profile. This refers to a predefined set of renditions with varying resolutions and bitrates that a player can switch between during playback. For instance, a user on a slower connection may receive a 480p stream, while someone with a high-speed connection and a 4K screen gets a higher-resolution version. Well-designed profiles help maintain smooth playback, conserve bandwidth, and reduce infrastructure costs without compromising quality.
Looking Ahead: Ads, AI, and Personalization
Streaming continues to evolve, moving beyond playback into more intelligent and monetizable workflows. Advertisement ingestion is now central to many platforms, requiring precise coordination between encoding segments and ad breaks. Using techniques like manifest manipulation and SCTE markers, platforms can insert mid-roll and pre-roll ads without disrupting playback. Server-side ad insertion allows this to happen in real time, even for live streams.
Artificial intelligence is beginning to play a larger role in encoding decisions. AI models can analyze scene complexity, detect motion levels, and recommend encoding settings for individual videos or scenes. Some platforms use AI to generate optimized profiles for every title, reducing delivery cost while maintaining perceived quality.
Personalization is also becoming a defining factor in modern streaming. Adaptive delivery is no longer just about device or bandwidth. It now includes viewer context, preferences, and behavior. Streaming platforms can tailor playback resolution, audio tracks, captions, and even encoding formats to suit the needs of each individual viewer.
In the next edition of Behind the Stream, we will dive deeper into how profiles are structured, how adaptive bitrate logic works under real-world conditions, and how to design scalable delivery pipelines that balance quality, efficiency, and cost.
Stay tuned as we continue unpacking the systems that power modern video delivery.
