Video Streaming with Adaptive Bitrate with Scott Wachtmann & Jen Brissman
Delivering Videos with the HTML Video Element & Video Player
This stage gives an overview of video on the web by comparing traditional, streaming & adaptive bitrate video strategies. Link to our blog about this topic:
Now let's take a quick minute to compare video strategies that we have available on the web. You may already be familiar with the HTML video element that allows you to embed video onto your webpage using containers and codecs, such as MP4, WEBM, or OGV. Cloudinary allows you to push this optimization a step further with transformations like our smart compression through q_ auto or other transformations that you might want to use to make creative adjustments to your content.
The main drawback of this approach to video, however, is that lengthy videos can take a longer time to buffer and become available to playback for your users. To get around this, you might consider streaming video. Streaming video is provided in chunks. This can help you avoid bandwidth consumption if your users happen to bounce before the entire video has played back and downloaded to the browser.
However, this is often unnecessary for short duration videos because the total size of an individual chunk is larger than the entire video itself. To push streaming a step further, we can turn to adaptive bitrate streaming. This allows quality to scale up or down based off of network conditions that are available to your users at the time of playback.
However, it does create a trade off between quality and speed for your users if their network conditions happen to take a turn during playback.
For our purposes, working with adaptive bitrate streaming, we're going to be using HLS or HTTP live streaming. The HLS protocol consists of two primary types of files: m3u8 files, and .ts files.
M3u8's are used for two purposes. One m3u8 will contain a master list of all of the different streams that are available for playback. Once the browser has selected a specific stream and additional m3u8 file will be downloaded as well. That contains a list of all the individual chunks that make up the stream or the .ts files for our video. ABR ,or advanced bitrate, gives the browser the ability to request different bitrates for different network conditions or bandwidths.
Profiles are composed of chunks of video, and each video is given at a specific resolution or height and width measurement. In order to make use of adaptive bitrate, we're going to need to install a more robust video player than what's available in the video HTML element. If your needs are simple, you will be able to use the HTML element to do things like show and hide controls, control autoplay conditions, automatically loop and mute video, and show an initial poster.
However, if you want to push things further, like being able to style the player itself, play additional formats or make use of ABR or playlist, you'll need an advanced player such as the one offered by Cloudinary or video.JS. If we look inside of HLS and ABR, we can take a look at the files that allow playback.
We'll start with the m3u8 manifest file. If we look at this file, you'll notice that each resolution listed inside the file corresponds to a specific streaming profile that we've created. Those resolutions each contain their own m3u8 playlist. Inside of this playlist, we have the individual binary representations or chunks that make up our file.
As the browser plays back video, you may see it make changes to which stream it plays to the user based on the conditions that are available. Typically a browser will start with a mid-level stream. However, if the buffer fills quickly and network conditions allow, it may choose to upscale to a higher resolution.
This could also happen if the end user themselves specifically requests a resolution through the video player that you've integrated. However, this could lead to buffering if their network conditions aren't able to keep up with the quality that they've requested and they may then drop back to a lower quality stream in order to smooth out playback.
We can actually see the behaviour of these files if we look in our browser's network console. If you look at this example that was taken during page load, you can see the network panel has downloaded three files, two m3u8 files, and one .ts. The first m3u8 is our master playlist file that contains all of the resolutions available for this video.
Second, we've selected a particular resolution that we're going to use for our initial start point for playback. The browser has also downloaded the first .ts file. So the first chunk of video is available for playback the moment the user hits their play button. Once playback has started, behaviour will vary based off of network conditions.
At higher speeds, you'll see larger .ts files that typically download very quickly to the browser's buffer as the video plays back. The video might even finish downloading before playback is finalized. However, under poor network conditions, you'll see smaller .ts files being moved across the network, and the video may take a longer time overall to download if we're using something like a 3g mobile connection to play back this piece of content. We're also receiving the content at a lower bitrate or a lower quality due to that decreased bitrate.