Synchronized live streaming for HLS and DASH

That’s a race against the clock, so everybody should have the same clock.

It’s no secret, traditional OTT live feeds are not just delayed, the delay is arbitrary and each screen will be slightly out of sync with any other. It’s not even a surprise if you understand the inner workings of internet streaming. Basically, the client will download chunks of content in a sequence and start playing that as soon as it’s long enough to be considered safe to further play without interruption. No more waiting as that would worsen the user experience. Yet for others, the download time won’t be the same, so they’d start playback somewhat earlier or later. 

Still, there’s times when you really need your viewers to see the same thing at the same time, otherwise it may offer some an unfair advantage, if seeing something even narrowly ahead of others. Think gambling or sports betting, auctions or trivia shows.

There are so many approaches to synchronizing your users’ video feed that I’ll have to dare categorize them; that is for fear of having to go into detail. As they all need an immutable reference to synchronize to, that may be the best criteria:

  1. Sync to NTP or equivalent
    You can’t assume your users’ devices’ clock is accurate. In fact, it’s often not. So one solution will be to get a sample of the “good” time via a reliable clock sync protocol like NTP; of course, you won’t “fix” the user’s clock but rather use it internally for video synchronization; for this to work, video has to carry similar wall-clock information with timestamps obtained from the same NTP source
  2. Sync to video/media server
    Socket-based video transmissions like RTMP or SLDP can benefit from the intimate connection with the server to swiftly get a reasonably accurate time sample directly from it; in the end, it’s not important what wall-clock you sync to, just that incoming and outgoing video can use the same absolute time reference 
  3. Sync implicitly for low/ultra-low latency and near-realtime streaming
    While for some sub-second delivery mechanisms the delay is so small that its fluctuation is inherently within certain acceptable margins for many applications, several providers will build fixed-delay capabilities right into the protocol and even advertise frame-accurate sync. Won’t further elaborate on the topic for now but some of it has been addressed here.

Interesting to note, the sync isn’t always perfect. That’s because, unlike broadcast television that uses dedicated data channels, OTT makes large use of the public internet and unpredictable private networks. Rather, the solutions will aim to bring the gap under a threshold that serves the purpose. 100 or 200 ms will be acceptable for gambling etc., while a 25-50ms will be fine even for displaying 2 TVs side by side in the same room.

Also important, the larger the tolerable live delay, the better you’ll be able to sync. Having a playback buffer helps deal with bandwidth fluctuations, but in the case of synchronized play this buffer will need to have the same length for all. 

And just so you know, being in sync just won’t work for every one of your viewers. Even with ABR, simply offering a playable stream at all is challenging enough for some device/bandwidth combinations; not having the luxury to pause and buffer (because it would go out of sync) will be too much for a few and a drop-out strategy will need set in place (i.e. notify the user they can’t support playback). You’ll actually have to trade off between how many can watch and how strict of a sync you are imposing. You’d need to employ certain analytics to know for sure, but expect to cover at least 95% in areas with good high speed coverage like Europe and North America for a 200ms tolerance. 

In light of the above, customer needed a proof-of-concept sync strategy to deploy on top of their HLS and DASH setup. What’s been delivered is pretty basic but also quite flexible. It allows adjusting the live delay (i.e. 10 seconds) and the sync tolerance (i.e. 100ms) to experimentally adjust to the optimal setup for a given use case. They may have further built upon or never have used it (can’t tell) but agreed to share the initial deliverable. It is here for anyone to use or hack with. 

Does it scale?

Yeah. The sync is handled in the client browser so there’s nothing burdening the delivery. Any scaling scheme you’d deploy for HLS or DASH would still be compatible. For what it’s worth, the NTP server needs to keep up with all the requests but that should be trivial. 

Is it stable?

Not to perfection. I’ve seen it glitching numerous times (it’s a POC mind you), especially when bringing the numbers down to a certain limit. Ideally, you’d know your audience and (at least) emulate their worst network capabilities to roughly tune your initial product. Going further, consider gathering QoS statistics and further improve upon it. 

Is it worth it?

No idea 🙂 As with most open source solutions, you will eventually run into issues you can’t blame anyone for. If you have a skilled dev/qa team, manipulating this should be a breeze. Otherwise you may want to opt for an enterprise solution, there’s at least a handful out there.

Is it expensive?

Figure it out yourself 🙂 There’s no added data or infrastructure costs, just the work of altering, tuning and maintaining the “synchronizing” player.