Building Romulus

When Wistia began, Flash video ruled the world. It wasn’t just the best way to stream video online — it was the only way. Eventually, the HTML5 video standard was conceived by the free web, and though it was young and imperfect, the community understood that it would one day be king.

At Wistia, we’ve been falling back to HTML5 for as long as it’s been around. There have been wars over codecs, conflicting implementations, and bugs along the way. But after years of development, the technology has finally improved to a point where we can use it as the basis of our default player.

This post exists to give clarity on the technical details of our new HTML5 player, Romulus, and the logic behind them.

Why make the change now?

I’m glad you asked! Here are some key events in HTML5 video land.

• 2008–2009: Chrome, Safari, and Firefox get HTML5 video support early on.
• March 2011: Internet Explorer 9 adds support for HTML5 video, marking the dawn of modern IE.
• June 2011: Chrome supports both webm and h.264. Says it will drop h.264 support, but never does.
• 2011–2012: There is much gnashing of teeth over webm vs. h.264.
• April 2013: Mozilla relents and allows h.264 support for Firefox, making it the last major browser to do so.
• April-October 2013: Chrome, Safari, and Firefox implement early drafts of the HTML5 fullscreen API.
• November 2013: Cisco purchases rights to h.264 and open sources it.
• November 2013: IE11 adds HTML5 fullscreen API support, the last major browser to do so.

Chart courtesy of CanIUse.com.

The three major requirements for Romulus are HTML5 video, h.264 decoding, and fullscreen API support. On top of those requirements, browsers are becoming generally more hostile to Flash. For example:

• Safari no longer comes with Flash installed.
• Tablet versions of modern IE no longer allow Flash.
• Firefox now requires “click-to-flash” by default.

Looking at the circumstances as a whole, it’s pretty clear that now is the time to start the move to HTML5-by-default.

Improvements in Romulus

The move to a new HTML5 player enabled us to improve a lot of things. Enjoy this long and rambling list!

Frame by frame seeking

We now support frame by frame seeking. If you pause the video and hover over it with your mouse, you can use the arrow keys to step frame by frame. This improvement also allows smoother stream switching when switching to HD. Finally, because stream switching is built-in to HTML5 video, playlists can transition between videos without leaving fullscreen. I’ve been waiting for that one for a long time.

Faster load times

Since we don’t need to load the Flash player and wait for the Javascript interface to come up, it loads super fast. That is, on the order of 10ms instead of 600ms. Oh, and there’s also no more flicker on load because the Flash player doesn’t need to initialize!

Changeable playback rate

Using the Player API, you can change the playback rate of the video. One caveat: it isn’t in the documentation yet since Flash doesn’t support it at all!

Fullscreen support for plugins & labs

We use the HTML5 fullscreen API to support plugins and Labs in fullscreen. However, we still only do it where it makes sense. The fullscreen API isn’t all-powerful. For example, Safari doesn’t allow any keyboard input in fullscreen mode.

A minor change that we really like: you can finally fullscreen before play.

Saves bandwidth

Some browsers will save bandwidth by throttling. For example, Google Chrome tries to only buffer 25 seconds ahead of the current frame. We don’t control this feature, but it’s a cool bonus that comes with HTML5!

Custom menu

Since the video is pure HTML, we can make a custom context menu on it. Right now, that means you can jump straight to the video in your Wistia account. Don’t worry, this option is only available if you’re logged in and own the video.

More stable videos

The intersection of Flash, graphics drivers, and the browser makes for some pretty gnarly bugs. HTML5 video cuts out one party, so there are fewer weird interactions. By simplifying, we improved the chances that your video will just work.

We’re better at the technologies behind HTML5

Finally and, in my estimate, most importantly, our expertise at Wistia is in HTML, Javascript, and CSS — not Flash. We know enough Flash to get by, but with our default player using HTML5, we can iterate faster and do better work.

A little more about the downsides of Flash

Flash had a lot of weird issues.

To be fair, it’s not entirely Adobe’s fault. Flash had to dance around third party drivers, operating systems, and different hardware. Each browser needed to maintain its own Flash integration. It’s impossible to account for all the possible permutations, and it’s impressive that Flash integrates as well as it does.

But in the end, people don’t care about that. They just want their videos to work. Here are some of the most aggravating bugs HTML5 will help us avoid:

Flash can’t be hidden without the Javascript interface breaking

I don’t know exactly why this happens — some optimization, I guess. But because of this issue, we monitor the visibility state of our Flash embeds, and rebuild them when it changes. That makes sure the Javascript works, but it doesn’t look great because the video flickers on rebuild.

Occasionally, Flash fails to load or causes errors

I’m not sure how it gets into this weird state, but when it happens, the only fix is to restart your browser or computer. That’s not a good experience for anyone.

Unpredictability with CSS transforms

Modern browsers have been implementing CSS transforms, which frequently use hardware acceleration to speed up computation. But when you combine Flash and hardware-accelerated CSS transforms, you get strange and unpredictable results. The video may appear 600px below its real location, or get rendered upside down and backwards. Seriously.

A lot of these situations occur when people are building dynamic sites with sliders or custom popups. We expect it will be way easier to do cool things like that with Romulus!

Where HTML5 video can still improve

On the whole, HTML5 is a huge upgrade, but there are some minor inconveniences that we want to work out.

Required “allowfullscreen” attributes

IFrame embeds require “allowfullscreen” attributes for full Romulus support. There’s not exactly anything wrong with that — it’s a good security feature — but a lot of content management systems are built with whitelist HTML validation, and all but the newest validators don’t include those attributes. Twitter cards have a similar problem: we just provide the IFrame src attribute, and they don’t include “allowfullscreen” attributes by default. This situation should improve with time and familiarity.

MOOV atom size increases with video length

A bigger issue is that our HTML5 assets are encoded in an MP4 container, and the MP4 format requires that you download a full MOOV atom before the video can be played. The MOOV atom’s size increases with the video’s length, which means a longer wait after clicking play for long videos.

Buffering immediately is a good way to lose a viewer, which is why we only use Romulus for videos under 15 minutes by default. In the meantime, we’re looking for different ways to get around this limitation.

Some browsers rely on native h.264 decoding

Last, and hopefully least, some browsers, like Firefox on Windows, rely on the h.264 decoding implementation that’s native to the operating system. Whether that implementation works correctly 100% of the time is yet to be seen. We’re keeping our eye on it just in case. It may be that they start using Cisco’s h.264 implementation instead, which should normalize behavior across computers.

Flash is still good at some things

Flash is much maligned, but it still has its uses in the foreseeable future.

• It’s the best (and only) delivery method for browsers without HTML5 video. I’m looking at you, IE8 and below. Also you browsers that have neglected your updates for years. Grr.
• The FLV format — that’s what Flash uses instead of MP4 — handles long videos well because it doesn’t need to download all the metadata before it can play.
• Security? Flash doesn’t care about “allowfullscreen” attributes, so you can still fullscreen from those old (and possibly mangled) IFrame embeds.
• Also, it pains me to say it, but there are still some environments that are Flash-only. No IFrames, no scripts, just object/embed tags. I’m looking at you, Facebook! Get with the times!

Intelligently Choosing a Player

I’ve mentioned in a few places that we are trying to intelligently choose which player, HTML5 or Flash, is used for each embed. I want to give more clarity on exactly how we plan to do that.

First, if you are familiar with our Player API, you might have seen an option called “platformPreference,” which allowed you to choose between “flash” and “html5” — that is, Flash and the native HTML5 player, not Romulus.

Image courtesy of Opera.

We are deprecating that option in favor of one called “playerPreference,” which allows you to choose between “auto,” “html5,” and “flash.” The difference is that, in this scheme, “html5” means Romulus if it’s supported, and the native HTML5 player if it’s not (notably IE9 and IE10). NOTE: “platformPreference” is deprecated, but still works and technically overrides “playerPreference.”

Here are browsers that, at time of posting, have basic Romulus support:

• Google Chrome
• Safari
• IE11+
• Firefox on Windows

For full Romulus support, the video needs to meet the following requirements:

• It’s less than 15 minutes.
• It has high quality MP4 encodes. All new videos have these, but old ones probably don’t.
• It can be fullscreened. (If it’s in an IFrame, it must have “allowfullscreen” attributes.)

By default, “playerPreference” is set to “auto,” which means it will try to use Romulus when it detects full support. If it does not detect support, it will fall back to Flash, then native HTML5.

You can choose to manually set playerPreference to “html5” in your embed code. If you do that, it will bypass the additional checks for full support, using Romulus if it detects just basic support. If it does not detect Romulus support, it will fall back to native HTML5, then Flash.

Rolling out Romulus

We were careful and took our time with this launch. The plan was to enable Romulus slowly for interested accounts. Then, we’d gradually ramp up the user base, catch any major bugs, and release to everyone once it was polished.

We turned Romulus on for our own account in October 2013. For the next few months, as we talked to people in support, we enabled Romulus for those we thought could benefit from it. This turned out to be a good way to prove that Romulus would help solve some of the annoying issues we had with Flash.

After a while, we reached out to customers with large user bases and turned on Romulus for them. After a couple more months of small bug fixes, we figured out the logic for our “auto” player preference. That logic reduced our risk of deploying, so we decided it was time to accelerate the launch.

Painting courtesy of Jean Auguste Dominique Ingres and Wikipedia.

Two weeks before launch, we switched all new accounts to use Romulus by default. This let us test Romulus in the wild on a wide range of relatively small, low-risk accounts. Seeing no major issues, we continued on.

We were confident, so starting a week before launch, we enabled Romulus for a large chunk of our customers each day. All the accounts got up-to-date quickly, but we still had some leeway to address issues without affecting our entire user base.

There were a few small snags — for example, fullscreen mid-roll Turnstile had to be canned due to Safari’s restrictions on keyboard input — but on the whole, they didn’t affect playback and were easy to fix.

Romulus in the long term

We’ve put a lot of thought into this change. With any luck, the current trends will continue. Browser technology will improve, users will upgrade to get HTML5 video support, and all will be right with the world.

Then, maybe one day, somewhere in the future, we can retire our old workhorse, Flash. It deserves a break.