Thursday, March 1, 2012

Video and audio in Win8/IE10 Metro

Yesterday Microsoft released the Windows 8 Consumer Preview (beta), so I downloaded it and installed it on my laptop. One of the first things I did was use the new Metro interface to look at how Gallery Server Pro looked in Internet Explorer 10. Overall, things looked great until I tried to view one of the videos in the demo gallery:

IE_metro_mp4_with_flash_cropped

That really wasn’t a surprise since the browser templates in GSP are configured to use either Flash or Silverlight for video and audio in IE. However, Microsoft has decided not to support any plug-ins in IE10 when running in Metro. (Plug-ins *are* supported when IE is running in desktop mode.) Instead, Microsoft is encouraging use of native HTML5 <video> and <audio> tags for media. That is great, except out of the box IE only supports H.264 (MP4) for video and MP3/AAC for audio. Any other video or audio format, such as your collection of Flash Video files (FLV), are simply not going to work in IE10 Metro. Period.

As drastic as this seems, I welcome the move away from plug-ins, as I can’t tell you how many hours I have spent fiddling with Flash/FlowPlayer/QuickTime/DIVX/Silverlight scripts and configurations over the years. That has easily been the most difficult challenge with GSP – how to best render any type of file (video, audio, document, image, etc) in any kind of browser.

A plug-in free world, however, brings a different kind of mess. Different browsers support different video and audio codecs, and there isn’t a single format that works in all browsers. But I digress.

Let’s get back to IE10 Metro in Win8. Since IE10 supports H.264 (MP4), we can tweak the browser template so that newer versions of IE that support MP4 get the HTML5 <video> tag, while older versions (before IE9) continue to use Flash. To make this change, run the following SQL scripts against your Gallery Server Pro database:

INSERT INTO [gs_BrowserTemplate] ([MimeType],[BrowserId],[HtmlTemplate],[ScriptTemplate]) VALUES (N'video/mp4',N'ie1to8',N'<a href="{MediaObjectUrl}" style="display:block;width:{Width}px;height:{Height}px" id="gsp_player"></a>',N'window.gspRunFlowPlayer=function(){jQuery("#gsp_player").attr("href",function(){return this.href.replace(/&/g,"%26")});flowplayer("gsp_player",{src:"{GalleryPath}/script/flowplayer-3.2.7.swf",wmode:"opaque"},{clip:{autoPlay:{AutoStartMediaObjectText},scaling:"fit"}})};if(window.flowplayer){gspRunFlowPlayer()}else{jQuery.getScript("{GalleryPath}/script/flowplayer-3.2.6.min.js",gspRunFlowPlayer)};');
GO

INSERT INTO [gs_BrowserTemplate] ([MimeType],[BrowserId],[HtmlTemplate],[ScriptTemplate]) VALUES (N'video/mp4',N'ie',N'<video src="{MediaObjectUrl}" controls autobuffer {AutoPlay}><p>Cannot play: Your browser does not support the <code>video</code> element or the codec of this file. Use another browser or download the file by clicking the download toolbar button above (available only when downloading is enabled).</p></video',N'');
GO

This SQL works against GSP 2.6.0 or higher. Older versions use a different version of FlowPlayer so you’ll have to adjust the script accordingly.

After adding these rows, restart the IIS application pool to clear the cache. When refreshing the page above, it now looks like this:

IE_metro_mp4_with_html5_cropped

That’s better. The built-in video controls offer basic playback features like play and pause, but unfortunately it is missing a full-screen mode. Hopefully Microsoft includes it in the final IE10 release.

This solves the issue of rendering H.264 (MP4) videos. A similar SQL script can be used for handling MP3/AAC audio (sorry, but I don’t have time to put that one together). But how to handle other video and audio types? Unfortunately, I know of no way to get other types to work in IE10 Metro. The good news is that you can still see this content in Win8; you just have to move over to desktop mode first. Microsoft is well aware that tons of plug-in dependent sites will break in IE10 Metro and has added a button on the screen to easily switch to desktop mode for the current page.

Should you run this SQL script against your current gallery? Probably not. All it does is make it possible to view MP4 video in IE10 Metro. Other IE10 scenarios, such as Win8’s desktop mode and IE10 on Vista/Win7 already work just fine (as long as Flash is installed). And there is even a slight downside in that IE9 users will now use the HTML5 video player, which is not as nice as the Flash one (e.g. it is missing a full-screen button).

I will likely incorporate the browser template changes described in this post in the next version of GSP. And I will continue to monitor the current state of all browsers and their video/audio support so that GSP takes advantage of their latest capabilities while working in as many older browsers as possible.

2 comments:

Dan F said...

Hello, have you tested the video playback on Windows RT (Surface) devices? My surface does not play mp4 videos so I was curious on your opinion on the direction of video playback on this device. Thanks - Dan F

Roger Martin said...

I don't have a Surface, but I would expect it to behave the same way as IE10/Metro on Win8, which is what I tested in this post. Did you apply the fix described here?