Monday, October 31, 2011

Version 2.6 released

Gallery Server Pro 2.6 is now available. It sports a brand new file upload widget, the ability to encode video and audio files, and much improved SQL CE performance. Full details about this release are posted on the release history page.

To upgrade an existing version, just copy the files from the upgrade package over the ones in your existing web directory. If you are upgrading from 2.3 or from .NET 3.5 or earlier, there are a few extra steps – full details are in the Admin Guide. If you are using the DotNetNuke module, just follow the standard module install steps.

I covered the new features in my previous blog post, so here I’ll just cover what is missing from that post.

New file upload experience

The ComponentArt Upload control has been replaced with Plupload. There are several benefits:

  • HTML 5 drag and drop support
  • Ability to create a compressed version of images before uploading them
  • Takes advantage of advanced browser features while gracefully degrading when necessary
  • Progress feedback in medium trust scenarios
  • Multiple file selection

When the add objects page loads, the upload control detects your browser’s capabilities and chooses the most appropriate runtime, based in this order of preference: HTML5, Silverlight, Flash, BrowserPlus, and HTML4. The runtime is shown in a tooltip when you hover over the upload area, as seen in the screen shot below.

The exact features available depend on your browser’s capabilities, including whether it has Flash or Silverlight installed. For example, drag and drop works only in Chrome and Firefox. Client side image resizing requires Flash or Silverlight to work in IE.

Client side image resizing

To request that images be resized on the client before being uploaded, select the ‘Discard original file’ option below the file selection window, as seen here:


When that option is selected and the browser supports client-side resizing, a compressed copy of each image is created in the browser and then uploaded to the server. This can dramatically reduce the length of time it requires to upload files. If the browser can’t create a compressed file, the original is uploaded and the server takes care of creating the compressed version and discarding the original file, just like how it worked in previous versions of Gallery Server Pro.

In most cases, the image’s metadata is copied to the compressed version so that it can be extracted on the server. However, there is one unfortunate exception. The Silverlight implementation, which is often used in Internet Explorer, fails to copy the image’s metadata to the compressed file, resulting in the metadata being lost. There are two workarounds:

1. Do not select the ‘Discard original file’ option when uploading files. Instead, delete the original files after they have been uploaded with the ‘Delete original files’ option in the Actions menu.

2. Modify the file upload’s control definition so that it never uses the Silverlight implementation, or at least prefers the Flash one to the Silverlight one. Do this by opening the file gs\pages\task\addobjects.ascx in a text editor and looking for this line:

runtimes: 'html5,silverlight,flash,browserplus,html4',

Either delete Silverlight from the list or rearrange it to be after Flash. However, in my experience the Flash runtime is not as robust as the Silverlight one, so that is why I choose to prefer Silverlight.

Video / audio encoding

Version 2.6 adds support for encoding video and audio files to other formats after uploading. For example, you can upload an AVI file and have it automatically converted to a Flash Video file (FLV). The original AVI file is still available by clicking the view original button, much like original images are always a click away, even though web-friendly ones are shown by default.

The encoding is done by FFmpeg, a popular open source utility that Gallery Server Pro was already using to generate thumbnail images from videos. FFmpeg is included in the Gallery Server Pro Binary Pack. Note that your gallery must be running at full trust in order to execute FFmpeg.

You have complete control over which files are converted, to what they are converted, and how it is done. Here are the default encoder settings you’ll see when you look at the Video & Audio page in the site admin area:


These settings are interpreted as:

  • Do not encode MP3, MP4, FLV, or M4A files.
  • Try to convert all video to FLV. If that fails, try converting it to MP4.
  • Convert all audio to M4A.

Feel free to adjust these as needed. I experimented with several variations and settled on this as the best compromise. Originally, I preferred converting to MP4 H.264 first, using FLV only when that failed. But, while MP4 files have several advantages, such as being a (fairly) open standard, small files sizes, and the ability to play without a plug-in in some browsers, it has one big limitation: The MP4 files created by FFmpeg must be fully downloaded by the browser before they begin playing, so you might have to wait a while before your video starts. This is a limitation of FFmpeg, not MP4 H.264, so I am hopeful that a future version resolves this. For now, Gallery Server Pro will default to converting video to Flash Video.

Viewing the encoding progress

An encoding’s progress is monitored as it processes each file. You can see the latest information about an encoding by refreshing the web page and then clicking the Expand link in the encoding queue.

Adjusting the encoding timeout

During QA testing, I noticed that certain video files would hang, continuing to use processor resources but not making any progress. Worse, when this happened, the encoding process didn’t respond to requests to cancel it, unless you used Task Manager to kill it. This is a potentially nasty situation because encoding uses a lot of server resources, and you don’t want it running for hours hammering away on your server’s processor. Your hosting company could get quite upset.

The solution I came up with was to set a timeout when kicking off the encoding process. The timeout default to 15 minutes (900,000 milliseconds) and can be set in the site admin area as seen in the screen shot above. If the encoder is still running when the timeout fires, it is automatically cancelled.

You may need to increase this timeout if converting large files. Whenever a timeout is hit that causes an encoding to be abandoned, an entry will be written to the error log.

Rebuilding encoded files

Gallery Server Pro treats the encoded video and audio files just like it does the optimized images that are created for image files. The encoded versions are shown by default when you view the media object, but the original is available by clicking the ‘View original’ button in the toolbar.

To build encoded versions of your media objects after you upgrade to 2.6 or anytime you change an encoder setting, run a synchronization with the ‘Rebuild optimized versions’ option selected. The existing encoded version will be replaced by a freshly created one.

iOS support

The old handler that served media files didn’t support HTTP range requests, which is a technique some browsers use to request a portion of a media file. iOS devices require this support, so for version 2.6, the handler was rewritten to support these types of requests. Now any iOS supported file can be played in your device from the gallery. I believe the current supported file types are MP4, MP3, M4V, MOV, and M4A.

If you have media files in your gallery that are not supported in iOS, you can tweak the encoder settings to create an iOS-compatible version. One easy way to do this is to reposition the MP4 encoder setting to come before the FLV setting in the encoder settings section. Just remember the one big flaw with MP4 files – they must be fully downloaded in the browser before they begin playing.

Monday, October 10, 2011

Version 2.6 is faster and brings new uploader, iOS support, video/audio encoding, and more

You know, I wasn’t planning to release a version 2.6. Right after 2.5 was released a few months ago, I started cranking away on version 3, which is a significant release that adds all the top features you voted for in the poll. It was coming along great when something happened.

A fan of GSP came along and was willing to pay good money if I immediately added two features: (1) on-the-fly encoding of AVI videos to the web-friendly MP4 format, and (2) the ability to select multiple files in one step on the add objects page. So I added those features to the 2.5 codebase and gave it to the client as a custom build. These are features many of you would find useful, so I will soon be releasing version 2.6 to bring them – and a few bonus features - to you. Here is the complete list:

  • 15X faster SQL CE performance
  • New file upload experience
  • Client-side image resizing
  • iOS support
  • Improved browser caching
  • Video and audio encoding

The release is at least three weeks away, but hopefully not more than a couple months. I am putting the finishing touches on it now and will get it to you as soon as I can.

15X faster SQL CE performance

Version 2.5 added SQL CE as a data storage option while retiring SQLite. There were many benefits to SQL CE over SQLite, such as medium trust support, 32/64-bit support in a single package, and strong data typing and referential integrity. But, as many of you noticed, it was slower. One test I did showed that it took about eight seconds to load an album with 250 media objects, while SQL Server could do it in about 0.1 seconds. Although subsequent loads of the same album were almost instantaneous due to caching in the application, first impressions are important, and an eight-second page load is clearly unacceptable.

After profiling the application and database, I found that most of the page load time was spent retrieving the metadata for each media object, one SQL statement at a time. For example, a page with 250 media objects would make 251 calls to the database (one request to the media objects table and 250 to the metadata table). SQL Server does the same thing, but since it does it with a stored procedure whose SQL is compiled and cached, it is very fast. The SQL CE implementation uses Entity Framework 4.1 Code First, which does not cache the query plan (although I hear that is coming).

The solution was to modify the query logic to return the media objects and the metadata in a single SQL call. Instead of 251 trips to the database, it is now one. This change reduced the load time from eight seconds to 0.5 seconds, a 15x improvement. While that is much better, it is *still* about five times slower than SQL Server. For most users, though, it is quite acceptable. If you want the fastest performance and don’t mind the additional responsibilities that come with installing and managing SQL Server, than use SQL Server (even SQL Server Express, which is free, is faster than SQL CE). The rest of you can enjoy 15x faster SQL CE performance.

New file upload experience

Providing a great file upload experience has been challenging for web developers since the dawn of time. The first version of GSP used a nice open source component called NeatUpload. When ComponentArt added an upload component to its suite of controls in 2008, I moved to that one. Both, however, have annoying limitations: (1) You can choose only one file at a time (not exactly true with NeatUpload, but it required a Flash dependency), (2) They require the app to run in full trust.

When I was asked to support multi-file selection for a client, I knew I had to replace the ComponentArt control with something else. Like manna from heaven, I found a wonderful open source component called Plupload, which offers a dream list of features: HTML5 support (like drag and drop!), graceful degradation, support for medium trust, multi-file selection, and more. For example, here is what the add objects page looks like after I drag four image files from Window Explorer onto the page:


While uploading the files, the vast majority of users will see a nice progress bar indicating the status of the upload, even when the app is running in medium trust. The only ones who won’t are those running an old browser without Flash, Silverlight, Google Gears, or BrowserPlus. Not even your grandmother should fit into that category.


The real beauty of this upload control is that it takes advantage of advanced browser capabilities while gracefully degrading when necessary. So, enjoy drag and drop if you’re using Chrome or Firefox, but IE6 users can still upload files using a file selection dialog.

Client-side image resizing

I saved my favorite Plupload feature for its own section - client-side image resizing. Rather than waiting for a hundred 5 MB digital camera photos to crawl their way to the server, you can let the browser create smaller, web-friendly versions for you, resulting in images that upload in a fraction of the time. Now you’re done in seconds instead of minutes (or even hours).

I’ve been wanting to add this feature for a while now, and even went so far as to ask a friend of mine earlier this year to build a prototype using Silverlight and fjcore. But now we get this feature for free, and without any dependencies on plug-ins. Sweet!

The one downside - this feature requires Firefox or Chrome. But as more browsers become HTML5 compliant, expect support to grow.

iOS support

The current version of GSP won’t play video or audio on iOS devices (iPhone, iPod, iPad), even when specifying a supported file type like MP4 and using the HTML 5 video and audio tags. This was because iOS requires HTTP range support, and GSP’s HTTP handler didn’t support that. For 2.6, the handler has been re-written from the ground up to support HTTP range requests.

GSP 2.6 will support any media file in iOS that is natively supported. Currently, that means MP4, MP3, M4V, quicktime, and M4A.

Improved browser caching

Rewriting the HTTP handler provided an opportunity to take another look at how media files are cached in the browser. The 2.5 handler sends HTTP header values to the browser that instruct it to cache the images for thirty days. But in my experience some browsers seem to ignore this advice and request the images from the server anyway, except for after an image rotation, in which case the browser stubbornly likes to show the original, unrotated image, even after the user hits the refresh button or F5. I say that slightly tongue in cheek, but the basic idea is that there was an opportunity to improve the caching.

Here is an example set of response headers for an image in 2.4.8:


The same image in 2.6:


In 2.6, we got rid of the Expires setting and instead use Last-Modified and the Etag. By doing this, the browser will automatically be notified of rotated images and other changes to the media file. Even this isn’t perfect, though, as some browsers don’t check with the server when a user very recently loaded an image. But even in this case a click of the refresh button will force the browser to send the Last-Modified header, which will then cause the updated image to load.

Another change is that the file name is sent in the Content-Disposition tag. This is nice when you right-click an image to save to your hard drive. The value here will be used for the default filename, which is an improvement over getmediaobject.ashx, which is the default filename users saw in 2.5 and earlier versions.

Video/audio encoding

Version 2.6 adds support for encoding video and audio files to other formats after uploading. For example, you can upload an AVI file and have it automatically converted to an HTML5-friendly MP4 video. The original AVI file is still available by clicking the view original button, much like original images are always a click away, even though web-friendly ones are shown by default.

The encoding is done by FFmpeg, a popular open source utility that Gallery Server Pro already uses to generate thumbnail images from videos. FFmpeg is included in the Gallery Server Pro Binary Pack. Note that your gallery must be running at full trust in order to execute FFmpeg.

You have complete control over which files are converted, to what they are converted, and how it is done. For example, look at the Encoder Settings section in this screen shot:


The order of the encoder settings is important – the first one that matches is the one that is used. If it fails for any reason, the next matching one will be used. So, the screen shot shows that the preferred conversion for AVI files is to convert them to MP4 using these command line arguments for FFmpeg:

-i "{SourceFilePath}" -y -vcodec libx264 -fpre "{BinPath}libx264-galleryserver.ffpreset" "{DestinationFilePath}

It basically says to apply H.264 encoding using the options in a file named libx264-galleryserver.ffpreset in the bin directory. The text in brackets {} are parameters that are replaced at runtime. You can use any FFmpeg option to give yourself complete control. I have not yet decided on the exact encoding settings and parameters to use as default values – the ones that ship will likely look different than what you see here.

Continuing with our example, if the AVI to MP4 conversion fails for any reason, the next matching rule instructs the encoder to try converting to FLV. In my experience, this conversion often succeeds when the MP4 conversion fails, so this is a good fallback setting. The “all video” and “all audio” rules are buckets that match on all files with a video or audio MIME type.

The screen shot also shows the encoder queue, which is a list of media files that are waiting to be processed, are being processed, or are completed. Encoding can take a while, so this will be useful to see how things are going and to investigate any encoding errors that occur. The text returned by FFmpeg is shown in the Result column, and it contains all the conversion details – and errors – related to the process. You can also delete completed items or cancel pending ones.