[ NOSSDAV 2005 Home Page | Conference Program ]
Anyone who tried to play the NOSSDAV 2005 material in the past couple of weeks discovered that audio played correctly, but not video. This note describes the problem introduced when Apple Released Quicktime v7.1.3, and how I fixed the problem. I am grateful to the many people who helped diagnose and fix the problem including Andrea Barbieri of Moving Image Research, Alex Zavatone of 3HZ Group which produces 3ivx Crush, and friends who work at Apple who wish to remain anonymous.
I encoded the files using a Beta Release of Quicktime Pro Version 7 (v7.0.2) on a Windows 2K PC during August 2005. The media files and accompanying web pages were published on the Internet at the beginning of September. The material played fine until a problem occurred with the release of v7.0.3 which changed the way streaming transport was specified. We fixed that problem with the HTML web pages in December 2005, and the material played fine until the release of Quicktime v7.1.3 in September 2006. More details on this problem are given in the paper we wrote about the experiment.
According to the Apple Press Releases at the time, Quicktime v7.1.3 fixed several bugs and closed seven critical security holes. Using the new Quicktime Player the NOSSDAV 2005 files would only play audio when played from a local file or streamed from the Darwin Streaming Server (DSS) that hosts the material at U.C. Berkeley.
So, why did Quicktime stop playing the video? The short answer is that the media track header did not have the correct specification of the profile and level for the H.264 video stream. The Beta Release H.264 encoder in Quicktime Pro V7 put zero bytes (0x00) into the two fields in the header. Players before v7.1.3 ignored the profile/level specification in the header. Release v7.1.3 enforced the restriction that you specify a valid profile/level or the player would ignore the track.
When I figured out that this was the problem, it was "deja vu all over again!" My research group developed the Berkeley MPEG-1 video decoder that was widely distributed on the Internet back in the 1990's. At one point we tightened the player to only allow playback of valid streams, which unfortunately broke a number of files that had been coded earlier by non-conforming encoders. Needless to say, we too received a number of bug reports and complaints.
My fear when diagnosing the problem was that we might have to re-encode all the files. It took approximately 170 hours (i.e., over 7 days of continuous computing) and many manual operations to transcode the captured material to H.264. I really didn't want to do that again.
The good news for me is that Apple has a tool, called Dumpster, that will display the structured data in a file (e.g., media and track headers) and allow you to change selected fields. I used that tool to update the profile/level fields in the over 80 files. The files would then play correctly from a local file.
Sadly, they did not play when we uploaded them to the DSS server. Further
emails and analysis showed that they would play correctly if we re-hinted
the files. For those unfamiliar with streaming media transport, hint
tracks provide information to the server about how the media data should
be packaged into an IP packet for delivery over the Internet. More details
on the Apple hint track structure is available at the following
documentation web page:
Quicktime Player Pro has features to extract tracks from a file and re-hint them. So, I just re-hinted the files. Re-hinting does not take as long as encoding so I was able to do it in a couple of hours, after which the files were re-loaded onto the DSS server. As far as I know, everything works well now. However, while solving this problem I noticed that the higher bit rate material (i.e., the 1200 Kbs files) did not look as good as the lower bit rate material (i.e., the 600 Kbs files). This problem, which is caused by the image scaling applied to the source material, is discussed in more detail elsewhere.
What then do I conclude from this experiment and experience? The original purpose of the NOSSDAV 2005 experiment was to test the feasibility of webcasting technology to capture conference presentations and discussion for on-demand replay through the ACM Digital Library. We used innovative technology to capture direct to disk that we believed could be done inexpensively and that could be easily uploaded to a digital library. An implicit test was also being made of the infrastructure for supporting the webcast library, which in this case used the Darwin Streaming Server and Quicktime playback.
Based on my experience in the past 15 months, the Quicktime webcast infrastructure is not stable enough yet to use for a large digital media library. Two out of the eight Quicktime software updates (25%) have broken the NOSSDAV 2005 website. And in both cases, we discovered the problem only after the update was released and the website stopped working.
Contrast this with the use of Adobe PDF for publication of proceedings through digital libraries. I am unaware of any problems that ACM or other major digital libraries have had where a new release of Acrobat broke all the existing files. What this tells me is that Quicktime streaming media technology is still too immature to be used for a large digital library.
Caveats:
Some of these problems were caused by my use of the beta release V7 H.264 codec. The most recent problem (profile/level specification) was undoubtably an oversight in the original encoder that has since been fixed. The fact that the player worked correctly for a while but then stopped working is not their problem, rather it was my mistake. However, it would have been nice if Apple had notified content providers about the change. Maybe they did in the detailed release notes, but I didn't see it.
The transport change, however, was in my opinion Apple's responsibility. I followed the instructions and examples provided in the Quicktime plug-in documentation, but sadly they changed the specification and provided no backward compatibility. In fact, the new transport specification seems to work better than the old approach although I still think it can be improved. The old approach required the user to specify TCP streaming explicitly rather than UDP streaming. For those unfamiliar with the problem, UDP packets are not forwarded correctly through gateways and NAT boxes. On the other hand, UDP packets are delivered without delay while TCP packets may be delayed due to congestion control. If you are streaming live material, you cannot deal with variable delay. On-demand material can buffer several packets to smooth out the delay.
The current approach allows the user to specify "automatic" selection of transport. However it works, it seems to discover that UDP doesn't work and switches to TCP. Frankly, anytime one is playing back material on-demand, TCP is probably preferred. The Quicktime plug-in could provide an optional argument that allowed the content author to specify TCP transport. Automatic selection is best if it works and the overhead of discovering which transport to use is low. Right now, the NOSSDAV 2005 material takes roughly five seconds to start. While that is better than some streaming server technology (e.g., the UC Berkeley course webcast system uses Real Networks streaming and it takes roughly 20 seconds to launch a playback), it is far from the standard channel change time in TV systems which must be under 0.5 seconds.
Finally, some readers might be thinking that I brought the problem on myself by updating to the newest software release rather than testing it beforehand. Unfortunately, that does not work in some settings because Quicktime automatically updates the software without any intervention by the user. In this case, users that were not updated could still play the material without problems.
I have worked with several streaming media technologies including the commercial packages produced by Apple, Microsoft, and Real. I am still impressed with the Apple Quicktime technology, and I would recommend that others use it for several reasons. First, Quicktime has adopted open standards (e.g., RTP, RTSP, and ITU codecs) and encourages developers to produce cross platform solutions. Compare this with Microsoft technology that really only works on Windows and is closed, meaning that a third party developer cannot build a player that decodes a Windows Media Stream using Microsoft VC-1 codecs.
Second, the Darwin Streaming Server is provided as open source so it is inexpensive to build a media server to produce a large scale media distribution service. Compare this with the Real Networks technology that requires you to purchase the server technology if you want to build a large scale media distribution system. As a practical matter, few websites such as the NOSSDAV 2005 material are likely to be widely viewed.
Finally, the one thing that would make this technology better is support for the Unix platform. I know that the material plays on Macs and Windows, and I suspect it can be downloaded and played with one of the open source players. But, it would be better if someone could implement the Quicktime plug-in on Unix. The big advantage that Flash and the video host sites like YouTube have is easy playback on any platform.
For further information contact Larry Rowe at
rowe(at)bmrc(dot)berkeley(dot)edu.