CS 294-3: Multimedia Systems and Applications

Multimedia Systems and Applications

"MPEG Coding" Assignment

This assignment will introduce you to some of the issues in MPEG coding and quality. The assignment will have three parts. You will have to install and run several programs that encode, decode, and analyze MPEG video streams. You will also have to make a small modification to an MPEG2 decoder and write a small program to compute signal-to-noise (SNR) quality measures and error images.

Programs

You will have to run a variety of programs, modify a program, and write a program from scratch to complete this assignment. Here are the programs you will need to run and where you can find them. Some of the programs are installed on the Unix Software Warehouse (sww).

mpeg_play - Berkeley mpeg1 video software-only decoder. Source code is available here.
mpeg_stat - mpeg1 video analysis program. Source is available here.
mpeg_encode - Berkeley mpeg1 video software-only encoder. Source code is available here.
mpeg2play - Stefan Eckart's mpeg2 video software-only decoder. This code is included in the MPEG Software Simulation Group Distribution which includes both an encoder and decoder for mpeg2 video. The code is available here. You should download the file mpeg2v12.zip if you run on a PC/Windows. The complete source code for the encoder and decoder are in the file mpeg2vidcodec_v12.tar.gz.

You can use mpeg_play to decode sequences and display them through an X Window or to output them to files in ppm format. This feature will be useful for computing SNR statistics. mpeg_stat is useful for determining frame sizes and other detailed information (e.g., block type and motion vectors).

You will have to modify mpeg2play to output information about block types. The routine is getMBs in the file getpic.c. Look for the constants MB_FORWARD, MB_BACKWARD, MB_INTRA which designate forward, backward, and intra macroblocks, respectively. To locate skip macroblocks, look for the variable MBAinc which starts at one and increments by one if the next macroblock in the stream is the next macroblock in the image. MBAinc can increment by any number of macroblocks, say three, which means two macroblocks were skipped.

Finally, you have to write a program to compute peak SNR (PSNR) given a source image and a reconstructed image. Both images will be in ppm format, so you can use the Portable Bitmap library (pbm) to write a simple program that will read an image or images and output the required statistics. The following web page describes the computation of PSNR.

Analyzing MPEG Coded Material

You will be given a collection of coded MPEG videos including both MPEG1 and MPEG2 sequences. There are two source sequences (i.e., Flower Garden and Football). The sequences are CIF pictures and contain only 28 frames. They have been coded by two encoders (i.e., Berkeley MPEG1 Video Encoder and the MPEG Software Simulation Group MPEG2 Video Encoder). We have included a couple of MPEG2 encodings, but most are MPEG1 bitstreams. We used a parameter to the MPEG2 encoder that allows us to generate MPEG1 bitstreams. More details on the sequences are available elsewhere. The bitstreams are available at the following location.

You are to analyze these bitstreams and do the following.

Compare two MPEG1 encoders using the same sequences. Pick either the Football or the Flower Garden bitstreams.

Plot the bits per frame (b/f) for all bitstreams coded by the two encoders. In other words, pick one sequence and do two plots, one for each encoder, with information on the five bitstreams.
Plot the PSNR per frame (psnr/f) for the bitstreams for each encoder.
Compute the correlation between PSNR and bits per pixel (b/p) for the two encoders. This computation should only include the bit rate limited sequences. In other words, you should compute two correlations, one for each encoder.
Construct a data set for footb-3 coded by each encoder that has all motion vectors (i.e., the horizontal and vertical offset) and the frame type. Produce a 3D histogram plot (i.e., horizontal offset, vertical offset, and number) for all frames, for all P-frames, and all B-frames for the two bitstreams.

Compare MPEG1 and MPEG2.

Plot b/f for the footb-6 bitstreams. The plot should include three lines, one each for ucb, mssg1, and mssg2.
What is the distribution of block types across all frames in the footb-6-ucb and footb-6-mssg2 bitstreams? What is the distribution of block types for B-frames in those sequences? You can show this distribution as a histogram.
What can you infer about the difference between MPEG1 and MPEG2 bitstreams?

Compare MPEG1 and MPEG2 motion vectors. Construct a data set for footb-6 coded by each encoder that has all motion vectors (i.e., horizontal and vertical offsets) and the frame type. Produce a 3D histogram plot (i.e., horizontal offset, vertical offset, and number of occurrences) for all frames, for all P-frames, and all B-frames for the three bitstreams.

Understanding the MPEG Encoding Process

To be filled in.

Links to Useful Information

There are numerous web sites with useful information on MPEG. Here are a couple that you might want to look at for information and help.

Berkeley MPEG Web Site

Tristan Savatier's MPEG Web Site