j.ello | options

j.ello | weather

j.ello | subscribe

If you're not already
receiving j.ello | report
just enter your e-mail
address and get it
delivered FREE!

February 23, 2000

j.ello | by the byte

     We're wrapping up MPEG today and moving on... I have a couple extra things to give away so I think we'll have a couple short contests starting in the next issue. :)


MPEG Video, part 4

7.  Run-Length Encoding (Temporal Redundancy)

THE SHORT VERSION
     Up until now, the steps we take to make an MPEG are pretty much the same as we use to make a JPEG... and the only serious space savings come when we "Quantize" out all the color and brightness information we don't need.  It turns out that we can save more space by getting rid of redundant information which would normally be in every single frame of video.  Run-Length Encoding, or RLE is a process that results in only recording the things that change from frame to frame, rather than recording whole frames, saving gobs of space.

THE LONG VERION
     If you recall from step 1, the MPEG encoder actively selects frames to be used as I-frames and encodes them in a JPEG-ish format.  We could use every frame as an I-frame... and that would save some space... but not enough to make the video small enough to distribute on DVD or via satellite.

     This is where the heart of MPEG lies, and it's where the I-frames become important.  The qualification of being an I-frame is that there needs to be a big change from the frame before it... like the moment something enters the scene, or a flash, an explosion... etc.  If nothing too drastic changes, we get an I-frame every 12th frame of the video.  So what happens to the 11 frames in between?

Enter the P-frames
     Well... the MPEG encoder looks at each frame and compares it to the one before.  When there are only small changes, the encoder might decide that it can reproduce the look of that frame without actually recording the whole frame itself.  It can apply a "mask" over the frame before it... changing only the Macroblocks that are very different from the frame before.  An example would be a monologue in a movie.  The character speaking to the camera often doesn't move, so the Macroblocks covering the background don't change at all from frame to frame... only the Macroblocks covering the face and mouth really change.  So... in between I-frames, we can "approximate" the look of the rest of the frames by using a P-frame... or "predicted" frame.  A P-frame is overlayed onto the original I-frame.  The Macroblocks that don't change in the P-frame are transparent, and the ones that do are opaque, so what you wind up seeing is an image which is partly made up of P-frame Macroblocks, and partly made up of I-frame Macroblocks on the layer below.  Because we don't include all the Macroblocks of a P-frame, P-frames are much smaller in size than I-frames... giving us big space saving.

     By default, a P-frame is put in every 4th frame before and after I-frames... so in a video it looks like this:

Frame #   -    1   2   3   4   5   6   7   8   9   10  11  12  13
Frame type -  I             P             P              P               I

Enter the B-frames
     There is, however, one more step.  P-frames are predicted from the I-frame before it... the I-frame after it has no effect, so if an error occurs in the first I-frame or in any of the P-frames, that error is propagated down the line until a new I-frame comes along.  This could lead to noticeable errors over the course of 1/3 of a second and if the change is great enough, there could be an eye-jarring "pop" as the last P-frame is replaced by the next I-frame.  What we want is a way to smooth the transitions between I-Frames and the P-frames in between.  So we have B-frames... or "bi-directional" frames filling the space between all the others.  B-Frames are predicted like P-frames, however they are predicted from BOTH the I-frame and the P-frame closest to it.  Most of the time, very few Macroblocks change between I and P frames, so the B-frames are usually very, very small.

Frame #   -    1   2   3   4   5   6   7   8   9   10  11  12  13
Frame type -  I   B   B   P   B   B   P   B    B   P    B   B    I



For illustration, below is what regular video looks like, or even MPEG I-frame format or M-JPEG format.... each frame is recorded, no matter how much redundant information there is:




But in MPEG-1 and MPEG-2 (Including DVD), only the parts that change in each successive frame are shown... so in frame #2 you see some new macroblocks, but are still seeing some macroblocks from frame #1.  In frame #3, you still see some blocks from frames #2 and #1... and so on until a new I-frame is reached:



Stick with me here, cuz this is where things get weird....
     The MPEG technique dumps tons of information we never needed in the first place, saving lots of space... but remember, the B- and P- frames are not recorded as full images like the I-frames, they are recorded mostly as mathematical equations which are applied to the I-frame when the video is decoded by your computer, or by a DVD player.  The player literally "creates" P-frame images and B-frames WHILE the movie is playing!  B-frames are dependant on the P-frames, even though they are displayed before the P-frame.  
  
"So the DVD Player reads frame #1, then #4, then #2 and #3?  That can't be!"  
     Well, cope and adjust.  P-frames are built based on the information in the I-frame #1... but B-frames are built as "between" frames, meaning they need to know what #1 looks like and what #4 is GOING to look like before they can be built.  So, frame #4 gets built and put into memory while the player creates and displays frames #2 and #3.  The player then shows frame #4 and the process keeps going.  This means that your DVD player isn't just a dumb device... because it is literally CREATING most of the video you see WHILE you watch.  Freaky, eh?
    This is also why you can't efficiently edit an MPEG file after it is created.  Every frame you change in an MPEG may be dependant on several other frames.  This is why MPEG I-frame format, or Motion JPEG format is used for editing.  You can edit any frame, because each frame is a full and separate picture, independent of the frames around it.  Converting to MPEG-1 or -2 format is almost always the last step in the video editing process.

     Now you know how a full-length, high quality movie can squeeze into a couple gigabytes.  Cool, no?

     -j.ello

j.ello | places & info

j.ello | software picks

j.ello | toys

To unsubscribe:  Go to http://www.jello.net/report, enter your e-mail and click UNSUBSCRIBE.
To change address or re-subscribe:  Go to http://www.jello.net/report enter your e-mail address and click SUBSCRIBE