Encoding to Xvid

Discuss about generic usage of MediaCoder.

Moderator: HuggiL

Posts: 3116
Joined: Tue Jun 23, 2009 2:45 pm
Location: Sydney, Australia

Encoding to Xvid

Post by meRobs » Tue Sep 13, 2011 3:29 pm

[Updated 16 Nov 2012]

The Xvid video Format/Codec is based on the MPEG-4 standard and, thus, is useful in creating files suitable for MPEG-4 Players such as DivX and XviD compatible devices. In particular, Xvid is based on the MPEG-4 standard Part 2, and uses MPEG-4 Advanced Simple Profile (ASP) compression (H.263) and any video that is encoded with it is termed "MPEG-4 ASP video".
On the other hand, H.264 encoding is also based on the MPEG-4 standard, but via the more-recent Part 10 – see the Guide on H.264 at: viewtopic.php?f=17&t=11026.

What follows are some notes that have been gleaned from the web, in July and August 2011, and some experiments I have done with MediaCoder build 5185. I have also done a few conversions in builds 4399, 5002 and 5130 to confirm the results from 5185.

1. Xvid Parameters
2. Settings for Xvid/AVI
3. FourCC requirement
4. Problems in MediaCoder
5. Tests for Xvid/AVI

1. Xvid Parameters
Xvid is an old Codec, and refers to the MPEG-4 Part 2 standard for video. It is the open-source alternative to DivX; and notice the spelling – XviD is DivX backwards.

To the right of the Video tab in MediaCoder is the Xvid tab. It offers choices for Profile (Unrestricted, by default), Motion Estimation (Medium Search by default), Motion Est. (6, by default) and B-Frames (0, by default). Notice that “B-Frames” is the maximum number of consecutive B-Frames to be allowed in any GOP in the output file.

It also has 5 check boxes, with High Quality Mode being the only one checked by default. The only changes I chose to consider were to the Profile and the value for “B-Frames”.

The "Simple" profile is best compared to VCDs, since it limits the resolution to 352x288, the video bit-rate to up to only 384 kbps and the frame rate to only 15 fps. On the other hand, an "Advanced Simple" profile allows more advanced features such as B-frames, Quarter Pixel and Global Motion Compensation. For example, Advanced Simple Profile Level 3 allows up to 768 kbps average bitrate and up to 30 fps, although, still no more than 352x288 pixels. Even better is the “Advanced Simple Profile Level 5”, which allows up to 720x576 pixels, up to 8000 kbps for video at up to 30 fps. There are also some special Profiles, from "Hand held" to "HDTV”.

There is a “B-Frames” setting (max number of consecutive B-frames), allowing a value from 0 (default) to 4 to be set. The difference between B-frames, P-frames and I-frames is explained in the Guide on setting the GOP length (viewtopic.php?f=17&t=8599&start=0).

B-frames improve the quality, but they need some cunning treatment when used in an old technology, such as the containers AVI and VFW (Video for Windows). These can handle only one-frame-in gives one-frame-out, yet B-frames must be generated from two frames, one I- and a P-frame, say. Thus, this adds a complication for either the Decoder, which must handle two frames at a time, or the encoder, which needs to do some nifty shuffling itself. The first requires that “Frame packed” be turned off, the default in MediaCoder, and the second requires it to be turned on. This may be why most profiles support no more than 1 B-frame. Also, having more than 1 B-frame in a row may cause problems with some standalone players, as may Global Motion Compensation and MPEG quantisation.

The effect of setting B-Frames may be seen in Section 4, where it is demonstrated that Xvid supports only up to 2 B-frames.

2. Settings for Xvid/AVI
In MediaCoder, Xvid is achieved by setting Format to Xvid on the Video tab. The associated encoder was best set as “Xvid” until build 5140 when it was replaced with Xvidcore. Also needing to be set is the “Source”, which refers to the decoder that interprets the source (input) file. I use MEncoder, which also occurs (usually) when Auto Select is checked.

On the Video tab, you may choose to convert for a set value of Video Bitrate or for a set value of Output Size. The bitrate is set at 1000 kbps in the Encoding Guide, as a suggested starting point, and was changed for the Tests conducted for Section 4, for example to 700 kbps. The output size option requires that Auto Bitrate be checked so that the Output Size checkbox be made available. Once selected, the Video Bitrate panel changes to suit size in MB. For both these options there is the choice of Average Bit Rate (ABR), which gives a single pass conversion, as well as 2-pass and 3-pass encoding.

Xvid may be used in different containers, such as MP4 and AVI. For example, you may mix and match the Video & Audio settings for Xvid/AVI and H.264/MP4 as given in the Encoding Guide ((viewtopic.php?f=17&t=9643). However, for simplicity, I shall confine my comments to Xvid in the AVI container. One advantage is that the conversion may be played in an MPEG-4 compatible player, such as a DivX player or a standalone DVD player that supports DivX and XviD.

However, for success in such a player, the Xvid/AVI file may also need the FourCC code to be set or overwritten – see Section 3, below.

So, based on the above comments, my choice for the various tabs in MediaCoder for this test, but see Section 5.5, are as follows.
* Video Tab: Format = Xvid, Encoder = Xvidcore (but see Section 5.5), Source = MEncoder, Video Bitrate = 1000 and Rate Mode = Average Bitrate (ABR).
* Xvid tab (at right): Advanced Simple Profile Level 5, Motion Estimation = Medium search, High Quality Mode checked,
Motion Est. = 6, B-Frames = 1 and all else at their defaults
* Audio tab: Format = MP3, Encoder = LAME MP3, Source = Auto Select and Resample = 44.1 kHz
* LAME MP3 tab (at right): Preset = Custom, Rate Mode = CBR, Channel = Stereo and Bitrate = 96 kbps
* Container tab: Container = AVI and Multiplexer = Auto Select or FFmpeg
* Picture tab: Resize = Original, Frame Rate = Original and all else at defaults (you should choose what suits)

3. FourCC requirement
So that Xvid or MPEG4 in AVI will play on a DivX compatible player it needs to have a suitable FourCC code in its header. FourCC (Four Character Code) is a sequence of four bytes used to uniquely identify data formats. Common identifiers include DIVX, XVID, DX50, FMP4 and H264.
In many builds, use of Xvid as video Format will result in “XVID” being written to the file as the FourCC. The code is displayed in MediaInfo as “Codec ID” and is on the Properties panel of most recent builds of MediaCoder.

If getting a suitable Code fails (see Section 4, below) or, as a safety precaution, manually set the code and save the setting in a Preset for all similar conversions. To set the FourCC in MediaCoder click on the Settings button then: Overall > Video > “Override FourCC of output video stream” and enter XVID or DX50. Alternatively, if the damage had been done and you wish to change it, use the App: 'AVI FourCC Changer', a small standalone.

4. Problems in MediaCoder
The above settings should give what is required. However, there are three problems that occur with the Xvid format in some builds of MediaCoder. They are:
(1) not getting the appropriate FourCC code written to the file header (Section 3).
(2) the output frame rate may be twice that set on the Picture tab!
(3) Xvidcore, seemingly the obvious choice of encoder for Xvid, may fail

The first problem (FourCC) is illustrated in the Test Report on the various builds, at: viewtopic.php?f=17&t=10383. Here it may be seen that builds 5197 to 5236 do the right thing, but the more recent builds, 5240 to 5262, fail. It seems that the Muxer (FFmpeg) writes the Codec string, so, changing the version of FFmpeg may fix the problem (see next paragraph). It was fixed in build 5197.

The second problem (frame rate doubling) was discussed in the Bug Report: viewtopic.php?f=28&t=11073. However, as pointed out by Stanley in this Topic, the doubling was due to the use of FFmpeg as Muxer and there is no actual duplication of the frames. Thus, it seems to be an error in the header as created by the Muxer and read by MediaInfo and, thus, the Properties panel in MediaCoder. Hence, just ignore it! Alternatively, if you want things to be tidy, copy over the version of FFmpeg from build 5230, which doesn't have the problem. In any case, the problem seems to be overcome since build 5275.

The third problem (Xvidcore) is described at the end of Section 5, in sub-section 5.5.

5. Tests with Xvid/AVI
In the following tests, I used the settings given in Section 2, above, except where indicated. I had the choice of three source files:
File A. a 100-sec VOB in NTSC (720x480 pixels), will all the frames full (no black bars)
File B. the first 1.0 GB VOB of a PAL DVD: duration 1541 sec (video at 4696 kbps)
File C. the first 1.0 GB VOB of a second PAL DVD: duration 1454 sec (video at 4923 kbps)

5.1 Effect of B-Frames
To begin with, I tested to what extent Xvid makes use of the “B-Frames” value.
I converted file A, above, with Video Bitrate = 700 kbps and “B-Frames” set at 0, 1, 2 or 3 in the x86 build 5185; and examined the frame types in Gspot.
As the number of B-frames was increased, the time it took to convert the 100-sec file decreased from 90 sec (B = 0) to 75 sec (B=2 or 3) and the file size decreased slightly as the average video bitrate fell a little (more B-frames).

In each case, there were the same number of I-frames (see Figure 1) and they were located at the same sites regardless of the number of B-frames. When, zero B-Frames were set, all other frames (99%) were P-frames and when B-frames = 1, there were 49% B-frames and 50% P-frames with all B-frames being alone (not beside another B-frame). On the other hand, with B-Frames set at 2, there were 58% B-frames and 41% P-frames and the B-frames were in groups of one or two (see Figure 1). The result for B-Frames = 3 was identical to that for B-Frames = 2, which demonstrates that Xvid does not support more than 2 consecutive B-frames.

5.2 Video Bitrate
In build 5185, I examined the result of setting different values of the Video Bitrate, with Auto Bitrate unchecked (the default) and the Rate Mode set at Average Bitrate (ABR). In each case, I compared the video bitrate for the output, as displayed in the Properties panel, with that calculated from the resultant file size, using equation (2) in the Guide at: viewtopic.php?f=17&t=12438.

I converted file A, above, to Xvid/AVI as per Section 2 with the Video Bitrate set at 400, 700, 1000 and 2000 kbps. The resultant file sizes were 6.22, 9.73, 13.24 and 24.71 MB and, hence, the calculated video bitrates are: 414, 701, 989 and 1928 kbps.The respective bitrates in the file header, as displayed in the Properties panel, were 407, 702, 997 and 1962 kbps, which follow the trend but are slightly closer to the set values. Note that both the displayed and the calc. bitrates tend to be greater than the set value when less than 700 kbps and smaller at large bitrates.

5.3 Output Size
I examined the ability to achieve a chosen Output Size in build 5185. For this to happen, you need to check (tick) both Auto Bitrate and Output Size on the Video tab. For this test the Rate Mode was set at Average Bitrate (ABR).

Using File A, above, I encoded with Output Size = 10 MB and the result was a file of 9.48 MB (stated video rate of 681 kbps). I then tried File C set for 150 MB, which gave 135.3 MB. I could not repeat this result or test other large files because of the Errors described in Section 5.5.

On the strength of these results, it looks as if the error in achieving a desired output file size may well be considerable if the required size is large?! The alternative is to set a Video Bitrate via the formula given in Section 4 of the Guide: viewtopic.php?f=17&t=8740&start=0 .
This indicates, for example, that a 2-h movie will fit on a CD of 700 MB if the video bitrate is 700 kbps and the audio rate is less than 91.7 kbps, say 80 kbps.

5.4 Multi-pass Encoding
Multi-pass encoding (2-pass and 3-pass encoding) is also available. It is as an alternative to Average Bitrate for “Rate Mode” on the Video tab. However, this is the subject of a separate Guide.

5.5 Failures with the Xvidcore encoder
The use of Xvidcore as the encoder for Xvid seems a logical choice, which is why it was chosen above and used in the Encoding Guide (viewtopic.php?f=17&t=9643). However, for a VOB source file of 425 MB (18:15 min, video at 3000 kbps) and the larger File B and File C, above, attempts with Xvid/Xvidcore/AVI failed. Usually, there was an error message that stated: “The decoding thread seems to be blocked …”, which timed out to a similar one for the encoding thread and finally: “There is one or more errors in transcoding process.”. The early ones are easily missed. This was first described in a bug report on build 5185 at: viewtopic.php?f=28&t=11006.

I tried changing numerous settings, such as MEncoder, FFmpeg, AviSynth and Auto as video Source, without success; and even tried the earlier builds 4399, 5002 and 5130, which used ‘Xvid’ as the encoder instead of Xvidcore (just a name change?). They also failed in the same way!

In Oct 2012, the problem was unravelled. All builds of MediaCoder from 5140, when Xvidcore was introduced, to 5292, fail for long source files in 32-bit versions (see: viewtopic.php?f=28&t=12250). On the other hand, the 64-bit versions of builds 5130 (Xvid encoder) to 5295, at least, have no problem (viewtopic.php?f=28&t=12250).

For users with 32-bit versions, I suggest you change the video Encoder from Xvidcore to FFmpeg, which works well, even though the various settings on the Xvid tab, at right, such as B-frames, are not available with this encoder. Fortunately, the video quality achieved with FFmpeg was considered excellent in a study that compared various video Codecs (see: viewtopic.php?f=17&t=11978).

6. Calculation of Bitrate
The calculation of bitrates and their relationship with the file size is dealt with in the Guide at: viewtopic.php?f=17&t=12438.
Have you checked out the Tips & Guides for MediaCoder? Try: http://forum.mediacoderhq.com/viewtopic.php?f=17&t=8061
Also, get older builds at: http://www.videohelp.com/tools/MediaCoder/old-versions#download

Posts: 3116
Joined: Tue Jun 23, 2009 2:45 pm
Location: Sydney, Australia

Re: Encoding to Xvid

Post by meRobs » Tue Sep 13, 2011 3:32 pm

The above post has been added to Tips & Guides.

If any users could make any suggestions or add useful comments, please start a new Topic that refers to this one.
Have you checked out the Tips & Guides for MediaCoder? Try: http://forum.mediacoderhq.com/viewtopic.php?f=17&t=8061
Also, get older builds at: http://www.videohelp.com/tools/MediaCoder/old-versions#download