Mencoder fails (Win 7 x64 GeForce 9800GT)

I am using Windows 7 Ultimate x64 on AMD Phenom II with GeForce 9800GT, I received the same message about incompatibility but I neglected it, the encoding seemed to be done by the GPU as the FPS ranged from 40 to 80. However, during the conversion Windows reported that MEncoder had no response and was quitted, I don't know if this is the problem of the program or the source video.

[mixer]

Sounds like your issues might be different than the one you originally posted to.

To narrow down the error, first try another source or video/audio file and see if the conversion fails again. It is possible that your original file can't be processed by Mencoder. I'm not familiar with the NT version of MediaCoder, but if you can select another encoder (different than Mencoder....like ffmpeg) then try that. Post back with your findings and hopefully someone else will respond.

Today I tried a low-res RMVB video as source video, the conversion ran successfully at average FPS of 242 and even 300FPS at some moments(really? i don't know ), another MPEG-4 video was converted at average FPS of 320FPS.