Feedbacks needed for Segmental Video Encoding

Discussing about advanced video encoding techniques including GPU encoding, distributive encoding and segmental encoding. Topics are not limited to MediaCoder.
stanley
Site Admin
Site Admin
Posts: 4135
Joined: Mon May 15, 2006 7:43 pm
Location: Sydney

Feedbacks needed for Segmental Video Encoding

Post by stanley » Thu May 23, 2013 12:13 am

I am so excited to release the new version of 0.8.22 with a new exciting technology named Segmental Video Encoding (SVE) was introduced in after a long time of thinking and implementation. The idea of SVE is straight-forward. On systems with a lot of processor cores or processors, a single instance of video encoding cannot utilize all the computation power of the systems, though modern encoders like x264 already has good support for multi-threading. This also happens when doing GPU-based encoding, whereas the GPU is quite often under-loaded. SVE is basically the mechansim of encoding of a video in temporal segmentations, by loading 1~2 minutes video frames into memory and dispatch the frames to multiple encoder instances. A series of encoded video segments will be generated and they are concatenated seamlessly in the muxing stage together with the audio. This improves paralellization regardless of the encoder’s support for multi-threading, so those encoders without good multi-threading implementation (Xvidcore, MPEG 1/2 encoder etc.) will benefit. Please be noted that SVE is still experimental. If you experience problem with it, please give me feedback. I really need some more testings especially on cutting-edge computer systems which I don’t have a chance to test on.

Image
When things work together, things work.

sharkbite
Amateur
Amateur
Posts: 37
Joined: Fri Feb 25, 2011 1:39 pm

Re: Feedbacks needed for Segmental Video Encoding

Post by sharkbite » Fri May 24, 2013 8:51 am

Hi Stanley -

Had a quick play last night...

Converting Mpeg2 .ts to h.264 using the Apple presets for ipad classic , and CUDA...

HW= 6 core phenom + GTX650
Source file is on one HDD and output is on another...

SVE appears to be functioning, as the indicators show 2 of the 4 buffers being used.
(however only one buffer fills at a time.....)

The conversions starts off being really fast - about 12x....
after about 30 seconds conversion speed drops off to about 7x - which
is the same speed i get with this configuration and SVE turned off.
Both CPU and GPU are way understressed whatever i use :-(

Can you publish what settings you tested with?
it would be handy to know if it is just my configuration....

Keep up the good work - its a great idea!

cheers,

stanley
Site Admin
Site Admin
Posts: 4135
Joined: Mon May 15, 2006 7:43 pm
Location: Sydney

Re: Feedbacks needed for Segmental Video Encoding

Post by stanley » Fri May 24, 2013 5:25 pm

Thanks for your feedback!
If one of the two buffer slots is always empty, that indicats the two instances of CUDA encoders drain the buffer faster than the decoder can fill up the buffer.
How much RAM do you have? If you don't have a big enough RAM, the allocated buffer will be too small, especially when the content is HD, so that they are drained more often and cause only one encoder instance working at one time.
The number underneath the buffer fullness indicators is the frames in the buffer. What's the biggest number you see?
When things work together, things work.

encoderdude
Beginner
Beginner
Posts: 6
Joined: Sat May 25, 2013 7:37 am

Re: Feedbacks needed for Segmental Video Encoding

Post by encoderdude » Sat May 25, 2013 7:56 am

i tried to do a segmental encode on a 720p avi file to high10 mp4 using x264 2-pass (i used a 1 minute cut clip, using the time option, from a full length movie of 90 mins.). Though the conversion was significantly fast & got a 1 min. output, it got only first 20-30 secs of the clip converted & the rest part a repetition of the first 20-30 secs. Tried changing the cocurrent setting from 4 to 1 trying all inbetweens (i got a quad core intel machine), but the results are more or less the same. The audio seems to Convert & play fine. It's the video part tht got the errors. & also when i converted the video using 'disabled' segmental encode setting, i got the file size half of tht compared to when i enabled the settings & it encoded perfectly. Have a look in to this issue.

meRobs
Expert
Expert
Posts: 3116
Joined: Tue Jun 23, 2009 2:45 pm
Location: Sydney, Australia

Re: Feedbacks needed for Segmental Video Encoding

Post by meRobs » Sat May 25, 2013 9:24 am

There was another problem reported by e1wh for SVE (build 5500) at: viewtopic.php?f=28&t=12653&p=37885#p37885.
It related to 3-pass encoding with H.264 (Error 14)
Have you checked out the Tips & Guides for MediaCoder? Try: http://forum.mediacoderhq.com/viewtopic.php?f=17&t=8061
Also, get older builds at: http://www.videohelp.com/tools/MediaCoder/old-versions#download

stanley
Site Admin
Site Admin
Posts: 4135
Joined: Mon May 15, 2006 7:43 pm
Location: Sydney

Re: Feedbacks needed for Segmental Video Encoding

Post by stanley » Sat May 25, 2013 6:08 pm

Multi-pass encoding is incompatbiel with SVE in 5500. This is fixed in 5506. Multi-pass CRF also works with SVE.
When things work together, things work.

dahagi
Beginner
Beginner
Posts: 2
Joined: Tue Jan 15, 2013 9:06 am
Location: Switzerland

Re: Feedbacks needed for Segmental Video Encoding

Post by dahagi » Sun May 26, 2013 10:01 am

It doesn't work for me in build 5505 (x64) when transcoding a H.264/DTS (MKV container) file into MPEG2/AC-3 (MPEG2 container) using ffmpeg as encoder/decoder for both audio and video. HW is Intel Core 2600 (4 Core with integrated Intel GPU). Tried other decoders like AviSynth but no change. Also concurrency variations make no diff.

Error occurs usually after about 20-40s into transcoding. In the UI I can see that it always appears quickly after cache fullness hits 100%.

Is SVE supposed to work in build 5505 for MPEG-2 target format?

sharkbite
Amateur
Amateur
Posts: 37
Joined: Fri Feb 25, 2011 1:39 pm

Re: Feedbacks needed for Segmental Video Encoding

Post by sharkbite » Sun May 26, 2013 11:42 am

Hi Stanley I am running 4g memory.....according to task manager media coder is using about 600meg all up and I have about 2.5g free when It is running......so I don't think memory is the problem.

The buffers fill and empty very quickly.....the numbers are around 16.....with an occasional spike to 427.....

If I let the encode continue all the way through it gradually gets slower and slower.

Will try 5505 and if it changes.

Cheers

BOG22
Beginner
Beginner
Posts: 1
Joined: Sun May 26, 2013 2:55 am

Re: Feedbacks needed for Segmental Video Encoding

Post by BOG22 » Sun May 26, 2013 1:18 pm

Hi Stanley.
The SVE with 2 pass on build 5505 has increased my processor usage from 35-55% to now 85-100% on Intel i7 980. The encoding speed has definitely increased when compared to not using SVE. Cache seems to be working accordingly as all 4 are being used. Thank you for a great program.

stanley
Site Admin
Site Admin
Posts: 4135
Joined: Mon May 15, 2006 7:43 pm
Location: Sydney

Re: Feedbacks needed for Segmental Video Encoding

Post by stanley » Sun May 26, 2013 8:58 pm

I am so excited to hear that!
BOG22 wrote:The SVE with 2 pass on build 5505 has increased my processor usage from 35-55% to now 85-100% on Intel i7 980. The encoding speed has definitely increased when compared to not using SVE. Cache seems to be working accordingly as all 4 are being used. Thank you for a great program.
When things work together, things work.

Post Reply