Feedbacks needed for Segmental Video Encoding

Discussing about advanced video encoding techniques including GPU encoding, distributive encoding and segmental encoding. Topics are not limited to MediaCoder.
stanley
Site Admin
Site Admin
Posts: 4130
Joined: Mon May 15, 2006 7:43 pm
Location: Sydney

Re: Feedbacks needed for Segmental Video Encoding

Post by stanley » Sun May 26, 2013 9:02 pm

Yes. Please give 5506 a try as an audio related bug when SVE is enabled is fixed.
dahagi wrote:Is SVE supposed to work in build 5505 for MPEG-2 target format?
When things work together, things work.

stanley
Site Admin
Site Admin
Posts: 4130
Joined: Mon May 15, 2006 7:43 pm
Location: Sydney

Re: Feedbacks needed for Segmental Video Encoding

Post by stanley » Sun May 26, 2013 9:05 pm

The maximum cache size is 900~1000 frames. MediaCoder uses no more than 1/4 of free memory as cache.
It is also likely that the decoding is the bottle-neck of your transcoding. You can see decoder CPU utilization percentage in the information sidebar after transcoding. You can also try increasing number of cache slots from default 2 to 3.
sharkbite wrote:Hi Stanley I am running 4g memory.....according to task manager media coder is using about 600meg all up and I have about 2.5g free when It is running......so I don't think memory is the problem.

The buffers fill and empty very quickly.....the numbers are around 16.....with an occasional spike to 427.....
When things work together, things work.

stanley
Site Admin
Site Admin
Posts: 4130
Joined: Mon May 15, 2006 7:43 pm
Location: Sydney

Re: Feedbacks needed for Segmental Video Encoding

Post by stanley » Sun May 26, 2013 10:10 pm

SVE works with MPEG-2 target. Please give latest build 5506 a try.
dahagi wrote:Is SVE supposed to work in build 5505 for MPEG-2 target format?
When things work together, things work.

sharkbite
Amateur
Amateur
Posts: 37
Joined: Fri Feb 25, 2011 1:39 pm

Re: Feedbacks needed for Segmental Video Encoding

Post by sharkbite » Mon May 27, 2013 8:40 am

Thanks Stanley - does this mean if i get more memory mediacoder will access more? (is there a way i can increase how much it uses as i have plenty free?)

Increasing the number of slots does not change anything - it just cycles around 3 or 4 buffers instead of 2.
decoder CPU utilization is at 30-50% even when i set the priority to decoder in tasking...

I tried this on a lower spec machine (2.66gh intel dual core/GTS250) and things seem to work as they should
i.e buffer stays full and CPU is at 100%) Sadly - Cuda alone on this machine is actually faster than with SVE.

Could my symptoms be because of the type of file i am decoding perhaps?
(576x720 Mpeg2 .ts file recorded from live FTA Australian TV)

Here are the steps i took:

1.) add file from disk 1
2.) Go to UI preferences and select Apple ipod defaults.
3.) go to Video and change to GPU encoding withe the radio button
4.) Go to Picture and change resize to original.
5.) change output directory to somewhere on disk 2
6.) go to tasking and turn SVE on.

This is so that i use as many defaults as possible - and ensures that the result is playable on all my devices.

I have played around with various combinations of CPU priorites and GPU offload parameters
which make no difference to the speed.

Is there a better series of settings i can use?

Cheers,

stanley wrote:The maximum cache size is 900~1000 frames. MediaCoder uses no more than 1/4 of free memory as cache.
It is also likely that the decoding is the bottle-neck of your transcoding. You can see decoder CPU utilization percentage in the information sidebar after transcoding. You can also try increasing number of cache slots from default 2 to 3.
sharkbite wrote:Hi Stanley I am running 4g memory.....according to task manager media coder is using about 600meg all up and I have about 2.5g free when It is running......so I don't think memory is the problem.

The buffers fill and empty very quickly.....the numbers are around 16.....with an occasional spike to 427.....

stanley
Site Admin
Site Admin
Posts: 4130
Joined: Mon May 15, 2006 7:43 pm
Location: Sydney

Re: Feedbacks needed for Segmental Video Encoding

Post by stanley » Mon May 27, 2013 3:01 pm

By default program allows use up to 1/4 of free memory for caching frames. This can be adjusted by setting an value at Settings->Overall->Video->Video data cache size.
Are you using CUDA encoding for all your tests? Maybe you can monitor the GPU load to see if it is increasing when switching SVE on. If it's not increasing, that means your CPU has already reach its limit so that SVE cannot boost the speed any more.
When things work together, things work.

sharkbite
Amateur
Amateur
Posts: 37
Joined: Fri Feb 25, 2011 1:39 pm

Re: Feedbacks needed for Segmental Video Encoding

Post by sharkbite » Mon May 27, 2013 8:39 pm

Thanks Stanley - i cranked up the Memory to 1.5M - no effect on speed, but the buffer allowed 900 frames.

With CUDA on:
- regardless of SVE or not - speed is 6.7x with CPU (all 6 cores) at 50% and GPU at 30%
With SVE on - only one buffer is used at a time and is filled and emptied in about half a second.
total Memory usage - 1.6 Gig

With CUDA off:
SVE 0ff CPU is at 100% - Speed is 4x
SVE on CPU is at 100% - Speed is 5x and all selected buffers fill
Interesting - total memory usage is 4Gig and then things start to slow down....

stanley wrote:By default program allows use up to 1/4 of free memory for caching frames. This can be adjusted by setting an value at Settings->Overall->Video->Video data cache size.
Are you using CUDA encoding for all your tests? Maybe you can monitor the GPU load to see if it is increasing when switching SVE on. If it's not increasing, that means your CPU has already reach its limit so that SVE cannot boost the speed any more.

encoderdude
Beginner
Beginner
Posts: 6
Joined: Sat May 25, 2013 7:37 am

Re: Feedbacks needed for Segmental Video Encoding

Post by encoderdude » Mon May 27, 2013 9:45 pm

Tried the new build 5506 on my quad core intel cpu. (Used the update package to update from prev. version)
Tried to convert an xvid avi file to mp4 x264 (used 60sec. clip from a 90 min. 720x304 avi file, used the time option to limit the cut area).
Used both high10 profile & high profile with levels auto & tune Film. i use 2-pass mode with first pass in turbo mode & opencl enabled.
Used sve with 4 caches then 3, 2 & 1.

The result i got,
A full garbage video file tht converts correctly towards the last few seconds. Yes, the last few seconds has correct video & audio but the portion prior to tht has only audio correctly converted, the video is either fully green or gray scale (used potplayer for playback)
The file size seems to be correct now. (prev. build gave doubled file size)
The prev. build gave me converted video with first 20-30s correct & the rest a repeat of the first 20-30s part. That seems to have been fixed, but the output now has green/grayscale garbage video in the area before the last 3-5 secs.
Wht could be the problem?

stanley
Site Admin
Site Admin
Posts: 4130
Joined: Mon May 15, 2006 7:43 pm
Location: Sydney

Re: Feedbacks needed for Segmental Video Encoding

Post by stanley » Tue May 28, 2013 1:11 am

I didn't test SVE with H.264 high10 profile. I believe it should work if you are not using 2-pass or 3-pass mode.
encoderdude wrote:Used both high10 profile & high profile with levels auto & tune Film. i use 2-pass mode with first pass in turbo mode & opencl enabled.
When things work together, things work.

stanley
Site Admin
Site Admin
Posts: 4130
Joined: Mon May 15, 2006 7:43 pm
Location: Sydney

Re: Feedbacks needed for Segmental Video Encoding

Post by stanley » Tue May 28, 2013 1:16 am

sharkbite wrote:Thanks Stanley - i cranked up the Memory to 1.5M - no effect on speed, but the buffer allowed 900 frames.
With CUDA on:
- regardless of SVE or not - speed is 6.7x with CPU (all 6 cores) at 50% and GPU at 30%
With SVE on - only one buffer is used at a time and is filled and emptied in about half a second.
total Memory usage - 1.6 Gig
I am more sure now that the bottleneck is at the decoding. The decoder just can't perform any faster and two CUDA encoders have to wait for frames to be filled in the cache. Try changing video source from MEncoder to FFmpeg which normally performs faster.

With CUDA off:
SVE 0ff CPU is at 100% - Speed is 4x0000
SVE on CPU is at 100% - Speed is 5x and all selected buffers fill
Interesting - total memory usage is 4Gig and then things start to slow down....
This is because x264 itself consumes very large amount of system memory (much more than CUDA encoder), not mentioning two or three instances of them. And that's why I choose to use no more than 1/4 of free system memory for video frame cache. ;-)
When things work together, things work.

encoderdude
Beginner
Beginner
Posts: 6
Joined: Sat May 25, 2013 7:37 am

Re: Feedbacks needed for Segmental Video Encoding

Post by encoderdude » Tue May 28, 2013 7:06 am

i'm getting garbage video output with high profile as well. Which profile do u personally prefer for sve, stanley?

Post Reply