Re: Limit of 4 threads in the VfW GUI

Hi,

well, maybe you could indeed provoke a crash with "way too many" threads.
But that shouldn't be an issue yet with a bit more than 4 threads.

The problem is rather that xvidcore doesn't scale well for many threads. So
using many threads doesn't necessarily make things faster, especially not
with low vertical input resolution.

The threading algorithm in xvidcore has a "warm-up" and a "fade-out" phase
so to say, where the threads do not all run fully in parallel. So there are
interdependencies between the threads, which cause waiting and syncing
phases with other threads. This overhead becomes bigger with increasing
number of threads, so that the benefits from launching more threads also
becomes smaller and smaller.

Further, only the ME and RD-opt code is multi-threaded in xvidcore. The
write-out of the final bitstream for a frame is still serialized (as is
the bitstream itself). So with more threads and faster ME and R-D analysis,
the bitstream writing will gain more impact relatively. This also limits
the scalability of xvidcore over many threads.

The limit of "4" in VfW is there because xvidcore scales quite well to this
number of threads. Increasing it to say "8" will likely yield a further
speed-up for HD input video but it will not nearly be double the speed
(because of the additional sync and wait overhead). So you might overall get
a better performance when running xvidcore with just the "4" threads (which
still scale ok) and use your remaining CPU to run some other common tasks
in parallel, like audio encoding, image prefiltering or decoding of your
input video (which should often be compressed also). Therefore, the VfW
limit of "4" threads seemed like a good compromise.

If you want to use many threads effectively for xvid encoding, have a look
at xvid_encraw. The xvid_encraw program can combine the xvidcore threading
algorithm with multi-threading at the app-level and scales rather well to
e.g. 8 threads.

Best regards,
Michael


Quoting Marcin Jaworski <[email protected]>:

> Greetings.
>
> The VfW GUI limits the number of threads to 4
> (xvidcore/vfw/src/config.c, line 1478), but the core library has no such
> limitation. I know that the vertical resolution of the input effectively
> limits the number of threads that can be used, but is there a reason for
> hardcoding the limit in VfW to just 4? SysKin once warned about using
> "way too many" threads. Can >4 be too many for the algorithm to handle
> (can it cause crashes etc.)?
>
> Best regards,
> Marcin Jaworski
> _______________________________________________
> Xvid-devel mailing list
> [email protected]
> http://list.xvid.org/mailman/listinfo/xvid-devel
>
>