Re: Solution for segfault when using export_dv

On 10/24/07, Kees Bakker <[email protected]> wrote:
>
> Hi guys,
>
> On my system transcode gives a segfault when export to dv.
> After analyzing the problem I found to cause to be here:
>
> static unsigned char *bufalloc(size_t size)
> {
>
> #ifdef HAVE_GETPAGESIZE
>    long buffer_align=getpagesize();
> #else
>    long buffer_align=0;
> #endif
>
>    char *buf = malloc(size + buffer_align);
>
>    long adjust;
>
>    if (buf == NULL) {
>        fprintf(stderr, "(%s) out of memory", __FILE__);
>    }
>
>    adjust = buffer_align - ((long) buf) % buffer_align;
>
>    if (adjust == buffer_align)
>       adjust = 0;
>
>    return (unsigned char *) (buf + adjust);
> }
>
> Notice the "((long) buf) % buffer_align". On my system it
> turns out that buf is in the high range, something like:
>    0xb2c2c008
> with the result that 'adjust' becomes 0x1ff8. And so the result
> of bufalloc is misalligned AND too far into the buffer.
> Next libdv will crash inside memset somewhere.
>
> The solution is to use unsigned long. Here is a small patch
> (hoping it survives the email).
>
> Greetings, Kees Bakker
>
> diff -ru transcode-1.0.4.orig/export/export_dv.c transcode-1.0.4
> /export/export_dv.c
> --- transcode-1.0.4.orig/export/export_dv.c  2007-10-23 17:06:24.000000000+0200
> +++ transcode-1.0.4/export/export_dv.c  2005-07-04 09:23:00.000000000+0200
> @@ -61,13 +61,13 @@
>
>     char *buf = malloc(size + buffer_align);
>
> -   long adjust;
> +   unsigned long adjust;
>
>     if (buf == NULL) {
>         fprintf(stderr, "(%s) out of memory", __FILE__);
>     }
>
> -   adjust = buffer_align - ((long) buf) % buffer_align;
> +   adjust = buffer_align - ((unsigned long) buf) % buffer_align;
>
>     if (adjust == buffer_align)
>        adjust = 0;
>
>

ack
expect followup discussion as soon as is possible