GraphicsMagick: Added support for a Read resource limit.
GraphicsMagick Commits <[email protected]>
| Newsgroups | gmane.comp.video.graphicsmagick.cvs |
|---|---|
| Message-ID | <mailman.76.1641683708.2413.graphicsmagick-commit@lists.sourceforge.net> |
changeset 121de89bed95 in /hg/GraphicsMagick details: http://hg.GraphicsMagick.org/hg/GraphicsMagick?cmd=changeset;node=121de89bed95 summary: Added support for a Read resource limit. diffstat: ChangeLog | 10 +++ VisualMagick/installer/inc/version.isx | 4 +- doc/environment.imdoc | 10 +++ doc/options.imdoc | 66 ++++++++++++++-------- fuzzing/utils.cc | 1 + magick/blob.c | 99 ++++++++++++++++++++++----------- magick/enum_strings.c | 4 +- magick/resource.c | 29 ++++++--- magick/resource.h | 5 +- magick/version.h | 4 +- utilities/gm.1 | 91 +++++++++++++++++++----------- www/Changelog.html | 9 +++ www/GraphicsMagick.html | 79 ++++++++++++++++++-------- www/animate.html | 2 +- www/compare.html | 2 +- www/composite.html | 2 +- www/convert.html | 2 +- www/display.html | 2 +- www/gm.html | 97 +++++++++++++++++++++----------- www/identify.html | 2 +- www/import.html | 2 +- www/mogrify.html | 2 +- www/montage.html | 2 +- 23 files changed, 350 insertions(+), 176 deletions(-) diffs (truncated from 1083 to 500 lines): diff -r eae611832371 -r 121de89bed95 ChangeLog --- a/ChangeLog Sat Jan 01 12:38:34 2022 -0600 +++ b/ChangeLog Sat Jan 08 17:14:10 2022 -0600 @@ -1,3 +1,13 @@ +2022-01-08 Bob Friesenhahn <[email protected]> + + * magick/resource.c: Add support for setting a read resource limit + via the MAGICK_LIMIT_READ environment variable, or similar to + '-limit read 5mb'. + + * magick/resource.h: Added a ReadResource limit, which is a limit + on how many uncompressed file bytes may be read while decoding an + input file. + 2022-01-01 Bob Friesenhahn <[email protected]> * ChangeLog.2021: Rotate ChangeLog for 2022. Happy New Year! diff -r eae611832371 -r 121de89bed95 VisualMagick/installer/inc/version.isx --- a/VisualMagick/installer/inc/version.isx Sat Jan 01 12:38:34 2022 -0600 +++ b/VisualMagick/installer/inc/version.isx Sat Jan 08 17:14:10 2022 -0600 @@ -10,5 +10,5 @@ #define public MagickPackageName "GraphicsMagick" #define public MagickPackageVersion "1.4" -#define public MagickPackageVersionAddendum ".020220101" -#define public MagickPackageReleaseDate "snapshot-20220101" +#define public MagickPackageVersionAddendum ".020220108" +#define public MagickPackageReleaseDate "snapshot-20220108" diff -r eae611832371 -r 121de89bed95 doc/environment.imdoc --- a/doc/environment.imdoc Sat Jan 01 12:38:34 2022 -0600 +++ b/doc/environment.imdoc Sat Jan 08 17:14:10 2022 -0600 @@ -195,6 +195,16 @@ number of pixels due to multiple image frames/pages (e.g. multi-page document or an animation).</abs> +<opt>MAGICK_LIMIT_READ</opt> + +<abs>Maximum number of uncompressed bytes which may be read while +decoding an image. Each read by the software from the input file is +counted against the total, even if it has been read before. Decoding +fails when the limit is reached. This limit helps defend against +highly compressed files (e.g. via gzip), or files which use complex +looping structures, or when data is being read from a stream +(pipe).</abs> + <opt>MAGICK_LIMIT_WIDTH</opt> <abs>Maximum pixel width of an image read, or created.</abs> diff -r eae611832371 -r 121de89bed95 doc/options.imdoc --- a/doc/options.imdoc Sat Jan 01 12:38:34 2022 -0600 +++ b/doc/options.imdoc Sat Jan 08 17:14:10 2022 -0600 @@ -3210,30 +3210,45 @@ <utils apps=animate,compare,composite,convert,display,identify,import,mogrify,montage> <dopt>-limit <type> <value></opt> -<abs>Disk, File, Map, Memory, Pixels, Width, Height or Threads resource limit</abs> +<abs>Disk, File, Map, Memory, Pixels, Width, Height, Read, or Threads resource limit</abs> <pp> By default, resource limits are estimated based on the available -resources of the system. The resource limits are <s>Disk</s>, maximum -total disk space consumed; <s>File</s>, maximum number of file -descriptors allowed to be open at once; <s>Map</s>, maximum total -number of file bytes which may be memory mapped; <s>Memory</s>, -maximum total number of bytes of heap memory used for image storage; -<s>Pixels</s>, maximum absolute image size (per image); <s>Width</s>, -maximum image pixels width; <s>Height</s>, maximum image pixels -height; and <s>Threads</s>, the maximum number of worker threads to -use per OpenMP thread team.</pp> - -<pp> -These resource limits are used to decide if (for a given image) the -decoded image ("pixel cache") should be stored in heap memory (RAM), -in a memory-mapped disk file, or in a disk file accessed via -read/write I/O. The number of total pixels in one image, and/or the -width/height, may also be limited in order to force the reading, or -creation of images larger than the limit (in pixels) to intentionally -fail. The disk limit establishes an overall limit since using the disk -is the means of last resort. When the disk limit has been reached, no -more images may be read.</pp> +resources and capabilities of the system. The resource limits are +<s>Disk</s>, maximum total disk space consumed; <s>File</s>, maximum +number of file descriptors allowed to be open at once; <s>Map</s>, +maximum total number of file bytes which may be memory mapped; +<s>Memory</s>, maximum total number of bytes of heap memory used for +image storage; <s>Pixels</s>, maximum absolute image size (per image); +<s>Width</s>, maximum image pixels width; <s>Height</s>, maximum image +pixels height; <s>Read</s>, maximum number of uncompressed bytes to +read; and <s>Threads</s>, the maximum number of worker threads to use +per OpenMP thread team.</pp> + +<pp> +The <s>Disk</s> and <s>Map</s> resource limits are used to decide if +(for a given image) the decoded image ("pixel cache") should be stored +in heap memory (RAM), in a memory-mapped disk file, or in a disk file +accessed via read/write I/O.</pp> + +<pp> +The number of total pixels in one image (<s>Pixels</s>), and/or the +width/height (<s>Width</s>/<s>Height</s>), may be limited in order to +force the reading, or creation of images larger than the limit (in +pixels) to intentionally fail. The disk limit (<s>Disk</s>) +establishes an overall limit since using the disk is the means of last +resort. When the disk limit has been reached, no more images may be +read.</pp> + +<pp> +The amount of uncompressed data read when reading one image may be +limited by the <s>Read</s> limit. Reading the image fails when the +limit is hit. This option is useful if the data is read from a stream +(pipe) or from a compressed file such as a gzipped file. Some files +are very compressable and so a small compressed file can decompress to +a huge amount of data. This option also defends against files which +produce seemingly endless loops while decoding by seeking backwards in +the file.</pp> <pp> The value argument is an absolute value, but may have standard binary @@ -3249,10 +3264,11 @@ environment variables <s>MAGICK_LIMIT_DISK</s>, <s>MAGICK_LIMIT_FILES</s>, <s>MAGICK_LIMIT_MAP</s>, <s>MAGICK_LIMIT_MEMORY</s>, <s>MAGICK_LIMIT_PIXELS</s>, -<s>MAGICK_LIMIT_WIDTH</s>, <s>MAGICK_LIMIT_HEIGHT</s>,and -<s>OMP_NUM_THREADS</s> may be used to set the limits for disk space, -open files, memory mapped size, heap memory, per-image pixels, image -width, image height, and threads respectively.</pp> +<s>MAGICK_LIMIT_WIDTH</s>, <s>MAGICK_LIMIT_HEIGHT</s>. +<s>MAGICK_LIMIT_READ</s>, and <s>OMP_NUM_THREADS</s> may be used to +set the limits for disk space, open files, memory mapped size, heap +memory, per-image pixels, image width, image height, and threads +respectively.</pp> <pp> Use the option <tt>-list resource</tt> list the current limits.</pp> diff -r eae611832371 -r 121de89bed95 fuzzing/utils.cc --- a/fuzzing/utils.cc Sat Jan 01 12:38:34 2022 -0600 +++ b/fuzzing/utils.cc Sat Jan 08 17:14:10 2022 -0600 @@ -22,6 +22,7 @@ MagickLib::SetMagickResourceLimit(MagickLib::WidthResource, 2048); MagickLib::SetMagickResourceLimit(MagickLib::HeightResource, 2048); MagickLib::SetMagickResourceLimit(MagickLib::DiskResource, 0); + MagickLib::SetMagickResourceLimit(MagickLib::ReadResource, 900000); } }; diff -r eae611832371 -r 121de89bed95 magick/blob.c --- a/magick/blob.c Sat Jan 01 12:38:34 2022 -0600 +++ b/magick/blob.c Sat Jan 08 17:14:10 2022 -0600 @@ -1,5 +1,5 @@ /* -% Copyright (C) 2003 - 2021 GraphicsMagick Group +% Copyright (C) 2003 - 2022 GraphicsMagick Group % Copyright (C) 2002 ImageMagick Studio % Copyright 1991-1999 E. I. du Pont de Nemours and Company % @@ -104,6 +104,8 @@ block_size, /* I/O block size */ length, /* The current size of the BLOB data. */ extent, /* The amount of backing store currently allocated */ + read_limit, /* Limit on data to return to user */ + read_total, /* Amount of data read thus far */ quantum; /* The amount by which to increase the size of the backing store */ unsigned int @@ -269,10 +271,11 @@ return 0; } *data=(void *)(blob->data+blob->offset); - available=Min(length,blob->length-blob->offset); + available=Min(Min(length,blob->read_limit-blob->read_total),blob->length-blob->offset); blob->offset+=available; - if (available < length) + if (available == 0) blob->eof=True; + blob->read_total += available; return available; } @@ -1318,38 +1321,41 @@ assert(image->blob != (BlobInfo *) NULL); assert(image->blob->type != UndefinedStream); blob=image->blob; - switch (blob->type) - { - case UndefinedStream: - break; - case FileStream: - case StandardStream: - case PipeStream: + if (!blob->eof) { - blob->eof=feof(blob->handle.std); - break; - } - case ZipStream: - { + switch (blob->type) + { + case UndefinedStream: + break; + case FileStream: + case StandardStream: + case PipeStream: + { + blob->eof=feof(blob->handle.std); + break; + } + case ZipStream: + { #if defined(HasZLIB) && !defined(DISABLE_COMPRESSED_FILES) - blob->eof=gzeof(blob->handle.gz); + blob->eof=gzeof(blob->handle.gz); #endif /* defined(HasZLIB) && !defined(DISABLE_COMPRESSED_FILES) */ - break; - } - case BZipStream: - { + break; + } + case BZipStream: + { #if defined(HasBZLIB) && !defined(DISABLE_COMPRESSED_FILES) - int - status; - - (void) BZ2_bzerror(blob->handle.bz,&status); - blob->eof=status == BZ_UNEXPECTED_EOF; + int + status; + + (void) BZ2_bzerror(blob->handle.bz,&status); + blob->eof=status == BZ_UNEXPECTED_EOF; #endif /* defined(HasBZLIB) && !defined(DISABLE_COMPRESSED_FILES) */ - break; + break; + } + case BlobStream: + break; + } } - case BlobStream: - break; - } return(blob->eof); } @@ -2387,7 +2393,8 @@ size_t block_size, - length; + length, + total=0; assert(image != (Image *) NULL); assert(image->signature == MagickSignature); @@ -2421,6 +2428,7 @@ count=write(file,buffer+i,(MAGICK_POSIX_IO_SIZE_T) (length-i)); if (count <= 0) break; + total += count; } if (i < length) break; @@ -2428,7 +2436,8 @@ (void) close(file); if (image->logging) (void) LogMagickEvent(BlobEvent,GetMagickModule(), - "Copyied %"MAGICK_SIZE_T_F"u bytes from Blob stream to \"%s\"",(MAGICK_SIZE_T) i,filename); + "Copied %"MAGICK_SIZE_T_F"u bytes from Blob stream to \"%s\"", + (MAGICK_SIZE_T) total,filename); MagickFreeMemory(buffer); return (i < length ? MagickFail : MagickPass); } @@ -2711,6 +2720,11 @@ " mode %s ...", image, image->blob, BlobModeToString(mode)); /* + Set read limits + */ + image->blob->read_limit = (size_t) GetMagickResourceLimit(ReadResource); + image->blob->read_total = 0; + /* Cache I/O block size */ image->blob->block_size=MagickGetFileSystemBlockSize(); @@ -3152,13 +3166,14 @@ % % */ -MagickExport size_t ReadBlob(Image *image,const size_t length,void *data) +MagickExport size_t ReadBlob(Image *image,const size_t req_length,void *data) { BlobInfo * restrict blob; size_t - count; + count, + length; assert(image != (Image *) NULL); assert(image->signature == MagickSignature); @@ -3167,6 +3182,9 @@ assert(data != (void *) NULL); blob=image->blob; + + length=Min(req_length,blob->read_limit-blob->read_total); + count=0; switch (blob->type) { @@ -3336,6 +3354,9 @@ } } assert(count <= length); + blob->read_total += count; + if (count == 0) + blob->eof=True; return(count); } @@ -3434,6 +3455,12 @@ blob=image->blob; + if (blob->read_total >= blob->read_limit) + { + blob->eof=1; + return EOF; + } + switch (blob->type) { case FileStream: @@ -3449,6 +3476,10 @@ blob->first_errno=errno; } } + else + { + blob->read_total++; + } break; } case BlobStream: @@ -3457,6 +3488,7 @@ { octet=*((unsigned char *)blob->data+blob->offset); blob->offset++; + blob->read_total++; c=octet; } else @@ -3472,6 +3504,7 @@ c=octet; } } + return c; } diff -r eae611832371 -r 121de89bed95 magick/enum_strings.c --- a/magick/enum_strings.c Sat Jan 01 12:38:34 2022 -0600 +++ b/magick/enum_strings.c Sat Jan 08 17:14:10 2022 -0600 @@ -1,5 +1,5 @@ /* -% Copyright (C) 2008-2019 GraphicsMagick Group +% Copyright (C) 2008-2022 GraphicsMagick Group % % This program is covered by multiple licenses, which are described in % Copyright.txt. You should have received a copy of Copyright.txt with this @@ -1660,6 +1660,8 @@ resource_type=WidthResource; else if (LocaleCompare("Height",option) == 0) resource_type=HeightResource; + else if (LocaleCompare("Read",option) == 0) + resource_type=ReadResource; return resource_type; } diff -r eae611832371 -r 121de89bed95 magick/resource.c --- a/magick/resource.c Sat Jan 01 12:38:34 2022 -0600 +++ b/magick/resource.c Sat Jan 08 17:14:10 2022 -0600 @@ -1,5 +1,5 @@ /* -% Copyright (C) 2003 - 2019 GraphicsMagick Group +% Copyright (C) 2003 - 2022 GraphicsMagick Group % Copyright (C) 2002 ImageMagick Studio % % This program is covered by multiple licenses, which are described in @@ -96,15 +96,16 @@ static ResourceInfo resource_info[] = { - { "", "", "", 0, 0, ResourceInfinity, AbsoluteLimit, 0 }, - { "disk", "B", "MAGICK_LIMIT_DISK", 0, 0, ResourceInfinity, SummationLimit, 0 }, - { "files", "", "MAGICK_LIMIT_FILES", 0, 32, 256, SummationLimit, 0 }, - { "map", "B", "MAGICK_LIMIT_MAP", 0, 0, ResourceInfinity, SummationLimit, 0 }, - { "memory", "B", "MAGICK_LIMIT_MEMORY", 0, 0, ResourceInfinity, SummationLimit, 0 }, - { "pixels", "P", "MAGICK_LIMIT_PIXELS", 0, 1, ResourceInfinity, AbsoluteLimit, 0 }, - { "threads", "", "OMP_NUM_THREADS", 1, 1, ResourceInfinity, AbsoluteLimit, 0 }, - { "width", "P", "MAGICK_LIMIT_WIDTH", 0, 1, PIXEL_LIMIT, AbsoluteLimit, 0 }, - { "height", "P", "MAGICK_LIMIT_HEIGHT", 0, 1, PIXEL_LIMIT, AbsoluteLimit, 0 } + { "", "", "", 0, 0, ResourceInfinity, AbsoluteLimit, 0 }, + { "disk", "B", "MAGICK_LIMIT_DISK", 0, 0, ResourceInfinity, SummationLimit, 0 }, + { "files", "", "MAGICK_LIMIT_FILES", 0, 32, 256, SummationLimit, 0 }, + { "map", "B", "MAGICK_LIMIT_MAP", 0, 0, ResourceInfinity, SummationLimit, 0 }, + { "memory", "B", "MAGICK_LIMIT_MEMORY", 0, 0, ResourceInfinity, SummationLimit, 0 }, + { "pixels", "P", "MAGICK_LIMIT_PIXELS", 0, 1, ResourceInfinity, AbsoluteLimit, 0 }, + { "threads", "", "OMP_NUM_THREADS", 1, 1, ResourceInfinity, AbsoluteLimit, 0 }, + { "width", "P", "MAGICK_LIMIT_WIDTH", 0, 1, PIXEL_LIMIT, AbsoluteLimit, 0 }, + { "height", "P", "MAGICK_LIMIT_HEIGHT", 0, 1, PIXEL_LIMIT, AbsoluteLimit, 0 }, + { "read", "B", "MAGICK_LIMIT_READ", 0, 4096, ResourceInfinity, AbsoluteLimit, 0 } }; /* @@ -411,7 +412,8 @@ max_pixels=-1, max_threads=1, max_width=-1, - max_height=-1; + max_height=-1, + max_read=-1; size_t index; @@ -610,6 +612,9 @@ if ((envp=getenv("MAGICK_LIMIT_HEIGHT"))) max_height=MagickSizeStrToInt64(envp,1024); + if ((envp=getenv("MAGICK_LIMIT_READ"))) + max_read=MagickSizeStrToInt64(envp,1024); + #if defined(HAVE_OPENMP) max_threads=omp_get_num_procs(); (void) LogMagickEvent(ResourceEvent,GetMagickModule(), @@ -713,6 +718,8 @@ (void) SetMagickResourceLimit(WidthResource,max_width); if (max_height >= 0) (void) SetMagickResourceLimit(HeightResource,max_height); + if (max_read >= 0) + (void) SetMagickResourceLimit(ReadResource,max_read); } /* diff -r eae611832371 -r 121de89bed95 magick/resource.h --- a/magick/resource.h Sat Jan 01 12:38:34 2022 -0600 +++ b/magick/resource.h Sat Jan 08 17:14:10 2022 -0600 @@ -1,5 +1,5 @@ /* - Copyright (C) 2003 - 2015 GraphicsMagick Group + Copyright (C) 2003 - 2022 GraphicsMagick Group Copyright (C) 2002 ImageMagick Studio This program is covered by multiple licenses, which are described in @@ -28,7 +28,8 @@ PixelsResource, /* Maximum number of pixels in single image (Pixels) */ ThreadsResource, /* Maximum number of worker threads */ WidthResource, /* Maximum pixel width of an image (Pixels) */ - HeightResource /* Maximum pixel height of an image (Pixels) */ + HeightResource, /* Maximum pixel height of an image (Pixels) */ + ReadResource /* Maximum amount of uncompressed file data to read */ } ResourceType; /* diff -r eae611832371 -r 121de89bed95 magick/version.h --- a/magick/version.h Sat Jan 01 12:38:34 2022 -0600 +++ b/magick/version.h Sat Jan 08 17:14:10 2022 -0600 @@ -38,8 +38,8 @@ #define MagickLibVersion 0x262300 #define MagickLibVersionText "1.4" #define MagickLibVersionNumber 26,23,0 -#define MagickChangeDate "20220101" -#define MagickReleaseDate "snapshot-20220101" +#define MagickChangeDate "20220108" +#define MagickReleaseDate "snapshot-20220108" /* The MagickLibInterfaceNewest and MagickLibInterfaceOldest defines diff -r eae611832371 -r 121de89bed95 utilities/gm.1 --- a/utilities/gm.1 Sat Jan 01 12:38:34 2022 -0600 +++ b/utilities/gm.1 Sat Jan 08 17:14:10 2022 -0600 @@ -1,4 +1,4 @@ -.TH gm 1 "2021/12/31" "GraphicsMagick" +.TH gm 1 "2022/01/08" "GraphicsMagick" .TP .in 15 .in 15 @@ -2987,28 +2987,41 @@ "Input Levels" interface. .TP .B "-limit \fI<type> <value>"\fP -\fRDisk, File, Map, Memory, Pixels, Width, Height or Threads resource limit +\fRDisk, File, Map, Memory, Pixels, Width, Height, Read, or Threads resource limit By default, resource limits are estimated based on the available -resources of the system. The resource limits are \fBDisk\fP, maximum -total disk space consumed; \fBFile\fP, maximum number of file