Re: Error on raster2pgsq - rt_band_from_wkb: Premature end of WKB on band offline path reading.

Hi Sean

Could you perhaps insert the following into postgresql.auto.conf file on
your machine (On Debian it
is /var/lib/postgresql/$PG_VERSION/main/postgresql.auto.conf), and restart
the server?


postgis.gdal_enabled_drivers = 'ENABLE_ALL'
postgis.enable_outdb_rasters = TRUE

Is the raster extension enabled?

Pieter du Plooy

On Wed, Feb 5, 2025 at 2:46 PM Chater, ST, Mnr [schater--- via
postgis-users <[email protected]> wrote:

> Hi,
>
>
>
> I am trying to ingest an out-DB raster into PostgreSQL 15 database using
> the raster2pgsql command. I have PostGIS 3.3.8 installed (recently upgraded
> from 3.2). This server is running Rocky Linux (will refer to this as server
> A). Please note this server is running a production application so I would
> prefer not to have to uninstall everything and restart from scratch.
>
> This is the command I have used in the past and have been using now:
>
>
>
> */usr/pgsql-15/bin/raster2pgsql -d -s 4326 -t auto -R -F -Y -e
> /home/<username>/Downloads/2016-01-01_40m_k30.tif public.test_raster_table
> | psql -U <postgres-user> -d <db-name>*
>
>
>
> After upgrading to PostGIS 3.2 I have been running into the following
> error when running my ingestion command:
>
>
>
> *[user@host Downloads]$ /usr/pgsql-15/bin/raster2pgsql ....*
>
> *Processing 1/1: /home/demsa/Downloads/2016-01-01_40m_k30.tif*
>
> *INFO: Using computed tile size: 256x256*
>
> *DROP TABLE*
>
> *CREATE TABLE*
>
> *COPY 50*
>
> *COPY 50*
>
> *COPY 50*
>
> *COPY 50*
>
> *COPY 50*
>
> *COPY 50*
>
> *COPY 50*
>
> *COPY 50*
>
> *COPY 50*
>
> *munmap_chunk(): invalid pointer*
>
> *ERROR:  rt_band_from_wkb: Premature end of WKB on band offline path
> reading*
>
> *CONTEXT:  COPY backfill_table_mean, line 42, column rast:
> "010000010043EEF8D9F5353A3F4075AD9570363ABF9A3CA74761E4334043AAD5064C5641C000000000000000000000000000..."*
>
>
>
> I have successfully ingested out-DB rasters to the same database in the
> past. In fact, the *2016-01-01_40m_k30.tif* in the above command is one
> of the files that I have successfully ingested into the same database in
> the past.
>
>
>
> *Attemps to resolve:*
>
>    - I have created a blank raster in QGIS (*test.tif*), which is quite
>    small (6x4 pixels) and attempted to ingest this with the above command. The
>    same error appears, which leads me to believe there is something wrong with
>    the install of PostGIS/postgis_raster package that I have.
>    - I copied both rasters (*2016-01-01_40m_k30.tif* and *test.tif*)
>    across to another linux server (server B) which is has PostGIS 3.3.4. Using
>    this command: /usr/pgsql-14/bin/raster2pgsql -d -s 4326 -t auto -R -F -Y -e
>    /home/<username>/Downloads/2016-01-01_40m_k30.tif public.test_raster_table
>    | psql -p 5432 -U <postgres-user> -d <dbname> -h <server A IP> I was
>    successfully able to ingest both rasters into the database on server A.
>       - This leads me to believe that there is something wrong with the
>       PostGIS version that I am using and not with the rasters themselves.
>    - Tried various combinations of the raster2pgsql flags
>
> Removed -t auto
>
> free(): invalid pointer
>
> Removed -e
>
> COPY 50
> COPY 50
> COPY 50
> free(): invalid pointer
> ERROR: rt_raster_from_hexwkb: Raster HEXWKB input must have an even number
> of characters
> CONTEXT: COPY backfill_table_mean, line 42, column rast:
> "010000010043EEF8D9F5353A3F4075AD9570363ABF9A3CA74761E4334043AAD5064C5641C00000..."
>
> Removed -Y
>
> ERROR: unterminated quoted string at or near "'2016-01-01_40m_k30."
>
> LINE 1: ...362D30312D30315F34306D5F6B33302E74696600'::raster,'2016-01-0...
>
> Removed -R
>
> munmap_chunk(): invalid pointer
> ERROR: rt_band_from_wkb: Premature end of WKB on band data reading
> (rt_wkb.c:201)
> CONTEXT: COPY backfill_table_mean, line 49, column rast:
> "010000010043EEF8D9F5353A3F4075AD9570363ABF8835813D97FE3340FE80203F676341C000000000000000000000000000..."
>
>
>
> *The results of running gdalinfo (on separate windows PC) on both of the
> rasters I tried to ingest:*
>
>
>
> *> gdalinfo 2016-01-01_40m_k30.tif*
>
> Driver: GTiff/GeoTIFF
>
> Warning 1: PROJ: proj_create_from_database: C:\Program
> Files\PostgreSQL\14\share\contrib\postgis-3.3\proj\proj.db contains
> DATABASE.LAYOUT.VERSION.MINOR = 0 whereas a number >= 2 is expected. It
> comes from another PROJ installation.
>
> Files: 2016-01-01_40m_k30.tif
>
> Size is 7265, 8338
>
> Coordinate System is:
>
> GEOGCRS["WGS 84",
>
>     DATUM["World Geodetic System 1984",
>
>         ELLIPSOID["WGS 84",6378137,298.257223563,
>
>             LENGTHUNIT["metre",1,
>
>                 ID["EPSG",9001]]]],
>
>     PRIMEM["Greenwich",0,
>
>         ANGLEUNIT["degree",0.0174532925199433,
>
>             ID["EPSG",9122]]],
>
>     CS[ellipsoidal,2],
>
>         AXIS["latitude",north,
>
>             ORDER[1],
>
>             ANGLEUNIT["degree",0.0174532925199433,
>
>                 ID["EPSG",9122]]],
>
>         AXIS["longitude",east,
>
>             ORDER[2],
>
>             ANGLEUNIT["degree",0.0174532925199433,
>
>                 ID["EPSG",9122]]]]
>
> Data axis to CRS axis mapping: 2,1
>
> Origin = (17.844391979228011,-31.500006410136233)
>
> Pixel Size = (0.000399944803551,-0.000399973379606)
>
> Metadata:
>
>   AREA_OR_POINT=Area
>
>   TIFFTAG_SOFTWARE=GDAL 3.7.0e (3070046) (4.5.1;1700;3.0.0;2.0.7;;;;),
> PROJ 9.2.1 (EPSG.VERSION=v10.088;EPSG.DATE=2023-05-13;ESRI.VERSION=ArcGIS
> Pro 3.1;ESRI.DATE=2023-19-01;PROJ_DATA.VERSION=1.14;)
>
> Image Structure Metadata:
>
>   COMPRESSION=DEFLATE
>
>   INTERLEAVE=BAND
>
>   PREDICTOR=3
>
> Corner Coordinates:
>
> Upper Left  (  17.8443920, -31.5000064) ( 17d50'39.81"E, 31d30' 0.02"S)
>
> Lower Left  (  17.8443920, -34.8349844) ( 17d50'39.81"E, 34d50' 5.94"S)
>
> Upper Right (  20.7499910, -31.5000064) ( 20d44'59.97"E, 31d30' 0.02"S)
>
> Lower Right (  20.7499910, -34.8349844) ( 20d44'59.97"E, 34d50' 5.94"S)
>
> Center      (  19.2971915, -33.1674954) ( 19d17'49.89"E, 33d10' 2.98"S)
>
> Band 1 Block=256x256 Type=Float32, ColorInterp=Gray
>
>   Min=19.340 Max=27.754
>
>   Minimum=19.340, Maximum=27.754, Mean=24.930, StdDev=1.194
>
>   NoData Value=-9999
>
>   Metadata:
>
>     STATISTICS_APPROXIMATE=YES
>
>     STATISTICS_MAXIMUM=27.753507614136
>
>     STATISTICS_MEAN=24.93024257498
>
>     STATISTICS_MINIMUM=19.340391159058
>
>     STATISTICS_STDDEV=1.1943478187917
>
>     STATISTICS_VALID_PERCENT=73.14
>
>
>
> *> gdalinfo test_grid.tif*
>
> Driver: GTiff/GeoTIFF
>
> Warning 1: PROJ: proj_create_from_database: C:\Program
> Files\PostgreSQL\14\share\contrib\postgis-3.3\proj\proj.db contains
> DATABASE.LAYOUT.VERSION.MINOR = 0 whereas a number >= 2 is expected. It
> comes from another PROJ installation.
>
> Files: test_grid.tif
>
> Size is 6, 4
>
> Coordinate System is:
>
> GEOGCRS["WGS 84",
>
>     DATUM["World Geodetic System 1984",
>
>         ELLIPSOID["WGS 84",6378137,298.257223563,
>
>             LENGTHUNIT["metre",1,
>
>                 ID["EPSG",9001]]]],
>
>     PRIMEM["Greenwich",0,
>
>         ANGLEUNIT["degree",0.0174532925199433,
>
>             ID["EPSG",9122]]],
>
>     CS[ellipsoidal,2],
>
>         AXIS["latitude",north,
>
>             ORDER[1],
>
>             ANGLEUNIT["degree",0.0174532925199433,
>
>                 ID["EPSG",9122]]],
>
>         AXIS["longitude",east,
>
>             ORDER[2],
>
>             ANGLEUNIT["degree",0.0174532925199433,
>
>                 ID["EPSG",9122]]]]
>
> Data axis to CRS axis mapping: 2,1
>
> Origin = (16.639998682000002,-31.334238383999999)
>
> Pixel Size = (1.000000000000000,-1.000000000000001)
>
> Metadata:
>
>   AREA_OR_POINT=Area
>
> Image Structure Metadata:
>
>   INTERLEAVE=BAND
>
> Corner Coordinates:
>
> Upper Left  (  16.6399987, -31.3342384) ( 16d38'24.00"E, 31d20' 3.26"S)
>
> Lower Left  (  16.6399987, -35.3342384) ( 16d38'24.00"E, 35d20' 3.26"S)
>
> Upper Right (  22.6399987, -31.3342384) ( 22d38'24.00"E, 31d20' 3.26"S)
>
> Lower Right (  22.6399987, -35.3342384) ( 22d38'24.00"E, 35d20' 3.26"S)
>
> Center      (  19.6399987, -33.3342384) ( 19d38'24.00"E, 33d20' 3.26"S)
>
> Band 1 Block=6x4 Type=Float32, ColorInterp=Gray
>
>   NoData Value=0
>
>
>
> *The result of running sudo journalctl --no-pager | tail -n 30 immediately
> after raster2pgsql fails.*
>
> Feb 05 12:46:42 SUNGIS18 systemd[1]: Started Process Core Dump (PID
> 265814/UID 0).
>
> Feb 05 12:46:42 SUNGIS18 systemd-coredump[265815]: Resource limits disable
> core dumping for process 265810 (raster2pgsql).
>
> Feb 05 12:46:42 SUNGIS18 systemd-coredump[265815]: Process 265810
> (raster2pgsql) of user 1000 dumped core.
>
> Feb 05 12:46:42 SUNGIS18 systemd[1]: [email protected]:
> Deactivated successfully.
>
> Feb 05 12:46:42 SUNGIS18 systemd[1]: Started Process Core Dump (PID
> 265819/UID 0).
>
> Feb 05 12:46:42 SUNGIS18 systemd-coredump[265820]: Resource limits disable
> core dumping for process 265813 (postmaster).
>
> Feb 05 12:46:42 SUNGIS18 systemd-coredump[265820]: Process 265813
> (postmaster) of user 26 dumped core.
>
> Feb 05 12:46:42 SUNGIS18 systemd[1]: [email protected]:
> Deactivated successfully.
>
> Feb 05 12:46:44 SUNGIS18 sudo[265831]:    demsa : TTY=pts/2 ;
> PWD=/home/demsa/Downloads ; USER=root ; COMMAND=/bin/journalctl --no-pager
>
> Feb 05 12:46:44 SUNGIS18 sudo[265831]: pam_unix(sudo:session): session
> opened for user root(uid=0) by demsa(uid=1000)
>
>
>
> Any advice as to how I could investigate this further or resolve the issue
> would be much appreciated.
>
>
>
> Many thanks
>
> Sean Chater
>
>
> The integrity and confidentiality of this email are governed by these
> terms. Disclaimer <https://www.sun.ac.za/emaildisclaimer/default.aspx>
> Die integriteit en vertroulikheid van hierdie e-pos word deur die volgende
> bepalings bereël. Vrywaringsklousule
> <https://www.sun.ac.za/emaildisclaimer/default.aspx>
>