Re: GMT 4: combining GRD files into one FILE with several LAYERS
"Walter (HF) Smith" <[email protected]>
| Newsgroups | gmane.comp.gis.gmt.user |
|---|---|
| Message-ID | <[email protected]> |
JJ, Although the number crunching might be parallelizable, it sounds like the I/O of the data to/from tiles is going to be the bottleneck. If you are going to the trouble of building a data structure to hold everything, you might consider also organizing the data by tiles that group lat,lon coordinates, such as by using a quadtree structure, rather than using a flat array of lon,lat as in a conventional GMT grid file. Seems to me that conventional storage would separate by large strides the elements you need to pull together for computation. Walter On Feb 11, 2014, at 6:04 PM, Joaquim Luis <[email protected]> wrote: > I never tried it but it's not impossible that you can do what you want by forcing the writing via GDAL. > Note that an alternative to a stack of 2D grids would be to have 256 grids written as individual grids inside the netCDF file (SUBDATASETS in GDAL parlance). > GMT could easily read those individual arrays and perhaps you could force the updating via a GDAL write. See this for an example of syntax for reading > > http://gmtrac.soest.hawaii.edu/doc/5.1.0/GMT_Docs.html#reading-more-complex-multi-band-images-or-grids > >> This data is essentially something along the lines of >> >> layer 1 = topogrpahy >> layer 2 = gravity >> layer 43= FFT(gravity) + sqrt (topo) (or something crazy like that)... >> >> and so on. >> >> In effect a global gird of all known “measurements", not so much “n dimensional” although that is certainly a good way to think of it. >> >> Does that change any facts, your opinion, etc… >> >> -jj >> >> >> On Feb 10, 2014, at 5:39 PM, Paul Wessel<[email protected]> wrote: >> >>> Hi JJ_ >>> >>> GMT can read from layer "a" but does not write 3-D grids. Given that I would not try to do a 256-layer file. >>> >>> -p >>> >>> On Feb 10, 2014, at 9:46 AM, Becker, Joseph J.<[email protected]> wrote: >>> >>>> I have 256 global grids at 15 second resolution; e.g. ~16 GB grd files * 256 grids = 4 TB of data. >>>> >>>> I want to automate some processing that uses tiny patches (16x16 nodes) of each and every one of these 256 files. And I step that over the globe. >>>> >>>> Off hand it seems like it might make sense (code re-use and simplicity) to combine all that data into a single grid with 256 layers. >>>> >>>> Is this practical? >>>> >>>> Anyone have some examples of doing things like this: >>>> >>>> grdmath on layer "a" writing result to layer "b" >>>> >>>> grdmask xy files> "layer" >>>> >>>> and so on. >>>> >>>> thanks! >>>> >>>> >>>> -jj >>>> >>>> To unsubscribe, send the message "signoff gmt-help" to [email protected] >>> To unsubscribe, send the message "signoff gmt-help" to [email protected] >> To unsubscribe, send the message "signoff gmt-help" to [email protected] >> >> > > To unsubscribe, send the message "signoff gmt-help" to [email protected] To unsubscribe, send the message "signoff gmt-help" to [email protected]