Re: [PROJ] RFC 102: Embedding resource files into libgdal (and PROJ)
Kurt Schwehr via gdal-dev <[email protected]> Wed, 2 Oct 2024 12:03:06 -0700
| Newsgroups | gmane.comp.gis.gdal.devel,gmane.comp.gis.proj-4.devel |
|---|---|
| Message-ID | <CACmBxyuvWsnPCPSwnPyvVS8YbY0VNPSQL6_4u51Ed6z3_khSSw@mail.gmail.com> |
--===============6592849117019579435==
Content-Type: multipart/alternative; boundary="0000000000003944a50623831734"
--0000000000003944a50623831734
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
So excited for this feature. We have custom code that does stuff like this.
e.g.
+ // BEGIN GOOGLE MODIFICATION
+ int flags =3D SQLITE_OPEN_READONLY | SQLITE_OPEN_NOMUTEX;
if (path.empty()) {
- path.resize(2048);
- const bool found =3D
- pj_find_file(pjCtxt(), "proj.db", &path[0], path.size() - 1)
!=3D 0;
- path.resize(strlen(path.c_str()));
- if (!found) {
- throw FactoryException("Cannot find proj.db");
- }
+ static const FileToc *toc =3D [] {
+ // Enable the memvfs extension.
+ const int init_code =3D sqlite3_memvfs_init(nullptr, nullptr,
nullptr);
+ if (init_code !=3D SQLITE_OK_LOAD_PERMANENTLY) {
+ throw FactoryException(
+ absl::StrCat("Can't initialize memvfs, with code ",
init_code));
+ }
+ // Re-register 'unix' as default filesystem. MemVFS can be used
+ // through URL parameters, or by passing as the fourth argument
to
+ // sqlite3_open_v2().
+ const int register_code =3D
+ sqlite3_vfs_register(sqlite3_vfs_find("unix"), /*makeDflt
=3D*/1);
+ if (register_code !=3D SQLITE_OK) {
+ throw FactoryException(absl::StrCat(
+ "Can't re-register the default VFS, with code ",
init_code));
+ }
+ return proj_db_create();
+ }();
+ // Load the database from a cc_embed_file, using the memvfs
+ // extension of sqlite.
+ path =3D
absl::StrFormat("file:/proj?ptr=3D0x%x&sz=3D%d&max=3D%d&vfs=3Dmemvfs",
+ reinterpret_cast<uintptr_t>(toc[0].data),
+ toc[0].size, toc[0].size);
+ flags |=3D SQLITE_OPEN_URI;
}
+ // END GOOGLE MODIFICATION
On Tue, Oct 1, 2024 at 10:28=E2=80=AFAM Even Rouault via PROJ <[email protected]=
sgeo.org>
wrote:
> Greg,
>
> What's unclear is to what extent things continue to be ok on compilers
> not supporting C23, and if they are, if those systems start to be second
> class. My overall feeling is that it is vastly too early to be
> depending on C23.
>
> The RFC is clear enough this is an optional feature ("C23 is not required
> if EMBED_RESOURCE_FILES is not enabled"). If you need it, you will need a
> C23 compiler, which there is already one available (clang 19), and anothe=
r
> one in a near future (gcc 15).
>
> C23 is only used to build the .c files where resources are embedded.
>
> If you don't need that new feature, you can use the current minimum build
> requirements of GDAL/PROJ.
>
> For once that GDAL/PROJ can use shiny new stuff, let's enjoy :-)
>
> But maybe c23 is c2x, and it's been supported in gcc for 6 years.
>
> No way a gcc from 6 years ago can support #embed: support has literally
> landed in GCC master 2 weeks ago.
>
> -- http://www.spatialys.com
> My software is free, but my time generally not.
>
> _______________________________________________
> PROJ mailing list
> [email protected]
> https://lists.osgeo.org/mailman/listinfo/proj
>
--0000000000003944a50623831734
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">So excited for this feature. We have custom code that does=
stuff like this.<div><br></div><div>e.g.</div><div><br></div><div>+ =C2=A0=
=C2=A0// BEGIN GOOGLE MODIFICATION<br>+ =C2=A0 =C2=A0int flags =3D SQLITE_=
OPEN_READONLY | SQLITE_OPEN_NOMUTEX;<br>=C2=A0 =C2=A0 =C2=A0if (path.empty(=
)) {<br>- =C2=A0 =C2=A0 =C2=A0 =C2=A0path.resize(2048);<br>- =C2=A0 =C2=A0 =
=C2=A0 =C2=A0const bool found =3D<br>- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0pj_find_file(pjCtxt(), "proj.db", &path[0], path.size()=
- 1) !=3D 0;<br>- =C2=A0 =C2=A0 =C2=A0 =C2=A0path.resize(strlen(path.c_str=
()));<br>- =C2=A0 =C2=A0 =C2=A0 =C2=A0if (!found) {<br>- =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0throw FactoryException("Cannot find proj.db&qu=
ot;);<br>- =C2=A0 =C2=A0 =C2=A0 =C2=A0}<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0sta=
tic const FileToc *toc =3D [] {<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0// E=
nable the memvfs extension.<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0const in=
t init_code =3D sqlite3_memvfs_init(nullptr, nullptr, nullptr);<br>+ =C2=A0=
=C2=A0 =C2=A0 =C2=A0 =C2=A0if (init_code !=3D SQLITE_OK_LOAD_PERMANENTLY) =
{<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0throw FactoryException(<br>=
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0absl::StrCat("=
;Can't initialize memvfs, with code ", init_code));<br>+ =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0}<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0// Re-r=
egister 'unix' as default filesystem. MemVFS can be used<br>+ =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0// through URL parameters, or by passing as =
the fourth argument to<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0// sqlite3_op=
en_v2().<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0const int register_code =3D=
<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sqlite3_vfs_register(=
sqlite3_vfs_find("unix"), /*makeDflt =3D*/1);<br>+ =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0if (register_code !=3D SQLITE_OK) {<br>+ =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0throw FactoryException(absl::StrCat(<br>+ =C2=A0=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"Can't re-registe=
r the default VFS, with code ", init_code));<br>+ =C2=A0 =C2=A0 =C2=A0=
=C2=A0 =C2=A0}<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return proj_db_creat=
e();<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0}();<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0/=
/ Load the database from a cc_embed_file, using the memvfs<br>+ =C2=A0 =C2=
=A0 =C2=A0 =C2=A0// extension of sqlite.<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0pa=
th =3D absl::StrFormat("file:/proj?ptr=3D0x%x&sz=3D%d&max=3D%d=
&vfs=3Dmemvfs",<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 reinterpret_cas=
t<uintptr_t>(toc[0].data),<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 toc[0=
].size, toc[0].size);<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0flags |=3D SQLITE_OPE=
N_URI;<br>=C2=A0 =C2=A0 =C2=A0}<br>+ =C2=A0 =C2=A0// END GOOGLE MODIFICATIO=
N<br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"g=
mail_attr">On Tue, Oct 1, 2024 at 10:28=E2=80=AFAM Even Rouault via PROJ &l=
t;<a href=3D"mailto:[email protected]">[email protected]</a>> wrot=
e:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0=
.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><u></u>
=20
=20
=20
<div>
Greg,
<blockquote type=3D"cite">
<pre>What's unclear is to what extent things continue to be ok on=
compilers
not supporting C23, and if they are, if those systems start to be second
class. My overall feeling is that it is vastly too early to be
depending on C23.</pre>
</blockquote>
<p>The RFC is clear enough this is an optional feature ("C23 is no=
t
required if EMBED_RESOURCE_FILES is not enabled"). If you need i=
t,
you will need a C23 compiler, which there is already one available
(clang 19), and another one in a near future (gcc 15).</p>
<p>C23 is only used to build the .c files where resources are
embedded.<br>
</p>
<p>If you don't need that new feature, you can use the current
minimum build requirements of GDAL/PROJ.</p>
<p>For once that GDAL/PROJ can use shiny new stuff, let's enjoy :-)=
<br>
</p>
<blockquote type=3D"cite">
<pre>But maybe c23 is c2x, and it's been supported in gcc for 6 y=
ears.</pre>
</blockquote>
<p>No way a gcc from 6 years ago can support #embed: support has
literally landed in GCC master 2 weeks ago.</p>
<span style=3D"white-space:pre-wrap">
</span>
<pre cols=3D"72">--=20
<a href=3D"http://www.spatialys.com" target=3D"_blank">http://www.spatialys=
.com</a>
My software is free, but my time generally not.</pre>
</div>
_______________________________________________<br>
PROJ mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">[email protected].=
org</a><br>
<a href=3D"https://lists.osgeo.org/mailman/listinfo/proj" rel=3D"noreferrer=
" target=3D"_blank">https://lists.osgeo.org/mailman/listinfo/proj</a><br>
</blockquote></div>
--0000000000003944a50623831734--
--===============6592849117019579435==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
gdal-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
--===============6592849117019579435==--