Re: gtags cannot generate more than 2G files

Hi. 

Will you check whether the attached patch solves your problem?
If off_t is wider than long, casting will be done before multiplication to avoid integer overflow

On Wed, 27 Sep 2017 13:08:02 +0000
"Wang, Lei AO. (NSB - CN/Qingdao)" <[email protected]> wrote:

> ok, thanks for your clear response.
> 
> Thanks,
> Zach from smart phone.
> 
> -----Original Message-----
> From: Hideki IWAMOTO [[email protected]]
> Received: 星期三, 27 9 月 2017, 9:05下午
> To: Wang, Lei AO. (NSB - CN/Qingdao) [[email protected]]
> CC: Shigio YAMAGUCHI [[email protected]]; [email protected] [[email protected]]
> Subject: Re: gtags cannot generate more than 2G files
> 
> Hi.
> 
> As long as you use the Berkeley DB format, Gtags can not handle tag files larger than
> 4G on 32 bit system.
> This is because the file offset is computed using unsigned 32 bits and then catsted to off_t.
> You can avoid the 2G limit by using 64 bit off_t, but 4G limit
> can not be avoided.
> 
> 
> On Wed, 27 Sep 2017 06:33:03 +0000
> "Wang, Lei AO. (NSB - CN/Qingdao)" <[email protected]> wrote:
> 
> > Supporter,
> > Seemed the gtags is not stability, after the gtags index expand to 4294967296. It will report “GTAGS seems older format. Please remake tag files”.
> > I could confirm the GTAGS index match with newest global.
> > I try to cut my project, gtags index may be 3294067296. The error report will be gone.
> >
> >
> >
> >
> >
> >
> > Thanks,
> > Zach
> > QD ONT Voice SW
> > x5606
> >
> > From: Wang, Lei AO. (NSB - CN/Qingdao)
> > Sent: 2017年9月27日 9:42
> > To: 'Shigio YAMAGUCHI'
> > Cc: [email protected]
> > Subject: RE: gtags cannot generate more than 2G files
> >
> > Supporter,
> > Seemed the gtags will use the parent GTAGS index to search code if there is.
> >
> > Normally, I think it should use the GTAGS index in current path.
> >
> > After I delete the GTAGS index of parent, it works now.
> >
> >
> >
> >
> >
> >
> >
> >
> > Thanks,
> > Zach
> > QD ONT Voice SW
> > x5606
> >
> > From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Shigio YAMAGUCHI
> > Sent: 2017年9月27日 5:54
> > To: Wang, Lei AO. (NSB - CN/Qingdao)
> > Cc: [email protected]<mailto:[email protected]>
> > Subject: Re: gtags cannot generate more than 2G files
> >
> > Hi,
> > If this report is unrelated to the 2G bug, could you
> > make a new thread? (with new subject, without Re:)
> > Thank you in advance.
> >
> > Regards,
> > Shigio
> >
> > 2017-09-26 23:54 GMT+09:00 Wang, Lei AO. (NSB - CN/Qingdao) <[email protected]<mailto:[email protected]>>:
> > Supporter,
> > Another but with version 6.5.7.
> > The GTAGS and GRTAGS are expanded to 4294967296 end.
> > All the file paths are not correct, they are the parent path.
> >
> > Like the file is “root/app/private/test.c”, the gtags path is “parent/root/app/private/test.c”.
> >
> > So, the test.c cannot display correctly.
> >
> >
> >
> >
> >
> >
> > Thanks,
> > Zach
> > QD ONT Voice SW
> > x5606
> >
> > From: Wang, Lei AO. (NSB - CN/Qingdao)
> > Sent: 2017年9月26日 16:36
> > To: 'Shigio YAMAGUCHI'
> > Cc: '[email protected]<mailto:[email protected]>'
> > Subject: RE: gtags cannot generate more than 2G files
> >
> > Supporter,
> > Finally, I found the solution that use the newest version and configure 64 bit.
> >
> >
> >
> >
> >
> >
> > Thanks,
> > Zach
> > QD ONT Voice SW
> > x5606
> >
> > From: Wang, Lei AO. (NSB - CN/Qingdao)
> > Sent: 2017年9月26日 14:22
> > To: 'Shigio YAMAGUCHI'
> > Cc: '[email protected]<mailto:[email protected]>'
> > Subject: RE: gtags cannot generate more than 2G files
> >
> > Hello supporter.
> > I tried to generate 4G file in my system, and it successes.
> > So, that means my system could support the larger files.
> >
> > May be the configuration of gtags is still not correct?
> >
> > ./configure CFLAGS='-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64'
> >
> >
> >
> >
> >
> >
> >
> > Thanks,
> > Zach
> > QD ONT Voice SW
> > x5606
> >
> > From: Wang, Lei AO. (NSB - CN/Qingdao)
> > Sent: 2017年9月25日 9:56
> > To: 'Shigio YAMAGUCHI'
> > Cc: [email protected]<mailto:[email protected]>
> > Subject: RE: gtags cannot generate more than 2G files
> >
> > Hi,
> > Seemed my GNU system is 32 bit.
> > Any solution for this?
> >
> >
> >
> >
> >
> >
> > Thanks,
> > Zach
> > QD ONT Voice SW
> > x5606
> >
> > From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Shigio YAMAGUCHI
> > Sent: 2017年9月25日 6:16
> > To: Wang, Lei AO. (NSB - CN/Qingdao)
> > Cc: [email protected]<mailto:[email protected]>
> > Subject: Re: gtags cannot generate more than 2G files
> >
> > Hi,
> > > After my project is larger, the gtags will report “dbop_put failedd
> > >  when the GTAGS exceed 2G.  Could you help to solve this?
> >
> > If you use GNU system, does the Q4 in the FAQ help you?
> > (FAQ file is included in GLOBAL package)
> >
> > ----------------------------------------------------------------------------
> > Q4. I'm using GNU system. In a large project, gtags fails in making tags
> >     like follows:
> >
> >         $ gtags
> >         gtags: cannot write to database.
> >         $ _
> >
> >     File system is not full. It seems that gtags cannot make a file over 2GB.
> >     Any solution?
> >
> > A4. If your GNU system supports 64-bit file offset then please try the
> >     following configuration.
> >
> >         $ ./configure CFLAGS='-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64'
> > ----------------------------------------------------------------------------
> >
> > Regards,
> > Shigio
> >
> >
> > 2017-09-24 14:14 GMT+09:00 Wang, Lei AO. (NSB - CN/Qingdao) <[email protected]<mailto:[email protected]>>:
> > Hi,
> > After my project is larger, the gtags will report “dbop_put failed” when the GTAGS exceed 2G.
> > Could you help to solve this?
> >
> >
> >
> >
> >
> >
> > Thanks,
> >
> >
> > _______________________________________________
> > Bug-global mailing list
> > [email protected]<mailto:[email protected]>
> > https://lists.gnu.org/mailman/listinfo/bug-global
> >
> >
> >
> > --
> > Shigio YAMAGUCHI <[email protected]<mailto:[email protected]>>
> > PGP fingerprint:
> > 26F6 31B4 3D62 4A92 7E6F  1C33 969C 3BE3 89DD A6EB
> >
> >
> >
> > --
> > Shigio YAMAGUCHI <[email protected]<mailto:[email protected]>>
> > PGP fingerprint:
> > 26F6 31B4 3D62 4A92 7E6F  1C33 969C 3BE3 89DD A6EB
> 
> --
> Hideki IWAMOTO <[email protected]>
> 

-- 
岩本 秀樹 <[email protected]>

_______________________________________________
Bug-global mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/bug-global

Index: configure.ac
===================================================================
RCS file: /sources/global/global/configure.ac,v
retrieving revision 1.195
diff -u -p -r1.195 configure.ac
--- configure.ac	15 May 2017 06:15:00 -0000	1.195
+++ configure.ac	27 Sep 2017 13:34:53 -0000
@@ -115,6 +115,8 @@ AC_CHECK_SIZEOF(char)
 if test ${ac_cv_sizeof_char} != 1; then
 	AC_MSG_ERROR([Char size isn't 8 bit.])
 fi
+AC_CHECK_SIZEOF(long)
+AC_CHECK_SIZEOF(off_t)
 
 dnl Checks for library functions.
 AC_FUNC_ALLOCA
Index: libdb/mpool.c
===================================================================
RCS file: /sources/global/global/libdb/mpool.c,v
retrieving revision 1.8
diff -u -p -r1.8 mpool.c
--- libdb/mpool.c	13 Oct 2012 07:01:21 -0000	1.8
+++ libdb/mpool.c	27 Sep 2017 13:34:53 -0000
@@ -229,7 +229,11 @@ mpool_get(mp, pgno, flags)
 #ifdef STATISTICS
 	++mp->pageread;
 #endif
+#if SIZEOF_OFF_T > SIZEOF_LONG
+	off = (off_t)mp->pagesize * (off_t)pgno;
+#else
 	off = mp->pagesize * pgno;
+#endif
 #ifdef HAVE_PREAD
 	if ((nr = pread(mp->fd, bp->page, mp->pagesize, off)) != mp->pagesize) {
 		if (nr >= 0)
@@ -425,7 +429,11 @@ mpool_write(mp, bp)
 	if (mp->pgout)
 		(mp->pgout)(mp->pgcookie, bp->pgno, bp->page);
 
+#if SIZEOF_OFF_T > SIZEOF_LONG
+	off = (off_t)mp->pagesize * (off_t)bp->pgno;
+#else
 	off = mp->pagesize * bp->pgno;
+#endif
 #ifdef HAVE_PWRITE
 	if (pwrite(mp->fd, bp->page, mp->pagesize, off) != mp->pagesize)
 		return (RET_ERROR);