Re: build for s390x system

On Thu, Sep 26, 2013 at 7:24 AM, Tsutomu Oyamada <[email protected]>wrote:

> Hi, Dave
>
> I've re-build of clamAV.
> As a result, confirmed that work just fine.
> Your advice was valid.
> Thank you very much.
>
> T.Oyamada
>
> On Thu, 26 Sep 2013 09:07:53 +0900
> Tsutomu Oyamada <[email protected]> wrote:
>
> > Hi, Dave.
> >
> > Thanks your advice.
> >
> > I changed type of fp_digit.
> > and I tried test program.
> >
> > fp_digit: 4
> > unsigned long: 8
> > FP_64BIT not defined
> > DIGIT_BIT: 32
> > CHAR_BIT: 8
> > FP_MAX_SIZE: 8448
> > FP_SIZE: 264
> > TFM_ASM not defined
> > fp_word: 8
> > ulong64: 8
> > unsigned long long: 8
> > CRYPT not defined
> >
> > It works good.
> > I will try re-compile clamav.
> > Later report the results.
> >
> > T.Oyamada
> >
> > On Wed, 25 Sep 2013 17:13:58 -0400
> > David Raynor <[email protected]> wrote:
> >
> > > On Wed, Sep 25, 2013 at 2:55 PM, David Raynor <[email protected]
> >wrote:
> > >
> > > >
> > > > On Tue, Sep 24, 2013 at 9:47 PM, Tsutomu Oyamada <
> [email protected]>wrote:
> > > >
> > > >> Hi, Dave.
> > > >>
> > > >> Thanks for your advice and quick respons.
> > > >> I tried it.
> > > >>
> > > >> ----
> > > >> fp_digit: 8
> > > >> unsigned long: 8
> > > >> FP_64BIT not defined
> > > >> DIGIT_BIT: 64
> > > >> CHAR_BIT: 8
> > > >> FP_MAX_SIZE: 8704
> > > >> FP_SIZE: 136
> > > >> TFM_ASM not defined
> > > >> fp_word: 8
> > > >> ulong64: 8
> > > >> unsigned long long: 8
> > > >> CRYPT not defined
> > > >> ----
> > > >>
> > > >> What should I just do in order to fix this problem?
> > > >> Please teach me how to set size of fp_word to 16.
> > > >>
> > > >> T.Oyamada
> > > >>
> > > >> On Tue, 24 Sep 2013 18:16:26 -0400
> > > >> David Raynor <[email protected]> wrote:
> > > >>
> > > >> > On Tue, Sep 24, 2013 at 4:37 PM, David Raynor <
> [email protected]
> > > >> >wrote:
> > > >> >
> > > >> > > On Tue, Sep 24, 2013 at 2:05 AM, Tsutomu Oyamada <
> > > >> [email protected]>wrote:
> > > >> > >
> > > >> > >> Hi,
> > > >> > >>
> > > >> > >> I investigated the value using the following programs.
> > > >> > >>
> > > >> > >> ----
> > > >> > >> #include "stdlib.h"
> > > >> > >> #include "bignum_fast.h"
> > > >> > >>
> > > >> > >> int main(int argc, char **argv) {
> > > >> > >>
> > > >> > >>         printf("fp_digit: %d\n",sizeof(fp_digit));
> > > >> > >>         printf("unsigned long: %d\n",sizeof(unsigned long));
> > > >> > >>
> > > >> > >> #ifdef FP_64BIT
> > > >> > >>         printf("FP_64BIT: %d\n",FP_64BIT);
> > > >> > >> #else
> > > >> > >>         printf("FP_64BIT not defined\n");
> > > >> > >> #endif
> > > >> > >>
> > > >> > >> #ifdef DIGIT_BIT
> > > >> > >>         printf("DIGIT_BIT: %d\n",DIGIT_BIT);
> > > >> > >> #else
> > > >> > >>         printf("DIGIT_BIT not defined\n");
> > > >> > >> #endif
> > > >> > >>
> > > >> > >> #ifdef CHAR_BIT
> > > >> > >>         printf("CHAR_BIT: %d\n",CHAR_BIT);
> > > >> > >> #else
> > > >> > >>         printf("CHAR_BIT not defined\n");
> > > >> > >> #endif
> > > >> > >>
> > > >> > >> #ifdef FP_MAX_SIZE
> > > >> > >>         printf("FP_MAX_SIZE: %d\n",FP_MAX_SIZE);
> > > >> > >> #else
> > > >> > >>         printf("FP_MAX_SIZE not defined\n");
> > > >> > >> #endif
> > > >> > >>
> > > >> > >> #ifdef FP_SIZE
> > > >> > >>         printf("FP_SIZE: %d\n",FP_SIZE);
> > > >> > >> #else
> > > >> > >>         printf("FP_SIZE not defined\n");
> > > >> > >> #endif
> > > >> > >>
> > > >> > >> #ifdef TFM_ASM
> > > >> > >>         printf("TFM_ASM: defined\n");
> > > >> > >> #else
> > > >> > >>         printf("TFM_ASM not defined\n");
> > > >> > >> #endif
> > > >> > >>
> > > >> > >>         exit(0);
> > > >> > >> }
> > > >> > >> ----
> > > >> > >>
> > > >> > >> The result was as follows.
> > > >> > >>
> > > >> > >> fp_digit: 8
> > > >> > >> unsigned long: 8
> > > >> > >> FP_64BIT not defined
> > > >> > >> DIGIT_BIT: 64
> > > >> > >> CHAR_BIT: 8
> > > >> > >> FP_MAX_SIZE: 8704
> > > >> > >> FP_SIZE: 136
> > > >> > >> TFM_ASM not defined
> > > >> > >>
> > > >> > >> Can you find a problem by this result?
> > > >> > >>
> > > >> > >> Thanks.
> > > >> > >>
> > > >> > >> --
> > > >> > >> T.Oyamada
> > > >> > >>
> > > >> > >> _______________________________________________
> > > >> > >> http://lurker.clamav.net/list/clamav-devel.html
> > > >> > >> Please submit your patches to our Bugzilla:
> http://bugs.clamav.net
> > > >> > >>
> > > >> > >
> > > >> > > I do not have an immediate fix, but that information does give
> me some
> > > >> > > leads.
> > > >> > >
> > > >> > > Basic issue: the tomsfastmath code must be falling through to
> the code
> > > >> > > block on lines 280-302 of fp_mul_comba.c.
> > > >> > > 1) The right shift causing the warning is DIGIT_BIT (64).
> > > >> > > 2) The datatype being shifted is fp_word.
> > > >> > > 3) fp_word defined as "typedef ulong64 fp_word;" from
> bignum_fast.h
> > > >> line
> > > >> > > 253
> > > >> > > 4) ulong64 defined as "typedef unsigned long long ulong64;" from
> > > >> > > bignum_fast.h line 248
> > > >> > >
> > > >> > > I think the problem is one of three issues:
> > > >> > > A) fp_word is not defined as a 64-bit datatype.
> > > >> > > B) line 301 of tomsfastmath/mul/fp_mul_comba.c is mistakenly
> > > >> downcasting t
> > > >> > > from fp_word to fp_digit before shifting.
> > > >> > > C) s390 is not allowing use of all 64 bits of fp_word.
> > > >> > >
> > > >> > > Problems A or B are easier fixed than C.
> > > >> > >
> > > >> > > Please add these lines to your test and re-run:
> > > >> > >
> > > >> > > printf("fp_word: %d\n",sizeof(fp_word));
> > > >> > > printf("ulong64: %d\n",sizeof(ulong64));
> > > >> > > printf("unsigned long long: %d\n",sizeof(unsigned long long));
> > > >> > > #ifdef CRYPT
> > > >> > >         printf("CRYPT: defined\n");
> > > >> > > #else
> > > >> > >         printf("CRYPT not defined\n");
> > > >> > > #endif
> > > >> > >
> > > >> > > I would like to see the config.log file generated by running
> > > >> configure. It
> > > >> > > would also be useful to have the full output from running make.
> The
> > > >> log
> > > >> > > snip shows line 91, but I expect that it first warned about
> line 15.
> > > >> > >
> > > >> > > The easiest way to continue and share logfiles is via Bugzilla.
> Please
> > > >> > > open a bug report on bugzilla.clamav.net on this issue. You
> can then
> > > >> > > attach the files to that bug.
> > > >> > >
> > > >> > > Hope this helps,
> > > >> > >
> > > >> > >
> > > >> > > Dave R.
> > > >> > >
> > > >> > > --
> > > >> > > ---
> > > >> > > Dave Raynor
> > > >> > > Sourcefire Vulnerability Research Team
> > > >> > > [email protected]
> > > >> > >
> > > >> >
> > > >> > I think I read something wrong, and I think I have an idea. It
> still
> > > >> needs
> > > >> > to be confirmed by the additional test lines above. Re-reading the
> > > >> output,
> > > >> > I think that the code is depending on fp_word being twice the
> size of
> > > >> > fp_digit. Based on fp_digit size of 8 that means it would expect
> it to
> > > >> be
> > > >> > 16. If sizeof(fp_word) is resolving to 8 and sizeof(fp_digit) is
> 8, that
> > > >> > could be the problem. Then the fix would depend on whether
> fp_word can
> > > >> be
> > > >> > made size 16 or must be constrained to size 8.
> > > >> >
> > > >> > Let me know what you find,
> > > >> >
> > > >> > Dave R.
> > > >> >
> > > >> > --
> > > >> > ---
> > > >> > Dave Raynor
> > > >> > Sourcefire Vulnerability Research Team
> > > >> > [email protected]
> > > >> > _______________________________________________
> > > >> > http://lurker.clamav.net/list/clamav-devel.html
> > > >> > Please submit your patches to our Bugzilla:
> http://bugs.clamav.net
> > > >> >
> > > >>
> > > >>
> > > >> _______________________________________________
> > > >> http://lurker.clamav.net/list/clamav-devel.html
> > > >> Please submit your patches to our Bugzilla: http://bugs.clamav.net
> > > >>
> > > >
> > > > From what I have read, gcc on s390 has a compiler option for setting
> the
> > > > size of "long double" type (64 or 128), but not "long long int". I
> expect
> > > > what we have to do instead is leave fp_word as 64 and tune down the
> > > > fp_digit to 32. That code block in bignum_fast.h is taken from
> tomsfastmath
> > > > and looks like it is aimed at 32-bit support anyway. This changes the
> > > > maximum ranges of the math calculations, but should be sufficient
> for our
> > > > needs (again, similar code runs on 32-bit).
> > > >
> > > > Please try changing this line: (bignum_fast.h:line 252)
> > > >    typedef unsigned long      fp_digit;
> > > > to this line:
> > > >    typedef unsigned int      fp_digit;
> > > >
> > > > If this is successful for you, we can determine the appropriate
> defines to
> > > > use to make this more portable.
> > > >
> > > > Dave R.
> > > >
> > > > --
> > > > ---
> > > > Dave Raynor
> > > > Sourcefire Vulnerability Research Team
> > > > [email protected]
> > > >
> > >
> > > Ideally, you should get these results from the test program after the
> > > change:
> > >
> > > fp_digit: 4
> > > unsigned long: 8
> > > FP_64BIT not defined
> > > DIGIT_BIT: 32
> > > CHAR_BIT: 8
> > > FP_MAX_SIZE: 8448
> > > FP_SIZE: 264
> > > TFM_ASM not defined
> > > fp_word: 8
> > > ulong64: 8
> > > unsigned long long: 8
> > > CRYPT not defined
> > >
> > > Except for the sizeof(unsigned long), this would match the values I
> see on
> > > a 32-bit test box.
> > >
> > > Dave R.
> > >
> > > --
> > > ---
> > > Dave Raynor
> > > Sourcefire Vulnerability Research Team
> > > [email protected]
> > > _______________________________________________
> > > http://lurker.clamav.net/list/clamav-devel.html
> > > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> > >
> >
> >
> > _______________________________________________
> > http://lurker.clamav.net/list/clamav-devel.html
> > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >
>
>
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>


Great! I am glad it works. I have recorded all of this in Bugzilla for
reference as bug #9017, including the fix.

This issue should also occur for any use of the tomsfastmath code on your
platform, based on the code in tomsfastmath/src/headers/tfm.h. For ClamAV
purposes, we do have type size checks run by configure that create defined
values we can reference.

Dave R.

-- 
---
Dave Raynor
Sourcefire Vulnerability Research Team
[email protected]
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net