Re: Winafsbld

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig626DF8C4C293F2D5D8BA8541
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Mickey:

I agree that the place to start is with a design discussion.  The code
that has been submitted to Gerrit had no such discussion and therefore
while it meets your personal needs it does not meet the general needs of
OpenAFS.

Some requirements:

1. The tool must be such that it can be used to drive the Windows
buildbot instances as well as the actual OpenAFS release build process.

2. The tool must not be edited by the user for a particular
configuration.  Configuration details should be configured external to
the script.  This can be in the form of site specific script that
generates the build environment in which winafsbld executes or in a
configuration file that is parsed by the winafsbld script.  This is
important because it permits "git clean -x -d -f" to be used to ensure
that each build starts from a clean build tree.

3. The tool should validate that all of the build dependencies are
present and consistent with one another.

 . compiler
 . windows sdk
 . windows ddk
 . IDN SDK
 . wix toolkit
 . nsis toolkit
 . perl
 . awk
 . docbook, xsltproc, etc.
 . signtool
 . other ...


4. The tool must be knowledgeable of the legal combination of compilers,
sdks, and ddks.  With explicit support for:

 . VS2003 (CL 1310)
 . VS2005 (CL 1400)
 . VS2008 (CL 1500)
 . VS2010.

 . XP SP2 SDK
 . 2003 SP1 SDK
 . 6.0/6.0a SDK (Vista)
 . 6.1 SDK (Vista SP1/2008)
 . 7.0 SDK
 . 7.1 SDK

 . 3790.1830 DDK
 . 6001.18002 WDK
 . 7600.16385.0 WDK

5. The tool should be knowledgeable of target platforms.  If the target
is Windows 2000 it should not permit VS2008, 7.1 SDK, and the
7600.16385.0 WDK to be used to build OpenAFS.

6. The tool should drive builds with/without codesigning/timestamping,
symbol storing, documentation, installers, etc.

---

There are two basic approaches that I can see using to meet these
requirements.  The first is the approach followed in gerrit 3332 where
an additional layer of batch scripting is used.  The other approach
would be a tool that is used to generate the ntbuild.bat script based
upon configuration inputs and an examination of the current system.

My personal preference would be to see a tool that examines a system,
auto-detects which packages are installed, and generates the best
configuration for the target platform.

Some comments on your bullets below:

On 11/29/2010 10:02 AM, Mickey Lane wrote:

> First) Having the URL of the time stamp server in the batch file.
>=20
> This is easy enough to move to an external environment variable. There
> is no need to use a specific server with a specific signing certificate=

> that I=E2=80=99m aware of. On at least one occasion, the preferred serv=
er was
> unavailable or taking too long to respond so I switched to another and
> everything worked fine. I left the initial server URL commented out so =
I
> wouldn=E2=80=99t have to look it up again.

There are already a set of environment variables that are defined and
used by the build system.  I'm not sure I see a reason to construct a
parallel name space that is only used to generate the variables that are
eventually used by the existing build system.

Why is there a WINAFSBLD__CODESIGN_TIMESTAMP_FOO that is later going to
be copied to CODESIGN_TIMESTAMP_FOO ?

Things that are site specific in nature should not be committed as part
of the script.


> Second) The requirement that a file needs to be edited to specify build=

> options.
>=20
> The design goal of the current code was to put all of the adjustable
> things at the top of the main script so the user only had part of one
> file to review and edit. All of the adjustments are in a single block o=
f
> code (lines 17-25). The main script doesn=E2=80=99t really do anything =
except
> call the subroutine scripts.
>=20
> I would like to know what would be preferable. I=E2=80=99m pretty sure =
the
> implementation wouldn=E2=80=99t be that difficult.
>=20
> Two options that come to mind are multiple top level files with the
> names reflecting the function and command line arguments to a single
> file. I=E2=80=99m sure many others exist.

It not just preferable.  Its sort of a requirement.  The scripts that
are committed to the git repository must not require that they be
altered in order to build the tree.  If they are, those changes will
either be lost during a git clean or accidentally checked in during a
git commit.

> Third) 32 vs 64-bit builds.
>=20
> Winafsbld works on either platform provided you do 64-bit builds on a
> 64-bit system and a 32 on a 32. The original design goal was to enable
> cross platform builds but there are a number of problems with that at
> the moment.

The underlying build system doesn't support cross platform builds so it
doesn't matter what you layer on top of it.

> The intent of the winafsbld_everything script is to replace the
> winafsbld script and iteratively set the variables and call the
> subroutine scripts. Jeffry Altman points out some issues in the
> everything script that need to be resolved.

I would punt on this as it is simply not practical.

> Fourth) Timestamping
>=20
> The line =E2=80=9Crem set WINAFSBLD_CONFIG_TIMESTAMP=3DNO    (not funct=
ional yet)=E2=80=9D
> in winafsbld.bat needs to be removed. This was added as part of a
> request by Mike Meffie to disable time stamping for build verifications=

> on a machine that apparently couldn=E2=80=99t see a public time stamp s=
erver.
> Setting =E2=80=9CWINAFSBLD_CONFIG_SIGN=3DNO=E2=80=9D serves the same pu=
rpose.

If the goal is to support disabling the use of time stamp servers during
the build while continuing to permit the use of code signing, this would
need to be done in the build system itself.  The build system at present
either supports code signing with timestamping or no code signing at
all.  Such a required change is "do not specify a timestamp server and
signtool should not be called with /t".

> Fifth) Environment variable WINAFSBLD__VC2005
> =20
> From Jeffery Altman=E2=80=99s comments=E2=80=A6
> =20
> In all the time I=E2=80=99ve been using this script, I never noticed th=
at =E2=80=9CVC=E2=80=9D
> should be =E2=80=9CVS.=E2=80=9D I will change it. At the moment, the VC=
2008 variable is
> not needed if the build is using Visual Studio 2005. Likewise, if Visua=
l
> Studio 2008 builds ever become supported, the VC2005 environment will
> not be needed.

My question wasn't whether one or the other is required when the other
is being used.  My question was what is the behavior when both are
specified?  A tool should be able to determine which visual studio
compilers are available by examining the system.


> Sixth) Possible combinations of VS, SDK & DDK
>=20
> This is a major issue and one that winafsbld seems to do rather well
> with. The setting of environment variables for include and library
> locations are dealt with by the winafsbld_include and winafsbld_lib
> subroutine scripts respectively.

Please note that the build system uses AFSBLD_TYPE/AFSDEV_BUILDTYPE,
AFSDEV_CRTDEBUG and AFSVER_CL to determine the type of build and
AFSDEV_BIN, AFSDEV_INCLUDE, and AFSDEV_LIB to specify the applicable
search paths.  AFSDEV_BIN is important because it means that the PATHs
constructed by the various visual studio, sdk, and wdk setup scripts are
not required.

> These files take a rather ponderous
> approach to the problem by defining specific sections to each of the
> major permutations.=20

In other words, it has the equivalent of a separate ntbuild.bat file for
each platform/vs/ddk/wdk combination that is supported.

> I haven=E2=80=99t looked at it in a while but in at least
> one case, the script goes hunting for the paths to use based on what=E2=
=80=99s
> installed on the build machine.
>=20
> Debugging this sort of thing is very tedious but at least what=E2=80=99=
s done
> for one configuration won=E2=80=99t impact any of the others. If winafs=
bld is
> accepted, work needs to be done in this area.

Jeffrey Altman


--------------enig626DF8C4C293F2D5D8BA8541
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)

iQEcBAEBAgAGBQJM8/pjAAoJENxm1CNJffh4a4cIAIMqbCTOB7xok/HiH+D7Tt/f
XnvYSTv292cINr3k2Pt1z4LFJxchZQoZNzHBsP9AAaGQuN+E3jsOKfDIGiCbhqEC
4ue2MWmERWEbx35DGf38l1Gh+7LK1EY+G4jYiJ5IdTBdUhtu3JaOT//IcThhT6RD
RIslyAydElVG3beoUZR1peiAt0gPK35O7Q8Sm2aZi8yFZ/LxEvUFDsQxjEDrI+Td
ShWw/on7pGJ/B8HeqYgOn/WzKr8nxC63L942u1B94ykhgPheq1PeHCfKXKMA11yr
sf98FpMBEBwNB36u/SQzhIjBdP/D8+czHR2FgYK5kykw9zNJG+WcLFpmItSgRSA=
=9Ea+
-----END PGP SIGNATURE-----

--------------enig626DF8C4C293F2D5D8BA8541--