Improve stack smashing protector check

Vicente Olivert Riera <[email protected]>
Newsgroups gmane.comp.tools.sudo.devel
Message-ID <[email protected]>
Hello,

The attached patch is an improvement for the stack smashing protector
check in the configure.ac file.

Regards,

Vincent.

____________________________________________________________
sudo-workers mailing list <[email protected]>
For list information, options, or to unsubscribe, visit:
http://www.sudo.ws/mailman/listinfo/sudo-workers
improve-stack-smashing-protector-check.patch (text/x-patch, 3.6 KB)
# HG changeset patch
# User Vicente Olivert Riera <[email protected]>
# Date 1446039926 0
# Node ID ed64c6fe934bb26dd73430f855275d8644c9be1c
# Parent  97ee37d905ceefa433e93a0f552c2a3e5926e2fb
Improve stack smashing protector check

GCC doesn't generate stack smashing protection check code if the program
we compile doesn't use the stack. Some toolchains do accept the
-fstack-protector{,all,strong} options, but they don't provide libssp.
In that case, our configure script will successfully pass the link test
when it shouldn't. This patch improves the stack smashing protector
check by compiling and linking a test program which does use the stack.

Signed-off-by: Vicente Olivert Riera <[email protected]>

diff -r 97ee37d905ce -r ed64c6fe934b configure.ac
--- a/configure.ac	Sun Oct 25 14:28:38 2015 -0600
+++ b/configure.ac	Wed Oct 28 13:45:26 2015 +0000
@@ -3979,26 +3979,76 @@
 dnl
 if test "$enable_hardening" != "no"; then
     if test -n "$GCC"; then
-	AX_CHECK_COMPILE_FLAG([-fstack-protector-strong], [
-	    AX_CHECK_LINK_FLAG([-fstack-protector-strong], [
-		SSP_CFLAGS="-fstack-protector-strong"
-		SSP_LDFLAGS="-Wc,-fstack-protector-strong"
-	    ])
-	])
+	tmpc=/tmp/tmp.c
+	tmpo=/tmp/tmp.o
+	tmpx=/tmp/tmp.x
+
+	echo -n "checking whether C compiler accepts -fstack-protector-strong... "
+	SSP_CFLAGS="-fstack-protector-strong"
+	echo "int main() { alloca(100); return(1); }" > $tmpc
+	$CC $SSP_CFLAGS -c $tmpc -o $tmpo > /dev/null 2>&1
+	if test $? -ne 0; then
+	    echo "no"
+	    SSP_CFLAGS=
+	else
+	    echo "yes"
+	    echo -n "checking whether the linker accepts -fstack-protector-strong... "
+	    SSP_LDFLAGS="-Wc,-fstack-protector-strong"
+	    $CC $SSP_CFLAGS $SSP_LDFLAGS $tmpo -o $tmpx > /dev/null 2>&1
+	    if test $? -ne 0; then
+		echo "no"
+		SSP_CFLAGS=
+		SSP_LDFLAGS=
+	    else
+		echo "yes"
+	    fi
+	fi
+	rm -f $tmpc $tmpo $tmpx > /dev/null 2>&1
+
 	if test -z "$SSP_CFLAGS"; then
-	    AX_CHECK_COMPILE_FLAG([-fstack-protector-all], [
-		AX_CHECK_LINK_FLAG([-fstack-protector-all], [
-		    SSP_CFLAGS="-fstack-protector-all"
-		    SSP_LDFLAGS="-Wc,-fstack-protector-all"
-		])
-	    ])
+	    echo -n "checking whether C compiler accepts -fstack-protector-all... "
+	    SSP_CFLAGS="-fstack-protector-all"
+	    echo "int main() { alloca(100); return(1); }" > $tmpc
+	    $CC $SSP_CFLAGS -c $tmpc -o $tmpo > /dev/null 2>&1
+	    if test $? -ne 0; then
+		echo "no"
+		SSP_CFLAGS=
+	    else
+		echo "yes"
+		echo -n "checking whether the linker accepts -fstack-protector-all... "
+		SSP_LDFLAGS="-Wc,-fstack-protector-all"
+		$CC $SSP_CFLAGS $SSP_LDFLAGS $tmpo -o $tmpx > /dev/null 2>&1
+		if test $? -ne 0; then
+		    echo "no"
+		    SSP_CFLAGS=
+		    SSP_LDFLAGS=
+		else
+		    echo "yes"
+		fi
+	    fi
+	    rm -f $tmpc $tmpo $tmpx > /dev/null 2>&1
 	    if test -z "$SSP_CFLAGS"; then
-		AX_CHECK_COMPILE_FLAG([-fstack-protector], [
-		    AX_CHECK_LINK_FLAG([-fstack-protector], [
-			SSP_CFLAGS="-fstack-protector"
-			SSP_LDFLAGS="-Wc,-fstack-protector"
-		    ])
-		])
+		echo -n "checking whether C compiler accepts -fstack-protector... "
+		SSP_CFLAGS="-fstack-protector"
+		echo "int main() { alloca(100); return(1); }" > $tmpc
+		$CC $SSP_CFLAGS -c $tmpc -o $tmpo > /dev/null 2>&1
+		if test $? -ne 0; then
+		    echo "no"
+		    SSP_CFLAGS=
+		else
+		    echo "yes"
+		    echo -n "checking whether the linker accepts -fstack-protector... "
+		    SSP_LDFLAGS="-Wc,-fstack-protector"
+		    $CC $SSP_CFLAGS $SSP_LDFLAGS $tmpo -o $tmpx > /dev/null 2>&1
+		    if test $? -ne 0; then
+			echo "no"
+			SSP_CFLAGS=
+			SSP_LDFLAGS=
+		    else
+			echo "yes"
+		    fi
+		fi
+		rm -f $tmpc $tmpo $tmpx > /dev/null 2>&1
 	    fi
 	fi
     fi
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.