autobuild: harden samba-xc test suite

Add more checks which directly test the behavior of
--cross-answers and --cross-execute.

Previous test tested things in a round-about way, checking
that running in all three modes (native, cross-execute,
cross-answers) yields the same result. It was vulnerable
to a degradation in which cross-compilation modes didn't
work at all and were running native tests, which is
what happened with the upgrade of waf.

The added tests check the following:
- That cross-excute with cross-answers sets the cross-answers file
- That the content of cross-answers file actually affects the build
  configuration
- That a missing line in cross-answers fails the build

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13846

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Uri Simchoni <uri@samba.org>
Autobuild-Date(master): Sun Oct 20 13:29:58 UTC 2019 on sn-devel-184

diff --git a/script/autobuild.py b/script/autobuild.py
index e4a36aab34e568f226f8ec873a853280ba25d8e5..cc2b86181dbe2499bdd214f91ec8b78d063afab1 100755 (executable)
--- a/script/autobuild.py
+++ b/script/autobuild.py
@@ -483,12 +483,22 @@ tasks = {
         ("configure-native", "./configure.developer --with-selftest-prefix=./bin/ab" + samba_configure_params),
         ("configure-cross-execute", "./configure.developer --out ./bin-xe --cross-compile --cross-execute=script/identity_cc.sh" \
             " --cross-answers=./bin-xe/cross-answers.txt --with-selftest-prefix=./bin-xe/ab" + samba_configure_params),
+        ("verify-cross-execute-output", "grep '^Checking value of NSIG' ./bin-xe/cross-answers.txt"),
         ("configure-cross-answers", "./configure.developer --out ./bin-xa --cross-compile" \
             " --cross-answers=./bin-xe/cross-answers.txt --with-selftest-prefix=./bin-xa/ab" + samba_configure_params),
         ("compare-results", "script/compare_cc_results.py "
             "./bin/c4che/default{} "
             "./bin-xe/c4che/default{} "
             "./bin-xa/c4che/default{}".format(*([CACHE_SUFFIX]*3))),
+        ("modify-cross-answers", "sed -i.bak -e 's/^\\(Checking value of NSIG:\\) .*/\\1 \"1234\"/' ./bin-xe/cross-answers.txt"),
+        ("configure-cross-answers-modified", "./configure.developer --out ./bin-xa2 --cross-compile" \
+            " --cross-answers=./bin-xe/cross-answers.txt --with-selftest-prefix=./bin-xa2/ab" + samba_configure_params),
+        ("verify-cross-answers", "test $(sed -n -e 's/VALUEOF_NSIG = \\(.*\\)/\\1/p' ./bin-xa2/c4che/default{})" \
+            " = \"'1234'\"".format(CACHE_SUFFIX)),
+        ("invalidate-cross-answers", "sed -i.bak -e '/^Checking value of NSIG/d' ./bin-xe/cross-answers.txt"),
+        ("configure-cross-answers-fail", "./configure.developer --out ./bin-xa3 --cross-compile" \
+            " --cross-answers=./bin-xe/cross-answers.txt --with-selftest-prefix=./bin-xa3/ab" + samba_configure_params + \
+            " ; test $? -ne 0"),
         ],
 
     # test build with -O3 -- catches extra warnings and bugs, tests the ad_dc environments