script: Add 'random-seed' option to traffic_replay

When using a traffic-model file to generate traffic, there is some
randomness in the actual packets that get generated. This means it's
hard to use the tool to detect an increase/decrease in Samba
performance - we don't know whether a decrease in packets sent is due
to a regression in the Samba codebase, or just due to the tool sending
different types of packets (i.e. ones that take longer to process).

This patch adds an option to seed the python random number generator.
This means that exactly the same traffic can be generated across
multiple test runs.

(Previously we were using the '--traffic-summary' option to avoid this
problem - we can generate a summary-file based on the model, and then
use the same summary file across multiple runs. However, this proved
impractical when you want to run multiple combinations of scale/rate
parameters, e.g. 21 x 8 different permutations just fills up disk space
with summary-files.)

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: William Brown <william@blackhats.net.au>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 16 13:53:26 CEST 2018 on sn-devel-144

diff --git a/docs-xml/manpages/traffic_replay.7.xml b/docs-xml/manpages/traffic_replay.7.xml
index d7d09a24ba9853550b57526df26367372e868e53..da8da5cf3c6b7417a312f309d726ea02b31a366c 100644 (file)
--- a/docs-xml/manpages/traffic_replay.7.xml
+++ b/docs-xml/manpages/traffic_replay.7.xml
@@ -30,6 +30,7 @@
                <arg choice="opt">-B, --badpassword-frequency &lt;frequency&gt;</arg>
                <arg choice="opt">--dns-rate &lt;rate&gt;</arg>
                <arg choice="opt">-t, --timing-data &lt;file&gt;</arg>
+               <arg choice="opt">--random-seed &lt;seed&gt;</arg>
                <arg choice="opt">-U, --username user</arg>
                <arg choice="opt">--password &lt;password&gt;</arg>
                <arg choice="opt">-W --workgroup &lt;workgroup&gt;</arg>
@@ -136,6 +137,16 @@
        </para></listitem>
        </varlistentry>
 
+       <varlistentry>
+       <term>random-seed</term>
+       <listitem><para>
+       A number to seed the random number generator with. When traffic is
+       generated from a model-file, use this option to keep the traffic
+       consistent across multiple test runs. This allows you to compare the
+       performance of Samba between different releases.
+       </para></listitem>
+       </varlistentry>
+
        <varlistentry>
        <term>Traffic Model Options</term>
        <listitem><para>
@@ -187,7 +198,7 @@
                <para>
                For taking performance measurements over several test runs,
                it's recommended to use this option and replay the traffic from
-               a traffic-summary file.
+               a traffic-summary file, or to use the --random-seed option.
                </para></listitem>
                </varlistentry>
        </itemizedlist>
@@ -474,7 +485,8 @@
        <para>The traffic generated is an approximation of what was seen in
        the network capture. The traffic generation involves some randomness,
        so running the same command multiple times may result in slightly
-       different traffic being generated.</para>
+       different traffic being generated (although you can avoid this, by
+       specifying the --random-seed option).</para>
 
        <para>As well as changing how long the model runs for, you can also
        change how many conversations get generated and how fast the traffic

diff --git a/script/traffic_replay b/script/traffic_replay
index 6f42f2d68cd5b0e535c46e23ef58b7a36c5c2ec5..764462e5be3c10d99c404aae31d2bea996de7865 100755 (executable)
--- a/script/traffic_replay
+++ b/script/traffic_replay
@@ -22,6 +22,7 @@ import os
 import optparse
 import tempfile
 import shutil
+import random
 
 sys.path.insert(0, "bin/python")
 
@@ -67,6 +68,8 @@ def main():
     parser.add_option('-c', '--clean-up',
                       action="store_true",
                       help='Clean up the generated groups and user accounts')
+    parser.add_option('--random-seed', type='int', default=0,
+                      help='Use to keep randomness consistent across multiple runs')
 
     model_group = optparse.OptionGroup(parser, 'Traffic Model Options',
                                        'These options alter the traffic '
@@ -155,6 +158,9 @@ def main():
                              " for the users created as part of this test"))
         sys.exit(1)
 
+    if opts.random_seed:
+        random.seed(opts.random_seed)
+
     lp = sambaopts.get_loadparm()
     creds = credopts.get_credentials(lp)
     creds.set_gensec_features(creds.get_gensec_features() | gensec.FEATURE_SEAL)