STEP20 - TODO smb2_tcon: report share as scale out if 'scale out share = yes'

TODO: behavioural changes

Pair-Programmed-With: Guenther Deschner <gd@samba.org>
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Pair-Programmed-With: Michael Adam <obnox@samba.org>

STEP20 - TODO smb2_tcon: report share as CA if 'continuously available share = yes'

TODO: Need to add change in behaviour

Pair-Programmed-With: Guenther Deschner <gd@samba.org>
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Pair-Programmed-With: Michael Adam <obnox@samba.org>

STEP20 - TODO smb2_tcon: report share as cluster share if "cluster share = true"

TODO: No further behaviour change yet...

Pair-Programmed-With: Guenther Deschner <gd@samba.org>
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Pair-Programmed-With: Michael Adam <obnox@samba.org>

STEP20 - param: add share parameter "scale out share"

Signed-off-by: Michael Adam <obnox@samba.org>

STEP20 - param: add share parameter "cluster share"

Signed-off-by: Michael Adam <obnox@samba.org>

STEP20 - param: add share parameter "continuously available share"

Pair-Programmed-With: Guenther Deschner <gd@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
Signed-off-by: Guenther Deschner <gd@samba.org>

STEP20 - TODO: s3:smb2_negprot: announce persistent handle support

if 'persistent handles' is set to yes

TODO: interactions with other options
(smb3, durable handles, kernel share modes, kernel oplocks, posix locking)

Pair-Programmed-With: Guenther Deschner <gd@samba.org>
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Pair-Programmed-With: Michael Adam <obnox@samba.org>

STEP20 - param: add parameter "persistent handles", defaults to off.

Guenther

Pair-Programmed-With: Michael Adam <obnox@samba.org>
Signed-off-by: Guenther Deschner <gd@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>

--- 00-STEPS-multi-channel: STEP12 - channel sequence/replay ---

STEP12? Revert "HACK debug replay/channel_sequence"

This reverts commit 46ce191a9bf9321251ea601a9b3ba7b48c5d5ccb.

STEP12? Revert "HACK disconnect after 2 writes"

This reverts commit eaa7023f865be73483a99ff203d1dbfb3e180ef9.

STEP12? HACK disconnect after 2 writes

STEP12? HACK debug replay/channel_sequence

--- 00-STEPS-multi-channel: STEP11 - lock sequence/replay detection ---

--- 00-STEPS-multi-channel: STEP05 - session lookup [DEBUG] ---

STEP05b debug session lookup

--- 00-STEPS-multi-channel: STEP00 : misc stuff ---

STEP02? use smbXsrv_client_update for >= smb2.1

selftest: mark samba3.base.delaywrite as flakey

We're getting failures like this, but also others

    [353/1594 in 23m14s] samba3.base.delaywrite(plugin_s4_dc)

    Running test_delayed_write_update1b
    Initial write time Wed May 21 03:27:02 2014 CEST
    write time Wed May 21 03:27:03 2014 CEST
    write time Wed May 21 03:27:03 2014 CEST
    write time Wed May 21 03:27:03 2014 CEST
    write time Wed May 21 03:27:03 2014 CEST
    write time Wed May 21 03:27:03 2014 CEST
    write time Wed May 21 03:27:03 2014 CEST
    write time Wed May 21 03:27:03 2014 CEST
    write time Wed May 21 03:27:03 2014 CEST
    write time Wed May 21 03:27:03 2014 CEST
    Server updated write time on close (correct)
    UNEXPECTED(failure): samba3.base.delaywrite.update of write time using
    SET_END_OF_FILE(plugin_s4_dc)
    REASON: _StringException: _StringException: After SET_END_OF_FILE truncate
    server updated write_time after 0.11 seconds(1 sec == 0.25)(wrong!)

    FAILED (1 failures, 0 errors and 0 unexpected successes in 0 testsuites)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

selftest: mark samba3.raw.oplock.batch1* as flakey

I saw this a few times:

    [432/1598 in 30m19s] samba3.raw.oplock(plugin_s4_dc)
    BATCH10: Open with oplock after a non-oplock open should grant level2
    write should trigger a break to none
    WARNING!: (../source4/torture/raw/oplock.c:1968): wrong value for break_info.count got 0x0 - should be 0x1

    Acking to none [0x00] in oplock handler
    UNEXPECTED(failure): samba3.raw.oplock.batch10(plugin_s4_dc)
    REASON: _StringException: _StringException: (../source4/torture/raw/oplock.c:1969): wrong value for break_info.fnum got 0x0 - should be 0x6490

    FAILED (1 failures, 0 errors and 0 unexpected successes in 0 testsuites)

or:

    [433/1594 in 38m13s] samba3.raw.oplock(plugin_s4_dc)
    BATCH11: Test if setpathinfo set EOF breaks oplocks.
    Acking to level II [0x01] in oplock handler
    WARNING!: (../source4/torture/raw/oplock.c:2046): wrong value for break_info.count got 0x1 - should be 0x2

    Acking to none [0x00] in oplock handler
    UNEXPECTED(failure): samba3.raw.oplock.batch11(plugin_s4_dc)
    REASON: _StringException: _StringException: (../source4/torture/raw/oplock.c:2048): wrong value for break_info.level got 0x1 - should be 0x0

    FAILED (1 failures, 0 errors and 0 unexpected successes in 0 testsuites)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

Revert "TMP: torture:smb2.session: only run bind1 test"

This reverts commit 75db98a26be2b11f35c29459f7ce9d4701e4b051.

Revert "HACK only test_session_reconnect1"

This reverts commit 3dbd1092c912f9436e0376250b9c33407672aff9.

HACK only test_session_reconnect1

TMP: torture:smb2.session: only run bind1 test

Revert "TODO make it possible to run smb2.session.bind1 test with -U% against IPC$ (windows)"

This reverts commit 173e6a5c289d3dcbf28cf73a61ade58e20f3b4e1.

TODO make it possible to run smb2.session.bind1 test with -U% against IPC$ (windows)

STATUS_INVALID_PARAMETER is returned

--- 00-STEPS-multi-channel: STEP07b - session bind [DEBUG] ---

STEP07 - TMP/DEBUG security >= user

STEP07 - TMP/DEBUG debug binding

STEP07 - DEBUG - smb2:session bind

STEP07 - TMP/DEBUG BIND

--- 00-STEPS-multi-channel: STEP02c - SMB connection passing [tests] ---

Revert "msgtest fd-passing"

This reverts commit b3acb9b155309f5459cd74c884a03e8b0a9b97f5.

msgtest fd-passing

-- 00-STEPS-multi-channel: STEP02b - SMB connection passing [DEBUG] ---

STEP02: smbXsrv_client: debug filter_pass

--- 00-STEPS-multi-channel: STEP02x - SMB connection passing - extra ---

STEP02: TODO smbXsrv_client: add commented out stuff for future purposes..

STEP02: smbXsrv_client: add commented out code related to local db_rec

STEP02: smbXsrv_client: add commented out code in ..pass_filter"

(prep for verifying client guid..)

STEP02: smbXsrv_client: add commented out store op in smbXsrv_client_create

STEP02: smbXsrv_client: add commented out table var to smbXsrv_client_connection_pass_loop"

--- 00-STEPS-multi-channel: STEP02a - SMB connection passing HACK ---

--- 00-STEPS-multi-channel: STEP07a enable multi channel ---

--- 00-STEPS-multi-channel: START, STEPs 01,05,06,07,10,02: [UPSTREAM] ---

update TODO.multi-channel

selftest: samba3.smb2.replay.replay9 fails (incomplete test)

selftest: samba3.smb2.replay.replay3 succeeds now

selftest: samba3.smb2.replay.replay4 succeeds now (channel sequence)

Signed-off-by: Michael Adam <obnox@samba.org>

TODO selftest: enable multi channel in the nt4_dc env

Signed-off-by: Michael Adam <obnox@samba.org>

TMP/DEBUG: debugging in smb2 dispatcher

SQ: torture:smb2: replay8 needs multi-channel

LATER? HACK add smb2.replay.replay9 test

SQ: torture:smb2: replay8 needs multi-channel

LATER? s4:torture: Add smb2.replay.replay8 test to examine server behaviour when Multiple Channels are utilized.

Signed-off-by: Anubhav Rakshit <anubhav.rakshit@gmail.com>
LATER? sq replay8 doesn't work (on a single CPU server?)

LATER? bla replay8...

s4-torture: add CHECK_CREATE_OUT_WITHOUT_HANDLE macro for checking two smb2_creates w/o file handle.

Guenther

Signed-off-by: Günther Deschner <gd@samba.org>

selftest:knownfail: smb2.lock.*replay succeed now

Signed-off-by: Michael Adam <obnox@samba.org>

s4-torture: in smb2.lock.replay try testing lock sequence checking against SMB3 samba servers.

Guenther

Signed-off-by: Guenther Deschner <gd@samba.org>

s4-torture: make smb2.lock.replay test work against Windows 2012 R2.

Guenther

Signed-off-by: Guenther Deschner <gd@samba.org>

s4-torture: make test_session_reauth5 work against Windows 2012R2.

The unlink of the file as anonymous is expected to fail.

Guenther

Signed-off-by: Guenther Deschner <gd@samba.org>

s3:smb2_lock: pass in_lock_sequence to smbd_smb2_lock_send()

Take the value from the client if the dialect is SMB2_10 or higher,
otherwise default to 0.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

s3:smbXsrv_open: initialize smbXsrv_open_global->lock_sequence_array with 0xFF

This does not match the current documentation, but is very likely the
right thing to do.

If we would match the documentation and initialize with 0x00,
we would return STATUS_SUCCESS without doing any locks.
If the client also follows the documentation and starts
with a lock_sequence of 0 for the first operation.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

s3:smbXsrv.idl: add lock_sequence_array to smbXsrv_open_global0

This is needed for lock replay detection.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

TODO.multi-channel: update plans for oplock breaks

Update TODO.multi-channel

TODO.multi-channel

STEP02a - HACK (avoid swrap fdpass..) accept unix sockets... source3/lib/util_sock.c

This lets selftest work without socket-wrapper supporting fd-passing...

ctdb-tests: Add some tool unit tests to ensure that timeouts work

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Feb 10 05:34:08 UTC 2020 on sn-devel-184

ctdb-tools: Allow shorter runtime limit to be specified

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tools: When in test mode set process group in top-level ctdb tool

If ctdbd hangs when shutting down in post-test clean-up then killing
the process group can kill the test.  When in test mode, create a
process group but only in the top-level ctdb tool - the natgw and lvs
helpers also run the ctdb tool.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Use $PWD/bin/ if it exists when running in-tree

When running tests from a top-level build, a stale build in ctdb/bin/
will be preferred and may cause confusing results.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Make $ctdb_dir absolute

This is used to set several variables so it might as well be cd-proof.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-daemon: Fork when not interactive and test mode is enabled

There is no sane way of keeping stdin open when using the shell to
background ctdbd in local_daemons.sh.  Instead, have ctdbd fork when
not interactive and when test mode is enabled.  become_daemon() can't
be used for this: if it forks then it also closes stdin.

For the interactive case, become_daemon() wasn't doing anything
special, so do nothing instead.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-daemon: Make some conditions more explicit

These don't need to depend on do_fork.  Child logging should be set up
whenever the daemon is not interactive.  The stdin handler should be
setup whenever test mode is enabled.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-daemon: Pass more information to ctdb_start_daemon()

No functional changes.

This is staging for a change that makes ctdbd fork when test mode is
enabled but interactive is not set.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Don't actually close stdin in fake ssh

A subsequent file descriptor allocation may return 0 and unexpected
things may then happen.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Redirect stdin from /dev/null when running a test

Otherwise, if the test is run via ssh it will "unexpectedly" find
itself at the other end of a pipe.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Revert "ctdb-tests: Enable job control when keeping stdin open"

This doesn't work when stdin is not a tty.

This reverts commit ea754bfdec9d537c500036d4d521bd41d34c0835.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

smbd: Remove overriding file_attributes with unix_mode in the VFS

Internally to open.c this is still used, but that can go away next.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Feb  7 22:27:48 UTC 2020 on sn-devel-184

smbd: Allow a Posix create context to override the unix mode

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Add posix create ctx to CREATE_FILE for posix operations

This will replace overloading file attributes

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Add a "done:" exit for get_posix_fsp()

We'll have another exit with the next commit

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Make unix_perms_from_wire() public

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Add make_smb2_posix_create_ctx()

Will be used internally to pass an artificial posix create context
into VFS_CREATE_FILE from the SMB1 unix extension calls

Pair programmed with: Jeremy Allison <jra@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>

smbd: Ignore incoming POSIX create context

We will use this internally and can only expose this once SMB3.11 unix
extensions are activated for the client.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Add smb2_create_blob_remove()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Allow passing in NULL to smb2_create_blob_find()

Will simplify callers a bit, and it does not change semantics
significantly. Zero create blobs won't find anything anyway.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Add posix create context definition

Pair programmed with: Jeremy Allison <jra@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>

libsmb: Add required #includes to smb_util.h

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Add required includes to smb2_create_blob.h

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Remove "const" from smb_create_blob->tag

I want to TALLOC_FREE that soon, and we do a talloc_strdup into this anyway.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3: lib: Now remote_machine is static, we can depend on it being non-NULL.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Feb  7 18:26:15 UTC 2020 on sn-devel-184

s3:lib: Remove unneded call to set_local_machine_name()

We return the netbios name by default if not set.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:lib: Use a static buffer for (local|remote)_machine

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libcli:smb: Don't use forward declartions for GnuTLS typedefs

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14271

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Feb  7 13:48:27 UTC 2020 on sn-devel-184

samba-tool gpo: tighter matching for ini names

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Fri Feb  7 12:03:34 UTC 2020 on sn-devel-184

python: use raw string for regex with escape

Python regards 'GPT\.INI$' as a string containing an invalid escape
sequence '\.', which is ignored (i.e. treated as the literal sequence
of those 2 characters), but only after Python has grumbled to itself,
and to you if you enabled DeprecationWarnings.

The proper thing to do here is use r-strings, like r'GPT\.INI$', which
tell Python that all backslashes are literal. Alternatively (as we do
once in this patch), the backslash can itself be escaped ('\\').

There are more problems of this nature in the build scripts.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>

pytests: heed assertEquals deprecation warning en-masse

TestCase.assertEquals() is an alias for TestCase.assertEqual() and
has been deprecated since Python 2.7.

When we run our tests with in python developer mode (`PYTHONDEVMODE=1
make test`) we get 580 DeprecationWarnings about this.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>

nmblib: avoid undefined behaviour in handle_name_ptrs()

If *offset is length - 1, we would read ubuf[(*offset)+1] as the lower
bits of the new *offset. This value is undefined, but because it is
checked against the valid range, there is no way to read further
beyond that one byte.

Credit to oss-fuzz.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14242
OSS-Fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20193

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Feb  7 10:19:39 UTC 2020 on sn-devel-184

librpc ndr: Change loop index to size_t

Change the loop index in ndr_check_padding to size_t.

Credit to OSS-Fuzz

REF: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20083
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14236

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>