join: Rename dc_join() so it looks like an object

dc_join() is creating an object, but it currently looks like it's
just a function call. Rename it to look more object-like.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

join: Pipe through dns_backend option for clones

Allow join_clone() calls to specify a dns_backend parameter for the new
cloned DB.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

provision: Small refactor to host-IP logic

Split out the code that determines the host-IP of the new server into
separate functions. This will allow us to re-use the same logic in the
backup/restore case.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

dbchecker: Fixing up incorrect DNs wasn't working

dbcheck would fail to fix up attributes where the extended DN's GUID is
correct, but the DN itself is incorrect. The code failed attempting to
remove the old/incorrect DN, e.g.

 NOTE: old (due to rename or delete) DN string component for
 objectCategory in object CN=alice,CN=Users,DC=samba,DC=example,DC=com -
 <GUID=7bfdf9d8-62f9-420c-8a71-e3d3e931c91e>;
   CN=Person,CN=Schema,CN=Configuration,DC=samba,DC=bad,DC=com
 Change DN to <GUID=7bfdf9d8-62f9-420c-8a71-e3d3e931c91e>;
   CN=Person,CN=Schema,CN=Configuration,DC=samba,DC=example,DC=com?
 [y/N/all/none] y
 Failed to fix old DN string on attribute objectCategory : (16,
 "attribute 'objectCategory': no matching attribute value while deleting
 attribute on 'CN=alice,CN=Users,DC=samba,DC=example,DC=com'")

The problem was the LDB message specified the value to delete with its
full DN, including the GUID. The LDB code then helpfully corrected this
value on the way through, so that the DN got updated to reflect the
correct DN (i.e. 'DC=example,DC=com') of the object matching that GUID,
rather than the incorrect DN (i.e. 'DC=bad,DC=com') that we were trying
to remove. Because the requested value and the existing DB value didn't
match, the operation failed.

We can avoid this problem by passing down just the DN (not the extended
DN) of the value we want to delete. Without the GUID portion of the DN,
the LDB code will no longer try to correct it on the way through, and
the dbcheck operation will succeed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13495

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>

dbcheck: Use symbolic control name for DSDB_CONTROL_DBCHECK_FIX_DUPLICATE_LINKS

While we do not wish to encourage use of this control, manually typed OIDs are
even more trouble, so pass out via pydsdb.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

ctdb-tests: Teach strace packet parser about non-octal escapes

strace output also encodes characters 7 to 13 as \a, \b, \t, \n, \v,
\f, \r.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Mon Jul  2 11:30:29 CEST 2018 on sn-devel-144

ctdb-daemon: Only consider client ID for local database attach

The comment immediately above this code says "don't allow local
clients to attach" and then looks up the client ID regardless of
whether the request is local or remote.

This means that an intentional remote attach from a client will not
work correctly.  No real client should ever do that since clients
attach so they an access databases locally.  Perhaps some sanity
checks should be added.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13500

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-docs: Fix the documentation for VNN map

It is incorrectly says that nodes not in the VNN map can not be
DMASTER.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-server: Rename CTDB_BROADCAST_VNNMAP -> CTDB_BROADCAST_ACTIVE

This broadcast is misnamed.  Both places where this type of broadcast
is used expect the broadcast to go to all active nodes.

Make the corresponding change to the semantics in the daemon by
sending to all active nodes.

There is a mismatch between the ideas of VNN map and active nodes.  A
node that is not in the VNN map but is active can still host database
records.  These were the same until the LMASTER capability was
introduced and then the logic was not updated.

The only place where the VNN map is relevant is when finding the
location master of a record in the migration code.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Add a simple test for database traverses

This tests that volatile databases traverse correctly, including the
case where a record was updated on a non-lmaster node.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Add check for non-lmaster node status in integration tests

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-client: Fix typo where CTDB_BROADCAST_ALL is repeated

Surely this is meant to be CTDB_BROADCAST_CONNECTED?

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-recoverd: Set the process name correctly

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-scripts: Drop 99.timeout event script

This is now implemented in local daemon testing.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Switch 90_debug_hung_script.sh to be a simple test

This test only runs against local daemons.  Configuration is done via
script.options, which simplifies things quite a bit.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Enable event script debugging in local daemon tests

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Support CTDB_RUN_TIMEOUT_MONITOR=yes in simple tests

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Clean up startup event in 00.test event script

Rewrite interface initialisation to avoid an error when there are no
interfaces configured.  Re-indent case label.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Add generic logging of event details in 00.test event script

No need for a separate case for each event just to log details.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Drop unnecessary code in 00.test event script

This script is only used with local daemons.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Clean up argument validation in 00.test event script

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Drop check for invalid event in 00.test event script

This isn't necessary and complicates the code.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

tdb: Fix build on AIX

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Here is the build error on AIX 7.1.

../../lib/tdb/tools/tdbtool.c:39:12: error: 'disable_lock' redeclared as different kind of symbol
 static int disable_lock;
            ^~~~~~~~~~~~
In file included from /usr/include/sys/gfs.h:24:0,
                 from /usr/include/sys/vfs.h:27,
                 from ../../lib/replace/system/filesys.h:48,
                 from ../../lib/tdb/tools/tdbtool.c:26:
/usr/include/sys/lock_def.h:314:5: note: previous declaration of 'disable_lock' was here
 int disable_lock(int,simple_lock_t);
     ^~~~~~~~~~~~

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb: Fix build on AIX

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-tests: Switch fake_ctdbd to use ctdb_get_peer_pid()

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move capture_socket functions to ctdb_socket.[ch]

The system_<os>.c files contain a lot of duplication, making
maintenance difficult.  These functions are being merged into
system_socket.c and system.c.

Bring across ctdb_sys_open_capture_socket(),
ctdb_sys_close_capture_socket() and ctdb_sys_read_tcp_packet().

Remove empty system_<os>.c files.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move ctdb_sys_send_tcp() to ctdb_socket.[ch]

The system_<os>.c files contain a lot of duplication, making
maintenance difficult.  These functions are being merged into
system_socket.c and system.c.

Bring across tcp_checksum(), renamed to ip_checksum().
uint16_checksum() becomes static.

Use the BSD struct tcphdr field names for portability.  See the
comment in the code for more details about how we get this to compile
on older glibc versions.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move ctdb_sys_send_arp() to ctdb_socket.[ch]

The system_<os>.c files contain a lot of duplication, making
maintenance difficult.  These functions are being merged into
system_socket.c and system.c.

Bring a copy of tcp_checksum6(), renamed to ip6_checksum().

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move ctdb_get_peer_pid() to system.[ch]

The system_<os>.c files contain a lot of duplication, making
maintenance difficult.  These functions are being merged into
system_socket.c and system.c.

This function doesn't need ctdb_sock_addr so put it with general
system utilities.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move ctdb_system_check_iface_exists() to system.[ch]

The system_<os>.c files contain a lot of duplication, making
maintenance difficult.  These functions are being merged into
system_socket.[ch] and system.[ch].

This function doesn't need ctdb_sock_addr so put it with general
system utilities.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move parse_ip_mask() to system_socket.[ch]

This uses ctdb_sock_addr so belongs here.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Rename system utility files

system_socket.[ch] will contain all the raw socket code and other
functions that use ctdb_sock_addr.  system.[ch] will contain other
platform dependent functions.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-build: Add ipv6 headers check for packet details

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-build: Add checks for raw pkt handling support

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-common: Use sin6_len only if the structure supports it

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

replace: Add test for sin6_len in sockaddr_in6 structure

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-common: Correctly handle conf->reload()

Configuration reload should reset the values of configuration options
missing from the config file to default.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Jun 29 15:12:37 CEST 2018 on sn-devel-144

ctdb: Improve robust mutex test

This avoids some of the undefined behaviour, like initializing the same mutex
twice which happens when the low and high priority processes start (both
do the initialization and that's dangerous.) Instead now we start an
"init" process to start the shared memory segment, and then everything
else just uses it without truncation or unlinking (same mutex).

Signed-off-by: Carlos O'Donell <codonell@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Jun 29 06:47:00 CEST 2018 on sn-devel-144

README.Coding: Fix link to Python coding style guide (PEP 8)

Signed-off-by: Björn Baumbach <bb@sernet.de>
Reviewed-by: David Mulder <dmulder@suse.com>
Autobuild-User(master): Björn Baumbach <bbaumbach@samba.org>
Autobuild-Date(master): Thu Jun 28 15:04:44 CEST 2018 on sn-devel-144

tests/ntacls: fix pep8 warnings

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Jun 28 12:14:59 CEST 2018 on sn-devel-144

tests/ntacls: use global vars to make code DRY

Move acl and dommain_sid to global vars so we don't repeat them in every
test.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

tests/ntacls_backup: register test

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

ntacls: add extra test file for ntacls backup

The ntacls backup tests have to run in ad_dc:local env, which is
different from existing ntacls tests. Add a separate file for backup
tests.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

ntacls: add functions to backup and restore ntacls

1. backup a share online from a smb connection with ntacls using pysmb API.
2. backup a share offline from service path with ntacls using pysmbd API.
3. restore from tarfile with pysmdb API.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

pysmbd: add py_smbd_create_file

Add create_file function to smbd API.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

pysmbd: extract init_files_struct function

Extract initialization code from set_nt_acl_conn for reuse.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

pysmbd: add py_smbd_mkdir

Add mkdir for smbd API.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

pysmb: add py_smb_unlink and test

Add unlink api to delete a file with a smb connection.
Test added.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

samba-tool showrepl tests: test all-good with --pull-summary

We test the all-good case with --pull-summary, which is the only one
we can be reasonably certain about.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu Jun 28 09:23:10 CEST 2018 on sn-devel-144

samba-tool drs showrepl: add --pull-summary and --notify-summary

These separate the two halves of --summary (which is still there),
allowing the repsto and repsfrom to be separately queried.

One motivation for this is testing: it is difficult to assert the
success of repsfrom (--notify-summary) in the test framework, because
we can't rely on the other end behaving properly and promptly.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

s4/torture/samba_tool_drs_showrepl: use assertRegexpMatches

rather than a local rewrite special to this file.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/torture/drs/python: don't double-call enable/disable replication

This is repeating work done in setup/teardown or doubling up in place (self._enable_all_repl includes self._enable_inbound_repl)

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool drs showrepl test: remove useless print

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool drs showrepl: Skip deleted DSAs when checking for success

The deleted DSAs are ignored by the server replication code, so ignore past failures
here also.

The repsFrom and repsTo entries will eventually be removed by the KCC.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

samba-tool drs show_repl: simplify the collection of DC lists

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool drs showrepl --summary for a quick local check

The default output ("classic") gives you a lot of very uninteresting
detail when everything is fine. --summary shuts up about things that
are fine but shouts a little bit when things are broken. It doesn't
provide any new information, just tries to present it in a more useful
format.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool drs showrepl: add a --color flag

Nothing is using it yet, but the next commit will

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

provision: set 'binddns dir' when making new smb.conf

When creating a new smb.conf from scratch during a join/clone/etc, the
'binddns dir' setting still uses the source smb.conf/default setting,
instead of the targetdir sub-directory.

I noticed this problem when trying to create a new testenv - the
provision() was trying to create /usr/local/samba/bind-dns directory,
which would fail if samba hadn't already been installed on the host
machine.

Now that this is fixed, we also need to fix tests that were explicitly
asserting that no unexpected directories were left behind after the test
completes.

This change also breaks the upgradeprovision script. The upgrade-
provision calls newprovision() to create a reference provision in a
temporary directory. However, previously this temporary provision was
creating the bind-dns directory in the actual upgrade directory as a
side-effect, e.g. it did a provision() with
targetdir=alpha13_upgrade_full/private/referenceprovisionLBKBh2 and this
ended up creating alpha13_upgrade_full/bind-dns as a side-effect.
The provision() now creates bind-dns in the specified targetdir, but
this means check_for_DNS() fails (it tries to create bind-dns sub-
directories, but the upgrade's bind-dns doesn't exist). I've avoided
this problem by making sure bind-dns exists as part of the
check_for_DNS() processing.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Jun 28 06:22:16 CEST 2018 on sn-devel-144

selftest: Update MAX_WRAPPED_INTERFACES comment to match code

Commit 19606e4dc657b0baf3ea84d updated the MAX_WRAPPED_INTERFACES define
in the C code from 40 to 64.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests: Add basic test for non-global LoadParm behaviour

Add a simple test to show that the new non-global LoadParm behaviour
works, i.e.
- by default all LoadParm objects are linked to the same underlying
  object
- using a non-global LoadParm creates a separate underlying object.
- using a non-global LoadParm with a bad filename fails.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

param: Add non-global smb.cfg option (support 2 different smb.confs)

The default behaviour is that there is only a single global underlying
LoadParm object. E.g. if you create 2 different LoadParm objects in
python, they both modify the same underlying object.

This patch adds a mechanism to override this and create a separate
non-global LoadParm object. The use-case is the backup tool, where we
want to manipulate 2 different smb.conf files (the one used to create
the backup, and the smb.conf in the backup itself).

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba: read backup date field on init and fail if present

This prevents a backup tar file, created with the new official
backup tools, from being extracted and replicated.

This is done here to ensure that samba-tool and ldbsearch can
still operate on the backup (eg for forensics) but starting
Samba as an AD DC will fail.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests: Add test that Samba cannot be started with a backup DB

We don't want users to take a backup file, and then simply untar it and
run Samba (Several modifications to the DB need to be made as part of
the restore process, so users should always run the 'backup restore'
command).

To enforce this, prime_ldb_databases() now refuses to start Samba if the
backupDate marker is present in the DB. This patch adds a test-case that
proves this basic behaviour works.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

docs: Add manpage for winbind_krb5_localauth.8

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13489

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Jun 27 18:45:56 CEST 2018 on sn-devel-144

docs: Move winbind_krb5_locator manpage to volume 8

The vfs and idmap manpages are in volume 8 too.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13489

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

krb5_plugin: Move krb5 locator plugin to krb5_plugin subdir

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13489

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

krb5_plugin: Install plugins to krb5 modules dir

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13489

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

Update .travis.yml to match package list used in docker container and Catalyst Cloud builds

The package list has some "strange" things in them, but more important is
using the same list everywhere.  We can hopefully harmonise the package lists
to a single file in Samba git soom, merging the docker and packer image creation.

Additionally, Travis CI will probably need to move to Docker once we change
beyond Ubuntu 14.04, so it will simple reference the gitlab.com image then.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Wed Jun 27 07:51:14 CEST 2018 on sn-devel-144

remove_dc: Fix removal of an old Windows DC

Windows has 'CN=DNS Settings' child object underneath the Server object.
This was causing the removal of the server object in remove_dc() to
fail.

Noticed this problem while testing the backup/restore tool manually
against a Windows VM.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13484

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Tue Jun 26 23:32:51 CEST 2018 on sn-devel-144

s3:tests: Add test for smbclient --quiet

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13485

Signed-off-by: Justin Stephenson <jstephen@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Björn Baumbach <bb@sernet.de>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Jun 26 20:29:19 CEST 2018 on sn-devel-144

s3:client: Add --quiet option to smbclient

Add quiet command-line argument to allow suppressing the help log
message printed automatically after establishing a smbclient connection

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13485

Signed-off-by: Justin Stephenson <jstephen@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Björn Baumbach <bb@sernet.de>

dsdb audit: Fix timestamp tests

Fix flapping test:
  [242(3560)/242 at 25m3s] samba4.dsdb.samdb.ldb_modules.audit_log
UNEXPECTED(failure):
  samba4.dsdb.samdb.ldb_modules.audit_log.test_operation_json_empty(none)
REASON: Exception: Exception: difftime(after, actual) >= 0
../source4/dsdb/samdb/ldb_modules/tests/test_audit_log.c:74: error:

The tests truncate the microsecond portion of the time, so the
difference could be less than 0.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Jun 26 06:09:46 CEST 2018 on sn-devel-144

dsdb-audit: Remove flapping part of the tests

Because we have tests for this in the auth audit code, we do not need to have
the complexity of checking that we got DCE/RPC over SMB as an authorization
message here.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

dsdb: Use GUID_zero() rather than memset in dsdb audit code

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

dsdb: Use customary variable names for the audit private context

The variable name "ac" typically implies the async context, and the long-life
private context is normally denoted private, not context.  This aligns better
with other modules.

talloc_get_type_abort() is now also used.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

dsdb: Use customary variable names for audit event contexts

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

dsdb: Use correct memory context for imessaging_client_init() in audit logging

This is only used for selftest, to send out the log messages for checking.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

audit_logging: Remove duplciate error printing

These errors are already logged at DBG_NOTICE in get_event_server()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

audit_logging: Initialise event_server

It is better if this is a known zero value to start, even if we check the errors
correctly.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

audit_logging: Remove incorrect check for NT_STATUS_OBJECT_NAME_NOT_FOUND

NT_STATUS_OBJECT_NAME_NOT_FOUND is not a case we can ignore, it would mean that event_server
is not initialised.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

audit_logging: Clarify debug messages

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

autobuild: Build samba-fileserver --without-json-audit

This build target is already --without-ad-dc and is the one we need to ensure is
compatible with a host without the Jansson JSON library.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Jun 26 02:03:30 CEST 2018 on sn-devel-144

dsdb: Ensure a build --without-json-audit --without-ad-dc compiles

We still build some of the ldb_modules even when we are not a DC, so we must
split up the DSDB_MODULE_HELPERS.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib/audit_logging: Only build audit_logging_test for --enable-selftest on the AD DC

This allows a --without-ad-dc --enable-selftest build to compile, still testing some
fileserver-only features.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib/audit_logging: Require jansson JSON library for building the AD DC

This combination is untested and it is reasonable to require this
broadly available library for the AD DC build.

Doing so keeps the combinational complexity down and ensures we test
what we ship.  (It was failing to compile).

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

build: Move --without-json-audit and json lib detection to lib/audit_logging

This is the common location of the audit logging code now

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

dsdb: Log the transaction duraton.

This is not a general purpose profiling solution, but these JSON
logs are already being generated and stored, so this is worth adding.

This will allow administrators to identify long running
transactions, and identify potential performance bottlenecks.

This complements a similar patch set to log authentication duration.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Jun 25 11:16:18 CEST 2018 on sn-devel-144

auth: For NTLM and KDC authentication, log the authentication duration

This is not a general purpose profiling solution, but these JSON logs are already being
generated and stored, so this is worth adding.

Some administrators are very keen to know how long authentication
takes, particularly due to long replication transactions in other
processes.

This complements a similar patch set to log the transaction duration.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

talloc_stack: Call talloc destructors while frame is still around

This fixes "samba-tool ntacl set -d10"

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Jun 23 04:56:44 CEST 2018 on sn-devel-144

check return value before using key_values

there are also mem leaks in this function

Signed-off-by: Andrej Gessel <Andrej.Gessel@janztec.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

ldb: check return values

Signed-off-by: Andrej Gessel <Andrej.Gessel@janztec.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

krb5_wrap: fix keep_old_entries logic for older kerberos libraries

MIT kerberos 1.13 and older only stores 8 bits of the KVNO. The change
from commit 35b2fb4ff32 resulted in breakage for these kerberos
versions: 'net ads keytab create' reads a large KVNO from AD, and only
the lower 8 bits are stored. The next check then removed the entry again
as the 8 bit value did not match the currently valid KVNO.

Fix this by limiting the check to only 8 bits.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13478

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Christof Schmitt <cs@samba.org>
Autobuild-Date(master): Sat Jun 23 00:57:47 CEST 2018 on sn-devel-144

ctdb-common: replace talloc / memcpy by talloc_memdup

Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Jun 22 11:57:19 CEST 2018 on sn-devel-144

krb5_plugin: Add winbind localauth plugin for MIT Kerberos

Applications (like OpenSSH) don't know about users and and
their relationship to Kerberos principals. This plugin allows that
Kerberos principals can be validated against local user accounts.

Administrator@WURST.WORLD -> WURST\Administrator

https://web.mit.edu/kerberos/krb5-latest/doc/plugindev/localauth.html

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13480

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Jun 21 15:52:02 CEST 2018 on sn-devel-144

s3: smbd: Don't use getcwd() directly. We must always go through the VFS.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

s3:winbind: Fix regression introduced with bso #12851

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12851

Pair-Programmed-With: Guenther Deschner <gd@samba.org>
Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Jun 21 01:03:52 CEST 2018 on sn-devel-144

s3:smbget: Fix buffer truncation issues with gcc8

../source3/utils/smbget.c: In function ‘smb_download_file’:
../source3/utils/smbget.c:97:27: error: ‘b’ directive output may be truncated writing 1 byte into a region of size between 0 and 19 [-Werror=format-truncation=]
   snprintf(buffer, l, "%jdb", (intmax_t)s);
                           ^

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13437

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

s3:registry: Fix buffer truncation issues issues with gcc8

../source3/registry/reg_perfcount.c: In function ‘reg_perfcount_get_hkpd’:
../source3/registry/reg_perfcount.c:337:29: error: ‘snprintf’ output may be truncated before the last format character [-Werror=format-truncation=]
   snprintf(buf, buflen,"%d%s", key_part1, key_part2);

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13437

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

samdb: Fix build error with gcc8

../source4/dsdb/samdb/ldb_modules/samldb.c: In function ‘samldb_add’:
../source4/dsdb/samdb/ldb_modules/samldb.c:424:6: error: ‘found’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
   if (found) {
      ^
../source4/dsdb/samdb/ldb_modules/samldb.c:348:11: note: ‘found’ was declared here
  bool ok, found;
           ^~~~~

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13437

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

Possible memory leak in map_info3_to_validation

In case of a failing call to copy_netr_SamInfo3, the allocated memory
for "validation" needs to be free'd before returning.

Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Jun 20 21:05:40 CEST 2018 on sn-devel-144