s3: smbd: Make change_file_owner_to_parent() static.

Only used inside open.c.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: smbd: change_dir_owner_to_parent_fsp(). Don't re-stat the pathref.

Optimization now becomes clear. We already have a valid stat of the parent
directory so we don't need to re-do a system call.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: smbd: Change change_dir_owner_to_parent() -> change_dir_owner_to_parent_fsp().

Operate on handles only.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: smbd: open_directory(). Cleanup. We don't need 'int flags' here.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

util/charset: warn loudly on unexpected E2BIG

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Jun 18 04:27:17 UTC 2021 on sn-devel-184

util/iconv: reject improperly packed UTF-8

If we allow a string that encodes say '\0' as a multi-byte sequence,
we are open to confusion where we mix NUL terminated strings with
sized data blobs, which is to say EVERYWHERE.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14684

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>

torture: talloc_string_sub tests for utf-8 brevity

If we allow overly long UTF-8 sequences (in the tests, encoding '\0'
as 2, 3, or 4 bytes), it might be possible for bad strings to slip
through.

We fail. But wait for the next commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14684

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>

netcmd: Incorrect arguments to Exception constructor

Discovered by Semmle code analysis:
https://lgtm.com/projects/g/samba-team/samba

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu Jun 17 05:12:03 UTC 2021 on sn-devel-184

upgradeprovision: Remove duplicate key

Discovered by Semmle code analysis:
https://lgtm.com/projects/g/samba-team/samba

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

perf_tests: Implicit string concatenation

Discovered by Semmle code analysis:
https://lgtm.com/projects/g/samba-team/samba

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

join: provision_fill does not return anything

Discovered by Semmle code analysis:
https://lgtm.com/projects/g/samba-team/samba

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

heimdal_build: Improve error and warning handling on old and new compilers

The previous commit 1eadeaed0a6ca3a58eb9fd176a7ae5bcc28f64ef had a couple of
errors, the unpicky flags were being set on all builds (not just old
compiler builds) due to confusing variable names, and Ubuntu 16.04
would not build (for fuzzing) because it thought some variables
were maybe-uninitialized.

This keeps stricter warnings->errors on modern compilers while
allowing the full build, even in the near future when a modern
Heimdal is imported.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Jun 16 14:43:17 UTC 2021 on sn-devel-184

s3: smbd: Fix smbd crash on dangling symlink with posix connection calling several non-posix info levels.

Tidy up fsp == NULL checks. Remove knownfail.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14742

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Wed Jun 16 11:58:00 UTC 2021 on sn-devel-184

s3: torture: Add POSIX-SYMLINK-SETPATHINFO regression test.

This ensure we never blunder into indirecting a NULL fsp pointer
in the server. Currently this crashes the server in several info
levels.

Add knownfail.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14742

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Noel Power <npower@samba.org>

mdssvc: avoid direct filesystem access, use the VFS

This ensures mdssvc uses the same FileIDs as the fileserver as well as Spotlight
can be used working on a virtual filesystem like GlusterFS.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14740

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Jun 16 05:59:13 UTC 2021 on sn-devel-184

mdssvc: chdir() to the conn of the RPC request

In preperation of calling VFS functions.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14740

Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

mdssvc: maintain a connection struct in the mds_ctx

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14740

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: add create_conn_struct_cwd()

Compared to create_conn_struct_tos_cwd() this takes a TALLOC_CTX and
tevent_context as additional arguments and the resulting connection_struct is
stable across the lifetime of mem_ctx and ev.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14740

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: pass tevent context to create_conn_struct_as_root()

The next commit will add another caller of create_conn_struct_as_root() that is
going to pass a long-lived tevent context.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14740

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

mdssvc: pass messaging context to mds_init_ctx()

This is needed in a subsequent commit. Note that I prefer to do the event
context unwrapping in the caller and pass both the event and messaging context
explicitly to mds_init_ctx().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14740

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

mdssvc: don't fail mds_add_result() if result is not found in CNID set

Just skip adding the result to the pending results set, don't return an
error. Returning an error triggers an error at the MDSSVC RPC error which is NOT
what we want here.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14740

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

mdssvc: use a helper variable in mds_add_result()

No change in behaviour.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14740

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib:ldb-samba: Migrate samba extensions to new cmdline option parser

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Jun 16 01:25:28 UTC 2021 on sn-devel-184

lib:ldb-samba: Use talloc_zero_array() and use ldb as the mem context

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

lib:ldb-samba: Improve calculate_popt_array_length()

Note that memcmp() doesn't work well with padding bytes. So avoid it!

(gdb) ptype/o struct poptOption
/* offset    |  size */  type = struct poptOption {
/*    0      |     8 */    const char *longName;
/*    8      |     1 */    char shortName;
/* XXX  3-byte hole  */
/*   12      |     4 */    unsigned int argInfo;
/*   16      |     8 */    void *arg;
/*   24      |     4 */    int val;
/* XXX  4-byte hole  */
/*   32      |     8 */    const char *descrip;
/*   40      |     8 */    const char *argDescrip;

                           /* total size (bytes):   48 */

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

lib:ldb: Use C99 initializers for builtin_popt_options[]

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture: Migrate masktest to new cmdline option parser

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture: Migrate locktest to new cmdline option parser

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture: Change -U|--user to --user1 and --user2

The '-U' option is already defined by the default cmdline parser!

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture: Migrate gentest to new cmdline option parser

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture: Change -U|--user to --user1 and --user2

The '-U' option is already defined by the default cmdline parser!

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

testprogs: Add smbtorture tests with new options

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture: Migrate smbtorture to new cmdline option parser

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture: Pass the pkinit ccache via a torture variable

Mixing -Uuser%password and --krb5-ccache doesn't really work on the
cmdline as -U overwrited the ccache.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture: For NTLM make sure we have CRED_USE_KERBEROS_DESIRED

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture: Write better error on invalid cmdline option

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture: Remove unused include

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:client: Migrate cifsdd to new cmdline option parser

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

testprogs: Use new kerberos options for smbclient(4) tests

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:client: Migrate smbclient4 to new cmdline option parser

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:client: Use a creds helper variable

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

testprogs: Remove --debuglevel from test_kinit_trusts_mit.sh

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

heimdal_build: Use lib/asn1/rfc2459.opt rather than hard-coded

Based on patch by Stefan Metzmacher in his Heimdal upgrade branch

lib/asn1/rfc2459.opt imported from
lorikeet-heimdal-abartlet/lorikeet-heimdal-201107241840-plus-recent-changes
which is the closest tree I could find, and matches the options being
removed from the wscript_build file.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Jun 15 23:25:27 UTC 2021 on sn-devel-184

heimdal_build: Add C99 struct initializer in source4/heimdal_build/krb5-glue.c

This avoids uninitiliased structure members in this dummy
structure we include to avoid including more of Heimdal.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

build: in SAMBA_BINARY use TO_LIST(cflags)

This avoids unfortunate issues when the cflags is
already a list, as then -fPIC becomes ['-f', 'P', 'I', 'C'].

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

heimdal_build: Provide C defines showing which Kerberos library is in use

Squashed from patches by Stefan Metzmacher as part of his Heimdal update branch

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

gse_krb5: Provide keytab name in fill_mem_keytab_from_dedicated_keytab() error strings.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

heimdal_build: check for secure_getenv

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

heimdal_build: Set up new build groups for the Heimdal hostcc components

This is based on various patches by Stefan Metzmacher in the patch set for
the Heimdal upgrade.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

heimdal_build: Rework Heimdal warning handling

If we have all the right -Wno-error flags then we can enable warnings
more generally, otherwise just set -Wno-strict-overflow (if available)

Adapted from patches by Stefan Metzmacher <metze@samba.org> in his
branch to update Heimdal.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

docs: Improve wording, fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Jun 15 19:02:18 UTC 2021 on sn-devel-184

libsmbclient: Avoid a call to SMBC_errno() in SMBC_mkdir_ctx()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Factor out cli_status_to_errno() from cli_errno()

cli_errno() calls far too many trivial but subtle functions, all
referencing cli->raw_status. This might be the first step towards
getting rid of that.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

rpc_server: Make get_domain_userlist() independent of errno

In the "num_users==0" case (previously just return NULL) we depended
on errno==0 implicitly. When list_sessions() above in this routine had
to open smbXsrv_session_global, it could however happen that errno was
set. If then there were no users, get_domain_userlist() returned NULL
with errno set, which the callers interpreted then as a real error.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

rpc_server: Make errno return of get_logged_on_userlist explicit

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

rpc_server: Don't rely on TCP-bind() to return EADDRINUSE

socket_wrapper can't do EADDRINUSE because unix domain sockets don't
do it.

This currently works correctly because right now all RPC servers
either use explicit ports or all listen on the same socket.

The new code uses a static variable, so it only helps if a single
process listens for multiple RPC sockets. It won't work if multiple
processes start listening. But in case samba-dcerpcd goes in this will
be exactly the right thing to do.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3: torture: Add POSIX-SYMLINK-GETPATHINFO regression test.

This ensure we never blunder into indirecting a NULL fsp pointer
in the server. We already pass this, but this test will ensure
we continue to do so as we make fileserver changes.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Noel Power<npower@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Tue Jun 15 11:06:23 UTC 2021 on sn-devel-184

Fix for https://bugzilla.samba.org/show_bug.cgi?id=9634

Add an option to smb.conf to list authorized zone transfer clients.
Implement restriction in dlz_bind9 module to allow transfers only to selected IPs.
Deny zone transfer by default in dlz_bind9.

Adds test for the restriction in DNZ zone transfer clients.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=9634

Signed-off-by: Julien ROPÉ <jrope@linagora.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Jun 11 19:28:10 UTC 2021 on sn-devel-184

s3: smbd: Remove erroneous TALLOC_FREE(smb_fname_parent) in change_file_owner_to_parent() error path.

Caller is still using this !

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14736

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Noel Power<npower@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Fri Jun 11 10:17:46 UTC 2021 on sn-devel-184

VFX: vxfs: Fixup some warnings

../../source3/modules/vfs_vxfs.c:343:6: error: unused variable ‘i’ [-Werror=unused-variable]
  int i, offset = 0;
      ^
../../source3/modules/vfs_vxfs.c:342:17: error: unused variable ‘n_id’ [-Werror=unused-variable]
  uint32_t e_id, n_id;
                 ^~~~
../../source3/modules/vfs_vxfs.c:342:11: error: unused variable ‘e_id’ [-Werror=unused-variable]
  uint32_t e_id, n_id;
           ^~~~
../../source3/modules/vfs_vxfs.c:341:35: error: unused variable ‘n_perm’ [-Werror=unused-variable]
  uint16_t e_type, n_type, e_perm, n_perm;
                                   ^~~~~~
../../source3/modules/vfs_vxfs.c:341:27: error: unused variable ‘e_perm’ [-Werror=unused-variable]
  uint16_t e_type, n_type, e_perm, n_perm;
                           ^~~~~~
../../source3/modules/vfs_vxfs.c: In function ‘vxfs_compare’:
../../source3/modules/vfs_vxfs.c:407:6: error: unused variable ‘i’ [-Werror=unused-variable]
  int i, count = 0;
      ^

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Jeremy Allison <jra@samba.org>

VFS: vxfs: ifdef out vxfs_sys_acl_set_fd

as the sys_acl_set_fd_fn definition for vxfs_sys_acl_set_fd is ifdef'ed
out we also need ifdef out the vxfs_sys_acl_set_fd implementation itself
otherwise we get the following error.

source3/modules/vfs_vxfs.c:484:12: error: ‘vxfs_sys_acl_set_fd’ defined but not used [-Werror=unused-function]
 static int vxfs_sys_acl_set_fd(vfs_handle_struct *handle,

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3/smbd: Remove unecessary 'else' block

This is an inconsequential cosmetic change, it just caught my eye
as looking a bit out of place compared to the surrounding code style.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3/smbd: dos_mode_check_compressed: remove smb_fname, conn fn parms

smb_fname is unused and we can get conn from the fsp passed in

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3/smbd: dos_mode_post: remove smb_fname param

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Jeremy Allison <jra@samba.org

s3/smbd: call dos_mode_post with fsp

Next commit can remove smb_name param from dos_mode_post
signature.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3: smbd: Protect dos_mode_at_send() from running into a symlink.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Noel Power<npower@samba.org>

pytests: add dns_aging, embracing and extending ageing tests

This incorporates tests from various dns*.py files, but makes them
correct.

All but one of these tests pass against Windows 2012r2.

Further patches will remove the broken tests in other files, and fix
Samba so it passes these.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Jun 11 09:29:23 UTC 2021 on sn-devel-184

py: samba.dnsserver: add helper for record buffers

We *always* make these steps when we get a record.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytest:dns_base: make_txt_update can set arbitrary TTL

Also, improve a variable name.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pydns: expose dns_records_match() as dsdb_dns.records.match()

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dns: merge dns_records_match and dns_record_match

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dlz: remove pretense of HINFO support

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dns_record_match: drop pretense of HINFO support

We don't support it really, and if we did there is no sense in which
it could be updated, which is the context in which this function is
used.

(modern HINFO returns the constant string "RFC8482". See RFC 8482).

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dns common: dns_records_match() matches tombstones

This will be needed by the RPC server. Other callers already filter
out tombstones, so this is OK.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dns: merge dlz/internal dns_records_match()

We have had three nearly identical functions called
dns_record[s]_match. This patch merges two of them, attempting to keep
the good bits and not the bugs.

That means:

1. We use the AAAA match from dlz, which is agnostic to all the
billions of ways you can write the same IPv6 address (case sensitivity
is just the beginning).

2. We lean more on the TXT match from dns_utils, because the dlz used
a weird bitwise &= operator, but we adjust to exit early.

3. Keep HINFO from dlz (for now).

4. Use the dns_name_equal() that was already in dns_common, which was
used by dlz. dns_utils had a strange one that probably did the same
thing.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dlz_bind9: remove redundant logging in b9_record_match()

This log message will never be seen. We know because:

1. Always (two places) we are comparing an incoming record against a
   database record.

2. The incoming record has come from b9_parse(), which makes the same
   check.

3. If the database record is bad, we will never get here because the
   first check is b9_record_match() is

       if (rec1->wType != rec2->wType) {
               return false;
       }

   and rec1->wType is not going to equal the corrupt database record's
   wType, because point 2.

OK, but why? So we can shift this into dnsserver_common.c, because
the internal dns server has an inferior record_match() and it could do
with sharing this one.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python:subunit: Avoid misleading "Test was never started" error message

subunithelper.py keeps track of tests that have been started, and
displays an error message if a test reports an outcome without having
previously been started. However, it makes the assumption that a test
has finished once it has reported a single outcome. This means that a
misleading error message will be displayed if it receives multiple
outcomes from the same test (which can happen if a test using the Python
unittest framework does not complete successfully, and the cleanup
subsequently fails), and any actual errors from the cleanup remain
undisplayed.

This commit ensures that only a single outcome is reported for each
test, and only after the test has finished. Outcomes are buffered up
until the stopTest() function is called, when a single outcome is
determined and all errors received for that test are output.

FilterOps still needs to output test outcomes immediately rather than
buffering them, otherwise they are never picked up and passed on to the
remote test case by subunithelper.parse_results(). This would result in
an error as the test would be considered to have never finished.

    Example subunitrun output before the change:

time: 2021-04-28 01:28:49.862123Z
test: samba.tests.example.ExampleTests.test
time: 2021-04-28 01:28:49.862215Z
failure: samba.tests.example.ExampleTests.test [
Traceback (most recent call last):
  File "bin/python/samba/tests/example.py", line 28, in test
    self.fail()
AssertionError: None
]
time: 2021-04-28 01:28:49.862407Z
failure: samba.tests.example.ExampleTests.test [
Traceback (most recent call last):
  File "bin/python/samba/tests/example.py", line 31, in tearDown
    self.fail()
AssertionError: None
]
time: 2021-04-28 01:28:49.862467Z
time: 2021-04-28 01:28:49.862510Z

    and after:

time: 2021-04-28 01:29:19.949347Z
test: samba.tests.example.ExampleTests.test
time: 2021-04-28 01:29:19.949440Z
time: 2021-04-28 01:29:19.949590Z
time: 2021-04-28 01:29:19.949640Z
failure: samba.tests.example.ExampleTests.test [
Traceback (most recent call last):
  File "bin/python/samba/tests/example.py", line 28, in test
    self.fail()
AssertionError: None
Traceback (most recent call last):
  File "bin/python/samba/tests/example.py", line 31, in tearDown
    self.fail()
AssertionError: None
]
time: 2021-04-28 01:29:19.949702Z

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python:subunit: Remove write_traceback()

This functionality is already present in the Python unittest framework,
and so is not necessary to include here.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python:subunit: Fix skipping a test with no reason given

Not specifying a reason means addSkip() is passed an empty string rather
than None. As a result, this condition was never hit, and the call to
_addOutcome() had an incorrect parameter.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dbcheck: formatting

Reduce the length of some lines to 79 characters or less.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Jun 11 08:28:28 UTC 2021 on sn-devel-184

dbcheck: Refactor RID Set check to use free_rid_bounds()

This function provides a simpler method of getting the bounds of the
range of RIDs we want to check. We also now check that the low bound is
less than the high bound for both rIDAllocationPool and
rIDPreviousAllocationPool.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

netcmd: Avoid conflicting SIDs when creating an offline backup

To allow the new DC object to be created in a restored domain while
avoiding conflicts with existing SIDS, we fetch a SID that is available
at the time of backing up and store it in the backed-up database.
However, if a new security principal is created on this DC during the
backup process, the stored SID may be reused for that object, resulting
in an error on restoration.

By getting the SID for restore only after all the database files have
been backed up, we ensure that the chosen SID does not conflict with any
objects in the backed-up database.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

ridalloc: Don't skip the first RID of a pool

Previously, if either of the rIDPreviousAllocation and rIDNextRID
attributes were not present in a RID Set, the first RID in
rIDAllocationPool was skipped over when determining their values.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

netcmd: Use next_free_rid() function to calculate a SID for restoring a backup

This means we won't get errors if the DC doesn't have a rIDNextRID
attribute, but we will still error if there is no RID Set or if all its
pools are exhausted.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python/tests/dsdb: Add tests for RID allocation functions

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

dsdb: Add next_free_rid() function to allocate a RID without modifying the database

If used to generate SIDs for objects, care should be taken, as the
possibility for having duplicate objectSIDs can arise.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

netcmd: Add tests for performing an offline backup immediately after joining a domain

This currently fails due to the DC not having a rIDNextRID attribute,
which is required for the restore process.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

dbcheck: check correct RID set attributes when looking for SID conflicts

The previous code would only work for the first rid set ever given to a DC
because the names are so misleading.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13632

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

netcmd: Refactor seizing DNS roles while restoring from a backup

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

netcmd: Use correct path for state directory during offline backup

During the restore process, we use make_smbconf() to create a new
smb.conf file with the default paths. The default location for 'state
directory' is 'state', but we currently rename this directory to
'statedir' on backing up, so it will end up pointing to a non-existent
directory. This commit ensures the names are consistent.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

tests: Specify additional modules for 'vfs objects' parameter

This helps to avoid a warning 'vfs objects specified without required AD
DC module'.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

netcmd: Ignore rIDUsedPool attribute in offline domain backup test

The RID Set of the newly created DC account has all its values
initialised to zero. If the rIDUsedPool attribute was previously
non-zero, then the restore process will cause its value to change.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

netcmd: Fix error-checking condition

This condition probably meant to check the argument of the most recently
thrown exception, rather than the previous one again.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

provision: Refactor another usage of create_dns_dir_keytab_link

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14181
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14535

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

sambadns: Create BINDDNS_DIR/dns.keytab link to PRIVATE_DIR/dns.keytab on DC join

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14181

Signed-off-by: Derek Lambert <dlambert@dereklambert.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

samba_upgradedns: Create binddns_dir if it doesn't already exist

Without doing this, the upgrade process can fail if the directory is not
present, e.g. after restoring from an offline backup (which specifically
ignores this directory).

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

testprogs: Test that dns.keytab is created after a dns upgrade

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

pyldb: Fix Message.items() for a message containing elements

Previously, message elements were being freed before the call to
Py_BuildValue(), resulting in an exception being raised. Additionally,
only the first element of the returned list was ever assigned to.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

pyldb: Add test for Message.items()

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

sambadns: Fix docstring for create_dns_dir()

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>