s3: torture: Add SMB1-TRUNCATED-SESSSETUP test.

Shows that we indirect through an uninitialized pointer and the client crashes
it's own smbd.

Add knownfail.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15420

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: smbd: Deliberately currupt an uninitialized pointer.

We will need this to show smbd crashing in the test code.
This will be removed once we're passing the test.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15420

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s4:kdc: Add get_claims_set_for_principal()

Add a new function, get_claims_set_for_principal(), that returns the
claims as a CLAIMS_SET structure rather than as a blob. To accommodate
this, move the call to encode_claims_set() out of get_all_claims() and
into get_claims_blob_for_principal().

Being able to get the unencoded claims will save us from having to
decode claims that we just needlessly encoded.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Aug 14 05:51:45 UTC 2023 on atb-devel-224

s4:kdc: Rename ‘claims_blob’ parameter to ‘claims_blob_out’

Just to make perfectly clear that it is an out parameter.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Rename get_claims_for_principal() to get_claims_blob_for_principal()

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Ensure that we don’t dereference a NULL pointer

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Properly allocate claims set on a talloc context

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Remove ldb_context parameter as being no longer needed

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Have encode_claims_set return NTSTATUS

This change will simplify things later. Probably.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:dsdb: Check for overflow in security_token_create()

Overflow is unlikely ever to occur, but you never know.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:dsdb: Make ‘sids’ parameter const

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:dsdb: Use uint32_t for ‘num_sids’

This matches the use of uint32_t for security_token::num_sids.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Free error message returned by krb5_get_error_message()

Also check whether the message is NULL. Passing NULL to vasprintf() is
undefined behaviour.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Use common out path in mit_samba_kpasswd_change_password()

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Inline samba_get_claims_blob()

Wrapping a function this simple doesn’t gain us very much.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Don’t enforce a server authentication policy for the krbtgt

As the server authentication policy will be non-NULL only for entries
looked up as servers, the krbtgt shouldn’t have an authentication policy
anyway. But we might as well be explicit.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Switch to using samdb_result_dom_sid_buf()

This function doesn’t require a heap allocation.

We also check the result of the function, which we weren’t doing before.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Return krb5_error_code

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Make some functions static

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Return (possibly) more appropriate error codes

This change ultimately won’t make much difference to responses, as
unrecognized codes are mapped to ERR_GENERIC in any case. But it might
provide some help for debugging.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:auth: Check return value of talloc_new()

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:auth: Fix leaks

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:auth: Add missing space to error message

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Use type bool for ‘is_tgs’

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Erase key data

If we’re going to zero the keys before freeing them, we might as well do
it properly.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Ensure the value of h->len is accurate

If we exited this function early due to an error, h->len would contain
the number of elements that *ought* to be in h->val, but not all of
those elements must have been initialized. Subsequently trying to free
this partially-uninitialized structure with free_Keys() could have bad
results.

Avoid this by ensuring that h->len accurately reports the actual number
of initialized elements.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Consistently zero HDB structures

To these conversion functions we sometimes pass malloc-allocated HDB
structures, which we free afterwards if conversion fails. If parts of
these structures are still uninitialized when we try to free them, all
sorts of fun things can result.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Fix leaks of sdb_entry’s members

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Fail PAC checksum verification if the krbtgt entry has no keys

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Correctly report length of KDC packet

If the data was received over TCP, it would have had four bytes
subtracted from its length already, in kdc_tcp_call_loop().

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Use portable format specifier

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Correct error message

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Return an error code if sdb_entry_to_hdb_entry() fails

This condition was written backwards — if samba_kdc_fetch() returned
zero, we would ignore any error code returned by
sdb_entry_to_hdb_entry().

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Fix code spelling

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Fix leaks

Use a temporary context to allocate these variables. Each variable that
needs to be transferred to the caller is stolen onto an appropriate
talloc context just prior to the function’s returning.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Move calls to talloc_steal() out of the ‘out’ paths

This simplifies the ‘out’ paths.

Every code path that reaches ‘out’ via a goto ensures that ‘ret’ is set
to a nonzero value.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Remove unnecessary talloc context

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Call krb5_free_principal() directly after to-be-freed principal is used

This simplifies the ‘out’ path.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Free samba_kdc_seq context on failure to allocate memory

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Check return value from ldb_dn_get_linearized()

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Fix leak of sdb_entry

We should take the common ‘out’ path to ensure that we call
sdb_entry_free() on the entry.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Ensure we don’t increase the value of entry->etypes->len

The value of entry->etypes->len ought to be equal to that of
entry->keys.len, and so should be nonzero. But it’s safer not to rely on
that assumption.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Check result of samdb_result_dom_sid()

We must not pass a NULL pointer into dom_sid_split_rid().

While we’re at it, switch to using samdb_result_dom_sid_buf(), which
doesn’t require a heap allocation.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Initialize entry->modified_by

If smb_krb5_make_principal() fails without setting the principal,
sdb_entry_free() will try to free whatever memory the uninitialized
member points to.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Don’t log secret keys

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Don’t issue forwardable or proxiable tickets to Protected Users

If an authentication policy enforces a maximum TGT lifetime for a
Protected User, that limit should stand in place of the four-hour limit
usually applied to Protected Users; we should nevertheless continue to
ensure that forwardable or proxiable tickets are not issued to such
users.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests/krb5: Test that neither forwardable nor proxiable tickets are issued to Protected Users

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests/krb5: Allow specifying KDC options when requesting a TGT

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests/krb5: Allow specifying additional details for a test account

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests/krb5: Split out new AuthnPolicyBaseTests class

This class provides a useful base for other tests to build upon.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests/krb5: Move some functions round to prepare for splitting the class

No effective code change.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests/krb5: Keep claim types for subsequent tests

We want to be able to reuse them across several tests.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests/krb5: Rename compatability_tests class

We should not have two unrelated classes both named SimpleKerberosTests.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests/krb5: Allow cached=True with an assigned silo or policy

We might want to create an account with an assigned silo or policy in
setUp() or setUpClass() to be reused in subsequent tests.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Refer to correct function in error messages

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture: Fix code spelling

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:rpc_server: Fix code spelling

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Correct comments mentioning Heimdal

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: Remove unnecessary casts

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:dsdb: Access correct member of union

Accessing the wrong member of a union invokes undefined behaviour.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:dsdb: Remove unnecessary casts

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:auth: Fix code spelling

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:registry: Fix code spelling

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:lib: Use portable format specifiers

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc:ndr: Format sizes as ‘size_t’

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tsocket: Fix code spelling

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

lib/replace: Ensure that __STDC_WANT_LIB_EXT1__ is set to 1

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

lib:krb5_wrap: Remove unnecessary cast

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb:ldb_sqlite3: Access correct member of union

Accessing the wrong member of a union invokes undefined behaviour.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb: Fix code spelling

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

docs-xml: Fix code spelling

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

auth: Use portable format specifier

Also, correctly format the value as unsigned.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

auth: Remove unnecessary casts

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

bootstrap: Install codespell

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Aug 14 04:40:03 UTC 2023 on atb-devel-224

krb5: Increase the minimum MIT Krb5 version to 1.21

This is the version we test with in CI after the image update
in the next commit.  This addresses the issues that were
fixed in CVE-2022-37967 (KrbtgtFullPacSignature) and ensures
that Samba builds against the MIT version that allows us to
avoid that attack.

The hooks to allow these expectations to be disabled in the tests
are kept for now, to allow this to be reverted or to test
older servers.

With MIT 1.21 as the new test standard for the MIT KDC build
we update the knownfail_mit_kdc - this was required regadless
after the CI image update.

Any update to the CI image, even an unrelated one, brings in
a new MIT Krb5, version 1.21-3 in this case.  This has new
behaviour that needs to be noted in the knownfail files or
else the tests, which haven't changed, will fail and
pipelines won't pass.

(The image generated by the earlier bootstrap commit brought
in krb5-1.21-2 which was buggy with CVE-2023-39975)

Further tweaks to tests or the server should reduce the number
of knownfail entries, but this keeps the pipelines passing for now.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15231

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>

selftest: Allow MIT Krb5 1.21 to still start to fl2000dc

This is the simplest way to keep this test environment alive.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>

bootstrap: Heimdal no longer requires perl-JSON

Heimdal after lorikeet-heimdal-202307040259
(commit 33d117b8a9c11714ef709e63a005d87e34b9bfde)
includes Heimdal master commit f62e2f278437ff6c03d2d09bd628381c795bba78.

This has PR https://github.com/heimdal/heimdal/pull/1176 and no
longer requires the external JSON module, as JSON::PP is builtin.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15394

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>

WHATSNEW: Remove unusual box around 'REMOVED FEATURES'

We do not normally put the ==== above the titles, per recent practice.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>

python/samba/netcmd/domain/schemaupgrade.py: fix missing newline

Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sun Aug 13 22:54:55 UTC 2023 on atb-devel-224

s4-rpc_server/drsupai: Avoid looping with Azure AD Connect by not incrementing temp_highest_usn for the NC root

We send the NC root first, as a special case for every chunk
that we send until the natural point where it belongs.

We do not bump the tmp_highest_usn in the highwatermark that
the client and server use (it is meant to be an opauqe cookie)
until the 'natural' point where the object appears, similar
to the cache for GET_ANC.

The issue is that without this, because the NC root was sorted
first in whatever chunk it appeared in but could have a 'high'
highwatermark, Azure AD Connect will send back the same
new_highwatermark->tmp_highest_usn, and due to a bug,
a zero reserved_usn, which makes Samba discard it.

The reserved_usn is now much less likely to ever be set because
the tmp_higest_usn is now always advancing.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-rpc_server/drsuapi: Ensure logs show DN for replicated objects, not (null)

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15407
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-rpc_server/drsuapi: Update getnc_state to be != NULL

This is closer to our READDME.Coding style

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-rpc_server/drsuapi: Rename ncRoot -> untrusted_ncRoot to avoid misuse

Because of the requirement to echo back the original string, we can
not force this to be a trustworthy value.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-rpc_server/drsuapi: Avoid modification to ncRoot input variable in GetNCChanges

This tries to avoid it appearing that ncRoot is a value that can
be trusted and used internally by not updating it and instead leaving
it just as an input/echo-back value.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-rpc_server/drsuapi: Fix indentation in GetNCChanges()

This avoids the indentation correction being in the previous patch.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-rpc_server/drsuapi: Only keep and invalidate replication cycle state for normal replication

This changes the GetNCChanges server to use a per-call state for
extended operations like RID_ALLOC or REPL_OBJ and only maintain
and (more importantly) invalidate the state during normal replication.

This allows REPL_OBJ to be called during a normal replication cycle
that continues using after that call, continuing with the same
highwatermark cookie.

Azure AD will do a sequence of (roughly)

* Normal replication (objects 1..100)
* REPL_OBJ (of 1 object)
* Normal replication (objects 101..200)

However, if there are more than 100 (in this example) objects in the
domain, and the second replication is required, the objects 1..100
are sent, as the replication state was invalidated by the REPL_OBJ call.

RN: Improve GetNChanges to address some (but not all "Azure AD Connect")
syncronisation tool looping during the initial user sync phase.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-torture/drs: Add test showing that if present in the set the NC root leads and tmp_highest_usn moves

The NC root, on any replication when it appears, is the first object to be
replicated, including for all subsequent chunks in the replication.

However the tmp_highest_usn is not updated by that USN, it must
only be updated for the non-NC changes (to match Windows exactly),
or at least only updated with the non-NC changes until it would
naturally appear.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-torture/drs: Add test demonstrating that a GetNCChanges REPL_OBJ will not reset the replication cookie

This demonstrates the behaviour used by the "Azure AD Connect" cloud sync tool.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-torture/drs: Add a test matching Azure AD Connect REPL_OBJ behaviour

Azure AD Connect will send a GUID but no DummyDN.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-torture/drs: Use addCleanup() in getchanges.py for OU handling

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-torture/drs: Create temp OU with a unique name per test

It is always better to keep the testing OUs unique if possible.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-torture/drs: Save the server dnsname on the DcConnection object

This object is used to hold one of many possible connections and
it is helpful for debugging and uniqueness to know which DC is being
connected to.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-rpc_server/drsuapi: Remove rudundant check for valid and non-NULL ncRoot_dn

This check was valuable before aee2039e63ceeb5e69a0461fb77e0f18278e4dc4
but now only checks things we know to be true, as the value has come
from Samba via drs_ObjectIdentifier_to_dn_and_nc_root() either on this
or a previous call.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-dsdb: Improve logging for drs_ObjectIdentifier_to_dn_and_nc_root()

At this layer we can make a reasonable assumption about being able
to read ldb_errstring() to print that for extra useful debugging.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-rpc_server/drsuapi: Improve debug message for drs_ObjectIdentifier_to_dn_and_nc_root() failure

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-rpc_server/drsuapi: Improve debugging of invalid DNs

This is still unreachable, so but improve the logging
to give more detail in this area anyway.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-rpc_server/drsuapi: Add tmp_highest_usn tracking to replication log

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

smbclient3: Get all reparse data for allinfo

If we hit a reparse point in point, it might be something but a
symlink.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Aug 10 14:36:40 UTC 2023 on atb-devel-224

libsmb: Retry with OPEN_REPARSE_POINT on IO_REPARSE_TAG_NOT_HANDLED

Eventually we'll have to make STOPPED_ON_SYMLINK special to handle the
symlink response, but for now they are the same.

STOPPED_ON_SYMLINK will tell us where the symlink is,
REPARSE_TAG_NOT_HANDLED won't. So if there's an unhandled reparse
point somewhere in the path, there's no really good way to handle
this. We'll get the REPARSE_TAG_NOT_HANDLED the second time as
well. Even SMB1 QPATHINFO gets this when you try to cross a NFS
reparse point.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Factor out cli_get_reparse_data() from cli_readlink()

Will be used in smbclient's allinfo command: Reparse points are more
than just symlinks.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>