Alexander Bokovoy [Mon, 3 Jul 2017 08:58:50 +0000 (11:58 +0300)]
smbldap: expose bind callback via API and increase smbldap ABI version
Until we fully migrate to use gensec in smbldap, we need to continue
exposing bind callback to allow FreeIPA to integrate with smbldap.
Since smbldap API is now lacking direct access to 'struct
smbldap_state' and new API functions were added to give access to
individual members of this structure, it makes sense to increase ABI
version too.
Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Jul 4 11:14:49 CEST 2017 on sn-devel-144
Andrew Bartlett [Mon, 3 Jul 2017 02:39:09 +0000 (14:39 +1200)]
samr: Disable NTLM-based password changes on the server if NTLM is disabled
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Tim Beale [Tue, 4 Jul 2017 01:40:31 +0000 (13:40 +1200)]
selftest: Disable NTLM authentication in ktest environment
This allows us to prove that "ntlm auth = disabled" works
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11923
Andrew Bartlett [Mon, 3 Jul 2017 02:16:50 +0000 (14:16 +1200)]
param: Add new "disabled" value to "ntlm auth" to disable NTLM totally
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11923
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Tim Beale [Tue, 4 Jul 2017 01:31:11 +0000 (13:31 +1200)]
selftest: Add test to confirm NTLM authentication is enabled
(or later, that it is disabled)
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11923
Andrew Bartlett [Mon, 3 Jul 2017 02:11:47 +0000 (14:11 +1200)]
param: Disable LanMan authentication unless NTLMv1 is also enabled
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11923
Andrew Bartlett [Mon, 3 Jul 2017 22:31:40 +0000 (10:31 +1200)]
selftest: Use new ntlmv2-only and mschapv2-and-ntlmv2-only options
This will allow the py_credentials test to tell if these are in use
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Mon, 3 Jul 2017 00:11:51 +0000 (12:11 +1200)]
auth: Allow NTLMv1 if MSV1_0_ALLOW_MSVCHAPV2 is given and re-factor 'ntlm auth ='
The ntlm auth parameter is expanded to more clearly describe the
role of each option, and to allow the new mode that permits MSCHAPv2
(as declared by the client over the NETLOGON protocol) while
still banning NTLMv1.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12252
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Based on a patch by Mantas Mikulėnas <mantas@utenos-kolegija.lt>:
Commit
0b500d413c5b ("Added MSV1_0_ALLOW_MSVCHAPV2 flag to ntlm_auth")
added the --allow-mschapv2 option, but didn't implement checking for it
server-side. This implements such checking.
Additionally, Samba now disables NTLMv1 authentication by default for
security reasons. To avoid having to re-enable it globally, 'ntlm auth'
becomes an enum and a new setting is added to allow only MSCHAPv2.
Signed-off-by: Mantas Mikulėnas <mantas@utenos-kolegija.lt>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Mon, 3 Jul 2017 05:28:05 +0000 (17:28 +1200)]
selftest: Add test for support for MSCHAPv2 and NTLMv1 on a server
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Sun, 2 Jul 2017 23:28:06 +0000 (11:28 +1200)]
s3-rpc_server: Disable the NETLOGON server by default
The NETLOGON server is only needed when the classic/NT4 DC is enabled
and has been the source of security issues in the past. Therefore
reduce the attack surface.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Tim Beale [Mon, 3 Jul 2017 21:31:54 +0000 (09:31 +1200)]
tests: Add simple check whether netlogon server is running
Netlogon only needs to run in DC environment. This is a simple test to
check whether the netlogon service is running. This will allow us to
disable the netlogon service on setups that don't require it.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Andrew Bartlett [Mon, 3 Jul 2017 01:10:35 +0000 (13:10 +1200)]
auth: Disable SChannel authentication if we are not a DC
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Tue, 4 Jul 2017 04:11:12 +0000 (16:11 +1200)]
dns_server: Only install common library if AD DC is enabled.
The library is used in selftest, so must still be built
This reverts commit
d32b66b40c931fe8214faa2e1d40b34b86667d4c and
replaces the behaviour.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Ralph Boehme [Wed, 28 Jun 2017 05:14:36 +0000 (07:14 +0200)]
net: add net cache samlogon list|show|ndrdump|delete
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Jul 4 00:12:46 CEST 2017 on sn-devel-144
Ralph Boehme [Tue, 27 Jun 2017 15:34:34 +0000 (17:34 +0200)]
samlogon_cache: add netsamlog_cache_for_all()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Ralph Boehme [Mon, 3 Jul 2017 10:38:22 +0000 (12:38 +0200)]
netlogon.idl: mark session keys with NDR_SECRET
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Ralph Boehme [Mon, 3 Jul 2017 13:16:13 +0000 (15:16 +0200)]
s4/torture: test fetching a resume key twice
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Sat, 10 Jun 2017 07:05:55 +0000 (09:05 +0200)]
s3/smbd: remove unneeded flags argument from SMB_VFS_OFFLOAD_WRITE_SEND
...and instead use the fsctl to infer required behaviour in the VFS
backends.
Note that this removes the check from vfs_default because there we only
handle FSCTL_SRV_COPYCHUNK(_WRITE) and must always perform the lock
checks.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Fri, 9 Jun 2017 15:27:17 +0000 (17:27 +0200)]
s3/smbd: get rid of files_struct.aapl_copyfile_supported
A previous commit removed the special hook from the SMB layer, so we
don't need this anymore.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Tue, 6 Jun 2017 12:36:38 +0000 (14:36 +0200)]
s4/torture: more tests for copy-chunk across shares
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Fri, 9 Jun 2017 11:02:49 +0000 (13:02 +0200)]
s3/vfs: make SMB_VFS_OFFLOAD_WRITE_SEND offload token based
Remove the source fsp argument and instead pass the offload token
generated with SMB_VFS_OFFLOAD_READ_SEND/RECV.
An actual offload fsctl is not implemented yet, neither in the VFS nor
at the SMB ioctl layer, and returns NT_STATUS_NOT_IMPLEMENTED
With these changes we now pass the copy-chunk-across-shares test.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Mon, 5 Jun 2017 06:31:19 +0000 (08:31 +0200)]
s4/torture: add a test for copy-chunk across shares
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Fri, 9 Jun 2017 14:50:05 +0000 (16:50 +0200)]
s3/smbd: redesign macOS copyfile copy-chunk
The copy-chunk request chunk_count can be 0 and Windows server just
returns success saying number of copied chunks is 0.
macOS client overload this after negotiating AAPL via their SMB2
extensions, meaning it's a so called copyfile request (copy whole file
and all streams).
We previously checked this at the SMB layer, with this patch we just
send this down the VFS, if vfs_fruit is loaded it implements the macOS
copyile semantics, otherwise we get Windows behavour..
No change in behaviour.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Fri, 9 Jun 2017 14:35:39 +0000 (16:35 +0200)]
s3/smbd: remove copy-chunk chunk merging optimisation
As we won't have the source fsp around with the coming token based
offload read/write based code, we can't merge chunks as that requires
checking against the source file size.
We could still merge chunks without checking, but getting the error
handling correct would require comlicated logic for the SMB2 ioctl
copy-chunk error reporting.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Fri, 9 Jun 2017 11:08:43 +0000 (13:08 +0200)]
s3/smbd: remove unused arg smb1req from copychunk_check_handles()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Fri, 9 Jun 2017 11:02:49 +0000 (13:02 +0200)]
s3/smbd: remove flags2 FLAGS2_READ_PERMIT_EXECUTE hack in the SMB2 code
By adding a SMB2 specific CHECK_READ_SMB2 macro called that always
grants read access if execute was granted, we can get rid of the flags2
hack.
All callers in the SMB2 code are converted to use the CHECK_READ_SMB2
macro.
Amongs other things, this later allows moving the handle checks in
copychunk_check_handles() down into the VFS layer where we don't have
access to the smbreq.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Fri, 9 Jun 2017 10:57:03 +0000 (12:57 +0200)]
s3/smbd: remove ununsed req arg from CHECK_READ_IOCTL macro
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Sun, 4 Jun 2017 11:50:33 +0000 (13:50 +0200)]
s3/vfs: rename SMB_VFS_COPY_CHUNK_SEND/RECV to SMB_VFS_OFFLOAD_WRITE_SEND/RECV
No change in behaviour, just a rename in preperation of more changes to
SMB_VFS_OFFLOAD_WRITE_SEND. It helps keeping the diff of the actual
changes smaller.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Tue, 6 Jun 2017 10:23:27 +0000 (12:23 +0200)]
s3/smbd: use SMB_VFS_OFFLOAD_READ_SEND/RECV
No change in behaviour, this just uses the new SMB_VFS_OFFLOAD_READ_SEND
in the duplicate extents and the resume key ioctls.
In the copy-chunk/resume-key case this means using
SMB_VFS_OFFLOAD_READ_SEND to create the resume-key token that is
returned to the client.
In the duplicate-extents case this ensures we can later call
offload-write, which requires a previous call to offload-read that
associates a token with a file-handle.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Sat, 3 Jun 2017 10:57:59 +0000 (12:57 +0200)]
s3/vfs: add SMB_VFS_OFFLOAD_READ_SEND/RECV
Add SMB_VFS_OFFLOAD_READ_SEND an SMB_VFS_OFFLOAD_READ_RECV.
This paves the way for supporting server-side copy-chunk with source and
destination file-handles on different shares. It can be used to
implement copy offload fsctl in the future, but for now this will be
used as a mere copy-chunk replacement.
SMB_VFS_OFFLOAD_READ generates a token that associates an fsp with the
token and stores the fsp in a in-memory db.
Initially only a copy-chunk resume key fsctl is supported. In the future
this can be enhanced to support real offload fsctl.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Tue, 6 Jun 2017 12:50:15 +0000 (14:50 +0200)]
s4/torture: pass destination tree to test_setup_copy_chunk
No change in behaviour, will be used in subsequent commits.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Fri, 2 Jun 2017 11:09:41 +0000 (13:09 +0200)]
librpc/idl: make use storage_offload_token
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Fri, 2 Jun 2017 11:06:31 +0000 (13:06 +0200)]
librpc/idl: fix STORAGE_OFFLOAD_TOKEN_TYPE_ZERO_DATA definition
STORAGE_OFFLOAD_TOKEN_TYPE_ZERO_DATA is defined as 0xffff0001 in MS-FSCC
2.3.79.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Fri, 2 Jun 2017 11:05:22 +0000 (13:05 +0200)]
librpc/idl: convert offload flags to a bitmap
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Fri, 26 May 2017 16:10:07 +0000 (18:10 +0200)]
lib/util: add more tfork tests
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Thu, 18 May 2017 10:02:22 +0000 (12:02 +0200)]
lib/util: adjust loglevel in tfork test with samba_runcmd_send()
No change in behaviour, this just ensures stdout and stderror are
logged with log level 0.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Tue, 16 May 2017 16:36:03 +0000 (18:36 +0200)]
lib/util: make use of tfork in samba_runcmd_send()
This makes it possible to use samba_runcmd_send() in processes like smbd
that install a SIGCHLD handler that reaps all terminated children.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Tue, 25 Apr 2017 22:48:39 +0000 (00:48 +0200)]
lib/util: enhanced tfork()
This function is a solution to the problem of fork() requiring special
preperations in the caller to handle SIGCHLD signals and to reap the
child by wait()ing for it.
Instead, tfork provides a pollable file descriptor. The caller gets the
file descriptor by calling tfork_event_fd() on the handle returned from
tfork_create() and the caller can then get the status of the child
with a call to tfork_status().
tfork avoids raising SIGCHLD signals in the caller by installing a
temporary SIGCHLD handler from inside tfork_create() and tfork_status().
The termination signal of other child processes not created with tfork()
is forwarded to the existing signal handler if any.
There's one thing this thing can't protect us against and that is if a
process installs a SIGCHLD handler from one thread while another thread
is running inside tfork_create() or tfork_status() and the signal
handler doesn't forward signals for exitted childs it didn't fork, ie
our childs.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Stefan Metzmacher [Fri, 30 Jun 2017 11:26:17 +0000 (13:26 +0200)]
s3:rpc_server: wrap make_auth4_context() into {become,unbecome}_root()
This need to create a temporary messaging context in order to do
the auth logging. This can only be done as root.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12850
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Mon Jul 3 08:15:29 CEST 2017 on sn-devel-144
Garming Sam [Mon, 3 Jul 2017 01:15:50 +0000 (13:15 +1200)]
WHATSNEW: Improved AD performance (particularly linked attributes)
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Garming Sam [Mon, 3 Jul 2017 01:09:26 +0000 (13:09 +1200)]
WHATSNEW: DNS at domain join improvements
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Garming Sam [Mon, 3 Jul 2017 00:46:09 +0000 (12:46 +1200)]
WHATSNEW: Additional hashes introduced with WDigest
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Garming Sam [Sun, 2 Jul 2017 23:51:10 +0000 (11:51 +1200)]
WHATSNEW: Improved RODC support
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Andrew Bartlett [Sat, 1 Jul 2017 10:20:17 +0000 (22:20 +1200)]
selftest: Prime the netlogon cache during test_idmap_rfc2307
This ensures that the group memberships just created are reflected in the test
comparison. Otherwise we are trusting that no caches are primed, which is
simply not safe in a test.
(The login will put a list of groups, as obtained by the login over NETLOGON or
via the PAC, into the samlogon cache).
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Sun Jul 2 21:59:18 CEST 2017 on sn-devel-144
Andrew Bartlett [Sat, 1 Jul 2017 09:44:38 +0000 (21:44 +1200)]
selftest: Bind rfc2307 tests to exactly one server
The tests make changes to the DC, do not wait for replication, then expect
those to be reflected in the client. If they bind to another server this
will not hold true.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Sat, 1 Jul 2017 09:34:44 +0000 (21:34 +1200)]
selftest: Use tree_delete control in idmap_rfc2307 test
This control removes an entire subtree, which was the intention of the previouse code
but much more effectively.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Bob Campbell [Mon, 9 Jan 2017 20:20:47 +0000 (09:20 +1300)]
selftest: Do not force run of kcc at start of selftest
This should help to avoid clashes between periodic and manual runs of
the KCC during autobuild.
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Fri, 30 Jun 2017 04:02:46 +0000 (16:02 +1200)]
WHATSNEW: Add entry for Multi-process LDAP Server
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Thu, 29 Jun 2017 00:50:03 +0000 (12:50 +1200)]
WHATSNEW: Add an entry for the LDB whole DB locking issue
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Mon, 17 Oct 2016 00:55:42 +0000 (13:55 +1300)]
ldap: Run the LDAP server with the default (typically standard) process model
This allows one LDAP socket to proceed if another fails, and reduces the
impact of a crash becoming a DoS bug, as it only impacts one socket.
This may mean we have a lot of idle tasks, but this should not be a big
issue
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Wed, 28 Jun 2017 05:34:05 +0000 (17:34 +1200)]
dsdb: Improve debugging on start transacton failure
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Mon, 26 Jun 2017 02:13:41 +0000 (14:13 +1200)]
dsdb: Teach the Samba partition module how to lock all the DB backends
The metadata partition (sam.ldb) lock is not
enough to block another process in prepare_commit(),
because prepare_commit() is a no-op, if nothing
was changed in the specific backend.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Mon, 26 Jun 2017 01:34:21 +0000 (13:34 +1200)]
dsdb: Add tests showing that the CN=CONFIGURATION partition is also locked
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Mon, 26 Jun 2017 01:16:01 +0000 (13:16 +1200)]
dsdb: Add new test adding a record to the top level sam.ldb file
This shows that locks are made on this file as well
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Stefan Metzmacher [Fri, 23 Jun 2017 10:13:19 +0000 (12:13 +0200)]
dsdb: Add more locking more tests, confirming blocking locks in both directions
These extended tests allow us to show that a search (read) blocks a
transaction commit (write), and that a transaction commit blocks a
search.
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Andrew Bartlett [Fri, 16 Jun 2017 03:49:45 +0000 (15:49 +1200)]
dsdb: Add test showing a search can't start while a transaction is already repared in a backend partition
Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Fri, 16 Jun 2017 03:49:45 +0000 (15:49 +1200)]
dsdb: Add test showing a search can't start while a transaction is already repared
Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Stefan Metzmacher [Tue, 11 Apr 2017 15:50:08 +0000 (17:50 +0200)]
ldb: version 1.2.0
* handle one more LDB_FLAG_INTERNAL_DISABLE_SINGLE_VALUE_CHECK
case in ldb_tdb
* fix ldb_tdb locking (performance) problems
* fix ldb_tdb search inconsistencies by adding
read_[un]lock() hooks to the module stack
(bug #12858)
* add cmocka based tests for the locking issues
* ldb_version.h provides LDB_VERSION_{MAJOR,MINOR,RELEASE} defines
* protect ldb_modules.h from being used by Samba < 4.7
Note: that this release (as well as 1.1.30 and 1.1.31)
may cause problems for older applications, e.g. Samba
See https://bugzilla.samba.org/show_bug.cgi?id=12859
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Stefan Metzmacher [Fri, 30 Jun 2017 06:14:02 +0000 (08:14 +0200)]
ldb:includes: protect ldb_modules.h from being used by Samba < 4.7
Samba versions before 4.7 are incompatible with the read_[un]lock()
behaviour introduced into ldb.
This makes sure older Samba versions fail to compile against
ldb >= 1.2.0.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12859
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Stefan Metzmacher [Fri, 30 Jun 2017 06:09:38 +0000 (08:09 +0200)]
ldb:wscript: define EXPECTED_SYSTEM_LDB_VERSION_{MAJOR,MINOR,RELEASE}
This indicates what feature set Samba assumes from the used
libldb from the system.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12859
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Stefan Metzmacher [Fri, 23 Jun 2017 08:50:54 +0000 (10:50 +0200)]
ldb:wscript: provide LDB_VERSION_{MAJOR,MINOR,RELEASE} in ldb_version.h
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Andrew Bartlett [Fri, 16 Jun 2017 03:49:16 +0000 (15:49 +1200)]
ldb:tests: Extend api.py testsuite to show transaction_commit() blocks against the whole-db read lock
The new ldb whole-db lock behaviour now allows this test
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Fri, 16 Jun 2017 03:44:46 +0000 (15:44 +1200)]
ldb:tests: Extend api.py testsuite to show transaction contents can not be seen outside the transaction
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Fri, 16 Jun 2017 00:19:00 +0000 (12:19 +1200)]
ldb:tests: Add test to show that locks are released on TALLOC_FREE(req)
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Fri, 16 Jun 2017 00:18:39 +0000 (12:18 +1200)]
ldb:tests: Correct comment about version numbers
(ldb releases have been made while this patch set was in train)
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Thu, 15 Jun 2017 01:56:46 +0000 (13:56 +1200)]
ldb: Lock the whole backend database for the duration of a search
We must hold locks not just for the duration of each search, but for the whole search
as our module stack may make multiple search requests to build up the whole result.
This is explains a number of replication and read corruption issues in Samba
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Thu, 11 May 2017 23:39:08 +0000 (01:39 +0200)]
ldb_tdb: Implement read_lock and read_unlock module operations
This allows Samba to provide a consistent view of the DB
despite the use of multiple databases via the partitions module
and over multiple callbacks via a module stack.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Thu, 15 Jun 2017 00:10:51 +0000 (12:10 +1200)]
ldb: Add read_lock and read_unlock to ldb_module_ops
This will be used to implement read locking in ldb_tdb
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Mon, 22 May 2017 04:18:20 +0000 (16:18 +1200)]
ldb:tests: Add test encoding current locking behaviour during ldb_search()
Currently, a lock is not held against modifications once the final
record is returned via a callback, so modifications can be made
during the DONE callback. This makes it hard to write modules
that interpert an ldb search result and do further processing
so will change in the future to allow the full search to be
atomic.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Tue, 25 Apr 2017 10:33:53 +0000 (22:33 +1200)]
ldb:tests: Show that writes do not appear during an ldb_search()
A modify or rename during a search must not cause a search to change
output, and attributes having an index should in particular not see
any change in behaviour in this respect
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Thu, 15 Jun 2017 01:56:46 +0000 (13:56 +1200)]
ldb:tests: don't assert the results before doing the final search finished
This is required to pass the test in future, because
otherwise the clean up will fail because we hold locks.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Garming Sam [Wed, 29 Mar 2017 23:03:17 +0000 (12:03 +1300)]
ldb:tdb: Ensure we correctly decrement ltdb->read_lock_count
If we do not do this, then we never take the all record lock, and instead do a lock
for every record as we go, which is very slow during a large search
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Fri, 30 Jun 2017 00:55:15 +0000 (12:55 +1200)]
ldb_pack: use ldb_dn_from_ldb_val() and avoid a duplicate strlen() call
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Stefan Metzmacher [Tue, 11 Apr 2017 15:27:33 +0000 (17:27 +0200)]
tdb: version 1.3.14
* allow tdb_traverse_read before tdb_transaction[_prepare]_commit()
* Improve documentation for tdb_transaction_start()
* Add new function tdb_transaction_active()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Andrew Bartlett [Wed, 26 Apr 2017 20:34:56 +0000 (08:34 +1200)]
tdb: Add new function tdb_transaction_active()
This will allow callers to avoid their own reference counting of transactions.
Additionally, this will always line up with the acutal transaction state, even
in the error cases where tdb can cancel the transaction
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Wed, 26 Apr 2017 20:51:08 +0000 (08:51 +1200)]
tdb: Improve documentation for tdb_transaction_start()
It now references the TDB_ALLOW_NESTING and TDB_DISALLOW_NESTING flags
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Fri, 31 Mar 2017 04:34:13 +0000 (17:34 +1300)]
tdb: Remove locking from tdb_traverse_read()
This restores the original intent of tdb_traverse_read() in
7dd31288a701d772e45b1960ac4ce4cc1be782ed
This is needed to avoid a deadlock with tdb_lockall() and the
transaction start, as ldb_tdb should take the allrecord lock during a
search (which calls tdb_traverse), and can otherwise deadlock against
a transaction starting in another process
We add a test to show that a transaction can now start while a read
traverse is in progress
This allows more operations to happen in parallel. The blocking point
is moved to the prepare commit.
This in turn permits a roughly doubling of unindexed search
performance, because currently ldb_tdb omits to take the lock due to
an unrelated bug, but taking the allrecord lock triggers the
above-mentioned deadlock.
This behaviour was added in
251aaafe3a9213118ac3a92def9ab2104c40d12a for
Solaris 10 in 2005. But the run-fcntl-deadlock test works also on Solaris 10,
see https://lists.samba.org/archive/samba-technical/2017-April/119876.html.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Wed, 3 May 2017 04:34:01 +0000 (06:34 +0200)]
drsuapi: Improve debugging in DsAddEntry()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Stefan Metzmacher [Fri, 30 Jun 2017 04:21:32 +0000 (06:21 +0200)]
wafsamba: add maxversion and version_blacklist to CHECK_BUNDLED_SYSTEM[_PKG]()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12859
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jeremy Allison [Fri, 30 Jun 2017 20:37:03 +0000 (13:37 -0700)]
s3: VFS: Change SMB_VFS_CONNECTPATH to take const struct smb_filename * instead of const char *.
We need to migrate all pathname based VFS calls to use a struct
to finish modernising the VFS with extra timestamp and flags parameters.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Richard Sharpe <realrichardsharpe@gmail.com>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Jul 1 07:20:28 CEST 2017 on sn-devel-144
Jeremy Allison [Fri, 30 Jun 2017 18:59:20 +0000 (11:59 -0700)]
S3: smbd: Finish plumbing struct smb_filename * through the check_name() stack.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Richard Sharpe <realrichardsharpe@gmail.com>
Jeremy Allison [Fri, 30 Jun 2017 18:34:13 +0000 (11:34 -0700)]
s3: smbd: Add missing out of memory check.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Richard Sharpe <realrichardsharpe@gmail.com>
Jeremy Allison [Fri, 30 Jun 2017 18:32:59 +0000 (11:32 -0700)]
s3: VFS: Change SMB_VFS_REALPATH to take and return struct smb_filename * instead of char *.
We need to migrate all pathname based VFS calls to use a struct
to finish modernising the VFS with extra timestamp and flags parameters.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Richard Sharpe <realrichardsharpe@gmail.com>
Jeremy Allison [Thu, 29 Jun 2017 21:32:47 +0000 (14:32 -0700)]
s3: VFS: Change SMB_VFS_GETWD to return struct smb_filename * instead of char *.
We need to migrate all pathname based VFS calls to use a struct
to finish modernising the VFS with extra timestamp and flags parameters.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Richard Sharpe <realrichardsharpe@gmail.com>
Jeremy Allison [Thu, 29 Jun 2017 18:29:33 +0000 (11:29 -0700)]
s3: VFS: Change SMB_VFS_CHDIR to use const struct smb_filename * instead of const char *.
We need to migrate all pathname based VFS calls to use a struct
to finish modernising the VFS with extra timestamp and flags parameters.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Richard Sharpe <realrichardsharpe@gmail.com>
Garming Sam [Fri, 23 Jun 2017 00:37:01 +0000 (12:37 +1200)]
show-deleted: Rename attr_filter to exclude_filter for clarity
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Jun 30 06:23:39 CEST 2017 on sn-devel-144
Garming Sam [Fri, 23 Jun 2017 00:35:56 +0000 (12:35 +1200)]
show-deleted: Simplify the code to require as little logic as needed
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Garming Sam [Fri, 23 Jun 2017 00:18:35 +0000 (12:18 +1200)]
show-deleted: Remove an unnecessary search during connect
This is only required if you supply SHOW_RECYCLED or SHOW_DELETED. Note
that any add does trigger this (through callbacks in the modules in acl,
objectclass etc.).
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Garming Sam [Tue, 27 Jun 2017 01:02:49 +0000 (13:02 +1200)]
show-deleted: Do not indicate an error if an object is missing.
This happens during provision, however due to the fact that the first
search in the rootDSE init does not check return codes, this was done
implicitly (and coincidentally).
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Andrew Bartlett [Wed, 28 Jun 2017 00:22:05 +0000 (12:22 +1200)]
dsdb: Add a dummy module to replace show_deleted
This helps when we improve show_deleted in a way that the fake database in samba3sam can not cover
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Fri, 5 May 2017 20:33:47 +0000 (22:33 +0200)]
travis-ci: Also build samba-systemkrb5
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Thu, 29 Jun 2017 23:11:05 +0000 (11:11 +1200)]
autobuild: Use new selftest.pl feature to run only some environments
This is cleaner than test filtering with regular expressions
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Mon, 27 Feb 2017 21:45:24 +0000 (10:45 +1300)]
selftest: Allow selftest.pl to run just some environments
This makes it easier to declare that some autobuild environments
only run some selftest environments.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Mon, 15 May 2017 20:32:03 +0000 (08:32 +1200)]
debug: new debug class for kerberos
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Stefan Metzmacher [Wed, 14 Jun 2017 01:29:58 +0000 (03:29 +0200)]
auth/spnego: do basic state_position checking in gensec_spnego_update_in()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Jun 29 20:15:05 CEST 2017 on sn-devel-144
Stefan Metzmacher [Tue, 13 Jun 2017 21:41:01 +0000 (23:41 +0200)]
auth/spnego: move gensec_spnego_update() into gensec_spnego_update_send()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Stefan Metzmacher [Fri, 30 Dec 2016 05:56:47 +0000 (06:56 +0100)]
auth/spnego: split out gensec_spnego_update_{client,server}() functions
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Stefan Metzmacher [Tue, 27 Jun 2017 16:05:04 +0000 (18:05 +0200)]
auth/spnego: remove unused out_mem_ctx = spnego_state fallback in gensec_spnego_update()
The only caller never passes NULL.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Stefan Metzmacher [Wed, 10 May 2017 12:44:48 +0000 (14:44 +0200)]
auth/spnego: add gensec_spnego_update_sub_abort() helper function
This helps to be consistent when destroying a unuseable sub context.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Stefan Metzmacher [Fri, 30 Dec 2016 08:06:33 +0000 (09:06 +0100)]
auth/spnego: remove useless spnego_state->sub_sec_ready check
The lines above make sure it's always true.
Check with git show -U15
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>