samba.git
4 years agoldb: Bump version to 2.1.3 ldb-2.1.3
Gary Lockyer [Thu, 30 Apr 2020 20:57:30 +0000 (08:57 +1200)]
ldb: Bump version to 2.1.3

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14330

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
4 years agolib ldb: lmdb init var before calling mdb_reader_check
Gary Lockyer [Tue, 31 Mar 2020 19:22:08 +0000 (08:22 +1300)]
lib ldb: lmdb init var before calling mdb_reader_check

Initilalise "stale" to zero before passing a pointer to it to
mdb_reader_check.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Apr  7 12:29:00 UTC 2020 on sn-devel-184

(cherry picked from commit d1f4002b914efb31aa34a59e7c93d80f3174727c)

4 years agolib ldb: lmdb clear stale readers on write txn start
Gary Lockyer [Sun, 29 Mar 2020 23:08:30 +0000 (12:08 +1300)]
lib ldb: lmdb clear stale readers on write txn start

In use process failures and Bind9 shut downs leave stale entries in the
lmdb reader table.  This can result in lmdb filling it's database file, as
the free list can not be reclaimed due to the stale reader.

In this fix we call mdb_reader_check at the start of each transaction,
to free any stale readers.  As the default maximum number of readers is
127, this should not impact on performance to any great extent.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14330

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Mar 31 01:26:07 UTC 2020 on sn-devel-184

(cherry picked from commit 89041a6d18a1d091ea713e6986cac5ca66c2b481)

4 years agoldb tests: Confirm lmdb free list handling
Gary Lockyer [Mon, 16 Mar 2020 02:18:12 +0000 (15:18 +1300)]
ldb tests: Confirm lmdb free list handling

Add cmocka tests to confirm lmdb's handling of the free list.

As a result of lmdb's MVCC (Multiversion Concurrency Control) long
running read transactions or stale readers (read transactions where the
process exited without ending the transaction) can cause the database to
run out of space.

Items in the free list are only reused when they would not be visible in
a read transaction.  So long running read transactions prevent entries
in the free list being reused, and the database can run out of space.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit fc13304d1575ad6bc6e04cdb3eedf46d3c3678c7)

4 years agolib:util: Fix smbclient -l basename dir
Amit Kumar [Wed, 22 Apr 2020 11:53:42 +0000 (06:53 -0500)]
lib:util: Fix smbclient -l basename dir

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14345

Pair-Programmed-With: Andreas Schneider <asn@samba.org>
Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
(cherry picked from commit c83ce5f4f99aef94530411ec82cc03e9935b352d)

4 years agoAdd a test for smbclient -l basename
Amit Kumar [Wed, 22 Apr 2020 11:53:33 +0000 (06:53 -0500)]
Add a test for smbclient -l basename

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14345

Signed-off-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
(cherry picked from commit 2a7fc40fb3f3ca994cecad3e2957433d7a411208)

4 years agos3: pass DCE RPC handle type to create_policy_hnd
Alexander Bokovoy [Tue, 28 Apr 2020 18:59:46 +0000 (21:59 +0300)]
s3: pass DCE RPC handle type to create_policy_hnd

Various RPC services expect policy handles of a specific type.

s3 RPC server did not allow to create policy handles with a specific
type while actually requiring that policy handle type itself in some
places.

Make sure we are able to specify the policy on-wire handle type when
creating the policy handle. The changes follow s4 DCE RPC server
implementation.

The original logic to always set on-wire handle type to 0 can be tracked
down to commit fdeea341ed1bae670382e45eb731db1b5838ad21 when we didn't
really know about differences in on-wire handle types.

All but LSA trusted domain RPC calls do not check the on-wire handle
type in s3 RPC server.

Fixes trusted domain operations when Samba RPC client attempts to call
s3 RPC server to perform lsa_lsaRSetForestTrustInformation in FreeIPA.
This fix is a pre-requisite for FreeIPA-FreeIPA forest trust.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14359

Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Apr 28 22:55:29 UTC 2020 on sn-devel-184

(cherry picked from commit c7a4578d06427a82ead287f0c5248c1a54cc9336)

Autobuild-User(v4-12-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-12-test): Mon May  4 13:06:07 UTC 2020 on sn-devel-184

4 years agodbwrap_watch: Set rec->value_valid while returning nested share_mode_do_locked()
Anoop C S [Mon, 20 Apr 2020 09:11:18 +0000 (14:41 +0530)]
dbwrap_watch: Set rec->value_valid while returning nested share_mode_do_locked()

As reported on samba-technical by Rouven WEILER <Rouven_Weiler@gmx.net>:
https://lists.samba.org/archive/samba-technical/2020-April/135116.html

Following backtrace was observed with vfs_fruit for time machine backup:

[2020/04/10 08:00:38.107917,  0] ../../lib/dbwrap/dbwrap.c:82(dbwrap_record_get_value)
  PANIC: assert failed at ../../lib/dbwrap/dbwrap.c(82): rec->value_valid
[2020/04/10 08:00:38.108499,  0] ../../source3/lib/util.c:830(smb_panic_s3)
  PANIC (pid 3427): assert failed: rec->value_valid
[2020/04/10 08:00:38.109541,  0] ../../lib/util/fault.c:265(log_stack_trace)
  BACKTRACE: 37 stack frames:
   #0 /usr/lib/samba/amd64/libsamba-util.so.0.0.1'log_stack_trace+0x26 [0xfffffd7fee51de66]
   #1 /usr/lib/samba/amd64/libsmbconf.so.0'smb_panic_s3+0x26 [0xfffffd7fedf5a596]
   #2 /usr/lib/samba/amd64/libsamba-util.so.0.0.1'smb_panic+0x1f [0xfffffd7fee51df3f]
   #3 /usr/lib/samba/private/amd64/libdbwrap-samba4.so'dbwrap_record_get_value+0x2a [0xfffffd7feccb627a]
   #4 /usr/lib/samba/private/amd64/libsmbd-base-samba4.so'get_share_mode_lock+0x109 [0xfffffd7fee7195c9]
   #5 /usr/lib/samba/private/amd64/libsmbd-base-samba4.so'smbd_contend_level2_oplocks_begin+0xa1 [0xfffffd7fee7f7761]
   #6 /usr/lib/samba/private/amd64/libsmbd-base-samba4.so'brl_lock+0x635 [0xfffffd7fee710f45]
   #7 /usr/lib/samba/private/amd64/libsmbd-base-samba4.so'do_lock_fn+0xa4 [0xfffffd7fee70d534]
   #8 /usr/lib/samba/private/amd64/libsmbd-base-samba4.so'share_mode_do_locked_fn+0x86 [0xfffffd7fee7174b6]
   #9 /usr/lib/samba/amd64/libsmbconf.so.0'dbwrap_watched_do_locked_fn+0xfa [0xfffffd7fedf622ca]
   #10 /usr/lib/samba/private/amd64/libdbwrap-samba4.so'db_tdb_do_locked+0x12f [0xfffffd7feccb95cf]
   #11 /usr/lib/samba/private/amd64/libdbwrap-samba4.so'dbwrap_do_locked+0x48 [0xfffffd7feccb69a8]
   #12 /usr/lib/samba/amd64/libsmbconf.so.0'dbwrap_watched_do_locked+0x6f [0xfffffd7fedf60d7f]
   #13 /usr/lib/samba/private/amd64/libdbwrap-samba4.so'dbwrap_do_locked+0x48 [0xfffffd7feccb69a8]
   #14 /usr/lib/samba/private/amd64/libsmbd-base-samba4.so'share_mode_do_locked+0xd2 [0xfffffd7fee719b82]
   #15 /usr/lib/samba/private/amd64/libsmbd-base-samba4.so'do_lock+0xf0 [0xfffffd7fee70dfe0]
   #16 /usr/lib/samba/amd64/vfs/fruit.so'fruit_create_file+0x7ba [0xfffffd7fe88855aa]
   #17 /usr/lib/samba/private/amd64/libsmbd-base-samba4.so'smbd_smb2_request_process_create+0xa07 [0xfffffd7fee7d3237]
   #18 /usr/lib/samba/private/amd64/libsmbd-base-samba4.so'smbd_smb2_request_dispatch+0xc8f [0xfffffd7fee7c985f]
   #19 /usr/lib/samba/private/amd64/libsmbd-base-samba4.so'smbd_smb2_connection_handler+0x621 [0xfffffd7fee7ca7e1]
   #20 /usr/lib/samba/private/amd64/libtevent.so.0.10.2'tevent_common_invoke_fd_handler+0x80 [0xfffffd7fecd3a580]
   #21 /usr/lib/samba/private/amd64/libtevent.so.0.10.2'epoll_event_loop_once+0x22c [0xfffffd7fecd4180c]
   #22 /usr/lib/samba/private/amd64/libtevent.so.0.10.2'std_event_loop_once+0x40 [0xfffffd7fecd3f8f0]
   #23 /usr/lib/samba/private/amd64/libtevent.so.0.10.2'_tevent_loop_once+0x95 [0xfffffd7fecd39bd5]
   #24 /usr/lib/samba/private/amd64/libtevent.so.0.10.2'tevent_common_loop_wait+0x23 [0xfffffd7fecd39e43]
   #25 /usr/lib/samba/private/amd64/libtevent.so.0.10.2'std_event_loop_wait+0x40 [0xfffffd7fecd3f870]
   #26 /usr/lib/samba/private/amd64/libsmbd-base-samba4.so'smbd_process+0x777 [0xfffffd7fee7b8677]
   #27 /usr/lib/samba/sbin/amd64/smbd'smbd_accept_connection+0x189 [0x40d5b9]
   #28 /usr/lib/samba/private/amd64/libtevent.so.0.10.2'tevent_common_invoke_fd_handler+0x80 [0xfffffd7fecd3a580]
   #29 /usr/lib/samba/private/amd64/libtevent.so.0.10.2'epoll_event_loop_once+0x22c [0xfffffd7fecd4180c]
   #30 /usr/lib/samba/private/amd64/libtevent.so.0.10.2'std_event_loop_once+0x40 [0xfffffd7fecd3f8f0]
   #31 /usr/lib/samba/private/amd64/libtevent.so.0.10.2'_tevent_loop_once+0x95 [0xfffffd7fecd39bd5]
   #32 /usr/lib/samba/private/amd64/libtevent.so.0.10.2'tevent_common_loop_wait+0x23 [0xfffffd7fecd39e43]
   #33 /usr/lib/samba/private/amd64/libtevent.so.0.10.2'std_event_loop_wait+0x40 [0xfffffd7fecd3f870]
   #34 /usr/lib/samba/sbin/amd64/smbd'main+0x1a0f [0x40f9ff]
   #35 /usr/lib/samba/sbin/amd64/smbd'_start_crt+0x83 [0x408e73]
   #36 /usr/lib/samba/sbin/amd64/smbd'_start+0x18 [0x408dd8]

In this particular nested share_mode_do_locked() invocation, callback
comes through dbwrap_watched_do_locked_fn() where it fails to update
rec->value_valid which further gets assigned to static_share_mode_record
within share_mode_do_locked_fn().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14352

Signed-off-by: Anoop C S <anoopcs@redhat.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Apr 21 17:37:43 UTC 2020 on sn-devel-184

(cherry picked from commit 5651fafe9856e69c93dba3efa6253780cf3c10a1)

4 years agolibsmb: Don't try to find posix stat info in SMBC_getatr()
Volker Lendecke [Thu, 19 Mar 2020 10:01:41 +0000 (11:01 +0100)]
libsmb: Don't try to find posix stat info in SMBC_getatr()

This wrongly used "frame" instead of "fname", which can never have
worked. A first attempt to fix in 51551e0d53fa6 caused a few followup
patches in an attempt to clean up the test failures 51551e0d53fa6
introduced. They were reverted after a few discussions. So rather than
changing behaviour, just remove the code that introduced the valgrind
error again.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Mar 20 05:06:07 UTC 2020 on sn-devel-184

(cherry picked from commit 39c910fd9cba3caf7414274b678b9eee33d7e20b)

4 years agoVERSION: Bump version up to 4.12.3.
Karolin Seeger [Tue, 28 Apr 2020 11:48:51 +0000 (13:48 +0200)]
VERSION: Bump version up to 4.12.3.

Signed-off-by: Karolin Seeger <kseeger@samba.org>
4 years agoMerge tag 'samba-4.12.2' into v4-12-test
Karolin Seeger [Tue, 28 Apr 2020 11:48:23 +0000 (13:48 +0200)]
Merge tag 'samba-4.12.2' into v4-12-test

samba: tag release samba-4.12.2

4 years agoVERSION: Disable GIT_SNAPSHOT for the 4.12.2 release. samba-4.12.2
Karolin Seeger [Wed, 22 Apr 2020 10:55:18 +0000 (12:55 +0200)]
VERSION: Disable GIT_SNAPSHOT for the 4.12.2 release.

o CVE-2020-10700: Use-after-free in Samba AD DC LDAP Server with ASQ
o CVE-2020-10704: LDAP Denial of Service (stack overflow) in Samba AD DC

Signed-off-by: Karolin Seeger <kseeger@samba.org>
4 years agoWHATSNEW: Add release notes for Samba 4.12.2.
Karolin Seeger [Wed, 22 Apr 2020 10:53:26 +0000 (12:53 +0200)]
WHATSNEW: Add release notes for Samba 4.12.2.

Signed-off-by: Karolin Seeger <kseeger@samba.org>
4 years agoCVE-2020-10704 libcli ldap: Check search request lengths.
Gary Lockyer [Tue, 7 Apr 2020 22:46:44 +0000 (10:46 +1200)]
CVE-2020-10704 libcli ldap: Check search request lengths.

Check the search request lengths against the limits passed to
ldap_decode.

Credit to OSS-Fuzz

REF: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20454
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14334

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
4 years agoCVE-2020-10704: libcli ldap_message: Add search size limits to ldap_decode
Gary Lockyer [Tue, 7 Apr 2020 20:49:23 +0000 (08:49 +1200)]
CVE-2020-10704: libcli ldap_message: Add search size limits to ldap_decode

Add search request size limits to ldap_decode calls.

The ldap server uses the smb.conf variable
"ldap max search request size" which defaults to 250Kb.
For cldap the limit is hard coded as 4096.

Credit to OSS-Fuzz

REF: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20454
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14334

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
4 years agoCVE-2020-10704: S4 ldap server: Limit request sizes
Gary Lockyer [Wed, 8 Apr 2020 03:32:22 +0000 (15:32 +1200)]
CVE-2020-10704: S4 ldap server: Limit request sizes

Check the size of authenticated and anonymous ldap requests and reject
them if they exceed the limits in smb.conf

Credit to OSS-Fuzz

REF: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20454
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14334

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
4 years agoCVE-2020-10704: smb.conf: Add max ldap request sizes
Gary Lockyer [Mon, 6 Apr 2020 21:09:01 +0000 (09:09 +1200)]
CVE-2020-10704: smb.conf: Add max ldap request sizes

Add two new smb.conf parameters to control the maximum permitted ldap
request size.

Adds:
   ldap max anonymous request size       default 250Kb
   ldap max authenticated request size   default 16Mb

Credit to OSS-Fuzz

REF: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20454
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14334

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
4 years agoCVE-2020-10704: ldapserver tests: Limit search request sizes
Gary Lockyer [Tue, 14 Apr 2020 01:32:32 +0000 (13:32 +1200)]
CVE-2020-10704: ldapserver tests: Limit search request sizes

Add tests to ensure that overly long (> 256000 bytes) LDAP search
requests are rejected.

Credit to OSS-Fuzz

REF: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20454
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14334

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
4 years agoCVE-2020-10704: lib util asn1: Check parse tree depth
Gary Lockyer [Wed, 8 Apr 2020 03:30:52 +0000 (15:30 +1200)]
CVE-2020-10704: lib util asn1: Check parse tree depth

Check the current depth of the parse tree and reject the input if the
depth exceeds that passed to asn1_init

Credit to OSS-Fuzz

REF: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20454
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14334

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
4 years agoCVE-2020-10704: libcli ldap: test recursion depth in ldap_decode_filter_tree
Gary Lockyer [Thu, 2 Apr 2020 02:25:53 +0000 (15:25 +1300)]
CVE-2020-10704: libcli ldap: test recursion depth in ldap_decode_filter_tree

Add tests to check that ASN.1 ldap requests with deeply nested elements
are rejected.  Previously there was no check on the on the depth of
nesting and excessive nesting could cause a stack overflow.

Credit to OSS-Fuzz

REF: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20454
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14334

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
4 years agoCVE-2020-10704: lib util asn1: Add ASN.1 max tree depth
Gary Lockyer [Thu, 2 Apr 2020 23:18:03 +0000 (12:18 +1300)]
CVE-2020-10704: lib util asn1: Add ASN.1 max tree depth

Add maximum parse tree depth to the call to asn1_init, which will be
used to limit the depth of the ASN.1 parse tree.

Credit to OSS-Fuzz

REF: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20454
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14334

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
4 years agoCVE-2020-10700: ldb: Bump version up to 2.1.2. ldb-2.1.2
Karolin Seeger [Fri, 17 Apr 2020 10:06:08 +0000 (12:06 +0200)]
CVE-2020-10700: ldb: Bump version up to 2.1.2.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14331

Signed-off-by: Karolin Seeger <kseeger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
4 years agoCVE-2020-10700: dsdb: Do not permit the ASQ control for the GUID search in paged_results
Andrew Bartlett [Wed, 11 Mar 2020 03:43:31 +0000 (16:43 +1300)]
CVE-2020-10700: dsdb: Do not permit the ASQ control for the GUID search in paged_results

ASQ is a very strange control and a BASE search can return multiple results
that are NOT the requested DN, but the DNs pointed to by it!

Thanks to Andrei Popa <andrei.popa@next-gen.ro> for finding,
reporting and working with us to diagnose this issue!

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14331

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
4 years agoCVE-2020-10700: ldb: Always use ldb_next_request() in ASQ module
Andrew Bartlett [Wed, 11 Mar 2020 03:41:34 +0000 (16:41 +1300)]
CVE-2020-10700: ldb: Always use ldb_next_request() in ASQ module

We want to keep going down the module stack, and not start from the top again.

ASQ is above the ACL modules, but below paged_results and we do not wish to
re-trigger that work.

Thanks to Andrei Popa <andrei.popa@next-gen.ro> for finding,
reporting and working with us to diagnose this issue!

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14331

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
4 years agoCVE-2020-10700: dsdb: Add test for ASQ and ASQ in combination with paged_results
Andrew Bartlett [Mon, 30 Mar 2020 09:44:20 +0000 (09:44 +0000)]
CVE-2020-10700: dsdb: Add test for ASQ and ASQ in combination with paged_results

Thanks to Andrei Popa <andrei.popa@next-gen.ro> for finding,
reporting and working with us to diagnose this issue!

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14331

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
4 years agoVERSION: Bump version up to 4.12.2...
Karolin Seeger [Tue, 7 Apr 2020 06:38:40 +0000 (08:38 +0200)]
VERSION: Bump version up to 4.12.2...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger@samba.org>
(cherry picked from commit a3157b958bfd9f1f77014fcfc6ca5ad1f38b0fbf)

4 years agoctdb: Fix a memleak
Volker Lendecke [Thu, 16 Apr 2020 12:38:34 +0000 (14:38 +0200)]
ctdb: Fix a memleak

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14348
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Apr 17 08:32:35 UTC 2020 on sn-devel-184

(cherry picked from commit ad4b53f2d95de3d3609f3a23d96602f3dd516705)

Autobuild-User(v4-12-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-12-test): Tue Apr 21 12:44:16 UTC 2020 on sn-devel-184

4 years agos3: VFS: Add cmocka test for vfs_full_audit to make sure all arrays are correct.
Jeremy Allison [Fri, 10 Apr 2020 21:14:25 +0000 (14:14 -0700)]
s3: VFS: Add cmocka test for vfs_full_audit to make sure all arrays are correct.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14343

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(Back-ported from commit 5e987e2f40e7698de489696d795ebe26b7e75c9b)

Autobuild-User(v4-12-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-12-test): Fri Apr 17 10:19:13 UTC 2020 on sn-devel-184

4 years agos3: VFS: full_audit. Add missing fcntl entry in vfs_op_names[] array.
Jeremy Allison [Fri, 10 Apr 2020 20:27:18 +0000 (13:27 -0700)]
s3: VFS: full_audit. Add missing fcntl entry in vfs_op_names[] array.

Found by yannick@in2ip.nl.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14343

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 977d3c7bab3319725abc418839d581489fd9bbe9)

4 years agos3/librpc/crypto: Fix double free with unresolved credential cache
Noel Power [Tue, 14 Apr 2020 10:21:22 +0000 (11:21 +0100)]
s3/librpc/crypto: Fix double free with unresolved credential cache

We free gse_ctx->k5ctx but then free it again in the
talloc dtor. This patch just lets the talloc dtor handle
things and removes the extra krb5_free_context

Failed to resolve credential cache 'DIR:/run/user/1000/krb5cc'! (No credentials cache found)
==30762== Invalid read of size 8
==30762==    at 0x108100F4: k5_os_free_context (in /usr/lib64/libkrb5.so.3.3)
==30762==    by 0x107EA661: krb5_free_context (in /usr/lib64/libkrb5.so.3.3)
==30762==    by 0x7945D2E: gse_context_destructor (gse.c:84)
==30762==    by 0x645FB49: _tc_free_internal (talloc.c:1157)
==30762==    by 0x645FEC5: _talloc_free_internal (talloc.c:1247)
==30762==    by 0x646118D: _talloc_free (talloc.c:1789)
==30762==    by 0x79462E4: gse_context_init (gse.c:241)
==30762==    by 0x794636E: gse_init_client (gse.c:268)
==30762==    by 0x7947602: gensec_gse_client_start (gse.c:786)
==30762==    by 0xBC87A3A: gensec_start_mech (gensec_start.c:743)
==30762==    by 0xBC87BC6: gensec_start_mech_by_ops (gensec_start.c:774)
==30762==    by 0xBC8167F: gensec_spnego_client_negTokenInit_step (spnego.c:633)
==30762==  Address 0x17259928 is 40 bytes inside a block of size 496 free'd
==30762==    at 0x4C2F50B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30762==    by 0x79462CA: gse_context_init (gse.c:238)
==30762==    by 0x794636E: gse_init_client (gse.c:268)
==30762==    by 0x7947602: gensec_gse_client_start (gse.c:786)
==30762==    by 0xBC87A3A: gensec_start_mech (gensec_start.c:743)
==30762==    by 0xBC87BC6: gensec_start_mech_by_ops (gensec_start.c:774)
==30762==    by 0xBC8167F: gensec_spnego_client_negTokenInit_step (spnego.c:633)
==30762==    by 0xBC813E2: gensec_spnego_client_negTokenInit_start (spnego.c:537)
==30762==    by 0xBC84084: gensec_spnego_update_pre (spnego.c:1943)
==30762==    by 0xBC83AE5: gensec_spnego_update_send (spnego.c:1741)
==30762==    by 0xBC85622: gensec_update_send (gensec.c:449)
==30762==    by 0x551BFD0: cli_session_setup_gensec_local_next (cliconnect.c:997)
==30762==  Block was alloc'd at
==30762==    at 0x4C306B5: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30762==    by 0x107EA7AE: krb5_init_context_profile (in /usr/lib64/libkrb5.so.3.3)
==30762==    by 0xB853215: smb_krb5_init_context_common (krb5_samba.c:3597)
==30762==    by 0x794615B: gse_context_init (gse.c:209)
==30762==    by 0x794636E: gse_init_client (gse.c:268)
==30762==    by 0x7947602: gensec_gse_client_start (gse.c:786)
==30762==    by 0xBC87A3A: gensec_start_mech (gensec_start.c:743)
==30762==    by 0xBC87BC6: gensec_start_mech_by_ops (gensec_start.c:774)
==30762==    by 0xBC8167F: gensec_spnego_client_negTokenInit_step (spnego.c:633)
==30762==    by 0xBC813E2: gensec_spnego_client_negTokenInit_start (spnego.c:537)
==30762==    by 0xBC84084: gensec_spnego_update_pre (spnego.c:1943)
==30762==    by 0xBC83AE5: gensec_spnego_update_send (spnego.c:1741)
==30762==

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14344
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Tue Apr 14 22:55:51 UTC 2020 on sn-devel-184

(cherry picked from commit 34f8ab774d1484b0e60dbdec8ad2a1607ad92122)

Autobuild-User(v4-12-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-12-test): Wed Apr 15 13:20:28 UTC 2020 on sn-devel-184

4 years agomit-kdc: Explicitly reject S4U requests
Isaac Boukris [Fri, 31 Jan 2020 21:34:21 +0000 (22:34 +0100)]
mit-kdc: Explicitly reject S4U requests

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14342

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Isaac Boukris <iboukris@samba.org>
Autobuild-Date(master): Tue Mar 10 14:46:04 UTC 2020 on sn-devel-184

4 years agoselftest: Set KRB5RCACHETYPE to none for selftest
Andreas Schneider [Fri, 6 Mar 2020 16:15:45 +0000 (17:15 +0100)]
selftest: Set KRB5RCACHETYPE to none for selftest

This is required that out tests work with MIT KRB5 1.18.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14342

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Isaac Boukris <iboukris@samba.org>
4 years agoSign and verify PAC with ticket principal instead of canon principal
Isaac Boukris [Thu, 16 Jan 2020 21:00:21 +0000 (22:00 +0100)]
Sign and verify PAC with ticket principal instead of canon principal

With MIT library 1.18 the KDC no longer set
KRB5_KDB_FLAG_CANONICALIZE for enterprise principals which allows
us to not canonicalize them (like in Windows / Heimdal).

However, it now breaks the PAC signature verification as it was
wrongly done using canonical client rather than ticket client name.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14342

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
4 years agoFix uxsuccess test with new MIT krb5 library 1.18
Isaac Boukris [Thu, 16 Jan 2020 20:36:24 +0000 (21:36 +0100)]
Fix uxsuccess test with new MIT krb5 library 1.18

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14155
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14342

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
4 years agoAdapt sign_authdata in our KDB module for krb5 v1.18
Isaac Boukris [Mon, 16 Sep 2019 13:40:12 +0000 (16:40 +0300)]
Adapt sign_authdata in our KDB module for krb5 v1.18

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14342

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
4 years agotestprogs: Add 'net ads join createupn' test also verifying the keytab
Andreas Schneider [Fri, 3 Apr 2020 13:40:48 +0000 (15:40 +0200)]
testprogs: Add 'net ads join createupn' test also verifying the keytab

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14336

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Apr  6 19:09:53 UTC 2020 on sn-devel-184

(cherry picked from commit c4be195da2845be4f64e47883e3c911dedd90e48)

Autobuild-User(v4-12-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-12-test): Tue Apr  7 09:36:45 UTC 2020 on sn-devel-184

4 years agos3:libads: Fix ads_get_upn()
Andreas Schneider [Fri, 3 Apr 2020 13:58:28 +0000 (15:58 +0200)]
s3:libads: Fix ads_get_upn()

This adds the userPrincipalName to ads_find_machine_acct() which
fetches the data for us.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14336

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
(cherry picked from commit ec69752cb963ae850568d3f4905d2941e485627e)

4 years agosmbd: let delayed update handler also update on-disk timestamps
Ralph Boehme [Sun, 15 Mar 2020 07:30:21 +0000 (08:30 +0100)]
smbd: let delayed update handler also update on-disk timestamps

Let delayed update handler also update on-disk timestamps by calling
trigger_write_time_update_immediate().

trigger_write_time_update_immediate() sets fsp->update_write_time_on_close to
false which prevents updating the write-time on close if there was ever only one
write to the file.

Besides resetting fsp->update_write_time_on_close and setting the on-disk timestamps
trigger_write_time_update_immediate() takes the same steps as the removed code.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14320

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Mar 19 03:05:40 UTC 2020 on sn-devel-184

(cherry picked from commit 81c1a14e3271aeed7ed4fe6311171b19ba963555)

4 years agosmbd: let mark_file_modified() always call trigger_write_time_update()
Ralph Boehme [Sun, 15 Mar 2020 07:30:21 +0000 (08:30 +0100)]
smbd: let mark_file_modified() always call trigger_write_time_update()

Preperatory change: the next commit will reset fsp->update_write_time_on_close
in the event handler, so this change ensures it gets set again for any
subsequent write.

This will NOT always result in a write-time update because
trigger_write_time_update() has its own only-once logic using the internal
variable fsp->update_write_time_triggered.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14320

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 53de2da7acfc24513082190502d93306c12b7434)

4 years agotorture/smb2: delayed timestamp updates test: more then one write
Ralph Boehme [Sun, 15 Mar 2020 15:46:16 +0000 (16:46 +0100)]
torture/smb2: delayed timestamp updates test: more then one write

Verify a close updates the write-time for subsequent writes after an initial
write started the delayed update logic.

This covers a scenario that will become relevant with the two subsequent
commits. The next commit:

  smbd: let mark_file_modified() always call trigger_write_time_update()

ensures that trigger_write_time_update() is not only called for the first write
on a file. Without that preaparatory change, the second commit:

  smbd: let delayed update handler also update on-disk timestamps

alone would cause this test to fail.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14320

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 60aecca9a727555847aa1412415c5bbd927df4ff)

4 years agotorture/smb2: delayed timestamp update test: single write
Ralph Boehme [Sat, 14 Mar 2020 15:43:48 +0000 (16:43 +0100)]
torture/smb2: delayed timestamp update test: single write

Verify close only updates write-time when a delayed update is actually pending.

This scenario is not covered by basic.delaywrite.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14320

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 58fa7b4fd7b53d3100459a0c9c7ef4ca7481b58a)

4 years agosmbd: remove stat call from mark_file_modified()
Ralph Boehme [Sun, 15 Mar 2020 14:51:18 +0000 (15:51 +0100)]
smbd: remove stat call from mark_file_modified()

This stat dates back to d03453864ab1bc5fd3b4a3abaf96176a006c102b where the call
to trigger_write_time_update() had been to the file IO codepath. It was present
there for other reasons: to setup the write-cache based on the file's size.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14320

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 2c19d27113036d607850f370bb9afd62856d671e)

4 years agos3: tests: Add samba3.blackbox.force-close-share
Jeremy Allison [Tue, 3 Mar 2020 21:31:18 +0000 (13:31 -0800)]
s3: tests: Add samba3.blackbox.force-close-share

Checks server stays up whilst writing to a force closed share.
Uses existing aio_delay_inject share to delay writes while
we force close the share.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sun Mar  8 19:34:14 UTC 2020 on sn-devel-184

(cherry picked from commit bb22be08b077b7d5911ccdeb1012f4dea85647e5)
(cherry picked from commit 566658d914176c41942e3c6aba404ae369aeb123)

4 years agosmbd: enforce AIO requests draining
Jeremy Allison [Mon, 2 Mar 2020 21:11:06 +0000 (13:11 -0800)]
smbd: enforce AIO requests draining

Assert we have no aio on a close.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Combined squash of commits:

(cherry picked from commit 410e7599bd2ae9b35429f60a529bb7c4aa88df25)
(cherry picked from commit acb0b01761330864a23932f643f7ad4e3d374634)
(cherry picked from commit f94cd10a211e2eae966ba4bd26921556bbe513fc)
(cherry picked from commit 0ae4f368c6c8d2c8c7aa34069007a984055df0da)
(cherry picked from commit 86dd5a080969e14ab0d131d8cb1054ec624a41ba)

4 years agos3: smbd: Remove file_close_pid().
Jeremy Allison [Thu, 12 Mar 2020 18:02:19 +0000 (11:02 -0700)]
s3: smbd: Remove file_close_pid().

The old synchronous reply_exit() was the only user.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 584933439c70af3d2fd047e62a3456c1c2eca45e)

4 years agos3: smbd: Remove old synchronous SMB1 reply_exit().
Jeremy Allison [Thu, 12 Mar 2020 18:01:05 +0000 (11:01 -0700)]
s3: smbd: Remove old synchronous SMB1 reply_exit().

SMB1 exit is now fully async.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 1de0daa715f3324e3620ae8152b7fbaeb40ee9d9)

4 years agos3: smbd: Add async internals of reply_exit().
Jeremy Allison [Thu, 12 Mar 2020 17:59:16 +0000 (10:59 -0700)]
s3: smbd: Add async internals of reply_exit().

Waits until all aio requests on all fsp's owned by this
vuid are finished before returning to the client.

Charges the profile time in the done function. Not strictly
correct but better than the other SMB1 async code that
double-charges profiling in both send and done at the
moment.

Done this way (commented out) so it is a clean
diff and it's clear what is being added.

A later commit will remove the old synchronous version.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 8f58feab58afbc7aa214fac2a1728dda68303c6b)

4 years agos3: smbd: Remove old synchronous SMB1 reply_ulogoffX().
Jeremy Allison [Thu, 12 Mar 2020 17:19:31 +0000 (10:19 -0700)]
s3: smbd: Remove old synchronous SMB1 reply_ulogoffX().

SMB1 user logoff is now fully async.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 446b64ca66591d8ae5b4bf1aabdd46a1e8cb1c1c)

4 years agos3: smbd: reply_ulogoffX() Update to modern coding standards.
Jeremy Allison [Thu, 12 Mar 2020 17:16:42 +0000 (10:16 -0700)]
s3: smbd: reply_ulogoffX() Update to modern coding standards.

Minimizes the diff in the later commits.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 9cda76ad29db0cfbffa3dbb0764ec5dda24490f9)

4 years agos3: smbd: In reply_ulogoffX(), replace req -> smb1req.
Jeremy Allison [Thu, 12 Mar 2020 17:15:10 +0000 (10:15 -0700)]
s3: smbd: In reply_ulogoffX(), replace req -> smb1req.

Minimises the diff in later commits.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 5c073aa01b304f54a0039d9cd9dc74123191eb4b)

4 years agos3: smbd: Add async internals of reply_ulogoffX.
Jeremy Allison [Thu, 12 Mar 2020 17:11:14 +0000 (10:11 -0700)]
s3: smbd: Add async internals of reply_ulogoffX.

Waits until all aio requests on all fsp's owned by this
vuid are finished before returning to the client.

Charges the profile time in the done function. Not strictly
correct but better than the other SMB1 async code that
double-charges profiling in both send and done at the
moment.

Done this way (commented out) so it is a clean
diff and it's clear what is being added.

A later commit will remove the old synchronous version.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 4dd3012cb1b5e000ccf68d2601dbdbcb7ff538b5)

4 years agos3: smbd: Remove old synchronous SMB1 reply_tdis().
Jeremy Allison [Thu, 12 Mar 2020 17:03:27 +0000 (10:03 -0700)]
s3: smbd: Remove old synchronous SMB1 reply_tdis().

SMB1 tree disconnect is now fully async.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 7613998e10c5f13c896667257fdef33824a45d2a)

4 years agos3: smbd: reply_tdis() Update to modern coding standards.
Jeremy Allison [Thu, 12 Mar 2020 16:59:47 +0000 (09:59 -0700)]
s3: smbd: reply_tdis() Update to modern coding standards.

Minimizes the diff in the next commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 71725f1c4adaa04ef04c0dd400c49399952ef5fa)

4 years agos3: smbd: In reply_tdis(), replace req -> smb1req.
Jeremy Allison [Thu, 12 Mar 2020 16:55:30 +0000 (09:55 -0700)]
s3: smbd: In reply_tdis(), replace req -> smb1req.

Minimises the diff in the next commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit ca4521f1dd97bc5a05e381c652b05ae1eb8bd29b)

4 years agos3: smbd: Add async internals of reply_tdis().
Jeremy Allison [Thu, 12 Mar 2020 16:51:53 +0000 (09:51 -0700)]
s3: smbd: Add async internals of reply_tdis().

Waits until all aio requests on all fsp's under this
conn struct are finished before returning to the client.

Charges the profile time in the done function. Not strictly
correct but better than the other SMB1 async code that
double-charges profiling in both send and done at the
moment.

Done this way (commented out) so it is a clean
diff and it's clear what is being added.

A later commit will remove the old synchronous version.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 86cc67d5a7de0a81131b11447dad57b2681d8e01)

4 years agos3: smbd: Replace synchronous conn_force_tdis() with the async version.
Jeremy Allison [Thu, 12 Mar 2020 16:47:26 +0000 (09:47 -0700)]
s3: smbd: Replace synchronous conn_force_tdis() with the async version.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 7891302ab8eeba8261b92171a4d429e2f538b89a)

4 years agos3: smbd: Add async internals of conn_force_tdis().
Jeremy Allison [Thu, 12 Mar 2020 16:28:21 +0000 (09:28 -0700)]
s3: smbd: Add async internals of conn_force_tdis().

Commented out so it can be seen complete as
a diff. The next commit will replace the old
synchronous conn_force_tdis() code with the
new async code.

Uses a wait_queue to cause the force close
requests to stay pending until all outstanding
aio is finished on all file handles opened
on the connection.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 4f9e0459cd06f0332083a4a465f49b5f258838fa)

4 years agos3: smbd: Don't allow force disconnect of a connection already being disconnected.
Jeremy Allison [Tue, 10 Mar 2020 19:25:58 +0000 (12:25 -0700)]
s3: smbd: Don't allow force disconnect of a connection already being disconnected.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit ac800ca6bcb43c74a1a6ef508b900e2e6cb532dc)

4 years agos3: smbd: Every place we check fsp->deferred_close, also check for fsp->closing.
Jeremy Allison [Wed, 11 Mar 2020 22:16:35 +0000 (15:16 -0700)]
s3: smbd: Every place we check fsp->deferred_close, also check for fsp->closing.

Eventually this will allow us to remove fsp->deferred_close
from the fsp struct (and also source3/lib/tevent_wait.[ch]).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 4287ea138e82103cce0a939e504f9810636b4747)

4 years agos3: smbd: In async SMB1 reply_close() set fsp->closing = true, as we already do in...
Jeremy Allison [Thu, 12 Mar 2020 00:25:59 +0000 (17:25 -0700)]
s3: smbd: In async SMB1 reply_close() set fsp->closing = true, as we already do in SMB2 async close.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit b7d09b30ad14d51bbcbe368a11348754121f6ff8)

4 years agos3: smbd: Now we free fsp->aio_requests when it gets zero entries, talloc in chunks...
Jeremy Allison [Wed, 11 Mar 2020 21:47:50 +0000 (14:47 -0700)]
s3: smbd: Now we free fsp->aio_requests when it gets zero entries, talloc in chunks of 10 instead of 1.

Prevents incremental +1 tallocs, and the original
idea of this array was that it wasn't freed for
io efficiency reasons. Add paranoia integer wrap
protection also.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit b90bc0f28918133badbf6810d5e298fc326bd1aa)

4 years agos3: smbd: In aio_del_req_from_fsp() talloc_free(fsp->aio_requests[]) when fsp->num_ai...
Jeremy Allison [Tue, 10 Mar 2020 17:55:22 +0000 (10:55 -0700)]
s3: smbd: In aio_del_req_from_fsp() talloc_free(fsp->aio_requests[]) when fsp->num_aio_requests reaches zero.

The add code in aio_add_req_to_fsp() re-tallocs
this array on demand, and talloc freeing it here
allows it to be used as the parent for a tevent
wait queue, so callers can get notified when
all outstanding aio on an fsp is finished.

We'll deal with any performance issues in
the next commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 0c952bba1edf7c8173d05ccdc6fdaa7232d2c6aa)

4 years agos3: VFS: vfs_aio_pthread: Make aio opens safe against connection teardown.
Jeremy Allison [Thu, 5 Mar 2020 18:22:00 +0000 (10:22 -0800)]
s3: VFS: vfs_aio_pthread: Make aio opens safe against connection teardown.

Allocate state off fsp->conn, not NULL, and add a destructor
that catches deallocation of conn which happens
on connection shutdown or force close.

Note - We don't allocate off fsp as the passed in
fsp will get freed once we return EINPROGRESS/NT_STATUS_MORE_PROCESSING_REQUIRED.
A new fsp pointer gets allocated on every re-run of the
open code path.

The destructor allows us to NULL out the saved conn struct pointer
when conn is deallocated so we know not to access deallocated memory.
This matches the async teardown code changes for bug #14301
in pread/pwrite/fsync vfs_default.c and vfs_glusterfs.c

state is still correctly deallocated in all code
paths so no memory leaks.

This allows us to safely complete when the openat()
returns and then return the error NT_STATUS_NETWORK_NAME_DELETED
to the client open request.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 6b567e0c138d1cf2bcf58c84872ed2b0e89d628d)

4 years agos3: VFS: vfs_aio_pthread: Add a talloc context parameter to create_private_open_data().
Jeremy Allison [Fri, 6 Mar 2020 17:30:26 +0000 (09:30 -0800)]
s3: VFS: vfs_aio_pthread: Add a talloc context parameter to create_private_open_data().

Pass in NULL for now so no behavior change.
We will be changing this from NULL to fsp->conn in a later commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit e566066605981549b670a5392683fbd81ce93d18)

4 years agos3: VFS: vfs_aio_pthread. Move xconn into state struct (opd).
Jeremy Allison [Thu, 5 Mar 2020 00:39:39 +0000 (16:39 -0800)]
s3: VFS: vfs_aio_pthread. Move xconn into state struct (opd).

We will need this in future to cause a pending open to
be rescheduled after the connection struct we're using
has been shut down with an aio open in flight. This will
allow a correct error reply to an awaiting client.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit ddb9038fe776b1d8239e563a4c9a70b4097645f3)

4 years agos3: VFS: vfs_aio_pthread: Replace state destructor with explicitly called teardown...
Jeremy Allison [Wed, 4 Mar 2020 21:47:13 +0000 (13:47 -0800)]
s3: VFS: vfs_aio_pthread: Replace state destructor with explicitly called teardown function.

This will allow repurposing a real destructor to allow
connections structs to be freed whilst the aio open
request is in flight.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 8db831a318cd4a10ec9c1d629ebff4ca35b8acfe)

4 years agos3: VFS: vfs_aio_pthread. Fix leak of state struct on error.
Jeremy Allison [Wed, 4 Mar 2020 21:29:08 +0000 (13:29 -0800)]
s3: VFS: vfs_aio_pthread. Fix leak of state struct on error.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit a1e247c3ba579ecc6ee03f5aad9679ed79fac5ac)

4 years agosmbd: avoid double chdir() in chdir_current_service()
Ralph Boehme [Wed, 22 Jan 2020 09:52:39 +0000 (10:52 +0100)]
smbd: avoid double chdir() in chdir_current_service()

Since 8e81090789e4cc3ba9e5aa792d4e52971909c894 we're doing chdir() twice, first
into conn->connectpath, then into conn->origpath.

Before commit 8e81090789e4cc3ba9e5aa792d4e52971909c894 if
chdir(conn->connectpath) succeeded, we wouldn't do the second chdir().

While at it, simplify the logging logic: if chdir() fails in this core function,
just always log is as error including the unix token.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14256
RN: smbd does a chdir() twice per request

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Feb  6 11:44:07 UTC 2020 on sn-devel-184

(cherry picked from commit f705629a171c1411131164f3adff36175154c093)

4 years agotorture/smb2: Windows 2019 15 ms timestamp resolution
Ralph Boehme [Fri, 6 Mar 2020 15:21:47 +0000 (16:21 +0100)]
torture/smb2: Windows 2019 15 ms timestamp resolution

This test demonstrates that Windows has a timestamp resolution of ~15ms.

When a smaller amount of time than that has passed between modifying operations
on a file, it's not necessarily detectable on a Windows 2019 server that
implements immediate timestamp updates (no delayed magic).

Note that this test relies on a low latency SMB connection. Even with a low
latency connection of eg 1m there's a chance of 1/15 that the first part of the
test expecting no timestamp change fails as the writetime is updated.

Due to this timing dependency this test is skipped in Samba CI, but it is
preserved here for future SMB2 timestamps behaviour archealogists.

See also: https://lists.samba.org/archive/cifs-protocol/2019-December/003358.html

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 6f7d1d8a37bfb877b3f07423cbcffd15710e8d08)

4 years agosmbd: flush pending writetime update when setting timestamps file
Ralph Boehme [Thu, 12 Mar 2020 18:23:40 +0000 (19:23 +0100)]
smbd: flush pending writetime update when setting timestamps file

Cf the explanations in the previous commits.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14150

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 7b90fe69a865ae8648b6548eabbcf2fa8237ebd8)

4 years agosmbd: flush pending writetime update when flushing file
Ralph Boehme [Thu, 12 Mar 2020 18:23:40 +0000 (19:23 +0100)]
smbd: flush pending writetime update when flushing file

Cf the explanations in the previous commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14150

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit d99d5bf2c6d0a818ef2f3920e0c93fac38761c36)

4 years agosmbd: always flush pending write time update when setting filesize
Ralph Boehme [Thu, 12 Mar 2020 15:52:34 +0000 (16:52 +0100)]
smbd: always flush pending write time update when setting filesize

We need to flush a pending write time update even when we're setting the
filesize to current filesize.

Note that we're already doing it this way in the relevant places listed my
dochelp@MS in

https://lists.samba.org/archive/cifs-protocol/2019-December/003364.html

  Cleanup (= Close)
  SetBasicInfo
  SetAllocationInfo
  SetEndOfFileInfo
  SetValidDataLengthInfo
  Flush
  FSCTL_SET_ENCRYPTION
  FSCTL_OFFLOAD_WRITE

Cleanup (= Close):

  Already implemented by update_write_time_on_close() and friends.

SetBasicInfo:

  Currently doesn't flush pending updates. Fixed by a subsequent commit.

SetAllocationInfo:

  smb_set_file_allocation_info() when setting a file's allocation size.

SetEndOfFileInfo:

  Currently doesn't flush pending updates. Fixed by a subsequent commit.

SetValidDataLengthInfo:

  Not implemented, returns NT_STATUS_NOT_SUPPORTED which seems wrong btw, as
  SetValidDataLengthInfo IS listed in MS-SMB2 2.2.39.

Flush:

  Currently doesn't flush pending updates. Fixed by subsequent commit.

FSCTL_SET_ENCRYPTION:

  Windows 2016 doesn't flush a pending writetime update, verified with a
  smbtorture test.

FSCTL_OFFLOAD_WRITE:

  NT_STATUS_NOT_IMPLEMENTED

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14150

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 79d7d6b9d01b8547f16b74a62926d0b471f18c39)

4 years agotorture/smb2: add a test verifying a setinfo(basicinfo) flushes a pending writetime...
Ralph Boehme [Thu, 12 Mar 2020 15:48:09 +0000 (16:48 +0100)]
torture/smb2: add a test verifying a setinfo(basicinfo) flushes a pending writetime update

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14150

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 4e3c2afbd6f12159252405f6efc6528fa9345f08)

4 years agotorture/smb2: add a test verifying a flush flushes a pending writetime update
Ralph Boehme [Thu, 12 Mar 2020 15:48:09 +0000 (16:48 +0100)]
torture/smb2: add a test verifying a flush flushes a pending writetime update

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14150

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit c63d6c9e256cdf6a3620373ef0f595e61d43fa97)

4 years agotorture/smb2: mtime update logic with 2 handles: write io on handle 1, then set mtime...
Ralph Boehme [Fri, 6 Mar 2020 10:50:57 +0000 (11:50 +0100)]
torture/smb2: mtime update logic with 2 handles: write io on handle 1, then set mtime on handle 2

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14150

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 47508c5ecf0713ef404d5226499a4269651dc020)

4 years agos4/torture: fix a timestamps test to work on ext filesystem
Ralph Boehme [Tue, 10 Mar 2020 17:26:49 +0000 (18:26 +0100)]
s4/torture: fix a timestamps test to work on ext filesystem

ext filesystem has a time_t limit of 15032385535 (0x0x37fffffff). From
Documentation/filesystems/ext4/inodes.rst:

  If the inode structure size ``sb->s_inode_size`` is larger than 128 bytes and
  the ``i_inode_extra`` field is large enough to encompass the respective
  ``i_[cma]time_extra`` field, the ctime, atime, and mtime inode fields are
  widened to 64 bits. Within this “extra” 32-bit field, the lower two bits are
  used to extend the 32-bit seconds field to be 34 bit wide; the upper 30 bits
  are used to provide nanosecond timestamp accuracy. Therefore, timestamps
  should not overflow until May 2446. ...

Changing the test to use the value 0x37fffffff instead of 100000000000 allows
running the test locally on ext filesytems.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 73fedf014bbe02a408360d48e35bce4a6dbc9c36)

4 years agoVERSION: Bump version up to 4.12.2...
Karolin Seeger [Tue, 7 Apr 2020 06:38:40 +0000 (08:38 +0200)]
VERSION: Bump version up to 4.12.2...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger@samba.org>
4 years agoVERSION: Disable GIT_SNAPSHOT for the 4.12.1 release. samba-4.12.1
Karolin Seeger [Tue, 7 Apr 2020 06:37:55 +0000 (08:37 +0200)]
VERSION: Disable GIT_SNAPSHOT for the 4.12.1 release.

Signed-off-by: Karolin Seeger <kseeger@samba.org>
4 years agoWHATSNEW: Add release notes for Samba 4.12.1.
Karolin Seeger [Mon, 6 Apr 2020 08:03:43 +0000 (10:03 +0200)]
WHATSNEW: Add release notes for Samba 4.12.1.

Signed-off-by: Karolin Seeger <kseeger@samba.org>
4 years agos3/utils: Fix double free error with smbtree
Noel Power [Wed, 1 Apr 2020 11:26:09 +0000 (12:26 +0100)]
s3/utils: Fix double free error with smbtree

==3632==
WORKGROUP
\\ATP
\\ATP\IPC$            IPC Service ()
==3632== Invalid read of size 8
==3632==    at 0x773C926: poptResetContext (in /usr/lib64/libpopt.so.0.0.0)
==3632==    by 0x773E5DD: poptFreeContext (in /usr/lib64/libpopt.so.0.0.0)
==3632==    by 0x10A8BC: main (smbtree.c:354)
==3632==  Address 0x16085e00 is 640 bytes inside a block of size 784 free'd
==3632==    at 0x4C2F1AD: free (vg_replace_malloc.c:530)
==3632==    by 0x773E6F7: poptFreeContext (in /usr/lib64/libpopt.so.0.0.0)
==3632==    by 0x10A84B: main (smbtree.c:342)
==3632==  Block was alloc'd at
==3632==    at 0x4C2FE45: calloc (vg_replace_malloc.c:711)
==3632==    by 0x773C79A: poptGetContext (in /usr/lib64/libpopt.so.0.0.0)
==3632==    by 0x10A829: main (smbtree.c:339)
==3632==

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14332
Signed-off-by: Noel Power <npower@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Wed Apr  1 18:51:43 UTC 2020 on sn-devel-184

(cherry picked from commit 24b03fd28ed19cea8f875f96b3e300f808dfc4ae)

Autobuild-User(v4-12-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-12-test): Fri Apr  3 12:24:36 UTC 2020 on sn-devel-184

4 years agonsswitch: fix use-after-free causing segfault in _pam_delete_cred
Günther Deschner [Fri, 27 Mar 2020 09:13:11 +0000 (10:13 +0100)]
nsswitch: fix use-after-free causing segfault in _pam_delete_cred

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14327

Guenther

Signed-off-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Mon Mar 30 13:01:20 UTC 2020 on sn-devel-184

(cherry picked from commit 047b0d8ab534c7a10a8572fd9f21e2456fd30710)

Autobuild-User(v4-12-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-12-test): Wed Apr  1 08:51:15 UTC 2020 on sn-devel-184

4 years agoctdb-recoverd: Avoid dereferencing NULL rec->nodemap
Martin Schwenke [Sun, 22 Mar 2020 02:46:46 +0000 (13:46 +1100)]
ctdb-recoverd: Avoid dereferencing NULL rec->nodemap

Inside the nested event loop in ctdb_ctrl_getnodemap(), various
asynchronous handlers may dereference rec->nodemap, which will be
NULL.

One example is lost_reclock_handler(), which causes rec->nodemap to be
unconditionally dereferenced in list_of_nodes() via this call chain:

  list_of_nodes()
  list_of_active_nodes()
  set_recovery_mode()
  force_election()
  lost_reclock_handler()

Instead of attempting to trace all of the cases, just avoid leaving
rec->nodemap set to NULL.  Attempting to use an old value is generally
harmless, especially since it will be the same as the new value in
most cases.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14324

Reported-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Tue Mar 24 01:22:45 UTC 2020 on sn-devel-184

(cherry picked from commit 716f52f68b248ae7cfd66479b3fc678c4a0d8b38)

Autobuild-User(v4-12-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-12-test): Mon Mar 30 09:36:01 UTC 2020 on sn-devel-184

4 years agoctdb-daemon: Don't allow attach from recovery if recovery is not active
Martin Schwenke [Tue, 25 Feb 2020 06:32:56 +0000 (17:32 +1100)]
ctdb-daemon: Don't allow attach from recovery if recovery is not active

Neither the recovery daemon nor the recovery helper should attach
databases outside of the recovery process.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 147afe77de372ddb9c180228d6fe1b04cca4610f)

4 years agoctdb-daemon: Remove more unused old client database functions
Martin Schwenke [Mon, 24 Feb 2020 19:20:32 +0000 (06:20 +1100)]
ctdb-daemon: Remove more unused old client database functions

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 052f1bdb9cf78f53f584edd32f81ae8e01e8e86e)

4 years agoctdb-recovery: Remove old code for creating missing databases
Martin Schwenke [Mon, 24 Feb 2020 08:51:19 +0000 (19:51 +1100)]
ctdb-recovery: Remove old code for creating missing databases

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 3a66d181b6f6199fca362fcb0aa06513645b589d)

4 years agoctdb-recovery: Create database on nodes where it is missing
Martin Schwenke [Mon, 24 Feb 2020 00:31:33 +0000 (11:31 +1100)]
ctdb-recovery: Create database on nodes where it is missing

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 76a8174279f42486b36cc41d5831d4e6613f172e)

4 years agoctdb-recovery: Fetch database name from all nodes where it is attached
Martin Schwenke [Sun, 23 Feb 2020 23:26:34 +0000 (10:26 +1100)]
ctdb-recovery: Fetch database name from all nodes where it is attached

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit e6e63f8fb8194634135bf34cda18f6cc8ff69a7c)

4 years agoctdb-recovery: Pass db structure for each database recovery
Martin Schwenke [Fri, 21 Feb 2020 05:51:10 +0000 (16:51 +1100)]
ctdb-recovery: Pass db structure for each database recovery

Instead of db_id and db_flags.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 1bdfeb3fdc06947a607957ab3d114f97bad5d7d7)

4 years agoctdb-recovery: GET_DBMAP from all nodes
Martin Schwenke [Fri, 21 Feb 2020 05:10:05 +0000 (16:10 +1100)]
ctdb-recovery: GET_DBMAP from all nodes

This builds a complete list of databases across the cluster so it can
be used to create databases on the nodes where they are missing.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit c6f74e590f602e2ed38fe293468770a5e669aefa)

4 years agoctdb-recovery: Replace use of ctdb_dbid_map with local db_list
Martin Schwenke [Fri, 21 Feb 2020 01:24:39 +0000 (12:24 +1100)]
ctdb-recovery: Replace use of ctdb_dbid_map with local db_list

This will be used to build a merged list of databases from all nodes,
allowing the recovery helper to create missing databases.

It would be possible to also include the db_name field in this
structure but that would cause a lot of churn.  This field is used
locally in the recovery of each database so can continue to live in
the relevant state structure(s).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 4c0b9c36050a0ed8a180d4ac1853224089528e8e)

4 years agoctdb-daemon: Respect CTDB_CTRL_FLAG_ATTACH_RECOVERY when attaching databases
Martin Schwenke [Wed, 26 Feb 2020 00:50:09 +0000 (11:50 +1100)]
ctdb-daemon: Respect CTDB_CTRL_FLAG_ATTACH_RECOVERY when attaching databases

This is currently only set by the recovery daemon when it attaches
missing databases, so there is no obvious behaviour change.  However,
attaching missing databases can now be moved to the recovery helper as
long as it sets this flag.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 7e5a8a4884ea87bb985fe0e2b65ff130fc2ba8aa)

4 years agoctdb-recovery: Use CTDB_CTRL_FLAG_ATTACH_RECOVERY to attach during recovery
Martin Schwenke [Fri, 21 Feb 2020 00:13:05 +0000 (11:13 +1100)]
ctdb-recovery: Use CTDB_CTRL_FLAG_ATTACH_RECOVERY to attach during recovery

ctdb_ctrl_createdb() is only called by the recovery daemon, so this is
a safe, temporary change.  This is temporary because
ctdb_ctrl_createdb(), create_missing_remote_databases() and
create_missing_local_databases() will all go away soon.

Note that this doesn't cause a change in behaviour.  The main daemon
will still only defer attaches from non-recoverd processes during
recovery.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 98e3d0db2bc5f33217e26fab1dfb4bb91eae534f)

4 years agoctdb-protocol: Add control flag CTDB_CTRL_FLAG_ATTACH_RECOVERY
Martin Schwenke [Fri, 21 Feb 2020 00:04:14 +0000 (11:04 +1100)]
ctdb-protocol: Add control flag CTDB_CTRL_FLAG_ATTACH_RECOVERY

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 17ed0425904a98624284d351ab7617b3e02c0f7b)

4 years agoctdb-daemon: Remove unused old client database functions
Martin Schwenke [Wed, 26 Feb 2020 06:03:49 +0000 (17:03 +1100)]
ctdb-daemon: Remove unused old client database functions

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit fc23cd1b9cdd1d70067491614b16e616291e8ff2)

4 years agoctdb-daemon: Fix database attach deferral logic
Martin Schwenke [Thu, 20 Feb 2020 02:48:13 +0000 (13:48 +1100)]
ctdb-daemon: Fix database attach deferral logic

Commit 3cc230b5eeca749ab68d19cfda969f72c269f1f6 says:

  Dont allow clients to connect to databases untile we are well past
  and through the initial recovery phase

It is unclear what this commit was attempting to do.  The commit
message implies that more attaches should be deferred but the code
change adds a conjunction that causes less attaches to be deferred.
In particular, no attaches will be deferred after startup is complete.
This seems wrong.

To implement what seems to be stated in the commit message an "or"
needs to be used so that non-recovery daemon attaches are deferred
either when in recovery or before startup is complete.  Making this
change highlights that attaches need to be allowed during the
"startup" event because this is when smbd is started.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit c6c89495fbe9b6f238d10a538eccc92b937a69de)

4 years agoctdb-recovery: Refactor banning a node into separate computation
Amitay Isaacs [Mon, 2 Mar 2020 05:16:26 +0000 (16:16 +1100)]
ctdb-recovery: Refactor banning a node into separate computation

If a node is marked for banning, confirm that it's not become inactive
during the recovery.  If yes, then don't ban the node.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(cherry picked from commit 1c56d6413f86cc15ebac232f39ef1e2a53ae4297)

4 years agoctdb-recovery: Don't trust nodemap obtained from local node
Amitay Isaacs [Tue, 18 Feb 2020 05:17:00 +0000 (16:17 +1100)]
ctdb-recovery: Don't trust nodemap obtained from local node

It's possible to have a node stopped, but recovery master not yet
updated flags on the local ctdb daemon when recovery is started.  So do
not trust the list of active nodes obtained from the local node.  Query
the connected nodes to calculate the list of active nodes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(cherry picked from commit c6a0ff1bed0265e44fd6135d16bfc41919fe5bf5)

4 years agoctdb-recovery: Consolidate node state
Amitay Isaacs [Mon, 2 Mar 2020 04:07:21 +0000 (15:07 +1100)]
ctdb-recovery: Consolidate node state

This avoids passing multiple arguments to async computation.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(cherry picked from commit 6e2f8756f1bce4dfc7fdc435e082f400116e29ec)

4 years agoctdb-recovery: Fetched vnnmap is never used, so don't fetch it
Amitay Isaacs [Mon, 2 Mar 2020 02:59:42 +0000 (13:59 +1100)]
ctdb-recovery: Fetched vnnmap is never used, so don't fetch it

New vnnmap is constructed using the information from all the connected
nodes.  So there is no need to fetch the vnnmap from recovery master.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(cherry picked from commit 072ff4d12b8f34766120ddae888d772e97bca491)

4 years agoselftest: test samba-tool group commands with groupnames with brackets and spaces
Björn Baumbach [Thu, 27 Feb 2020 10:06:34 +0000 (11:06 +0100)]
selftest: test samba-tool group commands with groupnames with brackets and spaces

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14296

Signed-off-by: Björn Baumbach <bb@sernet.de>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit 534809a0f09775390e89fa1cbfae3a1d0fafecb8)

Autobuild-User(v4-12-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-12-test): Fri Mar 27 13:19:20 UTC 2020 on sn-devel-184