VERSION: Disable GIT_SNAPSHOT for the 4.10.0rc4 release.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

WHATSNEW: Add release notes for Samba 4.10.0rc4.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

WHATSNEW: Add some detail on the changes to paged results

Signed-off-by: Garming Sam <garming@catalyst.net.nz>

fix samba-tool gpo backup syntax in WHATSNEW.txt

Signed-off-by: Denis Cardon <dcardon@tranquil.it>

vfs_ceph: remove ceph_fallocate/ceph_ftruncate fallback

Both libcephfs functions are supported and capable of extending files,
so fallback can be dropped.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13807

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
(cherry picked from commit 155f1289ba7a4802fbb99fbc9ea90d8bc6cff0c9)

vfs_ceph: fix strict_allocate_ftruncate()

The vfs_ceph "strict allocate = yes" ftruncate wrapper may attempt
*local* filesystem ftruncate(). Fix this.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13807

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
(cherry picked from commit 70329c36ed91dd0e50ff66f9b0a85c62ac8a621e)

vfs_ceph: add missing fallocate hook

SMB_VFS_FALLOCATE() calls atop a vfs_ceph share currently fall through
to vfs_default, which results in a local filesystem I/O attempt using a
libcephfs file-descriptor. Add the missing fallocate hook to vfs_ceph.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13807

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
(cherry picked from commit 5a7e7280813559fb70a9fc8e4238cb6015ee3b53)

s3: smbd: filenames - ensure we replace the missing '/' if we error in an intermediate POSIX path.

Previous regression test ensures we still return the correct
error code for POSIX pathname operations.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13803

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Mon Feb 25 09:33:27 CET 2019 on sn-devel-144

(cherry picked from commit 3f1a13a94a753c5cb3b9f2cf795df5adb0f74205)

s3: torture: Add additional POSIX mkdir tests.

Ensure that if POSIX_foo exists as a file
we return the correct error code NT_STATUS_OBJECT_PATH_NOT_FOUND
if we try and traverse it as a directory.

Also ensure creation/deletion of POSIX_foo/foo fails
for directories and files with NT_STATUS_OBJECT_PATH_NOT_FOUND
if the directory POSIX_foo/ doesn't exist.

knownfail is back :-).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13803

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 632d0db8c42d50f5eecd002d9573f739cd945960)

smbd: unix_convert: Ensure we don't call get_real_filename on POSIX paths.

For posix_pathnames don't blunder into the name_has_wildcard OR
get_real_filename() codepaths as they may be doing case insensitive lookups.
So when creating a new POSIX directory 'Foo' they might
match on name 'foo'.

Remove POSIX-MKDIR from knownfail.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13803

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sun Feb 24 14:04:14 CET 2019 on sn-devel-144

(cherry picked from commit 12da33e2bbce657f25abe9bc4cb936ca4615b30f)

smbd: SMB1-POSIX: Add missing info-level SMB_POSIX_PATH_OPEN for UCF_UNIX_NAME_LOOKUP flag.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13803

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 980db8debde68f9edce13fc01bc45a904379cffd)

s3: smbtorture3: Add POSIX-MKDIR test for posix_mkdir case sensitive bug.

Test does:

mkdir POSIX_foo
mkdir POSIX_Foo
mkdir POSIX_foo/Foo
mkdir POSIX_foo/foo
mkdir POSIX_Foo/Foo
mkdir POSIX_Foo/foo

Which should pass a SMB1 POSIX extensions server
as posix mkdir should always be case sensitive
no matter what the share is set to.

Mark as knownfail for now.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13803

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit f0decb31fff24ceb57d23bebbfb87aa0e5103947)

winbindd: set idmap cache entries as the last step in async wb_xids2sids

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13802

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sat Feb 23 09:23:22 CET 2019 on sn-devel-144

(cherry picked from commit 9b9565c3e69b92c298c7168e516387bb249c9e36)

winbindd: track whether a result from xid2sid was coming from the cache

This is needed in preparation of moving the step to update the idmap
cache from the per-idmap-domain callback wb_xids2sids_dom_done() to the
top-level callback wb_xids2sids_done().

Currently the sequence of action is:

* check cache, if not found:
  * ask backends
  * cache result from backend
* return results

Iow, if we got something from the cache, we don't write the cache.

The next commit defers updating the cache to the top-level callback, so
the sequence becomes

* check cache, if not found:
  * ask backends
* cache results
* return results

This has two problems:

* it needlessly writes to the cache what we just got from it

* it possibly overwrites the ID_TYPE_BOTH for a SID-to-xid mapping in
  the following case:

  - existing ID_TYPE_BOTH mapping in the cache, eg:

    IDMAP/SID2XID/S-1-5-21-2180672342-2513613279-2566592647-512 -> Value: 3000000:B

  - someone calls wb_xids2sids_send() with xid.id=3000000,xid.type=ID_TYPE_GID

  - cache lookup with idmap_cache_find_gid2sid() succeeds

  - when caching results we'd call idmap_cache_set_sid2unixid() with the
    callers xid.type=ID_TYPE_GID, so idmap_cache_set_sid2unixid() will
    overwrite the SID-to-xid mapping with ID_TYPE_GID

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13802

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 62f54229fced20102e11ad1da02faef45c2a7c2e)

winbindd: switch send-next/done order

In preparation of adding more logic to the done step. No change in
behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13802

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 8e9c2a1f6ceb06d695a6572701b96a3e3821ac42)

winbindd: update xid in wb_xids2sids_state->xids with what we got

In preparation of priming the idmap cache in the top-level
wb_xids2sids_done(), not in the per-idmap-domain callback
wb_xids2sids_dom_done().

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13802

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 7f23ef7b2cf7bd6e8dc087aa15137292b421a689)

winbindd: convert id to a pointer in wb_xids2sids_dom_done()

No change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13802

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit f8bf4fc608639695651f75c52b31f95e796a5a26)

winbindd: make xids a const argument to wb_xids2sids_send()

The previous commit made an internal copy of xids, this commit makes it
more obvious that we must not mess with the xids argument but treat it as
an in-parameter and don't write to it.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13802

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 5d277ea7ea258676b9ea5081a451a5874af115f6)

winbindd: make a copy of xid's in wb_xids2sids_send()

This is in preparation of setting the result of the mapping in the top-
level callback wb_xids2sids_done(), not in the per-idmap-domain callback
wb_xids2sids_dom_done().

When caching the mapping we need the id-type from the backend, so we
need a way to pass up that information from wb_xids2sids_dom_done() up
to wb_xids2sids_done()

The xids array copy gets passed from wb_xids2sids_send() to
wb_xids2sids_dom_send(), so wb_xids2sids_dom_done() can then directly
update the top-level copy.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13802

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit f5a8bc2f945be45cdade5f70d4f975bae8337f67)

ctdb-cluster-mutex: Separate out command and file handling

This code is difficult to read and there really is no common code
between the 2 cases.  For example, there is no need to split a
filename into words.  Separating each of the 2 cases into its own
function makes the logic much easier to understand.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Feb 25 03:40:16 CET 2019 on sn-devel-144

(cherry picked from commit c93430fe8fe530a55b9a04cf6cc660c3d420e333)

ctdb-tests: Add a test for configuring the recovery lock as a command

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit ebc082122fb34ffb8cbcafde9ad39bcc241d33ed)

ctdb-tests: Add -R option for local daemons to use recovery lock command

Under the covers, a command is always used.  However, there is no way
of testing of the code path where a command is explicitly configured.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit e74f5243fcb7594939769c16f3c79ab167dd1227)

ctdb-tests: Force test failure if local daemon setup fails

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit ce09d9c3e4c72ebec7a21686ae913398a5c9020f)

ctdb-recoverd: Time out attempt to take recovery lock after 120s

Currently this will wait forever.  It really needs a timeout in case
the cluster filesystem (or other lock mechanism) is completely wedged.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 13a1a4808935290dceb219daccd7aac3fda4e184)

ctdb-recoverd: Ban node on unknown error when taking recovery lock

We really shouldn't see unknown errors.  They probably represent a
misconfigured recovery lock or similar.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 45a77d65b2e39b4af94da4ab99575f4ee08a7ebd)

ctdb-recoverd: Make recoverd context available in recovery lock handle

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit c0fb62ed3954fc6e8667480aba92003fc270f257)

ctdb-recoverd: Clean up logging on failure to take recovery lock

Add an explicit case for a timeout and clean up the other messages.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 7e4aae6943291c3144c8a3ff97537e8d4c7dc7c9)

ctdb-recoverd: Free cluster mutex handler on failure to take lock

If nested events occur while the file descriptor handler is still
active then chaos can ensue.  For example, if a node is banned and the
lock is explicitly cancelled (e.g. due to election loss) then
double-talloc-free()s abound.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 621658cbed5d91d7096fc208bac2ff93a1880e7d)

ctdb-tests: Add test for ctdb_io.c

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13791

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Feb 22 03:51:37 CET 2019 on sn-devel-144

(cherry picked from commit 92a90524373a0348c1912d5019254dd18c07e207)

ctdb: buffer write beyond limits

In order to calculate the number of bytes correctly which
are to be read into the buffer, the buffer.offset must be taken
into account.

This patch fixes a regression introduced by 382705f495dd.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13791

Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(cherry picked from commit fa8e69ac9538980c441b7fbefe0979027ecc8eac)

ctdb-config: Change example recovery lock setting to one that fails

ctdbd will start without a recovery lock configured.  It will log a
message saying that this is not optimal.  However, a careless user may
overlook both this message and the importance of setting a recovery
lock.  If the existing example configuration is uncommented then the
directory containing it will be created (by 01.reclock.script) and the
failure (i.e. multiple nodes able to take the lock) will be confusing.

Instead, change the example setting to one that will result in banned
nodes, encouraging users to consciously configure (or deconfigure) the
recovery lock.  Tweak the corresponding comment.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13790

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 82e7f38214896c2c200132bc6dde3348cfac16cc)

messages_dgm: Properly handle receiver re-initialization

This only properly covers the small-message nonblocking case. Covering
the large-message and the blocking case is a much larger effort assuming
we want to re-send the failed message if parts of the message has gone
through properly. Don't do that for now.

This was found by sanba_dnsupdate constantly recreating its irpc handle to
winbindd in the RODC case.

The messaging_dgm code cached connected datagram sockets based on the
destination pid for 1 second. Which means the IRPC responses from
winbindd are never delivered to samba_dnsupdate,
which will then hit a timeout.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13786

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit 2543bba0364d8054e9ad316f5611621841bc061d)

torture3: Extend read3 for the "messaging target re-inits" failure

Do ping_pong a hundred times, re-initializing the msg_ctx every time.

https://bugzilla.samba.org/show_bug.cgi?id=13786

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit 8d8f62c4b9dea381ce9f5833bc794553ae358173)

messages_dgm: Use saved errno value

In this case this is just a cleanup, the value has just been set by
messaging_dgm_sendmsg. But as that already saves errno into a local
variable, use that.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13786

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit c27afc098398274abaed6dc9bef2019091c1b635)

man pages: document prefork process model

Document the process model options -M

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13765

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Tim Beale <timbeale@samba.org>
Autobuild-Date(master): Thu Jan 31 04:25:14 CET 2019 on sn-devel-144

(cherry picked from commit 4ecdcc4b462d03bec3ec36ba1b90f45d9977be8b)

notifyd: Fix SIGBUS on sparc

Problem is the structure "struct notify_instance" can lay in message buffer on
address not dividable by 8 but this structure begins by uint_64 (secs in
time-stamp). Structure should be re-packed to standalone buffer before the use.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13704
Signed-off-by: jiri.sasek@oracle.com
Reviewed-by: Volker Lendecke <vl@samba.org>

CVE-2019-3824 ldb: version 1.5.4

* Fix standalone build of ldb.
* C99 build fixes.
* CVE-2019-3824 out of bounds read in wildcard compare (bug 13773)

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(v4-10-test): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(v4-10-test): Tue Feb 26 16:09:12 CET 2019 on sn-devel-144

CVE-2019-3824 ldb: Add tests for ldb_wildcard_match

Add cmocka tests for ldb_wildcard_match.

Running test_wildcard_match under valgrind reproduces
 CVE-2019-3824 out of bounds read in wildcard compare (bug 13773)

 valgrind --suppressions=lib/ldb/tests/ldb_match_test.valgrind\
          bin/ldb_match_test

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit 45b75db50f5c1a7c8c38af59a62fccee5401c845)

CVE-2019-3824 ldb: wildcard_match end of data check

ldb_handler_copy and ldb_val_dup over allocate by one and add a trailing '\0'
to the data, to make them safe to use the C string functions on.

However testing for the trailing '\0' is not the correct way to test for
the end of a value, the length should be checked instead.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit 42f0f57eb819ce6b68a8c5b3b53123b83ec917e3)

CVE-2019-3824 ldb: wildcard_match check tree operation

Check the operation type of the passed parse tree, and return
LDB_INAPPROPRIATE_MATCH if the operation is not LDB_OP_SUBSTRING.

A query of "attribute=*" gets parsed as LDB_OP_PRESENT, checking the
operation and failing ldb_wildcard_match should help prevent confusion
writing tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit 34383981a0c40860f71a4451ff8fd752e1b67666)

CVE-2019-3824 ldb: ldb_parse_tree use talloc_zero

Initialise the created ldb_parse_tree with talloc_zero, this ensures
that it is correctly initialised if inadvertently passed to a function
expecting a different operation type.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit 8d34d172092f71baad0d777567e49aebfa07313d)

CVE-2019-3824 ldb: Improve code style and layout in wildcard processing

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
(cherry picked from commit 9427806f7298d71bd7edfbdda7506ec63f15dda1)

CVE-2019-3824 ldb: Extra comments to clarify no pointer wrap in wildcard processing

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
(cherry picked from commit 745b99fc6b75db33cdb0a58df1a3f2a5063bc76e)

CVE-2019-3824 ldb: Out of bound read in ldb_wildcard_compare

There is valgrind error in few tests tests/test-generic.sh
 91 echo "Test wildcard match"
 92 $VALGRIND ldbadd $LDBDIR/tests/test-wildcard.ldif  || exit 1
 93 $VALGRIND ldbsearch '(cn=test*multi)'  || exit 1
 95 $VALGRIND ldbsearch '(cn=*test_multi)'  || exit 1
 97 $VALGRIND ldbsearch '(cn=test*multi*test*multi)'  || exit 1

e.g.
  ==3098== Memcheck, a memory error detector
  ==3098== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
  ==3098== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info
  ==3098== Command: ./bin/ldbsearch (cn=test*multi)
  ==3098==
  ==3098== Invalid read of size 1
  ==3098==    at 0x483CEE7: memchr (vg_replace_strmem.c:890)
  ==3098==    by 0x49A9073: memmem (in /usr/lib64/libc-2.28.9000.so)
  ==3098==    by 0x485DFE9: ldb_wildcard_compare (ldb_match.c:313)
  ==3098==    by 0x485DFE9: ldb_match_substring (ldb_match.c:360)
  ==3098==    by 0x485DFE9: ldb_match_message (ldb_match.c:572)
  ==3098==    by 0x558F8FA: search_func (ldb_kv_search.c:549)
  ==3098==    by 0x48C78CA: ??? (in /usr/lib64/libtdb.so.1.3.17)
  ==3098==    by 0x48C7A60: tdb_traverse_read (in /usr/lib64/libtdb.so.1.3.17)
  ==3098==    by 0x557B7C4: ltdb_traverse_fn (ldb_tdb.c:274)
  ==3098==    by 0x558FBFA: ldb_kv_search_full (ldb_kv_search.c:594)
  ==3098==    by 0x558FBFA: ldb_kv_search (ldb_kv_search.c:854)
  ==3098==    by 0x558E497: ldb_kv_callback (ldb_kv.c:1713)
  ==3098==    by 0x48FCD58: tevent_common_invoke_timer_handler (in /usr/lib64/libtevent.so.0.9.38)
  ==3098==    by 0x48FCEFD: tevent_common_loop_timer_delay (in /usr/lib64/libtevent.so.0.9.38)
  ==3098==    by 0x48FE14A: ??? (in /usr/lib64/libtevent.so.0.9.38)
  ==3098==  Address 0x4b4ab81 is 0 bytes after a block of size 129 alloc'd
  ==3098==    at 0x483880B: malloc (vg_replace_malloc.c:309)
  ==3098==    by 0x491048B: talloc_strndup (in /usr/lib64/libtalloc.so.2.1.15)
  ==3098==    by 0x48593CA: ldb_casefold_default (ldb_utf8.c:59)
  ==3098==    by 0x485F68D: ldb_handler_fold (attrib_handlers.c:64)
  ==3098==    by 0x485DB88: ldb_wildcard_compare (ldb_match.c:257)
  ==3098==    by 0x485DB88: ldb_match_substring (ldb_match.c:360)
  ==3098==    by 0x485DB88: ldb_match_message (ldb_match.c:572)
  ==3098==    by 0x558F8FA: search_func (ldb_kv_search.c:549)
  ==3098==    by 0x48C78CA: ??? (in /usr/lib64/libtdb.so.1.3.17)
  ==3098==    by 0x48C7A60: tdb_traverse_read (in /usr/lib64/libtdb.so.1.3.17)
  ==3098==    by 0x557B7C4: ltdb_traverse_fn (ldb_tdb.c:274)
  ==3098==    by 0x558FBFA: ldb_kv_search_full (ldb_kv_search.c:594)
  ==3098==    by 0x558FBFA: ldb_kv_search (ldb_kv_search.c:854)
  ==3098==    by 0x558E497: ldb_kv_callback (ldb_kv.c:1713)
  ==3098==    by 0x48FCD58: tevent_common_invoke_timer_handler (in /usr/lib64/libtevent.so.0.9.38)
  ==3098==
  # record 1
  dn: cn=test_multi_test_multi_test_multi,o=University of Michigan,c=TEST
  cn: test_multi_test_multi_test_multi
  description: test multi wildcards matching
  objectclass: person
  sn: multi_test
  name: test_multi_test_multi_test_multi
  distinguishedName: cn=test_multi_test_multi_test_multi,o=University of Michiga
   n,c=TEST

  # returned 1 records
  # 1 entries
  # 0 referrals

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Lukas Slebodnik <lslebodn@fedoraproject.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
(cherry picked from commit 3674b0891afb016c83763520b87e9f190dcfe884)

ldb: The test api.py should not rely on order of entries in dict

Test failed on s390x but there is a simple reproducer for any
architecture.

The built-in function repr returns the canonical string representation
of the object. We needn't care about order attributes in string
representation. Therefore test should pass for any order.

    for i in {1..30}; do
        PYTHONHASHSEED=random \
        python2 -c 'import ldb; msg = ldb.Message(); msg.dn = ldb.Dn(ldb.Ldb(), "dc=foo29"); msg["dc"] = b"foo"; print(repr(msg)) '
    done

  ======================================================================
  FAIL: test_repr (__main__.LdbMsgTests)
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "tests/python/api.py", line 2322, in test_repr
      "Message({'dn': Dn('dc=foo29'), 'dc': MessageElement(['foo'])})")
  AssertionError: "Message({'dc': MessageElement(['foo']), 'dn': Dn('dc=foo29')})" != "Message({'dn': Dn('dc=foo29'), 'dc': MessageElement(['foo'])})"
  ----------------------------------------------------------------------
  Ran 1025 tests in 29.146s
  FAILED (failures=1)

Signed-off-by: Lukas Slebodnik <lslebodn@fedoraproject.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
(cherry picked from commit 7a7a5ccf501f89c530970bde072509ed86d7bd89)

lib:ldb: Use correct C99 initializer for 'struct tm'

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
(cherry picked from commit 146ed0894e640fe1791f75fa8f50167b6cdfc06e)

lib:ldb: Use C99 initializer for tdb_logging_context

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
(cherry picked from commit 3cd5d4b39713559d2f88e21a24f8190fe9ce075c)

lib:ldb: Use C99 initializer for PyGetSetDef in pyldb

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
(cherry picked from commit 85a5dc56e34298a60d7ef96f4178ccff20d40c82)

tevent: version 0.9.39

* py_tevent: add_timer takes float argument
* C99 build fixes.
* Fix standalone build of tevent.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit db58a50296041ca57675daee15caea8850f1d3f8)

py_tevent: add_timer takes float argument

We were already using it that way.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
(cherry picked from commit 8294e68a4175e6116b38869866c42a1b7ba55b6a)

lib:tevent: Use correct C99 initializer for tevent_req

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
(cherry picked from commit 0da2e9c215f882ea4c331ac4f17738913beb7404)

talloc: version 2.1.16

* Fix standalone build of talloc.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 3fe1551b5347934a20b9161a23e6a16220c3aeb6)

tdb: version 1.3.18

* Fix build problems with older python versions.
* C99 build fixes.
* Fix standalone build of tdb.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit f0d26dd1816f35a00abf52b640f42547ffdfa01b)

lib:tdb: Use C99 initializer for tdb_logging_context

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
(cherry picked from commit 47b57a2fc87664366099913305dd8d3783734a1a)

lib:tdb: Use C99 initializer for tdb_header

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
(cherry picked from commit a1ce666d68745a9bc65bd2709c581ed5105f656d)

lib:tdb: Use C99 initializer for PyGetSetDef in pytdb

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
(cherry picked from commit 651ee7f205f4a7e31c791f7bb235275816747463)

tdb: Fix compatibility of wscript with older python

Traceback (most recent call last):
  File "tdb-1.3.17/third_party/waf/waflib/Scripting.py", line 158, in waf_entry_point
    run_commands()
  File "tdb-1.3.17/third_party/waf/waflib/Scripting.py", line 251, in run_commands
    ctx = run_command(cmd_name)
  File "tdb-1.3.17/third_party/waf/waflib/Scripting.py", line 235, in run_command
    ctx.execute()
  File "tdb-1.3.17/third_party/waf/waflib/Context.py", line 204, in execute
    self.recurse([os.path.dirname(g_module.root_path)])
  File "tdb-1.3.17/third_party/waf/waflib/Context.py", line 286, in recurse
    user_function(self)
  File "tdb-1.3.17/wscript", line 225, in testonly
    cmd = "BINDIR={} {}".format(blddir, sh_test)
ValueError: zero length field name in format

Signed-off-by: Lukas Slebodnik <lslebodn@fedoraproject.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sun Jan 20 03:49:59 CET 2019 on sn-devel-144

(cherry picked from commit 0628ca2a558871402f9a17cc7f7a0c69d857ae19)

Search for location of waf script

When calling make from the ldb, talloc, tdb, and
tevent bundles, we need to first find the
location of the waf script. Currently the build
fails since it can't find waf.

Fixes regression caused by a660b7f.

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit 8ecd7f756f0fafe5fd79dc4deb445883450c7cca)

buildtools/wafsamba: Avoid decode when using python2

To avoid problematic type checking for 'str' types which fail
when result from str.decode is used.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13777

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit 244e2a02796b2ee85b9db01cbea7043a7448a110)

Autobuild-User(v4-10-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-10-test): Fri Feb 22 13:44:34 CET 2019 on sn-devel-144

VERSION: Bump version up to 4.10.0rc4...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

VERSION: Disable GIT_SNAPSHOT for the 4.10.0rc3 release.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

WHATSNEW: Add release notes for Samba 4.10.0rc3.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

waf: Check for libnscd

The check was in the old autoconf, but not in waf. As the code is still
in source3/lib/util_nscd.c, add the check for libnscd to allow building
and using the code.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13787

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Wed Feb 13 17:58:33 CET 2019 on sn-devel-144

(cherry picked from commit 3a793497796395ffa3efda5807bdb1ca8e09e35b)

Autobuild-User(v4-10-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-10-test): Thu Feb 21 16:26:51 CET 2019 on sn-devel-144

WHATSNEW: Add note that python2 support will be dropped on v4.11

Add a warning to Samba users that v4.10 will be the last Samba release
with python2 support.

I've reworked the existing text describing the different python2 build
options for 4.10. Hopefully this makes it slightly clearer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13785

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>

waf: Do not install internal header

We should not install header files without an public API:

- memory.h
- safe_strings.h
- talloc_stack.h

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13778

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 6d232f3f7c64f9b01439326e0e9b6d9df9a0bcbb)

lib:util: Move discard_const(_p) to own header for libndr.h

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13778

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 96df6878ed521b7e744d703abb32a585500d3b94)

pidl: Use NDR_ZERO_STRUCT(P) macros

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13778

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 532ce0d20a8016c1270ea689de627da8aa4abfdd)

librpc:ndr: Add NDR_ZERO_STRUCT(P) macros

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13778

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 666802a3db3115ca09f3ffed58c8e4a8cabd65de)

librpc:ndr: Implement ndr_zero_memory()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13778

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 2a7086f1ac4ad91fb7cd958e9386abb996794ed1)

tldap: avoid more use after free errors

See the previous commit for an explanation. :)

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13776

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Feb  6 10:19:12 CET 2019 on sn-devel-144

(cherry picked from commit bf91ee0a9727cc392583fe84ad069204be758515)

tldap: avoid a use after free crash

I saw the following crash in tldap in the winbindd idmap child on a
member server after messing with the LDAP server on the DC:

0  0x00007f77ea9a307a in __GI___waitpid (pid=9815, stat_loc=stat_loc@entry=0x7ffe77569eb0, options=options@entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29
1  0x00007f77ea91bfbb in do_system (line=<optimized out>) at ../sysdeps/posix/system.c:148
2  0x00007f77edd8c24b in smb_panic_s3 (why=0x7f77f08e6e88 "Bad talloc magic value - access after free") at ../source3/lib/util.c:828
3  0x00007f77f15afe85 in smb_panic (why=0x7f77f08e6e88 "Bad talloc magic value - access after free") at ../lib/util/fault.c:170
4  0x00007f77f08e2678 in talloc_abort (reason=0x7f77f08e6e88 "Bad talloc magic value - access after free") at ../lib/talloc/talloc.c:472
5  0x00007f77f08e268b in talloc_abort_access_after_free () at ../lib/talloc/talloc.c:477
6  0x00007f77f08e2710 in talloc_chunk_from_ptr (ptr=0x55da7605a020) at ../lib/talloc/talloc.c:494
7  0x00007f77f08e4a19 in _talloc_free (ptr=0x55da7605a020, location=0x7f77e181474d "../source3/lib/tldap.c:1918") at ../lib/talloc/talloc.c:1716
8  0x00007f77e180b65c in tldap_search_all_done (subreq=0x55da7605a020) at ../source3/lib/tldap.c:1918
9  0x00007f77f0af0fd0 in _tevent_req_notify_callback (req=0x55da7605a020, location=0x7f77e1813e50 "../source3/lib/tldap.c:47") at ../lib/tevent/tevent_req.c:125
10 0x00007f77f0af10a5 in tevent_req_finish (req=0x55da7605a020, state=TEVENT_REQ_USER_ERROR, location=0x7f77e1813e50 "../source3/lib/tldap.c:47") at ../lib/tevent/tevent_req.c:162
11 0x00007f77f0af1113 in _tevent_req_error (req=0x55da7605a020, error=9780923860630110289, location=0x7f77e1813e50 "../source3/lib/tldap.c:47") at ../lib/tevent/tevent_req.c:180
12 0x00007f77e180781a in tevent_req_ldap_error (req=0x55da7605a020, rc=...) at ../source3/lib/tldap.c:47
13 0x00007f77e180b2c4 in tldap_search_done (subreq=0x55da76058280) at ../source3/lib/tldap.c:1813
14 0x00007f77f0af0fd0 in _tevent_req_notify_callback (req=0x55da76058280, location=0x7f77e1813e50 "../source3/lib/tldap.c:47") at ../lib/tevent/tevent_req.c:125
15 0x00007f77f0af10a5 in tevent_req_finish (req=0x55da76058280, state=TEVENT_REQ_USER_ERROR, location=0x7f77e1813e50 "../source3/lib/tldap.c:47") at ../lib/tevent/tevent_req.c:162
16 0x00007f77f0af11cd in tevent_req_trigger (ev=0x55da760526c0, im=0x55da76058360, private_data=0x55da76058280) at ../lib/tevent/tevent_req.c:219
17 0x00007f77f0af0378 in tevent_common_loop_immediate (ev=0x55da760526c0) at ../lib/tevent/tevent_immediate.c:135
18 0x00007f77f0af8b8f in epoll_event_loop_once (ev=0x55da760526c0, location=0x7f77f0af92b0 "../lib/tevent/tevent_req.c:269") at ../lib/tevent/tevent_epoll.c:911
19 0x00007f77f0af5925 in std_event_loop_once (ev=0x55da760526c0, location=0x7f77f0af92b0 "../lib/tevent/tevent_req.c:269") at ../lib/tevent/tevent_standard.c:114
20 0x00007f77f0aef201 in _tevent_loop_once (ev=0x55da760526c0, location=0x7f77f0af92b0 "../lib/tevent/tevent_req.c:269") at ../lib/tevent/tevent.c:725
21 0x00007f77f0af1361 in tevent_req_poll (req=0x55da7605eed0, ev=0x55da760526c0) at ../lib/tevent/tevent_req.c:269
22 0x00007f77e180fec9 in tldap_gensec_bind (ctx=0x55da76051ec0, creds=0x55da76052250, target_service=0x7f77e18164b3 "ldap", target_hostname=0x55da7605d182 "dc1.sdom1.site", target_principal=0x0, lp_ctx=0x55da76052180, gensec_features=6) at ../source3/lib/tldap_gensec_bind.c:358
23 0x00007f77e1810d21 in idmap_ad_get_tldap_ctx (mem_ctx=0x55da76050510, domname=0x55da76051d50 "sdom1", pld=0x55da76050518) at ../source3/winbindd/idmap_ad.c:326
24 0x00007f77e1811056 in idmap_ad_context_create (mem_ctx=0x55da76059c00, dom=0x55da76059c00, domname=0x55da76051d50 "sdom1", pctx=0x7ffe7756a5f8) at ../source3/winbindd/idmap_ad.c:374
25 0x00007f77e18119c0 in idmap_ad_get_context (dom=0x55da76059c00, pctx=0x7ffe7756a640) at ../source3/winbindd/idmap_ad.c:554
26 0x00007f77e181275b in idmap_ad_sids_to_unixids (dom=0x55da76059c00, ids=0x55da760518a0) at ../source3/winbindd/idmap_ad.c:784
27 0x00007f77e1813217 in idmap_ad_sids_to_unixids_retry (dom=0x55da76059c00, ids=0x55da760518a0) at ../source3/winbindd/idmap_ad.c:947
28 0x000055da7459ce05 in _wbint_Sids2UnixIDs (p=0x7ffe7756a870, r=0x55da76050860) at ../source3/winbindd/winbindd_dual_srv.c:202
29 0x000055da7460aa5e in api_wbint_Sids2UnixIDs (p=0x7ffe7756a870) at default/librpc/gen_ndr/srv_winbind.c:391
30 0x000055da7459c7f4 in winbindd_dual_ndrcmd (domain=0x0, state=0x7ffe7756abb8) at ../source3/winbindd/winbindd_dual_ndr.c:369
31 0x000055da7459828c in child_process_request (child=0x55da74874bc0 <static_idmap_child>, state=0x7ffe7756abb8) at ../source3/winbindd/winbindd_dual.c:666
32 0x000055da7459ae58 in child_handler (ev=0x55da7602c2b0, fde=0x55da7603f8a0, flags=1, private_data=0x7ffe7756abb0) at ../source3/winbindd/winbindd_dual.c:1567
33 0x00007f77f0af85f1 in epoll_event_loop (epoll_ev=0x55da76048b00, tvalp=0x7ffe7756aab0) at ../lib/tevent/tevent_epoll.c:728
34 0x00007f77f0af8c29 in epoll_event_loop_once (ev=0x55da7602c2b0, location=0x55da74628b08 "../source3/winbindd/winbindd_dual.c:1766") at ../lib/tevent/tevent_epoll.c:930
35 0x00007f77f0af5925 in std_event_loop_once (ev=0x55da7602c2b0, location=0x55da74628b08 "../source3/winbindd/winbindd_dual.c:1766") at ../lib/tevent/tevent_standard.c:114
36 0x00007f77f0aef201 in _tevent_loop_once (ev=0x55da7602c2b0, location=0x55da74628b08 "../source3/winbindd/winbindd_dual.c:1766") at ../lib/tevent/tevent.c:725
37 0x000055da7459b9e9 in fork_domain_child (child=0x55da74874bc0 <static_idmap_child>) at ../source3/winbindd/winbindd_dual.c:1766
38 0x000055da74596e96 in wb_child_request_waited (subreq=0x0) at ../source3/winbindd/winbindd_dual.c:188
39 0x00007f77f0af0fd0 in _tevent_req_notify_callback (req=0x55da7604f820, location=0x7f77f0af90f8 "../lib/tevent/tevent_queue.c:355") at ../lib/tevent/tevent_req.c:125
40 0x00007f77f0af10a5 in tevent_req_finish (req=0x55da7604f820, state=TEVENT_REQ_DONE, location=0x7f77f0af90f8 "../lib/tevent/tevent_queue.c:355") at ../lib/tevent/tevent_req.c:162
41 0x00007f77f0af10cd in _tevent_req_done (req=0x55da7604f820, location=0x7f77f0af90f8 "../lib/tevent/tevent_queue.c:355") at ../lib/tevent/tevent_req.c:168
42 0x00007f77f0af0cc1 in tevent_queue_wait_trigger (req=0x55da7604f820, private_data=0x0) at ../lib/tevent/tevent_queue.c:355
43 0x00007f77f0af06f2 in tevent_queue_immediate_trigger (ev=0x55da7602c2b0, im=0x55da760466a0, private_data=0x55da76046580) at ../lib/tevent/tevent_queue.c:149
44 0x00007f77f0af0378 in tevent_common_loop_immediate (ev=0x55da7602c2b0) at ../lib/tevent/tevent_immediate.c:135
45 0x00007f77f0af8b8f in epoll_event_loop_once (ev=0x55da7602c2b0, location=0x55da74612630 "../source3/winbindd/winbindd.c:1803") at ../lib/tevent/tevent_epoll.c:911
46 0x00007f77f0af5925 in std_event_loop_once (ev=0x55da7602c2b0, location=0x55da74612630 "../source3/winbindd/winbindd.c:1803") at ../lib/tevent/tevent_standard.c:114
47 0x00007f77f0aef201 in _tevent_loop_once (ev=0x55da7602c2b0, location=0x55da74612630 "../source3/winbindd/winbindd.c:1803") at ../lib/tevent/tevent.c:725
48 0x000055da74561431 in main (argc=2, argv=0x7ffe7756c968) at ../source3/winbindd/winbindd.c:1803

subreq is a child of the state of req which will already be free by the
callback of req.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13776

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 9465292d4109f710f8fcd141a076f5c8278577bc)

s3:vfs: Correctly check if OFD locks should be enabled or not

Also the smb.conf options should only be checked once and a reload of
the config should not switch to a different locking mode.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13770

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Feb  9 03:43:50 CET 2019 on sn-devel-144

(cherry picked from commit 7ff94b18e2e39567ef7a208084cc5c914c39d3bd)

s3:vfs: Initialize pid to 0 in test_netatalk_lock()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13770

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 2ff2594b2bd878928cec30bc72a95a6d38bee154)

s4: torture: vfs_fruit. Change test_fruit_locking_conflict() to match the vfs_fruit working server code.

Originally added for BUG: https://bugzilla.samba.org/show_bug.cgi?id=13584
to demonstrate a lock order violation, this test
exposed problems in the mapping of SMB1/2 share modes
and open modes to NetATalk modes once we moved to OFD locks.

Change the test slightly (and add comments)
so it demonstrates working NetATalk share modes
on an open file.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13770

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Feb  8 23:26:46 CET 2019 on sn-devel-144

(cherry picked from commit 28990e4ba23695ecf264117efad90cc4e573302e)

s3: VFS: vfs_fruit. Fix the NetAtalk deny mode compatibility code.

This exhibited itself as a problem with OFD locks reported
as:

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13770

However, due to underlying bugs in the vfs_fruit
code the file locks were not being properly applied.

There are two problems in fruit_check_access().

Problem #1:

Inside fruit_check_access() we have:

flags = fcntl(fsp->fh->fd, F_GETFL);
..
if (flags & (O_RDONLY|O_RDWR)) {

We shouldn't be calling fcntl(fsp->fh->fd, ..) directly.
fsp->fh->fd may be a made up number from an underlying
VFS module that has no meaning to a system call.

Secondly, in all POSIX systems - O_RDONLY is defined as
*zero*. O_RDWR = 2.

Which means flags & (O_RDONLY|O_RDWR) becomes (flags & 2),
not what we actually thought.

Problem #2:

deny_mode is *not* a bitmask, it's a set of discrete values.

Inside fruit_check_access() we have:

if (deny_mode & DENY_READ) and also (deny_mode & DENY_WRITE)

However, deny modes are defined as:

/* deny modes */
define DENY_DOS 0
define DENY_ALL 1
define DENY_WRITE 2
define DENY_READ 3
define DENY_NONE 4
define DENY_FCB 7

so if deny_mode = DENY_WRITE, or if deny_mode = DENY_READ
then it's going to trigger both the if (deny_mode & DENY_READ)
*and* the (deny_mode & DENY_WRITE) conditions.

These problems allowed the original test test_netatalk_lock code to
pass (which was added for BUG: https://bugzilla.samba.org/show_bug.cgi?id=13584
to demonstrate the lock order violation).

This patch refactors the fruit_check_access()
code to be much simpler (IMHO) to understand.

Firstly, pass in the SMB1/2 share mode, not old
DOS deny modes.

Secondly, read all the possible NetAtalk locks
into local variables:

netatalk_already_open_for_reading
netatalk_already_open_with_deny_read
netatalk_already_open_for_writing
netatalk_already_open_with_deny_write

Then do the share mode/access mode checks
with the requested values against any stored
netatalk modes/access modes.

Finally add in NetATalk compatible locks
that represent our share modes/access modes
into the file, with an early return if we don't
have FILE_READ_DATA (in which case we can't
write locks anyway).

The patch is easier to understand by looking
at the completed patched fruit_check_access()
function, rather than trying to look at the
diff.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
(cherry picked from commit 3204dc66f6801a7c8c87c48f601e0ebdee9e3d40)

smbd: uid: Don't crash if 'force group' is added to an existing share connection.

smbd could crash if "force group" is added to a
share definition whilst an existing connection
to that share exists. In that case, don't change
the existing credentials for force group, only
do so for new connections.

Remove knownfail from regression test.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13690

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Fri Jan 25 16:31:27 CET 2019 on sn-devel-144

(cherry picked from commit e37f9956c1f2416408bad048a4618f6366086b6a)

s3: tests: Add regression test for smbd crash on share force group change with existing connection.

Mark as known fail for now.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13690

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 7b21b4c1f538650f23ec77fb3c02fe1e224d89aa)

printing: check lp_load_printers() prior to pcap cache update

Avoid explicit and housekeeping timer triggered printcap cache updates
if lp_load_printers() is disabled.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13766

Signed-off-by: David Disseldorp <ddiss@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Fri Feb  1 19:25:03 CET 2019 on sn-devel-144

(cherry picked from commit 6a77237c50dd258521f356af0b5dc9942dd5592e)

printing: drop pcap_cache_loaded() guard around load_printers()

Add the pcap_cache_loaded() check to load_printers() and return early
if it returns false. This simplifies callers in preparation for checking
lp_load_printers() in the printcap cache update code-path.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13766

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
(cherry picked from commit 0ae7c3144a30910adb1e54cf46d54d42a1036839)

s3-smbd: use fruit:model string for mDNS registration

With this change we now allow to modify the icon to represent Samba in
Finder. Possible values are at least:

fruit:model = iMac
fruit:model = MacBook
fruit:model = MacPro
fruit:model = Xserve
fruit:model = RackMac

Prior to this change we only displayed the correct icon when a mac
client negotiated the apple create context over SMB.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13746

Based on proposed patch from Rouven WEILER <Rouven_Weiler@gmx.net>

Guenther

Signed-off-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Tue Jan 15 21:27:20 CET 2019 on sn-devel-144

(cherry picked from commit 538ce72f1b2fa78450e3b711e58bd0e238faf466)

ldb: Release ldb 1.5.3

* Avoid inefficient one-level searches (bug 13762)
* The test api.py should not rely on order of entries in dict (bug 13772)

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13772

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
(cherry picked from commit 5e716c0256a6bec92e7855ccfc077a328320f2ea)

Autobuild-User(v4-10-test): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(v4-10-test): Wed Feb 13 16:24:32 CET 2019 on sn-devel-144

ldb: Add even more comments on what strict does to the list intersections

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
(cherry picked from commit e7f524fd2128aacb82e980652af8eb6fd275e1a8)

ldb: Rename variable

The old name confused me because it's not really related to the
one-level index at all. It's the result from evaluating the indexed
search specified in the ac->tree.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit 57a565b2fd680fc1a34f4ab91c6f6314f68ef67f)

ldb: Elaborate on ldb_kv_search_indexed() comments

Disclaimer: this is based on my limited understanding of what the code
is doing.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit 132600685b8c5d4964f20634cd7a64b14f41cfa7)

ldb: Remove comment that no longer makes sense

This comment was written before the GUID_index_attribute block of code
existed. So we now *do* load the index values and *do* check for a
strict intersect, so the comment is redundant.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit 72928444823c5b18ac9ef98e7432c999d70aa571)

ldb: Avoid inefficient one-level searches

Commit 88ae60ed186c9 introduced a problem that made one-level
searches inefficient if there were a lot of child objects in the same
level, and the requested object didn't exist. Basically, it ignored the
case where ldb_kv_index_dn() returned LDB_ERR_NO_SUCH_OBJECT, i.e. the
indexed lookup was successful, but didn't find a match. At which point,
there was no more processing we needed to do.

The behaviour after 88ae60ed186c9 was to fall-through and run the
ldb_kv_index_filter() function over *all* the children. This still
returned the correct result, but could be costly if there were a lot of
children.

The case 88ae60ed186c9 was trying to fix was where we could not do
an indexed search (e.g. trying to match on a 'attribute=*' filter). In
which case we want to ignore the LDB_ERR_OPERATIONS_ERROR and just run
ldb_kv_index_filter() over all the children. This is still more
efficient than the fallback of doing a full database scan.

This patch adds in a short-circuit for the NO_SUCH_OBJECT case, so we
can skip the unnecessary ldb_kv_index_filter() work.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit 9a893f9613bd6440abd8e487d22a39ab5b82a7b9)

VERSION: Bump version up to 4.10.0rc2...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

VERSION: Diable GIT_SNAPSHOT for the 4.10.0rc2 release.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

WHATSNEW: Add release notes for Samba 4.10.0rc2.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

s3-vfs: Use ENOATTR in errno comparison for getxattr

* ENODATA is not defined in FreeBSD
* ENOATTR is defined to be a synonym for ENODATA in Linux
* In its absence Samba already defines ENOATTR to either
  ENODATA or ENOENT

Thus it is safe and correct to compare with ENOATTR rather
than ENODATA.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13774

Signed-off-by: Anoop C S <anoopcs@redhat.com>
Reviewed-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Jan 23 21:59:10 CET 2019 on sn-devel-144

(cherry picked from commit c99402724a65f4e1f8ed4dcd236a43e0603bef0a)

Autobuild-User(v4-10-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-10-test): Tue Feb  5 23:59:06 CET 2019 on sn-devel-144

s3-vfs: add glusterfs_fuse vfs module.

This module only implements the get_real_filename function by accessing
a distinct extended attribute that is available over a glusterfs fuse
mount.

By implementing this vfs function users of a glusterfs fuse mount
achieve a much better performance in create based workloads where samba
then can avoid trying multiple case folding options to detect the real
filename.

Patch is based on an initial patch provided by
Poornima G <pgurusid@redhat.com>

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13774

Guenther

Signed-off-by: Günther Deschner <gd@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Tue Jan 22 18:37:56 CET 2019 on sn-devel-144

(cherry picked from commit adffe0dcf002aa4721dc7897261895e3486d5271)

selftest:Samba4: use 'smbcontrol samba shutdown'

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13752

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Björn Baumbach <bbaumbach@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Jan 30 01:51:48 CET 2019 on sn-devel-144

(cherry picked from commit d03991f569b54ae0a11911b622107fbae701715d)

s4:server: add support for 'smbcontrol samba shutdown'

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13752

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Björn Baumbach <bbaumbach@samba.org>
(cherry picked from commit 832776c0fcf7cc658c128765514755c2d15b06a6)

s4:server: avoid using pid=0 for the parent 'samba' process

It confuses the 'samba-tool processes' output and log messages.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13752

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Björn Baumbach <bbaumbach@samba.org>
(cherry picked from commit 5bd7a8e5685caa09067745b108ef7e53e3108e97)

s4:messaging: add support 'smbcontrol <pid> debug/debuglevel'

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13752

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Björn Baumbach <bbaumbach@samba.org>
(cherry picked from commit 3a0c1da432c53de234b54bac90a3fb84534994eb)

manpages/samba.7.xml: smbcontrol can also work with 'samba'

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13752

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Björn Baumbach <bbaumbach@samba.org>
(cherry picked from commit 12b9adec3ff48f4356f9ff865891dc3c652ff86b)

libcli: dns: Change internal DNS_REQUEST_TIMEOUT from 2 to 10 seconds.

Should make us more robust when dealing with slow DNS servers.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13750

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Jan 22 23:37:16 CET 2019 on sn-devel-144

(cherry picked from commit 36c42e6d629a0d4b0e5c60bdd68e08a8e60c477a)

python: dns_hub: Fix indentation of 'raise' on error.

Remove second socket.sendto().

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13750
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
(cherry picked from commit dd231a16a338cfe63d12cadf0fc3e2ebe7cac828)

join: Throw CommandError instead of Exception for simple errors

Throwing an exception here still dumps out the Python stack trace, which
can be a little disconcerting for users.

In this case, the stack trace isn't going to really help at all (the
problem is pretty obvious), and it obscures the useful message
explaining what went wrong.

Throw a CommandError instead, which samba-tool will catch and display
more nicely.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13747

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Rowland Penny <rpenny@samba.org>
Reviewed-by: Jeremy Allison <rpenny@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Jan 16 22:11:04 CET 2019 on sn-devel-144

(cherry picked from commit 9e4b08f4c384b8cae5ad853a7be7cf03e2749be5)

join: Fix TypeError when handling exception

When we can't resolve a domain name, we were inadvertently throwing a
TypeError whilst trying to output a helpful message. E.g.

ERROR(<class 'TypeError'>): uncaught exception - 'NTSTATUSError' object
does not support indexing

Instead of indexing the object, we want to index the Exception.args so
that we just display the string portion of the exception error.

The same problem is also present for the domain trust commands.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13747

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Rowland Penny <rpenny@samba.org>
Reviewed-by: Jeremy Allison <rpenny@samba.org>
(cherry picked from commit 3bb7808984c163a7bba66fb983411d1281589722)