VERSION: Disable GIT_SNAPSHOT for the 4.7.5 release.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

WHATSNEW: Add release notes for Samba 4.7.5.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

dbcheck: skip find_missing_forward_links_from_backlinks() if the db has the sortedLinks feature

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Mon Feb  5 18:32:51 CET 2018 on sn-devel-144

(cherry picked from commit 0c3348feb09f4f0ba85455b8c3ff5c5fa60d139b)

Autobuild-User(v4-7-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-7-test): Tue Feb  6 16:41:09 CET 2018 on sn-devel-144

dbcheck: add support for restoring missing forward links

This recovers broken databases with duplicate and missing
forward links.

See commit a25c99c9f1fd1814c56c21848c748cd0e038eed7 for
the fix that prevents to problem from happening.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 5bf823d68bd33ee3160175a18a3838eff4e3cbb2)

dbcheck: add find_missing_forward_links_from_backlinks()

find_missing_forward_links_from_backlinks() finds and returns missing forward-links by
searching all for all objects that link to the object in the backlink attribute.

This will be used in the next commit to restore forward links in a corrupted
forward link attribute by passing the missing backling objects to
err_recover_forward_links().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit d59f201388e8a16688adda145734dab8e27b785f)

dbcheck: make sure we ask for replPropertyMetaData if we need to process any forward link attributes

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 182fb3c4c9db8715d0dbcbc3d1aa0655b5cb29f1)

dbcheck: make sure we always ask for the objectGUID attribute explicitly

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 20598033866ca3d0fdad1edf3cb39e4614eae112)

dbcheck: add a helper function that checks is a value has duplicate links

Will be used in a subsequent commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit e258b4fb281d8577c425e05b35ce05cf128617ea)

dbcheck: add a dict where we remember attributes with duplicate links

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit e4cc062fa98f65369f3bde24a987c2651632cb06)

dbcheck: split out check_duplicate_links from check_dn

Refactoring, no change in behaviour.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 44a8782d71676517f0991f279f2472391ecede3b)

dbcheck: store fixed forward link attributes with the correct sorting

The corruption we're trying to fix messed up the sorting,
so there's no point in keeping the current order.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 7df17c0a8dffceb053ca806c9426d493b4837b1a)

dbcheck: remove ldb.FLAG_MOD_REPLACE when replacing search results for forward links

Search results don't have an ldb.FLAG_MOD_* flags set.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit b0bc3f60084e5998dd34aada2ac7377d390affc6)

dbcheck: rename err_duplicate_links() to err_recover_forward_links() and adjust the output message

It's really a fatal error to have duplicate values as it's very likely that
some forward links got lost.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit ec433f8531a822dd40b343fbf3244157a5ecd544)

dbcheck: add link direction to error message for duplicate links

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit dc43d31cd20fd12d2758b73ec0318215b8fbedfb)

dbcheck: rename err_duplicate_links arguments

In preperation of adding more arguments.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit a651cc79d64b9bcc1d5fee9b2ef8800a1579dea1)

dbcheck: only pass obj_dn to err_orphaned_backlink()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 9f47fe6c4a8bde4abfee3c774d9667e6a3439a45)

dbcheck: add forward_syntax argument to err_orphaned_backlink

Will be used in a subsequent commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 6f77503871fcb815e474cb76d14e22f7a8f083c9)

dbcheck: rename and reorder err_orphaned_backlink arguments

In preperation of adding more arguments.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 4a71394c6a30e8a1b5c6553f7410148dbf2e4a80)

selftest/dbcheck: add a test for corrupt forward links restoration

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 52bd0b09804621e6de9ee0a377a442a42e07ee05)

Revert "dbcheck: disable fixing duplicate linked attributes until we can recover lost forward links"

This reverts commit 43e3f79d54c5aeaea820865d298d4249cf47af99.

The real fix will follow in the next commits.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 8c01acd56274a5cb5926622cacab997cb62dd5a9)

python/common: add __cmp__ function to dsdb_Dn similar to parsed_dn_compare()

Linked attribute values are sorted by objectGUID of the link target.
For C code we have parsed_dn_compare() to implement the logic,
the same is now available on python dsdb_Dn objects.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 55d466549a3113f7625acdd6eb42f71cf63719b5)

python:tests: add test_dsdb_Dn_sorted() to "samba.tests.common"

Failing until dsdb_Dn implements the correct __cmp__() function.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit c56eb49119117a1a06afb0a76630ae5c7a1ca30c)

python:tests: remove test_dsdb_Dn() to test_dsdb_Dn_binary()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 1341780dcf9ec0c5d852fbbb77c5e00db2ad6564)

python:tests: use TestCaseInTempDir for "samba.tests.common"

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 681e0a1745b45c6ac22d394b9e78cb67007d7dc4)

vfs_fruit: set delete-on-close for empty finderinfo

We previously removed the stream from the underlying filesystem stream
backing store when the client zeroes out FinderInfo in the AFP_AfpInfo
stream, but this causes certain operations to fail (eg stat) when trying
to access the stream over any file-handle open on that stream.

So instead of deleting, set delete-on-close on the stream. The previous
commit already implemented not to list list streams with delete-on-close
set which is necessary to implemenent correct macOS semantics for this
particular stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13181

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Jan  9 17:09:12 CET 2018 on sn-devel-144

(cherry picked from commit e61e9e98e9ff461055daae2fe78f0202f7ed8663)

Autobuild-User(v4-7-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-7-test): Fri Jan 26 01:29:54 CET 2018 on sn-devel-144

vfs_fruit: filter out AFP_AfpInfo streams with pending delete-on-close

This is in preperation of fixing the implementation of removing the
AFP_AfpInfo stream by zeroing the FinderInfo out.

We currently remove the stream blob from the underyling filesystem
backing store, but that results in certain operations to fail on any
still open file-handle.

The fix comes in the next commit which will convert to backing store
delete operation to a set delete-on-close on the stream.

This commit adds filtering on streams that have the delete-on-close
set. It is only needed for the fruit:metadata=stream case, as with
fruit:metadata=netatalk the filtering is already done in
fruit_streaminfo_meta_netatalk().

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13181

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 70d8f7c5d25f35b58620c2db8f57c7c0758267b3)

vfs_fruit: factor out delete_invalid_meta_stream() from fruit_streaminfo_meta_stream()

No change in behaviour, just some refactoring before adding more code to
fruit_streaminfo_meta_stream() in the next commit.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13181

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit c41e1ea9247611473d30184efd953c61955ead15)

s4/torture/fruit: enhance zero AFP_AfpInfo stream test

This test more operations in the zeroed out FinderInfo test, ensuring
after zeroing out FinderInfo, operations on the filehandle still work
and that enumerating streams doesn't return the stream anymore.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13181

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit df31e94eb6241f5e5594f6fd0ec1ad7896e02e27)

s4/torture/fruit: ensure AFP_AfpInfo blobs are 0-initialized

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13181

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit a22833c2971dc7234b32741305f40ed62e232e0b)

dbcheck: disable fixing duplicate linked attributes until we can recover lost forward links

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 850a8027f32185e523614231cca76505134bb5e4)

repl_meta_data: fix linked attribute corruption on databases with unsorted links on expunge

This is really critical bug, it removes valid linked attributes.

When a DC was provisioned/joined with a Samba version older than 4.7
is upgraded to 4.7 (or later), it can happen that the garbage collection
(dsdb_garbage_collect_tombstones()), triggered periodically by the 'kcc' task
of 'samba' or my 'samba-tool domain tombstones expunge' corrupt the linked attributes.

This is similar to Bug #13095 - Broken linked attribute handling,
but it's not triggered by an originating change.

The bug happens in replmd_modify_la_delete()
were get_parsed_dns_trusted() generates a sorted array of
struct parsed_dn based on the values in old_el->values.

If the database doesn't support the sortedLinks compatibleFeatures
in the @SAMBA_DSDB record, it's very likely that
the array of old_dns is sorted differently than the values
in old_el->values.

The problem is that struct parsed_dn has just a pointer
'struct ldb_val *v' that points to the corresponding
value in old_el->values.

Now if vanish_links is true the damage happens here:

        if (vanish_links) {
                unsigned j = 0;
                for (i = 0; i < old_el->num_values; i++) {
                        if (old_dns[i].v != NULL) {
                                old_el->values[j] = *old_dns[i].v;
                                j++;
                        }
                }
                old_el->num_values = j;
        }

old_el->values[0] = *old_dns[0].v;
can change the value old_dns[1].v is pointing at!
That means that some values can get lost while others
are stored twice, because the LDB_FLAG_INTERNAL_DISABLE_SINGLE_VALUE_CHECK
allows it to be stored.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit a25c99c9f1fd1814c56c21848c748cd0e038eed7)

testprogs:blackbox: add regression test for unsorted links in tombstones-expunge.sh

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13228

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit c34c2dd55545b99fba46cf374a1653bad96cea9e)

samba: Only use async signal-safe functions in signal handler

Otherwise shutdown can hang

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13240

Signed-off-by: Björn Baumbach <bb@sernet.de>
(similar to commit 361ea743576cf125d7957a97ed78a0446dab1a19)

waf: Fix NFS quota support with libtirpc

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13238

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Jan 22 17:26:52 CET 2018 on sn-devel-144

(cherry picked from commit 39a6ea766dfe55d84ab2284b8d5ed01d66da11dd)

wafsamba: Allow passing 'lib' to CHECK_STRUCTURE_MEMBER

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13238

We need to be able to point it to the right header location, so we need
to be able to pass the 'lib' that it gets set.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
(cherry picked from commit 87f105d76ce074bff08fd507d72568be88d48d00)

build: deal with recent glibc sunrpc header removal

We need to rely on libtirpc or libntirpc to be around in that case.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13238
BUG: https://bugzilla.samba.org/show_bug.cgi?id=10976

Guenther

Pair-Programmed-With: Andreas Schneider <asn@samba.org>

Signed-off-by: Guenther Deschner <gd@samba.org>
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
(cherry picked from commit ee0be7eb723be1420fd601ea1abe0af748562953)

include: Create system/nis.h in libreplace

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13238

Pair-Programmed-With: Guenther Deschner <gd@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
(cherry picked from commit c29d087e1ea4c92717ef86e372fe80f410580fdc)

s3:waf: Move HAVE_NETGROUP to wscript

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13238

Pair-Programmed-With: Guenther Deschner <gd@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
(cherry picked from commit ca5eaf0cdcf8257ac52786aa7439c8f081a2fe0d)

vfs_default: use VFS statvfs macro in fs_capabilities

Currently the vfs_default fs_capabilities handler calls statvfs
directly, rather than calling the vfs macro. This behaviour may cause
issues for VFS modules that delegate fs_capabilities handling to
vfs_default but offer their own statvfs hook.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13208

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 4b25c9f4a4d336a16894452862ea059701b025de)

Autobuild-User(v4-7-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-7-test): Mon Jan 22 13:48:30 CET 2018 on sn-devel-144

vfs_ceph: add fs_capabilities hook to avoid local statvfs

Adding the fs_capabilities() hook to the CephFS VFS module avoids
fallback to the vfs_default code-path, which calls statvfs() against the
share path on the *local* filesystem.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13208

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 2724e0cac29cd1632ea28075a740fcc888affb36)

selftest: allow more time for tests

Maybe make test *should* run in under 4 hours, but it currently
doesn't.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Fri Dec 29 02:48:59 CET 2017 on sn-devel-144

(cherry picked from commit a0f810e7e39cde003c9c5e0cb701846472e80a74)

Autobuild-User(v4-7-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-7-test): Thu Jan 11 14:00:50 CET 2018 on sn-devel-144

s3: smbd: Use identical logic to test for kernel oplocks on a share.

Due to inconsistent use of lp_kernel_oplocks() we could miss kernel
oplocks being on/off in some of our oplock handling code, and thus
use the wrong logic.

Ensure all logic around koplocks and lp_kernel_oplocks() is consistent.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13193

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Jan  4 16:03:38 CET 2018 on sn-devel-144

(cherry picked from commit 114f5da2fab6f587de77e792274b396fb3d7ce71)

Autobuild-User(v4-7-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-7-test): Tue Jan  9 17:47:10 CET 2018 on sn-devel-144

smbd: Fix coredump on failing chdir during logoff

server_exit does an internal tree disconnect which requires a chdir to
the share directory. In case the file system encountered a problem and
the chdir call returns an error, this triggers a SERVER_EXIT_ABNORMAL
which in turn results in a panic and a coredump. As the log already
indicates the problem (chdir returned an error), avoid the
SERVER_EXIT_ABNORMAL in this case and not trigger a coredump.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13189

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Dec 16 01:56:06 CET 2017 on sn-devel-144

(cherry picked from commit 7fa91fc4791d076c609eaf119753e38dd3c50a1c)

Autobuild-User(v4-7-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-7-test): Tue Jan  2 14:19:44 CET 2018 on sn-devel-144

selftest: Add test for failing chdir call in smbd

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13189

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 0d3000be2af8f8c4a37892d95ae694ad834d7b3a)

selftest: Make location of log file available in tests

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13189

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit b0e1fc74fdacecb86f46b47e527b3fdf1906d27b)

selftest: Add share for error injection testing

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13189

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 8b6402f3e5ff98c2701e626e47246b2400f76e5f)

vfs_error_inject: Add new module

This module allow injecting errors in vfs calls. It only implements one
case (return ESTALE from chdir), but the idea is to extend this to more
vfs functions and more errors when needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13189

Signed-off-by: Christof Schmitt <cs@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 24623d53256c2424563709dedc19af1a106ccc73)

ctdb-recovery-helper: Deregister message handler in error paths

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13188

If PULL_DB control times out but the remote node is still sending the
data, then the tevent_req for pull_database_send will be freed without
removing the message handler.  So when the data is received, srvid
handler will be called and it will try to access tevent_req which will
result in use-after-free and abort.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

sysacls: change datatypes to 32 bits

The SMB_ACL_PERMSET_T and SMB_ACL_PERM_T were defined as
mode_t, which is 16-bits on some (non-Linux) systems. However,
pidl *always* encodes mode_t as uint32_t. That created a bug on
big-endian systems as sys_acl_get_permset() returns a SMB_ACL_PERMSET_T
pointer to an internal a_perm structure member defined in IDL as a mode_t,
which pidl turns into a uin32_t in the emitted header file.

Changing to 32 bits fixes that.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13176

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 75e7da9741c529f96fa409655ac5b326a6c071c5)

pysmbd: fix use of sysacl API

Fix pysmbd to use the sysacl (POSIX ACL support) as intended, and
not assume too much about the inner structure and implementation
of the permissions in the sysacl API.

This will allow the inner structure to change in a following commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13176

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit d6f5ee6707fa5404e2bef6fc81ae06b393ebd8ff)

HEIMDAL:kdc: fix dh->q allocation check in get_dh_param()

Thanks to Doug Nazar <nazard@nazar.ca> for spotting this!

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12986

Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from heimdal commit a79b59ba27070a015479e8d981b7e685dbe34310)
Reviewed-by: Andreas Schneider <asn@samba.org>
(cherry picked from commit 183e5d1e3dc306491c06f94c8c98e4882c64bc27)

Autobuild-User(v4-7-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-7-test): Sun Dec 24 01:36:48 CET 2017 on sn-devel-144

HEIMDAL: don't bother seeing q if not sent

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12986

Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from heimdal commit 19f9fdbcea11013cf13ac72c416f161ee55dee2b)

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Aug 28 15:10:54 CEST 2017 on sn-devel-144

(cherry picked from commit dd3e06f14ec1788a59e4c6ba4ea165fd77b9135e)

HEIMDAL: allow optional q in DH DomainParameters

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12986

Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from heimdal commit e8317b955f5a390c4f296871ba6987ad05478c95)

(cherry picked from commit 9f245aafdca8397df7dc050e25cfd858aeb1cc7f)

VERSION: Bump version up to 4.7.5...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

VERSION: Disable GIT_SNAPSHOT for the 4.7.4 release.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

WHATSNEW: Add release notes for Samba 4.7.4.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

s4:kdc: only map SDB_ERR_NOT_FOUND_HERE to HDB_ERR_NOT_FOUND_HERE

HDB_ERR_NOT_FOUND_HERE indicated a very specific error on an RODC.

We should not map any error to HDB_ERR_NOT_FOUND_HERE,
we should just pass errors along unmapped.

Otherwise we'll hit the logic bug in:

    if (ret == KDC_PROXY_REQUEST) {
        uint16_t port;

        if (!sock->kdc_socket->kdc->am_rodc) {
            DEBUG(0,("kdc_udp_call_loop: proxying requested when not RODC"));
                    talloc_free(call);
            goto done;
        }

And just don't send an error message to the client.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13132

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Dec  6 23:16:54 CET 2017 on sn-devel-144

(cherry picked from commit aaa946bb9eb8088389b8ffdec460023f1961616c)

Autobuild-User(v4-7-test): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(v4-7-test): Fri Dec 22 14:23:12 CET 2017 on sn-devel-144

s3:smb2_server: allow logoff, close, unlock, cancel and echo on expired sessions

Windows client at least doesn't have code to replay
a SMB2 Close after getting NETWORK_SESSION_EXPIRED,
which locks out a the client and generates an endless
loop around NT_STATUS_SHARING_VIOLATION.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13197

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Dec 21 23:28:42 CET 2017 on sn-devel-144

(cherry picked from commit c4919d4d5f78aeb54a438b95d4eab2f082a8174e)

s3:smbd: return the correct error for cancelled SMB2 notifies on expired sessions

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13197

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit cfaba684785529d656138df454165aa08a775a01)

s4:torture: add smb2.session.expire2 test

This demonstrates the interaction of NT_STATUS_NETWORK_SESSION_EXPIRED
and various SMB2 opcodes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13197

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit f60af3b61c4a374d7d1c575049a932d1824489b6)

g_lock: fix cleanup of stale entries in g_lock_trylock()

g_lock_trylock() always incremented the counter 'i', even after cleaning a stale
entry at position 'i', which means it skipped checking for a conflict against
the new entry at position 'i'.

As result a process could get a write lock, while there're still
some read lock holders. Once we get into that problem, also more than
one write lock are possible.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13195

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Wed Dec 20 20:31:48 CET 2017 on sn-devel-144
(similar to commit 576fb4fb5dc506bf55e5cf87973999dca444149b)

dns_server: Use the indexed "name" attribute in wildcard lookup

(the RDN, being 'dc' in this use case, does not have an index in
the AD schema).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13191

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
(cherry picked from commit 071ad56aef33c2bfb3840e1a114e17272e926890)

dns_server: Do not look for a wildcard for @

This query is made for every record returned via BIND9 DLZ.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13191

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
(cherry picked from commit 948791aca70ca973755adcef27dc02da4c46f267)

dns_server: Do the exact match query first, then do the wildcard lookup

The wildcard lookup is SCOPE_ONELEVEL combined with an index on the name
attribute.  This is not as efficient as a base DN lookup, so we try for
that first.

A not-found and wildcard response will still fall back to the ONELEVEL
index.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13191

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
(cherry picked from commit 3efc879d98ba136d4d70e0e2d77fac9614186ab3)

ldb: Release 1.2.3

 - Intersect the index from SCOPE_ONELEVEL with the index for the search expression

    This helps ensure we do not have to scan all objects at this level
    which could be very many (one per DNS zone entry).

    However, due to the O(n*m) behaviour in the LDB index code
    we only do this for small numbers of matches on the
    filter tree.

    This behaviour will only be for ldb 1.2 and will not be kept
    long-term in LDB, versions 1.3.1 and above will instead only
    intersect when the more efficient GUID index is in use.

    Finally, disallowDNFilter now applies to SCOPE_ONELEVEL banning
    dn= as a filter string when so configured.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13191

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

ldb: Intersect the index from SCOPE_ONELEVEL with the index for the search expression

This helps ensure we do not have to scan all objects at this level
which could be very many (one per DNS zone entry).

However, due to the O(n*m) behaviour in list_intersect() for ldb
1.2 and earlier, we only do this for small numbers of matches on the
filter tree.

This behaviour will only be for ldb 1.2 and will not be kept
long-term in LDB, versions 1.3.1 and above will instead only
intersect when the GUID index is in use.

NOTE WELL: the behaviour of disallowDNFilter is enforced
in the index code, so this fixes SCOPE_ONELEVEL to also
honour disallowDNFilter if there @IDXONE is enabled.  Again,
this will change again in 1.3.1 and above.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13191

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
(adapted from ef240aaca0ef693a96726ac2366c454294b87b96 in master)

ldb_tdb: Provide struct ltdb_private to index routines

This will make it easier to switch the GUID index mode on and off

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13191

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
(cherry picked from commit 3d952157d72b3a4635f3942449c1727c438c97c6)

selftest: Do not use dn= filter string

This accidentially worked with SCOPE_ONELEVEL against Samba but dn= filters are
not valid in AD.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
(cherry picked from commit 44eee9ce9e9818df8387b2b3782504408112f12c)

s4:samba: Fix default to be running samba as a deamon

Commit 8736013dc42c5755b75bbb2e843a290bcd545909 got the (confusing) sense of opt_fork
wrong.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13129

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Dec 19 11:24:29 CET 2017 on sn-devel-144

(cherry picked from commit 0806ff7dfd3f982226e4cd9b923a0e570b765f0c)

Autobuild-User(v4-7-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-7-test): Thu Dec 21 00:55:29 CET 2017 on sn-devel-144

messaging: Always register the unique id

The winbind child does not call serverid_register, so the unique id is not
registered. ctdbd_process_exists now calls CTDB_CONTROL_CHECK_PID_SRVID, which
then fails.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13180
Signed-off-by: Volker Lendecke <vl@samba.org>
Autobuild-User(v4-7-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-7-test): Fri Dec 15 15:35:25 CET 2017 on sn-devel-144

winbindd: idmap_rid: error code for failing id-to-sid mapping request

NT_STATUS_NO_SUCH_DOMAIN triggers complete request failure in the parent
winbindd. By returning NT_STATUS_NONE_MAPPED winbindd lets the individual
mapping fail but keeps processing any remaining mapping requests.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13052

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Oct 10 19:57:37 CEST 2017 on sn-devel-144

(cherry picked from commit 490c35df35bad6c2f1c4acd2f056d6fdc480ec1f)

Autobuild-User(v4-7-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-7-test): Thu Dec 14 16:32:49 CET 2017 on sn-devel-144

winbindd: idmap_rid: don't rely on the static domain list

The domain list in the idmap child is inherited from the parent winbindd
process and may not contain all domains in case enumerating trusted
domains didn't finish before the first winbind request that triggers the
idmap child fork comes along.

The previous commits added the domain SID as an additional argument to
the wbint_UnixIDs2Sids request, storing the domain SID in struct
idmap_domain.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13052

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 108675c4cf4c3d5bd29468255743423a56bd1471)

winbindd: pass domain SID to wbint_UnixIDs2Sids

This makes the domain SID available to the idmap child for
wbint_UnixIDs2Sids mapping request. It's not used yet anywhere, this
comes in the next commit.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13052

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 71f99cb132f4c26f9febac6cb7dcd79f4940216a)

winbindd: add domain SID to idmap mapping domains

Fetch the domain SID for every domain in the idmap-domain map. This is
in preperation of passing the domain SID as an additional argument to
xid2sid requests to the idmap child.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13052

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 59438bfd3d3551195582cf88bd1109c3cbc7e12a)

pthreadpool: Add a test for the race condition fixed in the last commit

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13179
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 53f7bbca0451e4f57cdbe8ab4f67f601fe8d40c1)

pthreadpool: Fix starvation after fork

After the race is before the race:

1) Create an idle thread
2) Add a job: This won't create a thread anymore
3) Immediately fork

The idle thread will be woken twice before it's actually woken up: Both
pthreadpool_add_job and pthreadpool_prepare_pool call cond_signal, for
different reasons. We must look at pool->prefork_cond first because otherwise
we will end up in a blocking job deep within a fork call, the helper thread
must take its fingers off the condvar as quickly as possible.  This means that
after the fork there's no idle thread around anymore that would pick up the job
submitted in 2). So we must keep the idle threads around across the fork.

The quick solution to re-create one helper thread in pthreadpool_parent has a
fatal flaw: What do we do if that pthread_create call fails? We're deep in an
application calling fork(), and doing fancy signalling from there is really
something we must avoid.

This has one potential performance issue: If we have hundreds of idle threads
(do we ever have that) during the fork, the call to pthread_mutex_lock on the
fork_mutex from pthreadpool_server (the helper thread) will probably cause a
thundering herd when the _parent call unlocks the fork_mutex. The solution for
this to just keep one idle thread around. But this adds code that is not
strictly required functionally for now.

More detailed explanation from Jeremy:

First, understanding the problem the test reproduces:

add a job (num_jobs = 1) -> creates thread to run it.
job finishes, thread sticks around (num_idle = 1).
num_jobs is now zero (initial job finished).

a) Idle thread is now waiting on pool->condvar inside
pthreadpool_server() in pthread_cond_timedwait().

Now, add another job ->

pthreadpool_add_job()
-> pthreadpool_put_job()
This adds the job to the queue.
Oh, there is an idle thread so don't
create one, do:

pthread_cond_signal(&pool->condvar);

and return.

Now call fork *before* idle thread in (a) wakes from
the signaling of pool->condvar.

In the parent (child is irrelevent):

Go into: pthreadpool_prepare() ->
pthreadpool_prepare_pool()

Set the variable to tell idle threads to exit:

pool->prefork_cond = &prefork_cond;

then wake them up with:

pthread_cond_signal(&pool->condvar);

This does nothing as the idle thread
is already awoken.

b) Idle thread wakes up and does:

Reduce idle thread count (num_idle = 0)

pool->num_idle -= 1;

Check if we're in the middle of a fork.

if (pool->prefork_cond != NULL) {

Yes we are, tell pthreadpool_prepare()
we are exiting.

pthread_cond_signal(pool->prefork_cond);

And exit.

pthreadpool_server_exit(pool);
return NULL;
}

So we come back from the fork in the parent with num_jobs = 1,
a job on the queue but no idle threads - and the code that
creates a new thread on job submission was skipped because
an idle thread existed at point (a).

OK, assuming that the previous explaination is correct, the
fix is to create a new pthreadpool context mutex:

pool->fork_mutex

and in pthreadpool_server(), when an idle thread wakes up and
notices we're in the prepare fork state, it puts itself to
sleep by waiting on the new pool->fork_mutex.

And in pthreadpool_prepare_pool(), instead of waiting for
the idle threads to exit, hold the pool->fork_mutex and
signal each idle thread in turn, and wait for the pool->num_idle
to go to zero - which means they're all blocked waiting on
pool->fork_mutex.

When the parent continues, pthreadpool_parent()
unlocks the pool->fork_mutex and all the previously
'idle' threads wake up (and you mention the thundering
herd problem, which is as you say vanishingly small :-)
and pick up any remaining job.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13179
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit f6858505aec9f1004aeaffa83f21e58868749d65)

third_party: Fix a typo in the option name

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13174

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
(cherry picked from commit e7e68958025937f97554cd956ca482dfe507f803)

third_party: Link th aesni-intel library with -z noexecstack

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13174

Signed-off-by: Björn Baumbach <bb@sernet.de>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
(cherry picked from commit 6015cfad6ebf46b9f311a069dd960ff5af5bdcd8)

s3: libsmb: Fix reversing of oldname/newname paths when creating a reparse point symlink on Windows from smbclient.

This happened as smbd doesn't support reparse points so we couldn't test.
This was the reverse of the (tested) symlink parameters in the unix extensions
symlink command.

Rename parameters to link_target instead of oldname so this is clearer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13172

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit abbc9b9ab793d22bca6a37828f4375ef38c56dd3)

Autobuild-User(v4-7-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-7-test): Wed Dec 13 14:19:59 CET 2017 on sn-devel-144

s3: client: Rename <oldname> to <link_target> in cmd_symlink() and cli_posix_symlink().

Stops us from mixing up the old and new names. Only behavior change
is correcting the names printed in the error messages.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13172

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 8448dcaa8da78bcb84fca6a000c75e256bce1e77)

pthreadpool: Fix deadlock

Christof's idea from

https://lists.samba.org/archive/samba-technical/2017-December/124384.html

was that the thread already exited. It could also be that the thread is
not yet idle when the new pthreadpool_add_jobs comes around the corner.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
Autobuild-User(master): Christof Schmitt <cs@samba.org>
Autobuild-Date(master): Wed Dec 13 04:46:12 CET 2017 on sn-devel-144

(cherry picked from commit dfc4670640341761b346065922a62a3e755e9e58)

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13170

pthreadpool: Add test for pthread_create failure

This is implemented using cmocka and the __wrap override for
pthread_create.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13170

Signed-off-by: Christof Schmitt <cs@samba.org
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Christof Schmitt <cs@samba.org>
Autobuild-Date(master): Fri Dec  8 13:54:20 CET 2017 on sn-devel-144

(cherry picked from commit 8cdb3995caf7a21d0c27a56a0bf0c1efd5b491e4)

wscript: Add check for --wrap linker flag

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13170

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
(cherry picked from commit 8e17be1c3df09c238560c8a7e62c17e9f9ff9bc7)

pthreadpool: Undo put_job when returning error

When an error is returned to the caller of pthreadpool_add_job, the job
should not be kept in the internal job array. Otherwise the caller might
free the data structure and a later worker thread would still reference
it.

When it is not possible to create a single worker thread, the system
might be out of resources or hitting a configured limit. In this case
fall back to calling the job function synchronously instead of raising
the error to the caller and possibly back to the SMB client.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13170

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 065fb5d94d25d19fc85832bb85aa9e379e8551cc)

pthreadpool: Move creating of thread to new function

No functional change, but this simplifies error handling.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13170

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 949ccc3ea9073a3d38bff28345f644d39177256f)

ctdb-daemon: Send STARTUP control after startup event

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13154

STARTUP control is primarily used to synchronise tcp tickles from running
nodes to a node which has just started up.  Earlier STARTUP control was
sent (using BROADCAST_ALL) after setup event.  Once the other nodes in
the cluster connected to this node, the queued up messages would be sent
and the tcp tickles would get synchronised.

Recent fix to drop messages to disconnected or not-yet-connected nodes,
the STARTUP control was never sent to the remote nodes and the tcp
tickles did not get synchronised.

To fix this problem send the STARTUP control (using BROADCAST_CONNECTED)
after startup event.  By this time all the running nodes in the cluster
are connected.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Nov 30 15:29:48 CET 2017 on sn-devel-144

(cherry picked from commit d7a5cd589b7b16d625dbc64dac21a1384519e32b)

ctdb-takeover: Send tcp tickles immediately on STARTUP control

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13154

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(cherry picked from commit 73e261b48c4abc91e00775ac7437752c9640e5bd)

ctdb-takeover: Refactor code to send tickle lists for all public IPs

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13154

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(cherry picked from commit 2b253f6b1bc4e765f3fcb614a3b67b14084a625d)

vfs_zfsacl: fix compilation error

Bug: https://bugzilla.samba.org/show_bug.cgi?id=6133

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Nov  2 03:16:11 CET 2017 on sn-devel-144

(cherry picked from commit 11da1e5c056c92fd7f51ecce0285628cac65f174)

Autobuild-User(v4-7-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-7-test): Thu Dec  7 14:19:57 CET 2017 on sn-devel-144

winbindd: let normalize_name_map() call find_domain_from_name_noinit()

Let normalize_name_map fetch the domain itself with
find_domain_from_name_noinit().

This removes two calls to find_domain_from_name_noinit() in the default
configuration of "winbind normalize names = no". The domain is only need
in normalize_name_map if "winbind normalize names" is enabled.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13173

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 1ce165a73350e802500c32435dbefe3639340435)

Autobuild-User(v4-7-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-7-test): Tue Dec  5 14:48:41 CET 2017 on sn-devel-144

s3: libsmb: Fix valgrind read-after-free error in cli_smb2_close_fnum_recv().

cli_smb2_close_fnum_recv() uses tevent_req_simple_recv_ntstatus(req), which
frees req, then uses the state pointer which was owned by req.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13171

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Nov 30 05:47:12 CET 2017 on sn-devel-144

(cherry picked from commit 5c8032b6b8ce4439b3ef8f43a62a419f081eb787)

testprogs: Test net ads keytab list

Test that correct keytab is picked up.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13166

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 4be05c835e9d8b8f13856d592aaf42b40ce397c2)

s3:libads: net ads keytab list fails with "Key table name malformed"

When keytab_name is NULL don't call smb_krb5_kt_open use ads_keytab_open
instead, this function will determine the correct keytab to use.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13166

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 3048ae318fc8b4d1b7663826972306372430a463)

vfs_fruit: proper VFS-stackable conversion of FinderInfo

This fixes the problem that conversion failed with
fruit:metadata=stream. Before we were calling ad_set() which stores the
metadata in the Netatalk compatible format.

Rewrite to fully go through the VFS by calling SMB_VFS_CREATE_FILE() and
SMB_VFS_PWRITE().

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Nov 29 08:38:06 CET 2017 on sn-devel-144

(cherry picked from commit 1da17204344a99a3bfa289355a996027a21814b8)

vfs_fruit: add AfpInfo prototypes

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 84976cb670847f199598995d48bd9c3f3dd0f035)

s4/torture: fruit: in test_adouble_conversion() also check stream list and AFPINFO_STREAM

This reveals that the conversion doesn't work properly with
fruit:metadata=stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 7b00b558761b6564928289efcf835e9b9cc86a89)

s4/torture: fruit: remove use of localdir from test_adouble_conversion test

The previous use of localdir and torture_setup_local_file() was
motivated by the fact that by default vfs_fruit rejects access to files
with a "._" prefix.

Since a previous commit allowed SMB access to ._ files, rewrite the
test_adouble_conversion() test to create the ._ AppleDouble file over
SMB.

This also renders torture_setup_local_file() obsolete.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit ebbffd80862d7d7b025eca219bc0c8f818585ac9)

selftest: add "fruit:veto_appledouble = no" to fruit shares

This is needed for a subsequent commit that modifies an existing test to
write a ._ file over SMB instead of using the ugly local creation hack.

SMB acces of ._ files requires "fruit:veto_appledouble = no", so let's
set it.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 3f9b45a410384904d64bdd0d68ff2a5bc25bd3e9)

s4/torture: let write_stream() deal with stream=NULL

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit ac880848a905a3840b69af2cb1d842d307401a07)

selftest: run AppleDouble sidecar-file conversion test runs against all fruit shares

This needs for work in all possible fruit configs, so test it.

This currently fails with stream_depot, as we don't propely copy over
the resourcefork data from the ._ file to the stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit e28dd6a0ce753a880b2512eb62661411b20f763b)