vfs:glusterfs_fuse: treat ENOATTR as ENOENT

The original implementation of the virtual xattr get_real_filename
in gluster was misusing the ENOENT errno as the authoritative anwer
that the file/dir that we were asking the real filename for does not
exist. But since the getxattr call is done on the parent directory,
this is a violation of the getxattr API which uses ENOENT for the
case that the file/dir that the getxattr call is done against does
not exist.

Now after a recent regression for fuse-mount re-exports due to
gluster mapping ENOENT to ESTALE in the fuse-bridge, the gluster
implementation is changed to more correctly return ENOATTR if the
requested file does not exist.

This patch changes the glusterfs_fuse vfs module to treat ENOATTR as
ENOENT to be fully functional again with latest gluster.

- Without this patch, samba against a new gluster will work correctly,
  but the get_real_filename optimization for a non-existing entry
  is lost.

- With this patch, Samba will not work correctly any more against
  very old gluster servers: Those (correctly) returned ENOATTR
  always, which Samba originally interpreted as EOPNOTSUPP, triggering
  the expensive directory scan. With this patch, ENOATTR is
  interpreted as ENOENT, the authoritative answer that the requested
  entry does not exist, which is wrong unless it really does not exist.

Signed-off-by: Michael Adam <obnox@samba.org>

vfs:glusterfs: treat ENOATTR as ENOENT

The original implementation of the virtual xattr get_real_filename
in gluster was misusing the ENOENT errno as the authoritative anwer
that the file/dir that we were asking the real filename for does not
exist. But since the getxattr call is done on the parent directory,
this is a violation of the getxattr API which uses ENOENT for the
case that the file/dir that the getxattr call is done against does
not exist.

Now after a recent regression for fuse-mount re-exports due to
gluster mapping ENOENT to ESTALE in the fuse-bridge, the gluster
implementation is changed to more correctly return ENOATTR if the
requested file does not exist.

This patch changes the glusterfs vfs module to treat ENOATTR as ENOENT
to be fully functional again with latest gluster.

- Without this patch, samba against a new gluster will work correctly,
  but the get_real_filename optimization for a non-existing entry
  is lost.

- With this patch, Samba will not work correctly any more against
  very old gluster servers: Those (correctly) returned ENOATTR
  always, which Samba originally interpreted as EOPNOTSUPP, triggering
  the expensive directory scan. With this patch, ENOATTR is
  interpreted as ENOENT, the authoritative answer that the requested
  entry does not exist, which is wrong unless it really does not exist.

Signed-off-by: Michael Adam <obnox@samba.org>

smbd: Slightly simplify delay_for_oplock()

We don't have to look at the leases.tdb record if it's our own lease.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu May 16 07:59:52 UTC 2019 on sn-devel-184

smbd: Fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

smbd: Use a direct struct assignment in reply_setatr()

Saves .text bytes with -O3

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

samba_autoconf: fix undefined behaviour sanitizer compile flags

Added the -g option, -fsaniize=null and -fsanitize=alignment, removed
the -fno-omit-frame-pointer option.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>

s3: net: Test of fuzzer problems with net rpc registry import.

Found by Michael Hanselmann using fuzzing tools

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13842

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 15 23:08:58 UTC 2019 on sn-devel-184

s3: net: Rewrite of reg_parse_fd() to harden against buffer overwrites.

Remove unused handle_iconv_errno(). Fix leaks of iconv handles.

Found by Michael Hanselmann using fuzzing tools

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13842

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3: net: Harden srprs_str() against memcmp overread.

Found by Michael Hanselmann using fuzzing tools

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13842

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3: net: Harden act_val_hex() act_val_sz() against errors.

Found by Michael Hanselmann using fuzzing tools

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13842

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3: net: Harden guess_charset() against overflow errors.

Found by Michael Hanselmann using fuzzing tools

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13842

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4 dsdb/repl_meta_data: fix use after free in dsdb_audit_add_ldb_value

Fix use after free detected by AddressSanitizer

AddressSanitizer: heap-use-after-free on address 0x61400026a4a0
                  at pc 0x7fd555c52f12 bp 0x7ffed7231180 sp 0x7ffed7231170
                  READ of size 1 at 0x61400026a4a0 thread T0
    #0 0x7fd555c52f11 in ldb_should_b64_encode
       ../../lib/ldb/common/ldb_ldif.c:197
    #1 0x7fd539dc9417 in dsdb_audit_add_ldb_value
       ../../source4/dsdb/samdb/ldb_modules/audit_util.c:491
    #2 0x7fd539dc9417 in dsdb_audit_attributes_json
       ../../source4/dsdb/samdb/ldb_modules/audit_util.c:651
    #3 0x7fd539dc6a7e in operation_json
       ../../source4/dsdb/samdb/ldb_modules/audit_log.c:305

The problem is that at the successful end of these functions
el->values is overwritten with new_values.  However get_parsed_dns()
points p->v at the supplied el and it effectively gets used
as a working area by replmd_build_la_val().  So we must duplicate it
because our caller only called ldb_msg_copy_shallow().

The reason this matters is that the audit_log module is
above repl_meta_data in the stack, and tries to log the
ldb_message it saw after the reply (to include the error code).
If that ldb_message is changed it is not only misleading,
it can point to memory that has since gone away.

In this case the memory for the full extended DN in the
member attribute ended up on 'ac', a context lost by
the time repl_meta_data has finished processing.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13941

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 15 05:35:47 UTC 2019 on sn-devel-184

s4 dsdb/repl_meta_data: allocate new extended DNs during ADD on a better context

Lower down in this function new_values is assigned over el->values and is
filled in with the values of all the parsed DNs.  Therefore it is the natural
talloc parent.

This will allow el->values to be allocated on tmp_ctx in the next commit for
a working area during the function call.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

ldap: Add a database open after fork to speed-up prefork binds

Removing this (system) database open from the later auth stack results
in a 15-30% improvement in different types of binds and on different
hardware (presumably better with slower disk speeds).

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

util/charset/torture: ensure each cp850 high bytes is 3 utf8 bytes

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

util/charset/convert: do not pretend to realloc

It seems very likely that our clever attempts to dynamically realloc
the output buffer were never triggered. Two lines of reasoning lead to
this conclusion:

1. We allocate 3 * srclen to start with, but no conversion we use will
   more than that. To be precise, from 8-bit charsets we will only deal
   with codepoints in the Unicode basic multilingual plane (up to 0xFFFF).

   These can all be expressed as 3 or fewer utf-8 bytes. In UTF16 they
   are naturally 2 bytes, while in the DOS codes they are 1 byte.

   We have checked the code tables, and can not find a plausible
   (e.g. not EBCDIC) DOS code page or unix charset that is outside
   this range.  Clients cannot chose the code page, the only code
   pages we will use come from 'unix charset' and 'dos charset'
   smb.conf parameters.

   Therefore the worst that can possibly happen is we expand 1 byte into 3
   (specifically, when converting some e.g. CP850 codepoints to UTF-8).

2. If the reallocation was ever used, the results would have been
   catastrophically wrong, as the input pointer was not reset.

Therefore we skip the complication of the goto loop and let E2BIG be
just another impossible error to report.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

util/charset/convert: when retrying, retry from the start

iconv() advances the inbuf pointer; if we decide to realloc and re-iconv,
we need to reset inbuf to the source string

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

util/charset/convert: do not overflow dest len in corner case

Now, if destlen were SIZE_MAX - 1, destlen * 2 would wrap to SIZE_MAX - 3,
which makes (destlen * 2 + 2) == SIZE_MAX - 1, the same number again.
So we need the <= comparison in this case.

As things stand, it is not actually possible for destlen to be
SIZE_MAX (because it is always an even number after the first round,
and the first round is constrained to be < SIZE_MAX / 2, but *if*
destlen was SIZE_MAX, destlen * 2 + 2 would be 0, so that case is OK.
Similarly the SIZE_MAX - 2 and smaller cases were covered by the
original formula.

We add the comment for people who are wondering WTF is going on with
all this destlen manipulation.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

util/charset/convert: do not overflow dest len

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

util/charset/convert_string: always set length

In failure cases the destination string pointer is set to NULL, but
the size is not changed. Some callers have not been checking the
return value and passing the destination pointer and uninitialised
length onto other functions. We can curse and blame those callers, but
let's also keep them safe.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

test_regfio.c: include stdint.h before cmoka.h

This fix the following build failure on mips64el:
/usr/lfs/v0/rc-buildroot-test/scripts/instance-2/output/host/mips64el-buildroot-linux-uclibc/sysroot/usr/include/stdint.h:122:27: error: conflicting types for 'uintptr_t'
 typedef unsigned long int uintptr_t;
                           ^
In file included from ../source3/registry/tests/test_regfio.c:23:0:
../third_party/cmocka/cmocka.h:126:28: note: previous declaration of 'uintptr_t' was here
       typedef unsigned int uintptr_t;

Fixes:
 - http://autobuild.buildroot.org/results/bae0508e84c905dc23ad7cf1153cd1e9d8e4d734

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb: Release ldb 2.0.1

* Rework pack routines to remove unused features:
  - ldb_unpack_data_only_attr_list_flags() is removed
  - LDB_UNPACK_DATA_FLAG_NO_DATA_ALLOC is now always implied

  This improves the unpack performace significantly.

* Improve search performance via new internal flag
  LDB_UNPACK_DATA_FLAG_READ_LOCKED which removes a memdup

* Improve search performance during full scan by ignoring
  index records early.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

ldb: remove unpack only attr list functionality

Unpack functions currently take an attribute list to restrict the set of
attributes to be returned in the constructed message. This
functionality is never used and complicates implementation of
forthcoming new pack format. This patch removes that functionality.
Using the unpack function then filtering the result turns
out not to be any slower.

NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

ldb: move ldb_kv's filter into pack code

This patch moves ldb_kv's filter code into the pack code to replace
'only attr list' functionality which will be removed in forthcoming
commit. Unpacking data then filtering the result is not any slower
than the removed 'only attr list' approach.
'only attr list' test repurposed to test unpack -> filter flow.

NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

ldb: removing msg and dn copying from filter attrs

Optimising filter_attrs by removing msg and dn allocation/copying. The
caller can construct the msg and possibly steal the dn.
Also giving the function an ldb for future use.

NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

ldb: perf test for pack format

Performance test for pack function and unpack function run with flag
LDB_UNPACK_DATA_FLAG_NO_DATA_ALLOC. Run this test before and after
forthcoming pack format changes to test throughput improvement. On my
machine, the unpack improvement is around 50%.
The test doesn't really belong in LDB torture but it's the only place
where all the required functions are available.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

ldb: removing alloc from unpack_data

Making unpack flag LDB_UNPACK_DATA_FLAG_NO_DATA_ALLOC required
behaviour, since allocating data during unpack is slow and unnecessary
in all current usages. In any future unpack usage, if editing of
returned memory is required, some function that duplicates the message
should be used, such as one of the filter_attrs functions, or msg_copy.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

ldb_kv: Use ldb_msg_add_steal_value() in msg_add_distinguished_name()

This more modern routine allocates a nice talloc tree.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

ldb_kv: Remove incorrect reference to LDB_UNPACK_DATA_FLAG_NO_DN

The DN is now exploded so as to improve other aspects of the search handling.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

s3:libsmb: Add missing OOM check in fill_quota_buffer()

Found by covscan (csbuild).

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Noel Power <nopower@suse.com>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed May 15 00:45:58 UTC 2019 on sn-devel-184

docs: add vfs_ceph_snapshots manpage

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): David Disseldorp <ddiss@samba.org>
Autobuild-Date(master): Tue May 14 23:31:27 UTC 2019 on sn-devel-184

vfs: add ceph_snapshots module

vfs_ceph_snapshots is a module for accessing CephFS snapshots as
Previous Versions. The module is separate from vfs_ceph, so that it can
also be used atop a CephFS kernel backed share with vfs_default.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_ceph: drop fdopendir handler

libcephfs doesn't currently offer an fdopendir equivalent, so the
existing implementation peeks at fsp->fsp_name->base_name, which can
break if vfs_ceph is used under a separate path-munging VFS module.

Return ENOSYS instead and rely on existing OpenDir_fsp() fallback.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

CVE-2018-16860 Heimdal KDC: Reject PA-S4U2Self with unkeyed checksum

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13685

Signed-off-by: Isaac Boukris <iboukris@gmail.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(master): Tue May 14 11:45:13 UTC 2019 on sn-devel-184

CVE-2018-16860 selftest: Add test for S4U2Self with unkeyed checksum

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13685

Signed-off-by: Isaac Boukris <iboukris@gmail.com>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>

ctdb-common: Fix memory leak in run_proc

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13943

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Tue May 14 08:59:03 UTC 2019 on sn-devel-184

ctdb-common: Fix memory leak

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13943

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-recoverd: Fix memory leak

state is always freed before exiting this function, so allocate fde
off it instead of long-lived ctdb context.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13943

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Fix logic error in simple ctdb reloadips test

There is a chance that restoring IP addresses to the test node will
result in different IP addresses being assigned to that node.
Removing a single IP address may then fail (or be a no-op) if it is
done after the restore.

So, swap the single IP address removal to happen first, then restore,
then remove all IP addresses.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13924

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Make ctdb reloadips tests more reliable

ctdb reloadips will fail if it can't disable takover runs.  The most
likely reason for this is that there is already a takeover run in
progress.  We can't predict when this will happen, so retry if this
occurs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13924

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Capture output in $out on failure as well

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13924

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

selftest: enable undefined behaviour sanitizer

Add a --undefined-sanitizer option to configure, this causes the tests
to be run with the undefined behaviout sanitizer enabled.

Errors can be suppressed by adding entries to selftest/ubsan.supp

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue May 14 07:20:28 UTC 2019 on sn-devel-184

s4 dsdb: fix use after free in samldb_rename_search_base_callback

Fix use after free detected by AddressSanitizer

AddressSanitizer: heap-use-after-free on address 0x60f0002b2738
                  at pc 0x7f89b1a213b5 bp 0x7ffce9528810 sp 0x7ffce9528800
                  READ of size 8 at 0x60f0002b2738 thread T0
    #0 0x7f89b1a213b4 in samldb_rename_search_base_callback
        ../../source4/dsdb/samdb/ldb_modules/samldb.c:4203
    #1 0x7f89d3a0db4a in ldb_module_send_entry
        ../../lib/ldb/common/ldb_modules.c:793
    #2 0x7f89b6f27356 in es_callback
        ../../source4/dsdb/samdb/ldb_modules/encrypted_secrets.c:1418

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13942

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ctdb-tests: Remove old socket wrapper state directory during setup

Otherwise, when looping tests for a long time, nodes are unable to
connect to each other.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13924

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon May 13 08:42:44 UTC 2019 on sn-devel-184

ctdb-tests: Actually restart if cluster doesn't become healthy

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13924

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Add dump-logs command for local daemons

Dump a single merged log to stdout.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Add reqid wrapping test

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13930

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-daemon: Never use 0 as a client ID

ctdb_control_db_attach() and ctdb_control_db_detach() assume that any
control with client ID 0 comes from another daemon and treat it
specially.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13930

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

vfs_ceph: fix cephwrap_flistxattr() debug message

The @list buffer may be uninitialised prior to ceph_flistxattr()
invocation, so only log the address.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13940

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): David Disseldorp <ddiss@samba.org>
Autobuild-Date(master): Fri May 10 12:59:05 UTC 2019 on sn-devel-184

s3 rpc_client: Fix Asan stack use after scope

Fixes Asan error:

==1924==ERROR: AddressSanitizer: stack-use-after-scope on address
    0x7ffe63f873d0 at pc 0x7fb99dae1733 bp 0x7ffe63f86a00 sp 0x7ffe63f861a8
READ of size 24 at 0x7ffe63f873d0 thread T0
    #0 0x7fb99dae1732  (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x79732)
    #1 0x7fb99cfe5549 in memcpy
        /usr/include/x86_64-linux-gnu/bits/string_fortified.h:34
    #2 0x7fb99cfe5549 in ndr_push_bytes
        ../../librpc/ndr/ndr_basic.c:729
    #3 0x7fb99cfe5646 in ndr_push_array_uint8
        ../../librpc/ndr/ndr_basic.c:754
    #4 0x7fb99a69dd1b in ndr_push_netr_ChallengeResponse
        librpc/gen_ndr/ndr_netlogon.c:462
    #5 0x7fb99a6c5fab in ndr_push_netr_NetworkInfo
        librpc/gen_ndr/ndr_netlogon.c:556
    #6 0x7fb99a6c749d in ndr_push_netr_LogonLevel
         librpc/gen_ndr/ndr_netlogon.c:783
    #7 0x7fb99a7222de in ndr_push_netr_LogonSamLogonEx
         librpc/gen_ndr/ndr_netlogon.c:16547
    #8 0x7fb99c982c97 in dcerpc_binding_handle_call_send
         ../../librpc/rpc/binding_handle.c:416

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13936

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri May 10 10:02:21 UTC 2019 on sn-devel-184

script/.gitlab-ci.yml: do not run o3 jobs if coverage enabled

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

.gitlab-ci.yml: add pages job to generate coverage report and publish to gitlab pages

- add new stage "report"
- add pages job in "report" stage to collect .info files, generate html report and publish to gitlab pages.
- only trigger when SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE == '--enable-coverage'

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

.gitlab-ci.yml: add var SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE

keep this var empty by default, since gcov will slow down the build process.
set to `--enable-coverage` to enable coverage on demand.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

script/autobuild.py: define LCOV_CMD and run it after each make test

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

script/autobuild.py: add ENABLE_COVERAGE placeholder in configure cmd option

The value will be "--enable-coverage" or "".

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

script/autobuild.py: replace more placeholders in cmds for coverage

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

script/autobuild.py: add --enable-coverage option

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

script/autobuild.py: mv optionparse to top

so the rest of the code can use the option values directly.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

script/autobuild.py: mv find_git_root and gitroot to top

so the reset of the code can use gitroot directly.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

script/autobuild.py: rename sdir to test_source_dir

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

wscript: mv --enable-coverage option to global

We want to enable gcov for all tasks, move it to global wscript.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest/target/Samba4.pm: increase max_wait from 60s to 120s to avoid timeout failure in samba-ad-dc-backup

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4 librpc rpc pyrpc: Ensure tevent_context deleted last

Ensure that the tevent_context is deleted after the connection, to
prevent a use after free.

Note: Py_DECREF calls dcerpc_interface_dealloc so the
TALLOC_FREE(ret->mem_ctx) calls in the error paths of
py_dcerpc_interface_init_helper needed removal.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13932

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba_dnsupdate: flush dns update cache file after write

While running samba python tests with `coverage.py`, `make test` failed
to start ad_dc env since dns_update_cache stays empty.
Flush the file after write fix the issue.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

s3/rpcclient: Fix bad (and illegal) func cast

../../source3/rpcclient/cmd_samr.c: In function ‘cmd_samr_get_usrdom_pwinfo’:
../../librpc/ndr/libndr.h:219:76: error: cast between incompatible function types from ‘void (*)(struct ndr_print *, const char *, uint32_t)’ {aka ‘void (*)(struct ndr_print *, const char *, unsigned int)’} to ‘void (*)(struct ndr_print *, const char *, const void *)’ [-Werror=cast-function-type]
 #define NDR_PRINT_STRUCT_STRING(ctx, type, p) ndr_print_struct_string(ctx, (ndr_print_fn_t)ndr_print_ ##type, #p, p)
                                                                            ^
../../source3/rpcclient/cmd_samr.c:2646:4: note: in expansion of macro ‘NDR_PRINT_STRUCT_STRING’
    NDR_PRINT_STRUCT_STRING(mem_ctx,

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri May 10 02:26:02 UTC 2019 on sn-devel-184

s3/registry: fix various 'cast between incompatible function' warnings

[3395/3524] Compiling source3/registry/reg_import.c
../../source3/registry/reg_import.c: In function ‘reg_import_adapter’:
../../source3/registry/reg_import.c:237:16: error: cast between incompatible function types from ‘int (*)(void *)’ to ‘WERROR (*)(void *, void *, const char *, void **)’ {aka ‘struct <anonymous> (*)(void *, void *, const char *, void **)’} [-Werror=cast-function-type]
   cb.openkey = (reg_import_callback_openkey_t)&nop;
                ^
../../source3/registry/reg_import.c:240:17: error: cast between incompatible function types from ‘int (*)(void *)’ to ‘WERROR (*)(void *, void *)’ {aka ‘struct <anonymous> (*)(void *, void *)’} [-Werror=cast-function-type]
   cb.closekey = (reg_import_callback_closekey_t)&nop;
                 ^
../../source3/registry/reg_import.c:243:18: error: cast between incompatible function types from ‘int (*)(void *)’ to ‘WERROR (*)(void *, void *, const char *, void **, _Bool *)’ {aka ‘struct <anonymous> (*)(void *, void *, const char *, void **, _Bool *)’} [-Werror=cast-function-type]
   cb.createkey = (reg_import_callback_createkey_t)&nop;
                  ^
../../source3/registry/reg_import.c:246:18: error: cast between incompatible function types from ‘int (*)(void *)’ to ‘WERROR (*)(void *, void *, const char *)’ {aka ‘struct <anonymous> (*)(void *, void *, const char *)’} [-Werror=cast-function-type]
   cb.deletekey = (reg_import_callback_deletekey_t)&nop;
                  ^
../../source3/registry/reg_import.c:249:18: error: cast between incompatible function types from ‘int (*)(void *)’ to ‘WERROR (*)(void *, void *, const char *)’ {aka ‘struct <anonymous> (*)(void *, void *, const char *)’} [-Werror=cast-function-type]
   cb.deleteval = (reg_import_callback_deleteval_t)&nop;

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3/registry: Fix incompatible func casts

[3425/3524] Compiling source3/registry/reg_parse.c
../../source3/registry/reg_parse.c: In function ‘reg_parse_new’:
../../source3/registry/reg_parse.c:223:12: error: cast between incompatible function types from ‘int (*)(void *)’ to ‘int (*)(void *, const char **, size_t,  _Bool)’ {aka ‘int (*)(void *, const char **, long unsigned int,  _Bool)’} [-Werror=cast-function-type]
   cb.key = (reg_parse_callback_key_t)&nop;
            ^
../../source3/registry/reg_parse.c:226:12: error: cast between incompatible function types from ‘int (*)(void *)’ to ‘int (*)(void *, const char *, uint32_t,  const uint8_t *, uint32_t)’ {aka ‘int (*)(void *, const char *, unsigned int,  const unsigned char *, unsigned int)’} [-Werror=cast-function-type]
   cb.val = (reg_parse_callback_val_t)&nop;
            ^
../../source3/registry/reg_parse.c:229:16: error: cast between incompatible function types from ‘int (*)(void *)’ to ‘int (*)(void *, const char *)’ [-Werror=cast-function-type]
   cb.val_del = (reg_parse_callback_val_del_t)&nop;
                ^
../../source3/registry/reg_parse.c:232:16: error: cast between incompatible function types from ‘int (*)(void *)’ to ‘int (*)(void *, const char *)’ [-Werror=cast-function-type]
   cb.comment = (reg_parse_callback_comment_t)&nop;

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3/registry: Fix func cast error (diff in params size_t vs uint32)

reg_parse_callback_key_t alread has size_t for number of elemens in key
but
reg_parse_callback_val_t uses uint32_t for length of value data.

Many internal function seem to take size_t as data len (see reg_format_value
etc.) It seems changing the signature of the callback aligns it more closely
with the existing api.

../../source3/registry/reg_format.c:438:10: error: cast between incompatible function types from ‘int (*)(reg_format *, const char *, uint32_t,  const uint8_t *, size_t)’ {aka ‘int (*)(struct reg_format *, const char *, unsigned int,  const unsigned char *, long unsigned int)’} to ‘int (*)(void *, const char *, uint32_t,  const uint8_t *, uint32_t)’ {aka ‘int (*)(void *, const char *, unsigned int,  const unsigned char *, unsigned int)’} [-Werror=cast-function-type]
   .val = (reg_parse_callback_val_t)&reg_format_value,

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/lib/policy/gp_ldap: use ldb API to find messages

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dsdb mod/linked_attributes: fix_link_slow(): clarify a comment.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dsdb/mod/extended_dn_out: use faster removal filters

When filtering out multiple elements, we end up memmove()ing the same
elements many times over. It is simpler to not do that by keeping track
of how many elements we are keeping.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb_msg: remove_element() checks element array bounds

Previously we half-heartedly checked one end.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb_kv: use ldb_msg_remove_element()

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb/tools/ldbtest: initialise msg object

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb_ldap: use ldb_msg API to add elements

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb_ldap: trust db_msg_new() to return empty message

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dsdb/replmd: use ldb_msg_remove_element()

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dsdb mods/extended_dn_out: remove element using ldb_msg api

The bare memmove is not strictly safe at the end of the list.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dsdb/mod/extended_dn_out: zero whole fake_msg struct

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4: use ldb_msg_new(), not talloc/talloc_zero

ldb_msg_new() is currently the same as talloc_zero(), but it might
not always be.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb: use ldb_msg_new(), not talloc/talloc_zero

ldb_msg_new() is currently the same as talloc_zero(), but it might
not always be.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

kdb_kv_search: spell 'linearized'

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dsdb/util: spell "equivalence"!

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb.h: spelling of 'means'

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb.h: improve comment for LDB_ATTR_FLAG_INDEXED

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/policy/gp_filesys: avoid SIZE_MAX smbcli write (CID 1034779)

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/messaging: do not deref NULL state (CID 1437973)

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/ldap_bind: notice backend init failure

This should silence CID 1418508 and CID 1418512

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

read_smb_length: Use correct function name in debug message

Signed-off-by: Michael Hanselmann <public@hansmi.ch>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

gitlab-ci: Install libtasn1-tools on Fedora based distributions

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3: SMB1: Don't allow recvfile on stream fsp's.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13938

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri May 10 01:14:02 UTC 2019 on sn-devel-184

dsdb/mod/count_attrs: set ldb var before using it (CID 1444979)

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu May  9 23:54:35 UTC 2019 on sn-devel-184

s4/dsdb/util_samr: check some return codes (CID 1444977)

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

s4/dnsserver: delay return when trying to log (CID 1444976)

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

s4/dnsserver: handle broken zone values in sort (CID 1414763, 1414769)

We sort NULL values to the end of the list. What happens to the after
that is another question.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

librpc/ndr: make push_charset_to_null UTF-16 safe (CID 1399648)

The length is in test units, not bytes, and includes terminating
nulls. For 16-bit character sets, the terminating null must be two
bytes.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

ldb modules: paged_search checks control is not NULL (CID 241355)

It is unlikely to be NULL, since we're in the callback.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

ldb_mdb: check fcntl return values (CID 1435851)

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

ldb_map: check a return value (CID 241354)

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

ldb_ldif: avoid NULL dereference with unexpected arguments (CID 1107195)

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>