ldb: Free memory when repacking database

The msg for each database record is allocated on the module context, but
never freed. The module seems like it could be a long-running context (as
the database would normally get repacked by the samba executable).

Even if it's not a proper leak, it shouldn't hurt to cleanup the memory.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

ldb: Log the partition we're repacking

Firstly, with Samba AD this looks a little weird because we log the same
message 5 times (once for every partition). If we log that we're doing
this to records in different partitions, hopefully someone with a little
Samba knowledge can figure out what's going on.

Secondly, the info about what partitions are actually changing might be
useful. E.g. if we hit a fatal error repacking the 3rd partition, and
the transaction doesn't abort properly, then it would be useful to know
what partitions were repacked and which ones weren't.

There doesn't appear to be a useful name for the partition
(ldb_kv->kv_ops->name() doesn't seem any more intelligible to a user),
so just log the first record that we update. We can use that to infer
the partition database).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

ldb: Log pack format in user-friendly way

The "format 0x26011968" log confused me (and I'm a developer).
We can subtract the base offset from the pack format to get a more
user-friendly number, e.g. v0 (not actually used), v1, v2, etc.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

ldb: Change pack format defines to enum

The main reason is so that any future pack formats will continue
incrementing this number in a sequential fashion.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

ldb: Move where we update the pack format version

Store it on the repack context so that we can log a more informative
message "Repacking from format x to format y".

While this is not really a big deal currently, it could be worth
recording for potential future scenarios (i.e. supporting three or more
pack versions), where upgrades could potentially skip an intermediary
pack format version.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

ldb: Always log when the database pack format changes

LDB_DEBUG_WARNING gets logged by Samba as level 2, whereas the default
log level for Samba is 0. It's not really fair to the user to change the
format of their database on disk and potentially not tell them.

This patch adds a log with level zero (using a alias define, as this
technically isn't a fatal problem).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

downgradedatabase: installing script

Installing downgrade script so people don't need the source tree for it.

Exception added in usage test because running the script without arguments
is valid. (This avoids the need to knownfail it).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

downgradedatabase: Add man-page documentation

A man-page is needed so that we can install this tool as part of the
Samba package.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

downgradedatabase: rename to samba_downgrade_db

Just so that it's slightly less of a mouthful for users.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

tests: Avoid hardcoding relative filepath

If we move the test file, the test will break.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

downgradedatabase: comply with samba.tests.source

In next commit we'll install the script, samba.tests.source picked up the
lack of a copyright message and some whitespace errors, so this patch
fixes that stuff first.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14059

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

third_party: Remove historical tombstone about zlib divergance

Now this has been recorded in the git history, we can remove it.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sat Aug 17 05:10:48 UTC 2019 on sn-devel-184

third_party: Remove zlib from third_party

We require zlib 1.2.3.

We stopped requring a patched zlib with 5631a1b9bc03d6cf31af66b13872255f18979fe8

As discussed on samba-technical here:

https://lists.samba.org/archive/samba-technical/2019-May/133476.html

In short, zlib contains some (old, now broken) crypto code that while not compiled
in Samba is best left out of our tarball to ease crypto audits.  It is also very
very out of date and is a slightly modified copy of something otherwise very
likely available on our supported host OSs.  It would be strange to say that
GnuTLS and dependencies are an acceptable burden to install but say zlib is
a step to far.

So it is removed from Samba's third_party with this commit.

The diff between zlib in Samba and official zlib 1.2.3 is included
in third_party/zlib/last-samba-from-1.2.3.diff

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

Prevent samba-tool online backup crash

On some GPOs, getting a files ntacl throws an
NT_STATUS_ACCESS_DENIED. Catch and log the
failure when this happens.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14088

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Tim Beale <timbeale@samba.org>

ctdb-tcp: Mark node as disconnected if incoming connection goes away

To make it easy to pass the node data to the upcall, the private data
for ctdb_tcp_read_cb() needs to be changed from tnode to node.

RN: Avoid marking a node as connected before it can receive packets
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14084

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Aug 16 22:50:35 UTC 2019 on sn-devel-184

ctdb-tcp: Only mark a node connected if both directions are up

Nodes are currently marked as up if the outgoing connection is
established.  However, if the incoming connection is not yet
established then this node could send a request where the replying
node can not queue its reply.  Wait until both directions are up
before marking a node as connected.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14084

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tcp: Create outbound queue when the connection becomes writable

Since commit ddd97553f0a8bfaada178ec4a7460d76fa21f079
ctdb_queue_send() doesn't queue a packet if the connection isn't yet
established (i.e. when fd == -1).  So, don't bother creating the
outbound queue during initialisation but create it when the connection
becomes writable.

Now the presence of the queue indicates that the outbound connection
is up.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14084

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tcp: Use TALLOC_FREE()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14084

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tcp: Move incoming fd and queue into struct ctdb_tcp_node

This makes it easy to track both incoming and outgoing connectivity
states.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14084

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tcp: Rename fd -> out_fd

in_fd is coming soon.

Fix coding style violations in the affected and adjacent lines.
Modernise some debug macros and make them more consistent (e.g. drop
logging of errno when strerror(errno) is already logged.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14084

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-daemon: Add function ctdb_ip_to_node()

This is the core logic from ctdb_ip_to_pnn(), so re-implement that
that function using ctdb_ip_to_node().

Something similar (ctdb_ip_to_nodeid()) was recently removed in commit
010c1d77cd7e192b1fff39b7b91fccbdbbf4a786 because it wasn't required.
Now there is a use case.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14084

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

s3: VFS: vfs_default. Convert underlying system call to renameat().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Aug 16 21:10:14 UTC 2019 on sn-devel-184

s3: VFS: Complete the replacement of SMB_VFS_RENAME() -> SMB_VFS_RENAMEAT().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_time_audit. Remove rename_fn(). No longer used.

NB, this will now fail smb_vfs_assert_all_fns()
until we remove the rename_fn() from the VFS definitions.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_full_audit. Remove rename_fn(). No longer used.

NB, this will now fail smb_vfs_assert_all_fns()
until we remove the rename_fn() from the VFS definitions.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_virusfilter. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_unityed_media. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_syncops. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_streams_xattr. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_streams_depot: Change to using SMB_VFS_RENAMEAT() instead of SMB_VFS_RENAME().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_streams_depot. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_snapper. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_shadow_copy2. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_netatalk. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_media_harmony. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_glusterfs. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_fruit. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_extd_audit. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_crossrename. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_ceph_snapshots. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_ceph. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_catia. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_cap. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_audit. Remove rename_fn(). No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_recycle. Use SMB_VFS_RENAMEAT() instead of SMB_VFS_RENAME().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: smbd: vfs_virusfilter: Use SMB_VFS_RENAMEAT() instead of SMB_VFS_RENAME.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: torture: Change cmd_vfs to call SMB_VFS_RENAMEAT().

Use conn->cwd_fsp as current src and dst fsp's.

No logic change for now.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: smbd: Change rename_internals_fsp() to call SMB_VFS_RENAMEAT().

Use conn->cwd_fsp as current src and dst fsp's.

No logic change for now.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_virusfilter. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_unityed_media. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_time_audit. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_syncops. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_streams_xattr. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_streams_depot. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_snapper. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_shadow_copy2. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_netatalk. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_media_harmony. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_glusterfs. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_full_audit. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_fruit. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_extd_audit. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_crossrename. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_ceph_snapshots. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_ceph. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_catia. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_cap. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: vfs_audit. Implement renameat().

Currently identical to rename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: Add SMB_VFS_RENAMEAT().

Currently identical to SMB_VFS_RENAME() - uses
AT_FDCWD for both src and dst directories.

Next, move add to all VFS modules that implement
rename and eventually remove rename.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: change connection_struct cwd_fname to cwd_fsp

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_full_audit: pass conn to smb_fname_str_do_log()

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: modernize conn_new()

Split the monstrous if into individual allocations. I'm going to add more talloc
allocations in a subsequent commit, so it's time to split this up.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3: smbd: Reformatting. Use helper variable for SMB_VFS_RENAME call.

Will make replacement with renameat() clearer later.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

vfs_glusterfs: initialize st_ex_file_id, st_ex_itime and st_ex_iflags

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14090
RN: vfs_glusterfs: initialize st_ex_file_id, st_ex_itime and st_ex_iflags

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Aug 16 01:07:23 UTC 2019 on sn-devel-184

vfs_default: use correct flag in vfswrap_fs_file_id

Luckily using the wrong flag ST_EX_IFLAG_CALCULATED_ITIME currently results in
the same semantics as using the correct ST_EX_IFLAG_CALCULATED_FILE_ID, as in
vfs_default the non-calculated file_id is based a non-calculated itime.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14089
RN: vfs_default: use correct flag in vfswrap_fs_file_id

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Make sys_disk_free static

The function is only called from the same file.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Christof Schmitt <cs@samba.org>
Autobuild-Date(master): Wed Aug 14 17:47:33 UTC 2019 on sn-devel-184

selftest: Add test for quota query on directory with SGID

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

vfs_fake_dfq: Add option to fake stat information

Add an option to allow faking the gid and the SGID bit in the returned
stat buffer. That will be used to verify quota queries on folders with
SGID set.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

vfs_gpfs: Check group quota for directory when SGID is set

On directories with the "set group id" (SGID) bit is set, new files and
subfolders will be created with the group of the directory, and not with
the primary group of the user. Checking for free space in this case
should query the group quota for the gid of the directory.

This is the same change as the common smbd code, but since vfs_gpfs
still has its own check for available space and quotas, add it here as
well. A future goal would be to use the common free space check instead
of duplicating code in vfs_gpfs.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

quotas: Check group quota for directory when SGID is set

On directories with the "set group id" (SGID) bit is set, new files and
subfolders will be created with the group of the directory, and not with
the primary group of the user. Checking for free space in this case
should query the group quota for the gid of the directory.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

auth:ntlmssp: Use generate_random_buffer() for session keys

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Alexander Bokovoy <ab@samba.org>
Autobuild-Date(master): Wed Aug 14 16:26:47 UTC 2019 on sn-devel-184

s3:passdb: Use generate_secret_buffer() for generating passwords

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

s4:samdb: Use generate_nonce_buffer() for AEC GCM nonce

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

lib:util: Use generate_secret_buffer() for long term passwords

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

Revert "s4:rpc_server: Use generate_secret_buffer() to create a session key"

This reverts commit 4b2480518bd3887be3a6cfb713523ac084e09fd5.

Reviewed-by: Alexander Bokovoy <ab@samba.org>

Revert "s4:rpc_server: Use generate_secret_buffer() for backupkey wap_key"

This reverts commit 5a62056b4530e4c509444be9164a1fca1dce193f.

Reviewed-by: Alexander Bokovoy <ab@samba.org>

Revert "s4:rpc_server: Use generate_secret_buffer() for netlogon challange"

This reverts commit a21770cfdffd2a21045a1bc87e489af0f4c6f130.

Reviewed-by: Alexander Bokovoy <ab@samba.org>

Revert "libcli:auth: Use generate_secret_buffer() for netlogon challenge"

This reverts commit c3ba556f52b15dd80efc26e4fb8f43ce2ee3a7f0.

Reviewed-by: Alexander Bokovoy <ab@samba.org>

lib:util: Fix documentation for random number functions

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

smbtorture: extend rpc.lsa to lookup machine over forest-wide LookupNames

Add a simple test to resolve DOMAIN\MACHINE$ via LSA LookupNames3
using LSA_LOOKUP_NAMES_UPLEVEL_TRUSTS_ONLY2 level. This level would pass
zero lookup flags to lookup_name().

Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Alexander Bokovoy <ab@samba.org>
Autobuild-Date(master): Wed Aug 14 13:07:42 UTC 2019 on sn-devel-184

lookup_name:  allow own domain lookup when flags == 0

In 2007, we've added support for multiple lookup levels for LSA
LookupNames family of calls. However, forest-wide lookups, as described
in MS-LSAT 2.2.16, never worked because flags passed to lookup_name()
were always set to zero, expecting at least default lookup on a DC to
apply. lookup_name() was instead treating zero flags as 'skip all
checks'.

Allow at least own domain lookup in case domain name is the same.
This should allow FreeIPA DC to respond to LSA LookupNames3 calls from a
trusted AD DC side.

For the reference, below is a request Windows Server 2016 domain
controller sends to FreeIPA domain controller when attempting to look up
a user from a trusted forest root domain that attemps to login to the
domain controller. Notice the level in the lsa_LookupNames3 call and
resulting flags in lookup_name().

[2019/08/03 07:14:24.156065,  1, pid=23639, effective(967001000, 967001000), real(967001000, 0), class=rpc_parse] ../../librpc/ndr/ndr.c:471(ndr_print_function_debug)
       lsa_LookupNames3: struct lsa_LookupNames3
          in: struct lsa_LookupNames3
              handle                   : *
                  handle: struct policy_handle
                      handle_type              : 0x00000000 (0)
                      uuid                     : 0000004c-0000-0000-455d-3018575c0000
              num_names                : 0x00000001 (1)
              names: ARRAY(1)
                  names: struct lsa_String
                      length                   : 0x000a (10)
                      size                     : 0x000c (12)
                      string                   : *
                          string                   : 'XS\ab'
              sids                     : *
                  sids: struct lsa_TransSidArray3
                      count                    : 0x00000000 (0)
                      sids                     : NULL
              level                    : LSA_LOOKUP_NAMES_UPLEVEL_TRUSTS_ONLY2 (6)
              count                    : *
                  count                    : 0x00000000 (0)
              lookup_options           : LSA_LOOKUP_OPTION_SEARCH_ISOLATED_NAMES (0)
              client_revision          : LSA_CLIENT_REVISION_2 (2)
[2019/08/03 07:14:24.156189,  6, pid=23639, effective(967001000, 967001000), real(967001000, 0), class=rpc_srv] ../../source3/rpc_server/rpc_handles.c:339(find_policy_by_hnd_internal)
  Found policy hnd[0] [0000] 00 00 00 00 4C 00 00 00   00 00 00 00 45 5D 30 18   ....L... ....E]0.
  [0010] 57 5C 00 00                                        W\..
[2019/08/03 07:14:24.156228,  4, pid=23639, effective(967001000, 967001000), real(967001000, 0)] ../../source3/smbd/sec_ctx.c:215(push_sec_ctx)
  push_sec_ctx(967001000, 967001000) : sec_ctx_stack_ndx = 2
[2019/08/03 07:14:24.156246,  4, pid=23639, effective(967001000, 967001000), real(967001000, 0)] ../../source3/smbd/uid.c:552(push_conn_ctx)
  push_conn_ctx(0) : conn_ctx_stack_ndx = 0
[2019/08/03 07:14:24.156259,  4, pid=23639, effective(967001000, 967001000), real(967001000, 0)] ../../source3/smbd/sec_ctx.c:319(set_sec_ctx_internal)
  setting sec ctx (0, 0) - sec_ctx_stack_ndx = 2
[2019/08/03 07:14:24.156273,  5, pid=23639, effective(967001000, 967001000), real(967001000, 0)] ../../libcli/security/security_token.c:53(security_token_debug)
  Security token: (NULL)
[2019/08/03 07:14:24.156285,  5, pid=23639, effective(967001000, 967001000), real(967001000, 0)] ../../source3/auth/token_util.c:865(debug_unix_user_token)
  UNIX token of user 0
  Primary group is 0 and contains 0 supplementary groups
[2019/08/03 07:14:24.156311,  5, pid=23639, effective(0, 0), real(0, 0), class=rpc_srv] ../../source3/rpc_server/lsa/srv_lsa_nt.c:244(lookup_lsa_sids)
  lookup_lsa_sids: looking up name XS\ab
[2019/08/03 07:14:24.156327, 10, pid=23639, effective(0, 0), real(0, 0)] ../../source3/passdb/lookup_sid.c:112(lookup_name)
  lookup_name: XS\ab => domain=[XS], name=[ab]
[2019/08/03 07:14:24.156340, 10, pid=23639, effective(0, 0), real(0, 0)] ../../source3/passdb/lookup_sid.c:114(lookup_name)
  lookup_name: flags = 0x00

Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

torture/rpc/lsa: allow testing different lookup levels

Convert torture/rpc/lsa LookupNames/LookupSids code to allow testing
different LSA_LOOKUP_NAMES_* levels. Keep existing level 1
(LSA_LOOKUP_NAMES_ALL) for the current set of tests.

Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

smbd: Fix CID 1452293 Incorrect expression (NO_EFFECT)

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Aug 14 11:48:42 UTC 2019 on sn-devel-184

ctdb-daemon: Avoid signed/unsigned comparison by casting

Compiling with -Wsign-compare complains:

 1047 |  &&  (call->call_id == CTDB_FETCH_WITH_HEADER_FUNC)) {
      |                     ^~

struct ctdb_call is a protocol element, so we can't simply change it.

Found by csbuild.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed Aug 14 10:29:59 UTC 2019 on sn-devel-184

ctdb-daemon: Avoid signed/unsigned comparison by declaring as unsigned

Compiling with -Wsign-compare complains:

ctdb/server/ctdb_call.c:831:12: warning: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Wsign-compare]
  831 |  if (count <= ctdb_db->statistics.hot_keys[0].count) {
      |            ^~

and

ctdb/server/ctdb_call.c:844:13: warning: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Wsign-compare]
  844 |   if (count <= ctdb_db->statistics.hot_keys[i].count) {
      |             ^~

Found by cs-build.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Return value of ctdb_queue_length() should be unsigned

Compiling with -Wsign-compare complains:

ctdb/server/ctdb_daemon.c: scope_hint: In function ‘daemon_queue_send’
ctdb/server/ctdb_daemon.c:259:40: warning: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Wsign-compare]
...

The struct ctdb_queue member out_queue_length is actually uint32_t, so
just return that type.

Found by csbuild.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Use select_test_node() in ctdb setdebug simple test

There is no requirement for IP addresses here.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Add function select_test_node()

Should be used when public IP addresses are not assigned.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tools: Fix usage for "ctdb cattdb"

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Don't print summary on failure if -e option is specified

If there is a failure it will always be the last test run.  Don't
obscure this by following it with a summary.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>