git.samba.org - samba-bkup.git/log

WHATSNEW: SMB Server performance improvements

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(v4-17-test): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(v4-17-test): Tue Aug 16 19:37:17 UTC 2022 on sn-devel-184

s3:vfs.h: add comment about VFS_OPEN_HOW_RESOLVE_NO_SYMLINKS

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 5adf051228b56c05fe1205e7a865a497b58e81d9)

s3: smbd: Add IS_VETO_PATH checks to openat_pathref_fsp_case_insensitive().

Returns NT_STATUS_OBJECT_NAME_NOT_FOUND for final component.

Note we have to call the check before each call to
openat_pathref_fsp(), as each call may be using a
different filesystem name. The first name is the
one passed into openat_pathref_fsp_case_insensitive()
by the caller, the second one is a name retrieved from
get_real_filename_cache_key(), and the third one is the name
retrieved from get_real_filename_at(). The last two
calls may have demangled the client given name into
a veto'ed path on the filesystem.

Remove knownfail.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15143

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Aug 16 08:26:54 UTC 2022 on sn-devel-184

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 1654eae11b9c13308b2b78f70309eb3a56960619)

s3: smbd: Add IS_VETO_PATH check to openat_pathref_dirfsp_nosymlink().

Returns NT_STATUS_OBJECT_PATH_NOT_FOUND for directory component.
Note IS_VETO_PATH only looks at the last component, so we must
do it during the directory walk on each component.

Note, we also have to check after a call to get_real_filename_at()
as it may have demangled the client sent name into a filesystem
name that matches the "veto files" parameter.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15143

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 1c293060204d96bf94427f91eb20eb9decc29a41)

s3: tests: Add samba3.blackbox.test_veto_files.

Shows we currently don't look at smb.conf veto files parameter
when opening a file or directory. Checks multi-component paths.
Also checks veto files that might be hidden behind a mangled
name.

Add knownfail.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15143

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit c6933673222ea9ae2eb74d5586c9495269f51ea0)

selftest/Samba3: let nt4_dc* use vfs_default:VFS_OPEN_HOW_RESOLVE_NO_SYMLINKS=no

We should always test the code path without openat2 being available,
even if the kernel supports it.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Mon Aug 15 16:00:26 UTC 2022 on sn-devel-184

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 076c22fbd7ecbf22dbfeb1711609f07fd42f88b0)

vfs_default: Use openat2(RESOLVE_NO_SYMLINKS) if available

This improves the following test:

time smbtorture //127.0.0.1/m -Uroot%test \
        smb2.create.bench-path-contention-shared \
        --option='torture:bench_path=Apps\1\2\3\4\5\6\7\8\9\10' \
        --option="torture:timelimit=600" \
        --option="torture:nprocs=1"

From:

   open[num/s=14186,avslat=0.000044,minlat=0.000042,maxlat=0.000079]
   close[num/s=14185,avslat=0.000027,minlat=0.000025,maxlat=0.000057]

to:

   open[num/s=16917,avslat=0.000038,minlat=0.000035,maxlat=0.000340]
   close[num/s=16916,avslat=0.000020,minlat=0.000019,maxlat=0.000104]

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 4708ba2f013c5f5ea5aa5dcf4873c2b4a86fb8ff)

vfs_default: prepare O_PATH usage with openat2()

When O_PATH is specified in flags, flag bits other than O_CLOEXEC,
O_DIRECTORY, and O_NOFOLLOW are ignored.

In preparation to use openat2(), which gives an error instead of
ignoring flags, we better remove unexpected flags, callers typically
pass O_RDONLY and O_NONBLOCK.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 8544f4490a0b5e54b807daedddb96778744b62ee)

s3:smbd: let openat_pathref_dirfsp_nosymlink() try VFS_OPEN_HOW_RESOLVE_NO_SYMLINKS first

This will reduce the amount of syscalls and the related cost drastically
for long path names.

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit d6653067b20e61af1f05423764c8486a1a5445c8)

s3:smbd: let openat_pathref_dirfsp_nosymlink() handle ELOOP similar to ENOTDIR

This is no likely to happen as we use O_NOFOLLOW with O_DIRECTORY,
but it's better to be prepared...

This will be more important in the upcoming openat2(RESOLVE_NO_SYMLINK)
case, but we should be consitent...

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 35b99c87ef92df006f8b0a41bbea051f0faeadb9)

s3:smbd: let openat_pathref_dirfsp_nosymlink() do a verification loop against . and .. first

I guess we should catch NT_STATUS_OBJECT_NAME_INVALID first,
currently the check is already done in check_path_syntax*,
but we may remove it in future.

But the most important reason for this is the
openat2(RESOLVE_NO_SYMLINK) optimization, which will
be introduced in the following commits.

Review with: git show -w

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 17484d069b92d08b0228fb509ea42ab4c3f496a8)

vfs: define VFS_OPEN_HOW_RESOLVE_NO_SYMLINKS

This will allow us to make use of openat2(RESOLVE_NO_SYMLINKS) soon.

The caller should check if connection_struct.open_how_resolve contains
VFS_OPEN_HOW_RESOLVE_NO_SYMLINKS before using it, this avoids waisting
cpu time. But even then the caller must be prepared to handle -1/ENOSYS.

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit f7dc27558329eea7d2c4d75ee101c7f9d3a7afe3)

lib/replace: let DISABLE_OPATH also undef __NR_openat2

The reason for DISABLE_OPATH is to simulate a non-linux
system, so we should not use openat2() either.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit ae1a84f7313bdf4702492451714eacc78ee7745f)

lib/replace: add fallback defines for __NR_openat2

sys/syscall.h might be older than the runtime kernel.

If the kernel has support for openat2() we should
try to use if anyway.

The callers have to deal with ENOSYS anyway,
so there's no difference if we get that from syscall(__NR_openat2)
or directly from rep_openat2().

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit f7618dd31a9f8f6c0dbfdedd1a664eed25e2e449)

lib/replace: use syscall(__NR_openat2) if available

There's no glibc wrapper for openat2() yet, so we need
to use syscall(__NR_openat2) ourself.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit b89001e9226ecb0f4e5c906f7195f0e53cd7d608)

lib/replace: always include <sys/syscall.h> in replace.c if available

It will be used for openat2() soon.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 37ba6df174d73b82e951de401cba7f839ad61ab5)

lib/replace: add a replacement for openat2() that returns ENOSYS

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit ce804b78164a3166a16ca3071028536761fd18d7)

vfs_btrfs: fix include order, includes.h or replace.h should be first

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 2369d0833361faf4a125431e735fce7efb6024d6)

vfs_io_uring: hide a possible definition of struct open_how in liburing/compat.h

liburing.h will include liburing/compat.h, which either includes
linux/openat2.h or defines struct open_how itself.

This will help with the following changes, which will provide
openat2() via libreplace's system/filesys.h, either including
linux/openat2.h or defining open_how ourself.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit cea9451f780d13e528f1722a67eccbbc78b2daf9)

wafsamba: allow cflags for CHECK_TYPE[_IN]()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 2b51bad747551605ba3b70ac3b692107a0cd7aad)

s3:tests: add a lot more tests to test_symlink_traversal_smb2.sh

We now also test more path components checking the difference between
OBJECT_NAME_NOT_FOUND and OBJECT_PATH_NOT_FOUND.

We also test with symlinks within the path instead of only checking
symlinks as final path components (at least for the dirfsp part).

This ensures the following commits won't introduce regressions
when adding the openat2(RESOLVE_NO_SYMLINK) optimization.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 085f14857531dab179af66a69962486c7dd2592c)

s3:utils: Fix NULL check

CID 1507864

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15140

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Aug 12 21:50:23 UTC 2022 on sn-devel-184

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit a38fad29803f9e2891b2264ac3258394152e8deb)

s3:util: Initialize json_object structures so we can call json_free()

CID 1507863
CID 1507865
CID 1507866
CID 1507867
CID 1507868
CID 1507869
CID 1507870

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15140

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 4a702cddaebf7e616706e0c728685567e141b493)

s3: smbd: Remove unix_convert() and associated functions.

All code now uses filename_convert_dirfsp() for race-free
filename conversion.

Best viewed with:
$ git show --patience

               ----------------
              /                \
             /       REST       \
            /         IN         \
           /         PEACE        \
          /                        \
          |                        |
          |      unix_convert      |
          |                        |
          |                        |
          |       9th August       |
          |          2022          |
          |                        |
          |                        |
         *|     *  *  *            | *
_________)/\\_//(\/(/\)/\//\/\///\/|_)_______

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Aug 12 19:18:25 UTC 2022 on sn-devel-184

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 78e4aac76df977cea6cdbcfdf082fd3acdffbd95)

s3: smbd: Remove the old dfs_path_lookup() code.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit cc638c25e0332d366016880d174d9349940cba3f)

s3: smbd: Switch get_referred_path() over to use the new dfs_path_lookup().

New function doesn't need a TWRP argument and returns NT_STATUS_OK
on successful redirect, not NT_STATUS_PATH_NOT_COVERED.

Comment out the old dfs_path_lookup().

There are now no more users of unix_convert().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 88e8bfec59412fdc0e83251fef60b45d2cc3a884)

s3: smbd: Add new version of dfs_path_lookup() that uses filename_convert_dirfsp().

Commented out as not yet used but it's easier to see the
new logic this way.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 22d4f62537199d9454be312a546e251f04022497)

s3: smbd: Remove dfs_redirect().

A moment of silence please.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 6b1224b22012b54b1ae20b682daf61c877362a7b)

s3: smbd: Remove call to dfs_redirect() from filename_convert_dirfsp_nosymlink().

Use dfs_filename_convert() instead. There are now no more callers of dfs_redirect().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit d20b60c3200b5e1881cdf4b59da154d1af7e3994)

s3: smbd: Remove call to dfs_redirect() from filename_convert_smb1_search_path().

Use dfs_filename_convert() instead. Code is now much simpler.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit fcf19d91c09edc6dfbf5bd7cbeedcd641030eb31)

s3: smbd: In filename_convert_dirfsp_nosymlink(), cope with an MS-DFS link as the terminal component.

If the terminal component was an MSDFS link, openat_pathref_fsp_case_insensitive() will
return NT_STATUS_OBJECT_NAME_NOT_FOUND with a VALID_STAT of a symlink.

If this is the case, check if we actually found a terminal MS-DFS link
at the end of the pathname and return NT_STATUS_PATH_NOT_COVERED.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit d80bedc3c418b6839b1bde78ba8d3db06611be2a)

s3: smbd: In filename_convert_dirfsp_nosymlink(), allow a NT_STATUS_PATH_NOT_COVERED error to be returned.

openat_pathref_dirfsp_nosymlink() can now return NT_STATUS_PATH_NOT_COVERED.
Don't convert this automatically into NT_STATUS_OBJECT_PATH_NOT_FOUND.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 07ef9e3029b8cca1b92d900d6ed684ca0ac6afe4)

s3: smbd: Allow openat_pathref_dirfsp_nosymlink() to return NT_STATUS_PATH_NOT_COVERED for a DFS link on a DFS share.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit b5f6809593524e7e9aca1c09ff379e02a1cde61b)

s3: smbd: In get create_junction(), make sure check_path_syntax() is called on returned reqpath.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit a92f4f7af0eaa035deebfb1c930ca0cc12d992d5)

s3: smbd: In get referred_path(), make sure check_path_syntax() is called on returned reqpath.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit da625e4ab4bc670e44fcb6ad7456aa64d0f1f9d2)

s3: smbd: Add dfs_filename_convert(). Simple wrapper around parse_dfs_path().

Not yet used.

This is what we will use to replace dfs_redirect() in the filename
conversion code. Keep as a wrapper for now as we might want to
add some error checking around the 'hostname' and 'service'
returns.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 245d07ab84852b829c029496618e56782d070e83)

s3: smbd: Use helper function msdfs_servicename_matches_connection() in dfs_redirect().

Replaces ugly complex logic.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit a3c9eb7931cb4da0dd5bc5d600125979dd1a7df5)

s3: smbd: Use helper function msdfs_servicename_matches_connection() in parse_dfs_path().

Replaces ugly complex logic.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit c0a1d7c7a8a7f24890e60c7a371498949dec11c2)

s3: smbd: Add helper function msdfs_servicename_matches_connection().

Not yet used so commented out.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 4f5d02f8c0efc1520b2113ce656c78483deb7826)

s3: smbd: Remove definition of struct dfs_path.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 6c83c674bab8e57ecaf6271eb3a403171bbbacca)

s3: smbd: Remove use of 'struct dfs_path'. Not needed for a (hostname, servicename, path) tuple.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit f92711f000a3cb658dfb8fffe92ae6bba78b4f91)

s3: smbd: Add TALLOC_CTX * parameter to parse_dfs_path().

Not yet used. Preparing to remove 'struct dfs_path'.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 2df8a8ab87a1372f2b67880be4454a0285b3104b)

s3: smbd: Ensure smb2_file_rename_information() uses the SMB2 pathname parsers, not the SMB1 parsers.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 0a4a27ce48bc7090aa821eea5e56f8d44c686716)

s3: smbd: Make sure we have identical check_path_syntax logic in smbd_smb2_create_durable_lease_check(), as for smb2_create.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit a2a097fc3d6a89fb970c1ea3ea75fde93ddb545e)

s3: smbd: In smbd_smb2_create_send() call the helper function check_path_syntax_smb2().

Previously for DFS names we were skipping this.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 4fafc3418931de06ea2d91baca1eef8d904cc4e6)

s3: smbd: Add helper function check_path_syntax_smb2().

Not yet used, but uses check_path_syntax_smb2_msdfs()
so remove the #ifdef's around that.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 7bd7fa0a0b46ad6826097a1987595e2ab6f83384)

s3: smbd: Add new function check_path_syntax_smb2_msdfs() for SMB2 MSDFS paths.

#ifdef'ed out as static and not yet used.

We can't just call check_path_syntax() on these as
they are of the form hostname\share[\extrapath]
(where [\extrapath] is optional).

hostname here can be an IPv6 ':' separated address,
which check_path_syntax() fails on due to the streamname
processing.

NB. This also has to cope with out existing (broken)
libsmbclient libraries that sometimes set the DFS
flag and then send a local pathname. Cope by just
calling the normal check_path_syntax() on the
whole pathname in that case.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit bcba5502282eb6dcc346d7c63aa3218cda2f9bb0)

s3: smbd: Fix cosmetic bug logging pathnames from Linux kernel clients using SMB1 DFS calls.

The Linux kernel SMB1 client has a bug - it sends
DFS pathnames as:

\\server\share\path

instead of:

\server\share\path

Causing us to mis-parse server,share,remaining_path here
and jump into 'goto local_path' at 'share\path' instead
of 'path'.

This doesn't cause an error as the limits on share names
are similar to those on pathnames.

parse_dfs_path() which we call before filename parsing
copes with this by calling trim_char on the leading '\'
characters before processing.

Do the same here so logging of pathnames looks better.

How did I find this ? Lots and lots of manual
testing with the Linux kernel client to make
sure all the recent changes haven't broken Linux
SMB1/2/3 DFS :-).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15144

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 2818fd6910201fd4a18b921933a0b7392a0a8995)

s4:torture/smb2: add smb2.bench.echo

This test calls SMB2_Echo in a loop per connection.

For 4 connections with 2 parallel loops use this:

time smbtorture //127.0.0.1/m -Uroot%test smb2.bench.echo \
--option="torture:timelimit=600" \
--option="torture:nprocs=1" \
--option="torture:qdepth=2"

Sometimes the bottleneck is the smbtorture process.
In order to bring the smbd process to 100% cpu, you can use
'--option="libsmb:client_guid=6112f7d3-9528-4a2a-8861-0ca129aae6c4"'
and run multiple instances of the test at the same time,
which both talk to the same smbd process.

This is a very useful test to show how many requests are possible
at the raw SMB2 layer.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Aug 11 19:23:37 UTC 2022 on sn-devel-184

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 23988f19e7cc2823d6c0c0f40af0195d0a3b81bf)

s4:torture/smb2: teach smb2.bench.path-contention-shared about --option="torture:qdepth=4"

This can now test more than one open/close loop per connection.

time smbtorture //127.0.0.1/m -Uroot%test \
smb2.create.bench-path-contention-shared \
--option='torture:bench_path=' \
--option="torture:timelimit=60" \
--option="torture:nprocs=1" \
--option="torture:qdepth=4"

The default is still 1, but it's very useful for tests.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 8ee783c4803d28cccc39144afa7b78c4b9e0cc2e)

s4:param: add --option="libsmb:client_guid=6112f7d3-9528-4a2a-8861-0ca129aae6c4" support...

We already handle this in the source3/libsmb code, but it's good to
have this also for torture tests.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 72caffbe1115c57ad38270eaeb951f6b97bf62b3)

s3:g_lock: use TDB_VOLATILE to avoid fcntl locks

This improves 'time smbtorture3 //foo/bar -U% local-g-lock-ping-pong -o 50000000'
from ~1.400.000 to ~3.400.000 operations per second any a testsystem.

As we also use TDB_VOLATILE for locking.tdb, this is a much more
realistic test now.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit cd01f5134696f7789fbc2933629ac2606feb0b5e)

smbd: avoid calling SMB_VFS_FGET_NT_ACL() if do_not_check_mask already covers all

This is inspired by 0d4cb5a641e1fea2d369bdc66470a580321366c2,
which avoids SMB_VFS_FGET_NT_ACL() for the root user again.

Opens with just FILE_READ_ATTRIBUTES are very common, so it's worth
optimizing for it.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit a0a97d27f7a60dbd86317b51bec0ece2476e8c8d)

s3:include: remove unused update_stat_ex_file_id() prototype

It was removed by commit 643da37fd139413651a6198fb0f6e550f7de6584

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 8c7e8c5f80f1488456f9dd6225020d29f74458d2)

smbstatus: Fix the 32-bit build on FreeBSD

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Aug 9 20:04:26 UTC 2022 on sn-devel-184

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit 4d015b4b6db20235d6d821204d03b0e1fce1c681)

smbd: Use dirfsp where we have it

One reference to conn->cwd_fsp less, makes "mkdir" look less ugly in
strace.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15146
(cherry picked from commit b1b513eebb0999cdfabab597927305be7d978605)

s3:tests: let smbstatus json tests fail if jq is not installed

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 4b91702a6ea2c6474e635d5ae5f7acd9aebbbab7)

manpages: add smbstatus option --json with sample output

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15147

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit ddbf1b29eee140b3112eb238852bfdc8285eb04f)

Autobuild-User(v4-17-test): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(v4-17-test): Tue Aug 16 17:37:28 UTC 2022 on sn-devel-184

WHATSNEW: announce new smbstatus json support

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

WHATSNEW: add section for new smbconf python api

Signed-off-by: John Mulligan <jmulligan@redhat.com>

VERSION: Bump version up to 4.17.0rc2...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Jule Anger <janger@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

VERSION: Disable GIT_SNAPSHOT for the Samba 4.17.0rc1 release.

Signed-off-by: Jule Anger <janger@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

WHATSNEW: Up to Samba 4.17.0rc1.

Signed-off-by: Jule Anger <janger@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

s3:tests: Add a test to check json output of smbstatus profile

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Jule Anger <janger@samba.org>
Autobuild-Date(master): Mon Aug 8 14:01:45 UTC 2022 on sn-devel-184

smbstatus: add JSON support for smbstatus --profile

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: fix indentation in profile_separator()

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jule Anger <janger@samba.org>

smbstatus: add a method to add profile items to json

The method changes the json item of a given traverse_state.
The root dictionary contains for each section a dictionary, which has
a dictionary for each subsection.

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3:tests: Add a test to check json output of smbstatus

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add JSON support for smbstatus

Adds the option --json for all informations except the profiling.
With --json sets the json_output variable to true, so that the json dictionary
can be created and printed.

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add machine readable creation_time to notify

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add server_id to notifies

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add a notifies dictionary

Adds an empty json dictionary under the key "notifies" and adds foreach
notify a dictionary with information to the notify dictionary. Uses the
pid as key.

Only print to stdout, if json_output is not set.

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add file_id information to byte-range locks in json output

Adds a dictionary with file_id information (devid, inode and extid) for
each byte-range locked file.

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add locks to byte-range locked files in json output

At the moment, there is only information about the byte-range locked files.
Adds a list of its locks for each file. An open is represented as
a dictionary. Contains all information (pid, dev_inode, read_write, start
and size) about the lock.

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add server_id to byte-range locks

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add a basic byte-range locks dictionary

Adds an empty json dictionary under the key "byte_range_locks"
and adds foreach locked file a dictionary with information
(path and filename) to the byte-range locks dictionary.

Only print to stdout, if json_output is not set.

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add service path to byte-range locks

The service/share path was added in 39ddd0a520bc and removed in
3a7d372e2eb5.

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add machine readable time info to locked files

Time has the format "2022-03-31T12:23:30+0200".

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add general caching information about open files to json output

Adds a dictionary named "caching" to a opens dictionary.
Represents both oplock and leases caching. The dictionary contains a
boolean for each type (READE, WRITE and HANDLE), the hex value and a string
representation.
If no oplocks are used, the dictionary is left empty.

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add sharemode information about open files to json output

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add server_id to open files dictionary

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add lease information about open files to json output

Adds a dictionary named "lease" to a opens dictionary.
If leases are used, the dictionary contains a boolean for each type
(READE, WRITE and HANDLE or UNKNOWN) and a string representation of
the lease. Otherwise the dict is left empty.

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add oplock information about open files to json output

Adds a dictionary named "oplock" to a opens dictionary.
Contains a string representation and booleans for each oplock type
(EXCLUSIVE, BATCH, LEVEL_II, LEASE).

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add access mode information about open files to json output

Adds a dictionary named "access_mask" to a open dictionary.
Contains the hex value of the mask, a text representation and booleans
for each attribute.

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add opens to files in json output

At the moment, there is only information about the open files.
Adds a list of its opens for each file. An open is represented as
a dictionary. Contains only the basic information (pid, uid and
time) about the open.

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add file_id information about open files to json output

Adds a dictionary with file_id information (devid, inode and extid) for
each locked file.

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add a basic dictionary with open files

Adds an empty json dictionary under the key "open_files" and adds foreach
locked file a dictionary with information (path, filename and pending
deletes) to the locked files dictionary. Uses path and filename as key.

Only print to stdout, if json_output is not set.

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add encryption and signing to sessions

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add server_id to sessions

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add a sessions dictionary

Adds an empty json dictionary under the key "sessions" and adds foreach
session a dictionary with information to the session dictionary. Uses the
session_id as key.
uid_str and gid_str are needed because both receive their own JSON field.

Only print to stdout, if json_output is not set.

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add encryption and signing to connections

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add machine readable time to connections

Time has the format "2022-03-31T12:23:30+0200".

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

conn_tdb: change type of connections_data.start to NTTIME

connections_data.start previously had the type time_t, but time_t
only had a precision for seconds. NTTIME has a higer precision,
which is useful for debugging.

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add session_id to connections dictionary

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

conn_tdb: add sess_id to struct connections_data

Save the session global id as sess_id in the connections_data struct.

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add server_id to connections

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbstatus: add a connections dictionary

Adds an empty json dictionary under the key "tcons" and adds foreach
connection a dictionary with information to the shares dictionary.

Only print to stdout, if json_output is not set.

Signed-off-by: Jule Anger <janger@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>