s3: smbd: cleanup. Change 'int referral_count' -> 'size_t referral_count' in struct junction_map.

This is a non-negative count. Fix remaing code to not mix int and size_t.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: smbd: msdfs: Cleanup, don't mix int and size_t types for a count variable.

Add integer wrap check.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib/replace: work around an API conflict between ncurses and XFS xattr API

Compile error:

  [4530/4693] Compiling source3/utils/regedit_list.c
  In file included from ../../source3/utils/regedit_list.h:24,
                   from ../../source3/utils/regedit_list.c:20:
  /usr/include/curses.h:611:28: error: conflicting types for ‘attr_get’
    611 | extern NCURSES_EXPORT(int) attr_get (attr_t *, NCURSES_PAIRS_T *, void *); /* generated */
        |                            ^~~~~~~~
  compilation terminated due to -Wfatal-errors.

Both ncurses and XFS xattr API provide a get_attr() function. As a workaround
avoid including <sys|attr/attributes.h> if <attr|sys/xattr.h> is present.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Dec 12 20:22:51 UTC 2019 on sn-devel-184

libndr: Return enum ndr_err_code from ndr_{pull,push}_steal_switch_value()

This breaks the ABI so we merge this into the unreleased libndr-1.0.0.

The advantage of the new functions is there (except for print, which
is unchanged) is an error raised when the token is not found, so
we can be confident in the changes to the token behaviour.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Dec 12 03:56:23 UTC 2019 on sn-devel-184

librpc/ndr: Remove ndr_{push,pull}_get_switch_value()

By removing this we know we do not need to worry about this list
growing without bounds.  We merge this into the recently created but
not yet released ABI 1.0.0

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

librpc: Remove last callers of ndr_push_get_switch_value()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

librpc: Remove last callers of ndr_pull_get_switch_value()

By removing this we know we do not need to worry about this list
growing without bounds.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

librpc: Do not follow a NULL pointer when calculating the size of a union

Found by Douglas Bagnall using Hongfuzz and the new fuzz_ndr_X
fuzzer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

pidl: Mismatch between set and get of relative base pointers

The set was within the switch, the get was before the switch.

The difference is shown when there is an empty default element.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

librpc: Do not follow a NULL pointer when calculating the size of a structure

Found by Douglas Bagnall using Hongfuzz and the new fuzz_ndr_X
fuzzer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

pidl: Mismatch between set and get of relative base pointers

The set was within the switch, the get was before the switch.

The difference is shown when there is an empty default element.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

pidl: Add and use ndr_print_steal_switch_value(), removing ndr_print_get_switch_value()

This avoids really long token lists for switch values
that will not be needed past this point.

The function name is changed to clarify what exactly is being
done here, and the old function is removed to ensure it is
not being used anywhere else.

Merge the removal of ndr_print_get_switch_value into
just-tagged librpc/ABI/ndr-1.0.0.sigs as this
has not been put into any release yet.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

librpc/ndr: Add ndr_push_steal_switch_value()

This will allow generated code to instead push and pop union values onto the
switch_list stack, which is more memory efficient than creating a single large
list to be scannned and eventually discarded.

Merge into unreleased ABI 1.0.0

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

pidl: Generate and consume the switch level token for both NDR_SCALARS and NDR_BUFFERS in ndr_pull()

This means what was previously a list becomes a single variable that
could be passed as a function paraemter, but this is avoided for now
because it would change the ABI and be more intrusive.

Before this, a client could cause a NDR token containing the swith level
to be allocated for each and every element in the array that they
promised they were sending (without having to actually send them).

Found by Michael Hanselmann using Honggfuzz and an fuzzer for Samba's
NDR layer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

librpc: Set the switch_value before NDR_BUFFERS to prepare for new libndr behaviour

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

negoex: Set the switch_value before NDR_BUFFERS to prepare for new libndr behaviour

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

s4-libcli/rap: Set the switch_value before NDR_BUFFERS to prepare for new libndr behaviour

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

ndr: Restrict size of ndr_token lists to avoid memory abuse by malicious clients

This is designed to stop a very large number of tokens from being stored for
arrays of structures containing relative pointers in particular.

This was one part of the minimum patch for CVE-2019-14908 before
being downgraded as not a security-release worthy issue.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

libndr: Do not overwrite token list with NULL on allocation failure

This was one part of the minimum patch for CVE-2019-14908 before
being downgraded as not a security-release worthy issue.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

selftest: Add test for ndr_size_struct() faulting on a NULL pointer

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

selftest: Add test for ndr_size_union() faulting on a NULL pointer

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

selftest: Add example xattr_NTACL packets to demonstrate switch/union behaviour

This is a good example with both buffers and scalars in the union.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

selftest: Add test for structure with NDR_BUFFERS only in a union

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

selftest: Do not force the endpoint for fsrvp tests

The test suite will bind to the srvsvc interface, let it find the
correct endpoint through the endpoint mapper.

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Dec 12 02:00:19 UTC 2019 on sn-devel-184

s4:torture/rpc: Fix torture comment in mdssvc.c

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pidl:NDR/Server: Allow to define endpoint server shutdown functions

The next commits will register legacy api_struct when the endpoint server
is initialized. This commit adds a shutdown function which will be used
to unregister the legacy api_struct.

The shutdown function will be also used to replace the rpc_srv_callbacks
struct shutdown member used, for example, by the spoolss service to
cleanup before exiting.

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc:core: Add a function to reinitialize the dcesrv_context

Clears all registered endpoints and interfaces, association groups and
broken connections.

To be used by S3 forked daemons.

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc:core: Add public functions to initialize endpoint servers

The dcesrv_init_registered_ep_servers() will be used by the S3 server to
initialize all registered endpoint servers (for embedded services), and
the dcesrv_init_ep_server() function will be used by the external
daemons to initialize the required ones.

As serveral S3 services may require to initialize another one before
itself (svcctl and eventlog for example require winreg) a boolean flag is
added to track the initialization status.

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc:core: Split dcesrv context init and endpoint servers init

The S4 server will initialize the endpoint servers specified in smb.conf,
but the S3 server need to initialize all registered endpoint servers (the
embedded ones).

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc/idl/dnsserver.idl: Ensure DnsProperty id matches what is pulled from the stored buffer

There are two concerns here, assuming the attacker can place arbitary values
in a dnsProperty attribute over LDAP (eg is a DNS administrator).

This comes from the fact that id is used as the switch value at the C layer
but at the NDR layer the wDataLength value is considered first.

One concern is that a pull/push round-trip could include server memory:

 The previous switch_is() behaviour could store the server memory back
 into the attribute.

 However this pattern of pull/push only happens in ndrdump and fuzzing tools, as
 dnsserver_db_do_reset_dword() operates only on the uint32/bitmap union
 arms, and fully initialises those.

The other is that a pull of the attacker-supplied value could
cause the server to expose memory.

 This would be over the network via DNS or the RPC dnsserver protocols.
 However at all times the ndr_pull_struct_blob is passed zeroed memory.

The final concern (which fuzz_ndr_X found) is that in the ndr_size_dnsPropertyData()
the union descriminent is only id.

 This has no impact as only zeroed memory is used so there will be a
 zero value in all scalars, including data->d_ns_servers.AddrArray.

 Therefore the server will not crash processing the attacker-supplied blob

[MS-DNSP] 2.3.2.1 dnsProperty has no mention of this special behaviour.
https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-dnsp/445c7843-e4a1-4222-8c0f-630c230a4c80

This was known as CVE-2019-14908 before being triaged back to a normal bug.

Found by Douglas Bagnall using Hongfuzz and the new fuzz_ndr_X fuzzer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14206
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>

librpc dnsp test: Ensure length matches union selector

Ensure that a dnsp_DnsProperty is rejected if the length data does not not
correspond to the length indicated by the union id.  It was possible for
the union to be referencing memory past the end of the structure.

Found by Douglas Bagnall using Hongfuzz and the new fuzz_ndr_X fuzzer.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14206
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

lib/krb5_wrap: Remove unused smb_krb5_get_allowed_weak_crypto()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Isaac Boukris <iboukris@samba.org>
Autobuild-User(master): Isaac Boukris <iboukris@samba.org>
Autobuild-Date(master): Wed Dec 11 22:18:47 UTC 2019 on sn-devel-184

lib/fuzzing: Fix argument order to ldb_filter_from_tree in fuzz_ldb_parse_tree

Found by the oss-fuzz CI tooling.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Dec 11 04:21:28 UTC 2019 on sn-devel-184

lib/fuzzing: Split up automatically build fuzzers into TYPE_{IN,OUT,STRUCT}

The advise is that a fuzz target should be as small as possible
so we split this up.  Splitting up by function would build too
many fuzzers, but this should help a little.

See for example:
https://github.com/google/fuzzing/blob/master/docs/good-fuzz-target.md#large-apis

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>

lib/fuzzing: Ensure mem_ctx is freed each time fuzz_ldb_parse_tree is run

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>

autobuild: extend autobuild with samba-fuzz job to build the fuzzers in AFL mode using oss-fuzz scripts

This helps ensure the build_samba.sh file keeps working and the fuzzers build
(because they are excluded from the main build).

This is not in the default autobuild because it uses too much
space on sn-devel (4GB).

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>

autobuild.py: Avoid listing jobs twice

We use the tasks table instead, to avoid the issue shown in the previous commit.

Now we just have to keep .gitlab-ci.yml and the tasks table in sync.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>

autobuild.py: Add missing samba-simpleserver job

This was missed when the job was split out in f0e8dd1a08698884209873bb84002d7b34db016c.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>

.gitlab-ci.yml: Align tasks with "pages" dependency to get comprehensive code coverage

These two lists can get out of skew very easily.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>

smbd: Fix a leases.tdb record leak

If we set e->stale=true in the share_mode_forall_entries() callback,
the share entry will be removed directly. Thus further down
share_mode_forall_leases() won't find anything anymore. Only find
possibly still connected entries in the first walk, and then remove
the share_entries.tdb record straight away after the leases and
brlocks have been removed.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Dec 10 21:57:05 UTC 2019 on sn-devel-184

torture: Run durable_v2_reconnect_delay_msec with leases

This will show a leases.tdb record leak. If you SIGSTOP the smbtorture
process while it's in the 10-second wait, you will find locking.tdb
and share_entries.tdb empty after the scavenger has cleaned up. But
there will be an entry in leases.tdb left.

I have no clue how to test this properly, or how to have a reasonably
cheap assert in smbd during normal operations. The problem is that
this leak can't really be distinguished from a "normal" leak that a
crashed smbd would leave behind. Possibly we need a background job
walking leases.tdb to clean this up properly.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

torture4: Use generate_random_u64() instead of random()

random() returns an int, which is not necessarily a uint64

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib:crypto: Build intel aes-ni only if GnuTLS doesn't provide AES CMAC

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Dec 10 20:30:57 UTC 2019 on sn-devel-184

lib:crypto: Only build AES code if we need AES CMAC

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

lib:crypto: Remove our implementation of AES GCM

We require GnuTLS >= 3.4.7 which provides AES GCM.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

lib:crypto: Remove our implementation of AES CCM

We require GnuTLS >= 3.4.7 which provides AES CCM.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pidl:NDR/Parser: only include structs in ndr_interface_public_struct

We only have ndrdump and the fuzzers set up for structures, not BITMAPS,
ENUMS etc.

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Dec 10 17:45:46 UTC 2019 on sn-devel-184

selftest: Confirm that ndrdump struct mode is not available for enums

These are not passed by pointer so the structure dump system does not work
for these.  It is best to dump the containing structure instead.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

CVE-2019-14870: mit-kdc: enforce delegation_not_allowed flag

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Autobuild-User(master): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(master): Tue Dec 10 10:44:01 UTC 2019 on sn-devel-184

CVE-2019-14870: heimdal: enforce delegation_not_allowed in S4U2Self

Signed-off-by: Isaac Boukris <iboukris@gmail.com>

CVE-2019-14870: heimdal: add S4U test for delegation_not_allowed

Signed-off-by: Isaac Boukris <iboukris@gmail.com>

samba-tool: add user-sensitive command to set not-delegated flag

Signed-off-by: Isaac Boukris <iboukris@gmail.com>

CVE-2019-14861: Test to demonstrate the bug

This test does not fail every time, but when it does it casues a segfault which
takes out the rpc_server master process, as this hosts the dnsserver pipe.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14138

Signed-off-by: Andrew Bartlett <abartlet@samba.org>

CVE-2019-14861: s4-rpc/dnsserver: Avoid crash in ldb_qsort() via dcesrv_DnssrvEnumRecords)

dns_name_compare() had logic to put @ and the top record in the tree being
enumerated first, but if a domain had both then this would break the
older qsort() implementation in ldb_qsort() and cause a read of memory
before the base pointer.

By removing this special case (not required as the base pointer
is already seperatly located, no matter were it is in the
returned records) the crash is avoided.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14138

Signed-off-by: Andrew Bartlett <abartlet@samba.org>

CVE-2019-14861: s4-rpc_server: Remove special case for @ in dns_build_tree()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14138

Signed-off-by: Andrew Bartlett <abartlet@samba.org>

CVE-2019-14861: s4-rpc/dnsserver: Confirm sort behaviour in dcesrv_DnssrvEnumRecords

The sort behaviour for child records is not correct in Samba so
we add a flapping entry.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14138

Signed-off-by: Andrew Bartlett <abartlet@samba.org>

bootstrap: Add chrpath as a required package

This is used to test build.sh, part of the oss-fuzz integration, and so also that we
correctly build our fuzzers.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Dec 10 09:15:43 UTC 2019 on sn-devel-184

build: Skip build of python bindings when in fuzzing mode

This will just save a bit of time and space.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

lib/fuzzing Truncate the original files after RUNPATH manipulation in build.sh

This saves space on the rackspace runners in particular.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

lib/fuzzing Add comments to explain RUNPATH manipulation in build.sh

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

lib/fuzzing: Support an oss-fuzz build with either address or undefined behaviour sanitizers

Add handler for $SANITIZER in build.sh

This allows a build with the undefined behaviour sanitizer.

Otherwise we fail the oss-fuzz CI because the UBSan build links with ASan.

Once this in in then https://github.com/google/oss-fuzz/pull/3094
can be merged to oss-fuzz.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

lib/fuzzing: Remove oss-fuzz build.sh stub from the Samba repo

We need to ship the stub build.sh in the oss-fuzz repo, not ours.
This is because otherwise the travis CI checks skip the build
(it thinks we are not set up yet, or have been disabled).

See https://github.com/google/oss-fuzz/pull/3094 for the PR
creating a similar file there.  This is very similar to how
janus-gateway operates, so this is an accepted pattern.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

oss-fuzz: Align build.sh sh parameters with pattern from the oss-fuzz project

We should run build_samba.sh with -eux to ensure we exit on failure,
refuse to use an unset varible and print the commands we are running.

(The suggested build.sh on the oss-fuzz side uses -eu).

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

fuzz/decode_ndr_X_crash: -f to filter crashes by regex

If you go:

$ ./lib/fuzzing/decode_ndr_X_crash -H HONGGFUZZ_REPORT.txt -f 'SIG[^V]' > ./crash.sh

you will get all the crashes and not the timeouts (which have SIGVTALARM).

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

lib/fuzzing: Add mode for the AFL fuzzer

This is helpful for ensuring the fuzzers still compile in autobuild as no
library support is required.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

lib/fuzzing: Also confirm we can make a string filter from the parsed tree in fuzz_ldb_parse_tree

This also avoids tree being an unused variable.

This is similar to doing an ndr_push() in ndr_fuzz_X, it
catches some of the cases where the parse is successful but
the application code could misinterpret the structure.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

lib/fuzzing: Tell the compiler we know we are ignoring errors in fuzz_reg_parse

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

lib/fuzzer: Allow building a fuzz binary for just one interface

This helps direct the fuzzer at a particular function that we are concerned about.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

lib/fuzzer: Remove rudundent install=False flag from fuzz_ndr_X build rule

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

lib/fuzzing: Link only the required NDR_ subsystems into ndr_fuzz_X binaries

This reduces the binary size and shows that we are linked against the correct
ndr_table_ global variable.  This might help the fuzzing engine know there
is not much more of the binary to find if unreachable code is not included.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

librpc: Fill in full deps for NDR_* subsystems

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

fuzzing/decode_ndr_X: read crashes from a HONGGFUZZ report

In theory, you should be able to run honggfuzz and go

$ lib/fuzzing/decode_ndr_X_crash -H HONGGFUZZ-REPORT.txt > crash-crash-crash.sh

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

lib/fuzzing/decode_ndr_X: print less by default, avoid pipe

ndrdump can now take base64 input directly.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

fuzzing: Add script decode_ndr_X_crash to decode crash results

This interprets a file that crashes an fuzz_ndr_X binary

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dcerpc: developer option to save ndr_fuzz_X seeds

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

lib/fuzzing: add fuzz_ndr_X

This NDR fuzzer links with each "interface" in the IDL files to
create avsingle binary.  This tries to matches what the fuzzing
engines desire.

It started as a copy of ndrdump but very little of that remains
in place.

The fancy build rules try to avoid needing a lof of boilerplate
in the wscript_build files and ensure new fuzzers are generated
and run when new IDL is added automatically.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pidl/tests/samba-ndr.pl: remove duplicate import

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Dec 10 04:16:31 UTC 2019 on sn-devel-184

pidl s4::Python: silence warnings

- do not redeclare variables in the same scope.
- use $1 instead of \1, which perl just prefers.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pidl Parse::Pidl::NDR: silence two warnings about undefined strings

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pidl Parse::Pidl::NDR: add HRESULT alignment

this is a guess

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pidl Parse::Pidl::NDR: warn of unknown scalar alignments

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pidl s4::NDR::Parser: silence a warning

At level 0 there is no previous level so $pl is undefined thus so is ->{TYPE}

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pidl s4::NDR::Parser: correct has_fast_array logic

Here we fix two bugs that cancelled each other out completely, so this
patch leaves us with exactly the same functionally as before.

Bug 1: In perl, return is *syntactically* a function.

That means 'return X or Y' is read as 'return(X) or Y', as in the
'open(X) or die "..."' construct -- Y is only evaluated if return
returns false. But return never returns, so Y is dead code. If in
doubt, try these:

perl -e "sub x {return 0 or die;} x"
perl -e "sub x {return (0 or die);} x"

What we *meant* here is 'return (X or Y)', BUT it turns out we were
confused -- the Y case was bogus.

Bug 2: string arrays never had "fast array logic" in the first place.

The fast array logic is for arrays of bytes, and can be fast (i.e.
memcpy) because there is no endianness to worry about. A string array
is an array of pointers not bytes.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pidl s4::NDR::Parser: read hex numbers as numbers for ranges

Hex numbers in IDL are not parsed as numbers, resulting in warnings
like

Argument 0x2000 isn't numeric in numeric lt (<) at /home/douglas/src/samba/pidl/lib/Parse/Pidl/Samba4/NDR/Parser.pm line 981

not to mention problematic code. We add a utility function to convert
these numbers to numbers.

A typical difference this makes is:

 --- old/default/librpc/gen_ndr/ndr_dcerpc.c     2019-11-30 23:40:32.915816967 +1300
 +++ new/default/librpc/gen_ndr/ndr_dcerpc.c     2019-11-30 17:00:09.055733660 +1300
 @@ -1893,7 +1893,7 @@
         if (ndr_flags & NDR_SCALARS) {
                 NDR_CHECK(ndr_pull_align(ndr, 4));
                 NDR_CHECK(ndr_pull_uint32(ndr, NDR_SCALARS, &r->ReceiveWindowSize));
 -               if (r->ReceiveWindowSize > 0x40000) {
 +               if (r->ReceiveWindowSize < 8192 || r->ReceiveWindowSize > 262144) {
                         return ndr_pull_error(ndr, NDR_ERR_RANGE, "value out of range");
                 }
                 NDR_CHECK(ndr_pull_trailer_align(ndr, 4));

Where the minimum ("0x2000" == 8192) was read as a string, thus
treated as zero.

The treatment as zero was introduced in 142b2a61f8a77b3065ce4c78b459ab714d6d190a
accidentially, which shows why warnings are important.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pidl: use perl warnings

Warnings are good. If we turn on warnings with 'use warnings', we will
see bugs that have lain latent for years.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc/ndr: Do not follow a client-supplied NULL pointer in _print_drsuapi_DsAttributeValue_attid()

This is not a security issue as it only happens when printing the structure
during debugging, not normal production.

Found by Michael Hanselmann using an NDR fuzzer and Hongfuzz.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

selftest: Add ndrdump tests for malformed drsuapi traffic

Thanks to Douglas Bagnall for the samples, produced from seeds
generated by Samba's make test traffic, fuzzed by ndr_fuzz_X
and Hongfuzz.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

pidl: Remove Parse/Yapp/Driver.pm

This file is provided by Parse::Yapp and on install we overwrite the
orignal file.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Dec 10 01:54:02 UTC 2019 on sn-devel-184

smbdes: remove old unused DES builtin-crypto

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

sess_crypt_blob can only crypt blobs whose size divides by 8

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

session: convert sess_crypt_blob to use gnutls

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

smbdes: convert des_crypt112_16 to use gnutls

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

smbdes: convert des_crypt112 to use gnutls

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

smbdes: convert E_old_pw_hash to use gnutls

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

smbdes: convert des_crypt128() to use gnutls

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

smbdes: convert E_P24() and SMBOWFencrypt to use gnutls

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

smbdes: remove D_P16() (not used)

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

smbdes: convert E_P16() to use gnutls

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

smbdes: convert sam_rid_crypt() to use gnutls

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

SMBsesskeygen_lm_sess_key: use gnutls and return NTSTATUS

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>