Update NEWS

Signed-off-by: Martin Schwenke <martin@meltin.net>

readonly: Do not use hard-coded value for readonly revoke timeout

In case of control timeouts, readonly revoke code currently aborts.  This
needs to be fixed.  Meanwhile, using control_timeout instead of 5 seconds,
increases the timeout to 60 seconds.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Mar 31 07:20:48 CEST 2014 on sn-devel-104

(Imported from commit 7667da659042d82d26bfdcbd79588a6b8e4e3edf)

tests: Do not mix bool and int data types

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Mar 28 07:56:18 CET 2014 on sn-devel-104

(Imported from commit 533ad246443bbea5116c06fb6478453ddf004080)

tests: Use ctdb_fetch_lock instead of ctdb_fetch_lock_readonly

This will test that ctdb_fetch_lock correctly revokes readonly
delegations.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(Imported from commit 30f7d7db8c0bf370464238133f75b5e0bdce37c6)

client: ctdb_fetch_lock should check for readonly delegations

When readonly delegations were added, ctdb_fetch_lock code should have
been modified to include the check for readonly flags.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(Imported from commit 78015320b60b0fd0d8c3dc65fbbe3e38e4a02993)

tests: Fix and extend read-only records test

This test currently counts the number of read-only-enabled databases
and expects there to only be 1.  It fails when there are existing
databases with read-only already enabled.  Instead, check just the
test database.

Clean up the test by adding some functions to check for precisely the
read-only flags that should be set on a node after each operation.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit aa7cd51bbc438555552584a7d71f5bae6909603f)

tests: Add a new NFS tickle test

This one ensures that a newly started node gets an up-to-date tickle
list.  Tweak some of the integration test functions to accommodate
this.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 234f8eb5712c38872444c5dd7a258903b389b062)

tests/integration: Decentralise the daemon restart code

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
(Imported from commit f67a1043b381f623ec9c97ac137568ca7ab6cd8a)

tests/integration: Update daemons shutdown pseudo-test

Don't scatter the TEST_LOCAL_DAEMONS logic around the code.  Limit it
to the local daemons file.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
(Imported from commit f05db5edc434da61bc7cd7233c5b9cb61cd888bf)

tests/simple: Local daemons version of setup_ctdb() overrides

setup_ctdb() doesn't need to do anything on a cluster.  To avoid a
conditional, just override it for local daemons.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
(Imported from commit 6b15fe247c4c026064a8f412011d484d3b74bfc7)

tests/integration: Remove some unused functions

This was the start of some refactorisation that was never completed.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
(Imported from commit 85a711f5a8e9aaea42595cc53998a250b0ff6469)

tests/simple: Move the local daemons code to its own file

This is just a straight move.  The clever stuff will follow.  :-)

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
(Imported from commit 28acce28626d17c27af831240a5ea593e0410616)

tests/integration: Be more careful when killing ctdbd

Also match $TEST_VAR_DIR in the socket name.  This means that we'll
only ever kill ctdbd process belong to our own test run.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
(Imported from commit 0ae3d009f36066c794e6f39c6dd45735e706df34)

tests: Add NAT gateway eventscript unit tests for static routes

Signed-off-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed Mar 26 06:24:01 CET 2014 on sn-devel-104

(Imported from commit 3d5b80f26ae65ce43ea25dd94274ec865f1c4026)

eventscripts: CTDB_NATGW_STATIC_ROUTES can specify gateways

Extend CTDB_NATGW_STATIC_ROUTES so that each network can have an
optional gateway that overrides CTDB_NATGW_DEFAULT_GATEWAY.

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit 70bbbbe448e96019c79832ac89563eefc25b1bae)

eventscripts: New configuration variable CTDB_NATGW_STATIC_ROUTES

This can be used to create more specific NATGW routes than the usual
NATGW default route.

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit 34682affe9f19432851d5f921b58e692e5c6274d)

eventscripts: Clarify that CTDB_NATGW_DEFAULT_GATEWAY is optional

This has been implied since the command to add the route has had
errors redirected to /dev/null.  If infrastucture (e.g. ADS, DNS) is
on the same network as CTDB_NATGW_PUBLIC_IP then no route is
necessary.

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit 7705efc35551a083873e2fed91ac42d021669df9)

eventscripts: Improve check in NATGW "startup" event

Although the dots in $CTDB_NATGW_PUBLIC_IP could probably only help
match an invalid public IP address, this is only executed once so do
as exact a check as possible.

Use CTDB_BASE instead of hardcoding /etc/ctdb.

Make the error message less redundant.

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit 8a3be1f1a979b8058cf7754632d60ab52d066d5c)

eventscripts: Reformat natgw_clear()

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit e22a22b1f72f75507941b5652bb9a0a2f42dd22f)

eventscripts: Rename some NAT gateway functions

delete_all() really needed renaming for clarity.  While doing this,
might as well rename some of the others that don't start with
"natgw_".

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit 3c839c60d15f11c538709c3e6b85d888131d6e30)

tests: Add a test for NAT gateway sanity checking

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit c0e239473b532cebd31ef5e53d8cbf23c262c18d)

eventscripts: Sanity check NAT gateway configuration

NAT gateway really can't operate unless most of the configuration
variables are set.

A check in delete_all() can be removed - strange that this isn't also
done in the add case.

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit 4ee4925d416a86341bd76c11fa99ec9173682a1d)

eventscripts: Improve readability of NAT gateway update code

Put the code into a couple of usefully named functions.

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit 0953f5799c8545d9afee27d2350f344f2480df02)

tests: Add some tests for 11.natgw eventscript

This includes adding support for:

* Configuring fake NATGW state in the eventscript unit tests

* "natgwlist" and "setnatgwstate" in ctdb command stub

* ip command stub to default to "main table" when no table specified,
  allow routes to be added without "dev" option (just add a default
  dev), support "metric" option

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit 7fdd6b7861a4767cd3e2dfa76a7d7e7790efd840)

eventscripts: Use set_proc() to update /proc

In case we want to write some unit tests in the future.

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit feeb9843bf46d00beb10f3e959ba0f73f5e33e8c)

Revert "eventscripts: Use set_proc() to update /proc"

This reverts commit 6587dc06cc5b697289af3888cd94a5c6d1e5ce8e.

Wrong imported commit information.

Revert "tests: Add some tests for 11.natgw eventscript"

This reverts commit 62268bf277dad3d79e33bc88d51dffc53a3a873d.

Wrong imported commit information.

Revert "eventscripts: Improve readability of NAT gateway update code"

This reverts commit e046b625e0f1461bfa84972f0b981111fce725bb.

Wrong imported commit information.

Revert "eventscripts: Sanity check NAT gateway configuration"

This reverts commit a15fbcc26c9b1753a525694c318d8f77c0f38dfa.

Wrong imported commit information.

Revert "tests: Add a test for NAT gateway sanity checking"

This reverts commit f4bd0dca4ad63268203ef123bcd1287a38583844.

Wrong imported commit information.

Revert "eventscripts: Rename some NAT gateway functions"

This reverts commit 8909030c2559b0e627db83937580d2fbf1227247.

Wrong imported commit information.

Revert "eventscripts: Reformat natgw_clear()"

This reverts commit 41b630097d0cabfe32bd2b3cf9cbcae9aeff5c5f.

Wrong imported commit information.

Revert "eventscripts: Improve check in NATGW "startup" event"

This reverts commit 2d616266a221e54019eaf09ff5276c2c455bbf57.

Wrong imported commit information.

Revert "eventscripts: Clarify that CTDB_NATGW_DEFAULT_GATEWAY is optional"

This reverts commit a4ff3c748d253162289f95c3fe94e935f8acc48f.

Wrong imported commit information.

Revert "eventscripts: New configuration variable CTDB_NATGW_STATIC_ROUTES"

This reverts commit ceea3beb65e524bc864ea17940bb846b28b302e3.

Wrong imported commit information.

Revert "eventscripts: CTDB_NATGW_STATIC_ROUTES can specify gateways"

This reverts commit 12ec379d6cfd58dc2416f9e212973ef04a2326e6.

Wrong imported commit information.

Revert "tests: Add NAT gateway eventscript unit tests for static routes"

This reverts commit ce7e1cecb831e9eb1c994aa4b86ce13299af436f.

Wrong import commit information.

locking: Instead of comparing key, compare key hash

This replaces memory comparison of the key with integer comparison.
In addition, this also avoids scheduling locks with the same hash.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Mar 28 05:28:58 CET 2014 on sn-devel-104

(Imported from commit f99a759ce7cb5492442c018f976619f0ba439a43)

locking: Check for talloc_memdup error

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(Imported from commit d1c8f8d63506c099e2ee9be7cb91f0fac4a8d848)

locking: Avoid memory leak

If lock_request could not be allocated, free lock_ctx since there can
only be a single lock_request per lock_ctx.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(Imported from commit ff1c85ae279a87d9ce23955b02a68814f1f846f1)

tests: Add NAT gateway eventscript unit tests for static routes

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit 2b83bec1041995478e613ebefb1035ff91e1b4e2)

eventscripts: CTDB_NATGW_STATIC_ROUTES can specify gateways

Extend CTDB_NATGW_STATIC_ROUTES so that each network can have an
optional gateway that overrides CTDB_NATGW_DEFAULT_GATEWAY.

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit a8fa3e780f6cd9505867cbd073c2333653bf4d32)

eventscripts: New configuration variable CTDB_NATGW_STATIC_ROUTES

This can be used to create more specific NATGW routes than the usual
NATGW default route.

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit 3a623223153657acf8ea1ab38652f729b204b918)

eventscripts: Clarify that CTDB_NATGW_DEFAULT_GATEWAY is optional

This has been implied since the command to add the route has had
errors redirected to /dev/null.  If infrastucture (e.g. ADS, DNS) is
on the same network as CTDB_NATGW_PUBLIC_IP then no route is
necessary.

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit d88ace388e7a5a9e04d0d0c7f3e55edc16d8771b)

eventscripts: Improve check in NATGW "startup" event

Although the dots in $CTDB_NATGW_PUBLIC_IP could probably only help
match an invalid public IP address, this is only executed once so do
as exact a check as possible.

Use CTDB_BASE instead of hardcoding /etc/ctdb.

Make the error message less redundant.

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit 1615c21fe0bbbae03f2e8e15c3527cd4bd3b2bf5)

eventscripts: Reformat natgw_clear()

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit 996b639ea76d0e5d3b2a4eaa1279e5efdd24ca8f)

eventscripts: Rename some NAT gateway functions

delete_all() really needed renaming for clarity.  While doing this,
might as well rename some of the others that don't start with
"natgw_".

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit 9ef20edcbe7331138cd4b2eb1aafc007c682a296)

tests: Add a test for NAT gateway sanity checking

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit 7c0b7f699900aff69723c1f27341946752ba2bb4)

eventscripts: Sanity check NAT gateway configuration

NAT gateway really can't operate unless most of the configuration
variables are set.

A check in delete_all() can be removed - strange that this isn't also
done in the add case.

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit d4f333f5e6186d88f436b0a747f500f3f32ed9bd)

eventscripts: Improve readability of NAT gateway update code

Put the code into a couple of usefully named functions.

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit a7b2794f7bbf601351cef1e18bfc8c05183b35de)

tests: Add some tests for 11.natgw eventscript

This includes adding support for:

* Configuring fake NATGW state in the eventscript unit tests

* "natgwlist" and "setnatgwstate" in ctdb command stub

* ip command stub to default to "main table" when no table specified,
  allow routes to be added without "dev" option (just add a default
  dev), support "metric" option

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit bfb615cfec4ebbdb75ebebc67087e0b43df386e7)

eventscripts: Use set_proc() to update /proc

In case we want to write some unit tests in the future.

Signed-off-by: Martin Schwenke <martin@meltin.net>
(Imported from commit 5519a05766ca9e0a8ace39c49661b9ad52440cac)

daemon: Optimise deletion of IPs

Previous commits maintained the ordering between
ctdb_remove_orphaned_ifaces() and ctdb_vnn_unassign_iface().  This
meant that ctdb_remove_orphaned_ifaces() needed to steal the orphaned
interfaces and they would be freed later.

Unassign the interface first and things get simpler.
ctdb_remove_orphaned_ifaces() is now self-contained.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Sun Mar 23 06:20:43 CET 2014 on sn-devel-104

(Imported from commit 20c719677a28afa1d1b912b9fadbf384e9e65de7)

daemon: Make delete IP wait until the IP is released

reloadips really expects deleted IPs to be released before completing.
Otherwise the recovery daemon starts failing the local IP check.  The
races that follow can cause a node to be banned.

To make the error handling simple, do the actual deletion in
release_ip_callback().

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 9b907536fb657fa15c02858caf0ffff633ecd478)

tests: Improve tickle tests

It is hard to diagnose failures in the NFS tickle test because there's
no way of telling if the test node doesn't have the tickle or if it
didn't get propagated.

Factor out check_tickles() into local.bash and give it some
parameters.

Have the NFS test call it first to ensure the tickle has been
registered.  Then use new function check_tickles_all() to ensure the
tickle has been propagated to all nodes.  Give this a bit of extra
time (double the timeout) just in case we're racing with the update.

Add a useful comment to the CIFS test so that I stop asking myself how
the test could ever have worked reliably.  :-)

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>

(Imported from commit bafb9151ccb5722df36f9ba168716f4f4fa01cdc)

daemon: Do not disable monitoring when running eventscripts

This is racy and cbffbb7c2f406fc1d8ebad3c531cc2757232690e makes it
unnecessary.

The eventscript code still knows that monitor events are special
compared to other events.  However, the general concept of monitoring
is no longer tangled up with running scripts.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit ecafbce1b1cf53ae8c3de9eb5201192f7fe1f67d)

eventscripts: Attach to persistent ctdb.tdb in "startup" event

"statd-callout notify" currently complains until an add-client or
del-client is done.

Given that we might use ctdb.tdb for something else in the future it
makes sense attach to it in the "startup" event.  This could be done
in the background but it should be so lightweight that a timeout will
indicate serious problems.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 87d58fd07b1294688b8fc6dbdf3dbb6cb12d3a80)

daemon: Fix tickle updates to recently started nodes

Commit 0723fedcedd4a97870f7b1224945f1587363c9bf added a cheap
implemention of ctdb_control_startup() that simply flags the recipient
node as needing to send updates for each IP when the tickle update
loop next fires.  Commit 026996550d726836091ff5ebd1ebf925bf237bb0
ensures that a node only sends tickle updates once being flagged to do
so.

CTDB_CONTROL_STARTUP is broadcast to all nodes, so this is a good
start.  However, the tickle updates are only broadcast to connected
nodes.  A recently started node may not yet be considered to be
connected because the keepalive monitoring loop may not yet have
marked the node as connected.  This means that the tickle update loop
races with the keepalive monitoring loop.  If the tickle update loop
wins then updates will not be sent to the recently started node.

The simplest improvement is to stop the tickle update from depending
on whether a node is connected or not.  So instead of broadcasting
tickle updates to connected nodes, they are broadcast to all nodes.
Since no reply is expected, this should work just fine.

While looking at this code, ctdb_ctrl_set_tcp_tickles() is named like
a client function.  It isn't a client function.  Also, 2 of the
arguments are ignored.  So rename this function to
ctdb_send_set_tcp_tickles_for_ip() and remove the ignored arguments.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>

(Imported from commit 4f79fa6c7c843502fcdaa2dead534ea3719b9f69)

tools-ctdb: Parse IP addresses when reading a list from a file

This way this logic is centralised.  It also means that the IP address
comparisons in the NAT gateway code are IPv6 safe.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 441e0998370bfd7b0de5dd9aed7e2abbcf64cf73)

tools-ctdb: Remove redundant filtering of trailing empty lines

There is a check for empty lines in the loop just below.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit fad2b6b074495eb1dc036cce293456857985f8f5)

tools-ctdb: Use DLIST_ADD_END() to avoid reversing the list

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 555aa06c41e7f77bf241e04ccf771009645e9c27)

tools-ctdb: Factor out function read_pnn_node_file()

Factor it from read_nodes_file().  Use it there and in
read_natgw_nodes_file().

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 91895b33c52c0a81904c3ea36042d4574422f5fd)

tests: Add "ctdb listnodes" and "ctdb xpnn" stub tests

Tests for xpnn need to implement a stub for ctdb_sys_have_ip().  The
cheapest way of doing this is to read a fake nodemap using the
existing code and check if the IP of the "current" node is the one
being asked about.  However, the fake state initialisation isn't
currently available to without_daemon commands because it is meant to
represent daemon state.  However, it can be made available by moving
the relevant code into a new stub for tevent_context_init().  The stub
still needs to initialise a tevent context - this can be done by
calling a lower level function.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 79d28000043bd463beecaeac47855d3a4970eaf2)

tools-ctdb: Read NAT gateway nodes from a separate function

Now it gets easier to refactor.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 9bede494743f0ce13493fe718ed8f0c3c5f2959c)

tools-ctdb: Add and use function filter_nodemap_by_natgw_nodes()

Add another filter function, like the ones for capabilities and flags
to, for filtering by NAT gateway nodes.  This makes the main
natgw_list function more readable.

Note that this drops the early filtering of disconnected nodes, so
they will now be listed in a NAT gateway group.  This makes sense.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit ba69742ccd822562ca2135d2466e09bf1216644b)

tools-ctdb: Update LVS commands to use filter_nodemap_by_flags()

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit e728a35dc19c397cb17e1bf434401df25c35f337)

tools-ctdb: Update LVS commands to use filter_nodemap_by_capabilities()

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 26c9a591e539e33dd0896ec1e2958192b3e4efd4)

tools-ctdb: Fixes for "lvs" and "lvsmaster" commands

The index of the nodes array in nodemap isn't the PNN.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 5fb7e386ac4452786512d077a00b4907ef39cb51)

tools-ctdb: Generalise find_natgw() -> filter_nodemap_by_flags()

Instead of just finding the first node that doesn't have any flags in
flag_mask set, change it into a function that filters a nodemap to
exclude nodes with the given flags.

This makes the NATGW code simpler but also provides a function that
can be used in other code.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 33b1fcbd7083668cfd58b1cfb1172b6134cd07ca)

tools-ctdb: Update natgwlist to filter nodes by NATGW capability

Check capabilities once to build a filtered node list instead of
repeatedly checking capabilities

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 798bd58370f6ea7bc70db96edd23ae86caf6bf79)

tests: New natgwlist tests where nodes capability not set

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 771005386403acf15a81be5de2a3798384a37d8a)

tests: Update ctdb stub LVS tests and add some new ones

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 75cf99b9da1677fa83197d111d757e14041dae05)

tests: Support fake capabilities in CTDB tool stub

... and add a test to make sure it works.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 3af858e6f4990599c23b54d05c42187400fd1426)

tests: Remove old, unused copy of a CTDB tool unit test

This looks to have got left behind a long time ago when things got
moved around...

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 263e5eabf8d55c7f53db597b8fcede831c211e45)

tools-ctdb: Don't close stderr when running without_daemon commands

It looks like the original without_daemon code still tried to
establish a client connection to the daemon.  Closing stderr looks to
be a cheap way of hiding the errors when this failed.

However, later cleanups avoid the client connection altogether, so do
not close stderr.  Now debug output from without_daemon commands
actually appears.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit a947cf6c0c3e1453ec833033dcd2edaa9490a55b)

pmda: Fix metric identifiers

The commit "pmda: Use upstream assigned PCP domain id" updated the
Performance Metrics Namespace (pmns) file, without changing the
corresponding metric identifiers used by the agent.

This change fixes the agent metric identifier values to match the pmns
definitions.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Mar 22 00:07:31 CET 2014 on sn-devel-104

(Imported from commit 7fdb21cc321bc0b9a759393467fe42f26cdc812a)

vacuum: fix delete list counts in delete_marshall_traverse_first

when bumping skipped, decrement left, so the sum is correct

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Thu Mar  6 03:32:33 CET 2014 on sn-devel-104

(Imported from commit d8e110ed7dacda18860cce0c86e4e44f0b83dd42)

vacuum: fix possible cause for delelete_list processing counts left records > 0

We need to have left records == 0 at the end of the delete list processing.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 5b81848e50b43b7ab7889f5217e05ca42e452c8f)

vacuum: systematize counters into various structs

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 551e9d791c146473b45f8a9fb1574e0ad7cca6b2)

vacuum: remove unused counter vdata->total

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit ec3f33c314207f93529d7c9a2bfb82bf05e3a08b)

vacuum: make ctdb_process_delete_list() void

The overall return code was not really used anyways.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 49eb503c5d3133e1476a15f9d11ce4269407e6c6)

vacuum: make ctdb_process_vacuum_fetch_lists() void.

This constantly returns 0 anyways.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 776d4e88f2a6068016dedd37003cdde10f8090a5)

vacuum: make ctdb_vacuum_traverse_db() void.

Failure in traversal of the DB should not
prevent further processing.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 19948702992c94553e1a611540ad398de9f9d8b9)

vacuum: don't stop in process_vacuum_fetch_lists when sending to one node fails.

We should try to continue vacuuming as much as possible.
Failure to send records to one lmaster doesn't mean the
others will fail too.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 7983946680ac0de8f82dfee6f0f849a11653d042)

vacuum: catch and log errors to traverse the delete list in ctdb_process_delete_list()

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit f3483de240987f05cc20f747ac467c8ed81bb03e)

vacuum: catch and log error of traverse in ctdb_process_delete_queue()

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 72ea9759930bd29b59c0f831d2cb2f4f1e2e643d)

vacuum: use tdb_parse_record instead of tdb_fetch in delete_marshall_traverse_first()

Spare malloc and free.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 83fa09e78c6ca8e08cb2659f013a05b4b340f0aa)

vacuum: simplify delete_marshall_traverse_first: use tdb_null

we know anyways the record to store is empty at this point.
So skip pointer calculations.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 974aa73160d50b7cf63b4a5e6dd7a7b1408ece42)

vacuum: remove VacuumLimit criterion for triggering a repack

With the new vacuuming, we consider it an error if there are
records left for deletion after processing the various lists.
All records that can be deleted should have been deleted by
tdb_delete calls.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 16837bc309aa9a86fc21d7f59a8fce0b947428a3)

vacuum: treat value 0 of tunable VacuumLimit as turning off repacking

I.e. no number of records found to delete will trigger the
repacking.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 6fdd25008f384408dfc103b90ab40b8e64ce18b0)

vacuum: add consistency check for counts at end of process_delete_list()

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 1b2fc1f096f80db5974eab021f12f0ad9af24882)

vacuum: log error if records are left for deletion after ctdb_process_delete_list()

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit cf407d208afdc70b31ce5013591c869b36f588f1)

vacuum: use tdb_parse_record instead of tdb_fetch in delete_record_traverse()

Spare malloc and free.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 23be632449524e74d73fcb6fd3875a6d5a428d89)

vacuum: update comment for ctdb_process_delete_queue

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit be2f1a0c790d08571dd757fb7b2941d367175008)

vacuum: rename ctdb_vacuum_db_fast --> ctdb_process_delete_queue

This describes more precisely what this does.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit cd877b6a21a5080b3d9ae8ee7ac8cf27c4fd9512)

vacuum: update comment for ctdb_vacuum_traverse_db

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit c4478bd40daee22bbe5696bb17d0a1bff164c7f7)

vacuum: rename ctdb_vacuum_db_full --> ctdb_vacuum_traverse_db

This describes more precisely what it actually is, nowadays.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 0309d5eda4f28c7e99e93e0da6c02757ea0cae8f)

vacuum: change full db traverse vacuuming to fill delete queue

This lets the "fast vacuum" delete queue traverse do the actual work.

On the positive side, we note that this lets the "full vacuuming"
treat the records that have never been migrated with data correctly.
These had previously been added to the delete list for complicated
cross-node deletion instead of directly deleting them.

On the other hand side, there might be a slight overhead
since the records are read again in the delete queu traverse,
but this is OK because this change is in preparation of
untangling the db traverse altogether from the vacuum run,
making it independent.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit ba49deb2344c0a9a8f76c9fd0136bdeadad6af89)

vacuum: run the fast vacuum after the db traverse

This in preparation of modifying the db traverse to
fill the delete_queue that is processed by the fast
vacuum run, instead of filling the same lists as the
fast vacuum run for further processing.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit d0b7b3882511769b1bfc1d0d4fdc0dba288e6ccd)

vacuum: rename private->private_data in repack_traverse

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit 04c2115b606f2346fc7315a503b3dae0189e0737)