Amitay Isaacs [Thu, 28 Aug 2014 06:59:52 +0000 (16:59 +1000)]
freeze: simplify code with TALLOC_FREE
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Thu, 28 Aug 2014 06:58:56 +0000 (16:58 +1000)]
freeze: Improve message
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Thu, 28 Aug 2014 05:28:34 +0000 (15:28 +1000)]
freeze: Use individual database locks in freeze priority
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Thu, 28 Aug 2014 05:30:39 +0000 (15:30 +1000)]
freeze: Move destructor function definition down in a file
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Thu, 21 Aug 2014 02:32:02 +0000 (12:32 +1000)]
daemon: Use ctdb_is_frozen() to check if databases are frozen
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Thu, 21 Aug 2014 02:26:58 +0000 (12:26 +1000)]
freeze: Refactor code to check if databases are frozen
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Tue, 5 Aug 2014 04:16:29 +0000 (14:16 +1000)]
daemon: Add controls to freeze/thaw single database
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Tue, 5 Aug 2014 04:42:00 +0000 (14:42 +1000)]
call: Convert pending calls list to per database list
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Tue, 5 Aug 2014 07:13:22 +0000 (17:13 +1000)]
freeze: Use ctdb_db_iterator to commit transaction on databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Tue, 5 Aug 2014 07:02:28 +0000 (17:02 +1000)]
freeze: Use ctdb_db_iterator to start transaction on databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Tue, 5 Aug 2014 07:02:07 +0000 (17:02 +1000)]
freeze: Use ctdb_db_iterator to cancel transaction on databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Tue, 5 Aug 2014 06:23:21 +0000 (16:23 +1000)]
freeze: Use ctdb_db_prio_iterator to cancel transaction on databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Tue, 5 Aug 2014 06:49:06 +0000 (16:49 +1000)]
locking: Add ctdb_db_iterator to iterate through all databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Tue, 5 Aug 2014 06:45:34 +0000 (16:45 +1000)]
locking: Expose ctdb_db_prio_iterator function
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Tue, 5 Aug 2014 06:43:11 +0000 (16:43 +1000)]
locking: Rename ctdb_db_iterator to ctdb_db_prio_iterator
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Tue, 5 Aug 2014 06:37:43 +0000 (16:37 +1000)]
locking: Remove unused priority field from db_handler_t
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Tue, 5 Aug 2014 06:23:47 +0000 (16:23 +1000)]
freeze: Remove commentented test code
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Tue, 5 Aug 2014 05:35:57 +0000 (15:35 +1000)]
freeze: Do an early exit if freeze is pending
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Wed, 30 Jul 2014 04:31:54 +0000 (14:31 +1000)]
daemon: Drop tunable that is no longer in use
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Wed, 30 Jul 2014 02:32:08 +0000 (12:32 +1000)]
recoverd: Fix typo in comment
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Mon, 23 Feb 2015 01:38:11 +0000 (12:38 +1100)]
io: Do not use sys_write to write to client sockets
When sending messages to clients, ctdb checks for EAGAIN error code and
schedules next write in the subsequent event loop. Using sys_write in
these places causes ctdb to loop hard till a client is able to read from
the socket. With real time scheduling, ctdb daemon spins consuming 100%
of CPU trying to write to the client sockets. This can be quite harmful
when running under VMs or machines with single CPU.
This regression was introduced when all read/write calls were replaced to
use sys_read/sys_write wrappers (
c1558adeaa980fb4bd6177d36250ec8262e9b9fe).
The existing code backs off in case of EAGAIN failures and waits for an
event loop to process the write again. This should give ctdb clients
a chance to get scheduled and to process the ctdb socket.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Tue Feb 24 12:29:30 CET 2015 on sn-devel-104
(Imported from commit
04a061e4d19d5bdbd8179fb0fab8b0875eec243e)
Martin Schwenke [Sat, 14 Feb 2015 01:53:08 +0000 (12:53 +1100)]
scripts: Improve messages about invalid tunables during "setup"
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed Feb 18 08:03:33 CET 2015 on sn-devel-104
(Imported from commit
dc32f11b871a7d4e8ea6fd1d01491d89103decf7)
Martin Schwenke [Sun, 8 Feb 2015 23:33:35 +0000 (10:33 +1100)]
tool: Print a warning when setting an obsolete tunable variable
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
c3706e7fb07bcb35f7d894c4e8e0c12b4a62d0db)
Martin Schwenke [Sun, 8 Feb 2015 23:32:47 +0000 (10:32 +1100)]
client: Return a value of 1 when setting obsolete tunable variable
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
54f0c39e5a33871847aa9fe2c070c7f638f54cc4)
Martin Schwenke [Sun, 15 Feb 2015 03:39:51 +0000 (14:39 +1100)]
tests: New tests for 00.ctdb "setup" event - set tunables from config
Unit test infrastructure tweaks to support.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
2c7c35377e5452e37925b970253b70875a8d7470)
Martin Schwenke [Mon, 16 Feb 2015 03:04:09 +0000 (14:04 +1100)]
scripts: Fix tunable setup code by making it shell-agnostic
All tunables set in configuration are currently set to 0 on system
where /bin/sh is dash (and perhaps other non-bash shells). dash puts
single quotes around all values in the output of the "set" builtin
command, whereas bash only puts them around values when something
needs to be quoted. Tunables always have a simple integer value so
dash will quote them and bash won't. The setup code currently passes
the raw value, including any quotes to "ctdb setvar ...". This
command does no error checking on the input, so "'1'" is converted to
0.
Change the code so that the value is determined from the shell
variable and is independent of the "set" output.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
39686f45056d942de5ebe3263a533a99ca17c79e)
Martin Schwenke [Tue, 27 Jan 2015 01:55:42 +0000 (12:55 +1100)]
recoverd: Abort when daemon can take recovery lock during recovery
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Feb 13 09:48:15 CET 2015 on sn-devel-104
(Imported from commit
39d2fd330a60ea590d76213f8cb406a42fa8d680)
Martin Schwenke [Wed, 17 Dec 2014 09:33:19 +0000 (20:33 +1100)]
recoverd: Improve error messages on recovery lock coherence fail
When the daemon is able to take the recovery lock during recovery we
might as well guess that the cluster filesystem has a lock coherence
problem and print a more useful message. This will be more helpful to
those trying out cluster filesystems that don't have lock coherence or
that are difficult to setup.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
432d6774891eba30a959cd2d8ee8469d189c7872)
Martin Schwenke [Tue, 9 Dec 2014 02:51:27 +0000 (13:51 +1100)]
recoverd: Don't release and re-take the recovery lock
Just continue to hold it, otherwise a broken node might win an
election and grab the lock.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
48c91407abd5e34463d3a10cb6fce47ec4a0d5f6)
Martin Schwenke [Tue, 9 Dec 2014 03:50:38 +0000 (14:50 +1100)]
recoverd: Simplify ctdb_recovery_lock()
Have it just silently take or fail to take the lock, except on an
unexpected failure (where it should log an error).
This means that when it is called we need to keep the old behaviour
and explicitly release the lock. In do_recovery() the lock is
released and a message is printed before attempting to take the lock.
In the daemon sanity check the lock must be released in the error path
if it is actually taken.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
1d6ed91f5518d462ba368bca03be923428710157)
Martin Schwenke [Tue, 9 Dec 2014 03:45:08 +0000 (14:45 +1100)]
recoverd: Remove check_recovery_lock()
This has not done anything useful since commit
b9d8bb23af8abefb2d967e9b4e9d6e60c4a3b520. Instead, just check that
the lock is held.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
be19a17faf6da97365c425c5b423e9b74f9c9e0c)
Martin Schwenke [Tue, 9 Dec 2014 03:09:40 +0000 (14:09 +1100)]
recoverd: Improve logging when recovery lock file is changed
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
668ed5366237b61f0ff618f32555ce29cca5e6f3)
Martin Schwenke [Tue, 9 Dec 2014 03:07:20 +0000 (14:07 +1100)]
recoverd: New function ctdb_recovery_unlock()
Unlock the recovery lock file. This way knowledge of the file
descriptor isn't sprinkled throughout the code.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
db32a2bce54b9618fe247b33d6de81bd5f7a3b62)
Martin Schwenke [Tue, 9 Dec 2014 02:50:22 +0000 (13:50 +1100)]
recoverd: New function ctdb_recovery_have_lock()
True if this recovery daemon holds the lock.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
72701be663ddb265320a022a22130a3437bbf6bc)
Martin Schwenke [Tue, 9 Dec 2014 02:49:06 +0000 (13:49 +1100)]
daemon: Log a warning when setting obsolete tunables
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
4d3b52f1cec46f66f8d0827bc8f458cd8c86b5a5)
Martin Schwenke [Tue, 9 Dec 2014 02:47:42 +0000 (13:47 +1100)]
daemon: Mark tunable VerifyRecoveryLock as obsolete
It is pointless having a recovery lock but not sanity checking that it
is working. Also, the logic that uses this tunable is confusing. In
some places the recovery lock is released unnecessarily because the
tunable isn't set.
Simplify the logic by assuming that if a recovery lock is specified
then it should be verified.
Update documentation that references this tunable.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
d110fe231849d76ecb83378c934627dc64b74c72)
Martin Schwenke [Tue, 3 Feb 2015 03:27:11 +0000 (14:27 +1100)]
doc: Improve documentation of the recovery lock
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
a01744c08ff5b8aca4af99842acfc78a87af9297)
Martin Schwenke [Mon, 2 Feb 2015 10:21:20 +0000 (21:21 +1100)]
tests: Add new "ctdb setreclock" test
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed Feb 4 05:40:55 CET 2015 on sn-devel-104
(Imported from commit
0e89d586b2b7fa6a165a49862d2dce0d13f8b157)
Martin Schwenke [Wed, 28 Jan 2015 07:51:42 +0000 (18:51 +1100)]
daemon: Fix SET_RECLOCK_FILE regression
If the recovery lock file is unset then this dereferences a NULL
pointer. The regression is due to commit
6f1ac7af0f87d85402d708231e45a69713bba026.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
5e00673f2d95b6257a05324d2ae068004e29ff85)
Martin Schwenke [Tue, 30 Dec 2014 05:04:00 +0000 (16:04 +1100)]
scripts: Call iptables/ip6tables directly from iptables_wrapper
Drops the iptables() and ip6tables() functions and, hence, the
hardcoding of paths /sbin/iptables and /sbin/ip6tables. The latter
avoids problems on openSUSE where (for example) /usr/sbin/iptables is
used instead.
This means that locking around ip*tables commands is only done when
iptables_wrapper is called directly. This is fine because the only
conflict is when "releaseip" or "takeip"/"updateip" events are run in
parallel. The other uses in 11.natgw and 70.iscsi are in events where
there will be no collisions.
Making 11.natgw support IPv6 is unnecessary. Just put a static IPv6
address on each interface - they're plentiful.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed Jan 28 08:29:55 CET 2015 on sn-devel-104
(Imported from commit
ab51f283e7a7f4fc82a94d39e7bb3a68e8aac554)
Martin Schwenke [Tue, 30 Dec 2014 06:07:09 +0000 (17:07 +1100)]
scripts: Error message, comment and whitespace cleanups
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
9b67c1fa3748678552400a81172d124e59d5eb79)
Martin Schwenke [Tue, 30 Dec 2014 06:03:46 +0000 (17:03 +1100)]
scripts: iSCSI eventscript should fail when PNN can't be determined
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
1a5414b6d25ed1b1abdafd8594183b84af33a6fb)
Martin Schwenke [Tue, 30 Dec 2014 06:01:21 +0000 (17:01 +1100)]
scripts: Make 70.iscsi IPv6-aware
Block iSCSI port for families of all address the node is configured to
host.
Could just unconditional add blocking using ip6tables instead.
However, this would produce errors when no IPv6 public addresses are
configured and ip6tables is not installed.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
d1bd26e5eb25aee2ce82ef178692a64073a99aa0)
Michael Adam [Thu, 8 Jan 2015 23:10:37 +0000 (00:10 +0100)]
improve helpfulness of debug message when taking reclock fails
Print out the errno if the fcntl call.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Richard Sharpe <rsharpe@samba.org>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Fri Jan 9 04:25:02 CET 2015 on sn-devel-104
(Imported from commit
a59fb322d60b7152110cc0638dd9b76dd259ac15)
Martin Schwenke [Tue, 9 Dec 2014 02:40:23 +0000 (13:40 +1100)]
daemon: Handle out-of-memory when setting recovery lock file
Log a message when the reclock file actually changes and avoid a
memory allocation when it doesn't change.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
(Imported from commit
6f1ac7af0f87d85402d708231e45a69713bba026)
Martin Schwenke [Fri, 19 Dec 2014 03:19:32 +0000 (14:19 +1100)]
scripts: Don't use the GNU awk gensub() function
This is a gawk extension and can't be used reliably if just running
"awk". It is simple enough to switch to using the standard sub() and
gsub() functions.
The alternative is to switch to explicitly running "gawk". However,
although the eventscripts aren't exactly portable, it is probably
better to move closer to portability than further away.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
(Imported from commit
4638010abb116aed0c180207aaa11475277aecb7)
Martin Schwenke [Mon, 1 Dec 2014 01:21:16 +0000 (12:21 +1100)]
scripts: Try to deal with Ubuntu having /usr/sbin/service
Falling back to running the initscript doesn't work because it detects
that upstart is being used and fails. This was observed when trying
to start winbind on Ubuntu 11.04.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
(Imported from commit
a5c5eee7d186d938c5b458cb6dbf0c78cb548b63)
Amitay Isaacs [Thu, 11 Dec 2014 02:16:47 +0000 (13:16 +1100)]
daemon: Use correct tdb flags when enabling robust mutex support
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11000
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(Imported from commit
e0bf5dd4566785b41ad1fa0492a9f215639f1685)
Led [Sun, 14 Dec 2014 19:06:44 +0000 (21:06 +0200)]
scripts: Fix bashism in ctdbd_wrapper script
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11007
Signed-off-by: Oleksandr Chumachenko <ledest@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
(Imported from commit
2c3672f4249a7d3453b7e82ca96b4cd666aae709)
Martin Schwenke [Sat, 20 Dec 2014 08:54:13 +0000 (19:54 +1100)]
tests: Fix tickle sniffing for IPv4
tcptickle_sniff_start() assumes that if $dst contains a ': then it
should use the IPv6 sniffing code. However, $dst is a socket, so has
a trailing ":<port>".
Strip the trailing ":<port>" before checking for ':' as a marker for
an IPv6 address.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
(Imported from commit
82aa6bd2f27775f3fd82931d613379d5d618ef70)
Martin Schwenke [Wed, 3 Dec 2014 01:10:07 +0000 (12:10 +1100)]
tests: Need to drop public IPs in kill-failover tests
These tests simulate a dead node rather than a CTDB failure, so drop
IP addresses when killing a "node" to avoid problems with duplicates.
To cope with a CTDB failure a watchdog would be needed to ensure that
the public IPs are dropped when CTDB dies. Let's not do that now.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Dec 5 23:29:39 CET 2014 on sn-devel-104
(Imported from commit
a07af1bb56a76d1a7ab856547c801499fc17b21b)
Martin Schwenke [Mon, 1 Dec 2014 23:57:12 +0000 (10:57 +1100)]
daemon: Gratuitous ARP equivalent for IPv6 is neighbor advertisement
Not neighbour solicitation. See:
https://tools.ietf.org/html/rfc4861#section-4.4
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
7f3f3b15d52c4047cbcb1c3e81f65675c8708f5f)
Martin Schwenke [Wed, 3 Dec 2014 01:09:12 +0000 (12:09 +1100)]
tests: More debug on SSH failure
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
3a2c8bb906fac4e2611a28ead6b4290ddc93de54)
Martin Schwenke [Mon, 1 Dec 2014 02:30:29 +0000 (13:30 +1100)]
tests: Make tcpdump output more verbose
This helps with debugging.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
49449f66fca52d5044d2e486570562df866adf57)
Martin Schwenke [Sat, 29 Nov 2014 09:01:20 +0000 (20:01 +1100)]
tests: Use ip neigh command instead of arp
Extend select_test_node_and_ips() to set $test_prefix in addition to
$test_ip.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
a7e7d95de9cf09652cef63d65484bbb0faa42e1c)
Martin Schwenke [Mon, 1 Dec 2014 03:07:57 +0000 (14:07 +1100)]
tests: Generalise the gratarp and tickle sniffing code for IPv6
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
0f3d9752c4677b2f3b5ee47a0b8f973b4260ef57)
Martin Schwenke [Mon, 1 Dec 2014 02:51:47 +0000 (13:51 +1100)]
tests: Match IPv6 connections in netstat output
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
3135a8c62551e09a3abada86335882a91d398747)
Martin Schwenke [Mon, 1 Dec 2014 02:50:42 +0000 (13:50 +1100)]
tests: Use ping_wrapper to do relevant ping or ping6
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
255705c030135bd54a1f7a7dc40cbf00c2fb39c9)
Martin Schwenke [Wed, 3 Dec 2014 04:58:20 +0000 (15:58 +1100)]
tests: Extend regexps to handle IPv6 address matching
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
62acf5986f52a3118ed4e3638c5ac8b1f9c0adf8)
Martin Schwenke [Wed, 3 Dec 2014 04:57:35 +0000 (15:57 +1100)]
tests: Bracket IP addresses in NFS mounts and scp command (for IPv6)
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
62164ec52fd1082863bf5017a5170f74f18a07c3)
Martin Schwenke [Wed, 26 Nov 2014 10:32:22 +0000 (21:32 +1100)]
tests: Try to handle IPv6 addresses for local daemons
If CTDB_USE_IPV6 is set then use IPv6 addresses for nodes and public
IPs. This can be useful for some simple tests. However, the node
address actually needs to be on lo so that ctdbd can bind to the port
on that address, so they actually need to be added as root before
running tests, like this:
for i in $(seq 1 10) ; do ip addr add "fc00:10::${i}/64" dev lo ; done
IPv4 127.0.0.0/8 addresses are somehow magic and only one needs to be
on lo so that many can be bound to.
Also change the IPv4 node addresses to be (slightly) more exotic.
For both IPv4 and IPv6, choose addresses that are compatible with
socket wrapper.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Signed-off-by: Amitay Isaacs <amitay@gmail.com> (socket wrapper fixes)
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net> (socket wrapper fixes)
(Imported from commit
d9d07fff34143d251b4987cdb1cff1e8b3384e64)
Martin Schwenke [Wed, 26 Nov 2014 10:31:42 +0000 (21:31 +1100)]
tests: Extend regexp to match IPv6 addresses
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
2a83b74597a3366536e1935e2e8ff23493503117)
Martin Schwenke [Fri, 28 Nov 2014 10:49:08 +0000 (21:49 +1100)]
tools: Bracket IP addresses in onnode (for IPv6)
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
2d480792e7d0b3f6628186dc77f54b708e315dd1)
Amitay Isaacs [Tue, 25 Nov 2014 01:38:23 +0000 (12:38 +1100)]
daemon: Fix IP address comparisons for IPv6 addresses
Before storing node IP address, convert into the correct abbreviated
string form for IPv6 addresses.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(Imported from commit
e3c59d83d0ace9d7421d40d33fe917fb82bb38d8)
Martin Schwenke [Fri, 21 Nov 2014 06:33:21 +0000 (17:33 +1100)]
scripts: Wait until IPv6 addresses are not "tentative"
There are a few potential failure modes when adding an IPv6 address.
It takes a little while of duplicate address detection to complete, so
wait for a while. After a timeout, also need to check to see if
duplicate address detection failed - if it did then actually drop the
IP address.
This really needs some careful thinking. If CTDB disappears on a node
but the node's IP addresses are still on interfaces then the above
failure mode could cause the takeover nodes to become banned.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
d0b2375c3d754da3cd68e34617ab3edd36e9317b)
Amitay Isaacs [Thu, 20 Nov 2014 10:58:31 +0000 (21:58 +1100)]
eventscripts: Specify broadcast optionally to ip addr add
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(Imported from commit
d4212bd6a533b4b54b56e376a9246f2396cba253)
Martin Schwenke [Fri, 21 Nov 2014 03:52:47 +0000 (14:52 +1100)]
daemon: Trust vnn->interface for an IP when releasing it
ctdb_sys_find_ifname() doesn't work for IPv6 addresses so don't use
it.
Trust the eventscript to do sanity checking on the interface. Current
warnings are replaced with equivalents generated by the eventscript.
The unlikely message:
Public IP %s is hosted on interface %s but we have no VNN
will be replaced by:
WARNING: Public IP %s hosted on interface %s but VNN says __none__
which is clear enough.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
108b1be0ee62af7ecb3c775f45c540dd07a527bf)
Martin Schwenke [Fri, 21 Nov 2014 03:46:00 +0000 (14:46 +1100)]
scripts: Make 10.interface IPv6-safe
Add checking to "releaseip" and "updateip" to ensure that the given IP
address is really on the given interface with the given netmask. If
reality doesn't match the given arguments then believe reality.
Use new function iptables_wrapper() instead of calling iptables()
directly.
Use new function flush_route_cache() instead of doing IPv4-specific
/proc magic.
Remove setting of otherwise unused variable "failed".
Fix a test for which the error message has changed.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
6471541d6d2bc9f2af0ff92b280abbd1d933cf88)
Martin Schwenke [Fri, 21 Nov 2014 03:39:43 +0000 (14:39 +1100)]
scripts: New functions ip6tables() and iptables_wrapper()
ip6tables() uses the same lock as iptables(). This is done on
suspicion.
iptables_wrapper() takes 1st argument "inet" or "inet6", and the rest
is passed to the correct iptables variant.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
c314ae0b2af4a902cdd003ec6d663fe5b62b003b)
Martin Schwenke [Fri, 21 Nov 2014 03:37:54 +0000 (14:37 +1100)]
scripts: Add IPv6 addresses support in ip_maskbits_iface()
It also prints a third word, the address family. This is either
"inet" or "inet6".
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
ed029ae0a1faa56bf882a71d10828e2a90ab0bc7)
Martin Schwenke [Thu, 20 Nov 2014 23:48:25 +0000 (10:48 +1100)]
utils: Update Nagios code to use ctdb -X
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
b41c1bdaa1df92ee6c510ae6749d0524b88ef828)
Martin Schwenke [Thu, 20 Nov 2014 23:47:22 +0000 (10:47 +1100)]
doc: Update examples to use ctdb -X
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
0d61b6137306ee0eab2981e3c213023c332da550)
Martin Schwenke [Thu, 20 Nov 2014 23:45:57 +0000 (10:45 +1100)]
tool: Fix "ctdb -Y ifaces" output to have trailing delimiters
In the CTDB CLI tool source code and the documentation example.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
550fb8ce877cf980b4fd3be4f964449b369cf61e)
Martin Schwenke [Thu, 20 Nov 2014 04:03:25 +0000 (15:03 +1100)]
tests: Update integration tests to use ctdb -X
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
2c4de75d8754616891e97222cfb2ec58fdd8eac2)
Martin Schwenke [Thu, 20 Nov 2014 03:39:59 +0000 (14:39 +1100)]
tools: Update onnode and ctdb-diagnostics to use ctdb -X
Also update onnode unit tests.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
55df9c86c19e261a2a384ffc4b77c596c84e53a0)
Martin Schwenke [Thu, 20 Nov 2014 03:32:46 +0000 (14:32 +1100)]
scripts: Update eventscripts to use ctdb -X instead of ctdb -Y
Also update associated eventscript unit tests and ctdb stub.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
4940f191d37b5deadc8b1edf0cd516674e5d5d64)
Martin Schwenke [Wed, 19 Nov 2014 07:19:50 +0000 (18:19 +1100)]
tools: Add -X option for machine parsable output with separator '|'
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
7428f809a76044fcbc98fd5f718e331ce183941d)
Martin Schwenke [Wed, 19 Nov 2014 06:15:21 +0000 (17:15 +1100)]
tools: Add -x option to specify delimiter for machine readable output
To support this, update printm() to replace ':' in format string with
options.machineseparator, which is a string but must contain a single
character.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
3b90e45bae555cc4a47fe9958b86628d41084868)
Martin Schwenke [Wed, 19 Nov 2014 05:35:35 +0000 (16:35 +1100)]
tools: Produce machine readable output with new function printm()
printm() is a printf(3) replacement and must be used to printing any
machine readable output. It currently just calls vprintf(3). Later
it will change the field delimiter.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
fbacbb9c7868e22c04980af3602bae59dd5fe34d)
Martin Schwenke [Wed, 24 Sep 2014 07:07:12 +0000 (17:07 +1000)]
tests: Make the fake log timestamp string easy to modify
Use a variable to allow easy change of this string in case future
logging changes modify the timestamp format or do not support
timestamping.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
(Imported from commit
1c8c0be6f97f74169c0b46be4df2245aa631186a)
Martin Schwenke [Fri, 12 Sep 2014 03:20:43 +0000 (13:20 +1000)]
tests: Clean up some tests where IP movement is checked
Some of this implements logic that exists in functions. Some of it is
overly complicated and potentially failure-prone.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
371543207e8955d6665fcec03b261d80cde2401b)
Martin Schwenke [Fri, 12 Sep 2014 05:21:49 +0000 (15:21 +1000)]
tests: Remove dependency on log ringbuffer from missing IP test
The log ringbuffer will probably be removed. The test can be
implemented just as reliably by checking IP assignments using "ctdb
ip".
Update wait_until_ips_are_on_node() to print a more useful log
message.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
81a8758b9b24bbb7200be4682016f989b904bda8)
Martin Schwenke [Wed, 17 Sep 2014 10:34:39 +0000 (20:34 +1000)]
tests: Make all_ips_on_node() do what it should
The "-n all" is wrong.
Simplify the implementation and tighten up some uses of this function.
_select_test_node_and_ips() can't use this function anymore.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
e3089d7da17e2f25d426005eca0fe55397f2689f)
Martin Schwenke [Fri, 12 Sep 2014 03:40:01 +0000 (13:40 +1000)]
tests: Factor out new function get_test_ip_mask_and_iface()
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
81213af32ae17eaae33f9d27c9b7ce63c84ce5df)
Martin Schwenke [Fri, 12 Sep 2014 03:34:51 +0000 (13:34 +1000)]
tests: Simplify and rename wait_until_ips_are_on_nodeglob()
The glob functionality is unsed so simplify the code by removing it.
Rename this function to wait_until_ips_are_on_node(). Update all
calls.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
4b8cfe4847477e3cfdb3f4dd7070226a6604bc38)
Amitay Isaacs [Tue, 8 Jul 2014 06:31:21 +0000 (16:31 +1000)]
tests: Do not run ip command if running against local daemons
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Tue Jul 8 12:48:06 CEST 2014 on sn-devel-104
(Imported from commit
bf7f2b06e8e36710d0f9bf0684e70538e809dd16)
Martin Schwenke [Fri, 14 Mar 2014 02:14:18 +0000 (13:14 +1100)]
eventscripts: Fix regression in IP add/delete functions
Commit
176ae6c704528c021fcc34a41878584f43a00119 caused these functions
to exit on failure. This is incorrect and broke NAT gateway.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
058e14cdb0b6eb2e999dca599961ae6734105f99)
Martin Schwenke [Tue, 28 Jan 2014 05:08:50 +0000 (16:08 +1100)]
tests-eventscripts: Testing support for promote_secondaries
Just enable this behaviour by default in the ip command stub, since
10.interface assumes/sets it. The rc.local replacement for set_proc()
doesn't do anything...
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
a226015990356ee989c5b9a472581bb3187de894)
Martin Schwenke [Tue, 28 Jan 2014 03:41:25 +0000 (14:41 +1100)]
eventscripts: Deleting IPs should use the promote_secondaries option
If a primary IP address is being deleted from an interface, the
secondaries are remembered and added back after the primary is
deleted. This is done under a lock shared by the add/del script code.
It is necessary because, by default, Linux deletes secondaries when
the corresponding primary is deleted.
There is a race here between ctdbd and the scripts, since ctdbd
doesn't know about the lock. If ctdbd receives a release IP control
and the IP address is not on an interface then it is regarded as a
"Redundant release of IP" so no "releaseip" event is generated. This
can occur if the IP address in question is a secondary that has been
temporarily dropped. It is more likely if the number of secondaries
is large.
Since Linux 2.6.12 (i.e. 2005) Linux has supported a
promote_secondaries option on interfaces. This option is currently
undocumented but that will change in Linux 3.14. With
promote_secondaries enabled the kernel will not drop secondaries but
will promote a corresponding secondary instead. The kernel does all
necessary locking.
Use promote_secondaries to simplify the code, avoid re-adding
secondaries, avoid re-adding routes and provide improved performance.
This could be done conditionally, with a fallback to legacy
secondary-re-adding code, but no supported Linux distribution is
running a pre-2.6.12 kernel so this is unnecessary.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
176ae6c704528c021fcc34a41878584f43a00119)
Martin Schwenke [Fri, 8 Aug 2014 01:42:51 +0000 (11:42 +1000)]
logging: Rename ctdb_log_handler() to ctdb_child_log_handler()
Now it is obvious that it has something to do with child processes.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
7d391b746695d7a262e4f939f057ee1d1685e12b)
Martin Schwenke [Wed, 8 Oct 2014 03:22:53 +0000 (14:22 +1100)]
logging: Remove debug levels DEBUG_ALERT and DEBUG_CRIT
Internally map them to DEBUG_ERR to limit code churn.
This reduces the unwieldy number of debug levels used by CTDB. ALERT
and CRIT aren't of much use as separate errors, since everything from
ERR up should always be logged. In future just ERR can be used.
This also improves compatibility with Samba's debug.c system priority
mapping.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
f4fc9a153c533968905b8c7945c6615dcd9253d1)
Martin Schwenke [Wed, 8 Oct 2014 03:19:22 +0000 (14:19 +1100)]
logging: Remove DEBUG_EMERG
It isn't used and shouldn't be. CTDB can't make the system unusable.
Update associated test to ensure that EMERG isn't attempted. Actually
test all remaining debug levels and modernise the test a bit.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(Imported from commit
0eabbb8c2b91b61a23f20e04605fdbd653c5cbcb)
Amitay Isaacs [Tue, 14 Oct 2014 06:52:55 +0000 (17:52 +1100)]
tools: Fix heap-use-after-free problem
Found by address sanitizer.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Oct 17 12:56:02 CEST 2014 on sn-devel-104
(Imported from commit
470af881479d1a1588dc23ef40622b4d8f006b61)
Amitay Isaacs [Wed, 23 Apr 2014 08:02:39 +0000 (18:02 +1000)]
recoverd: Process all the records for vacuum fetch in a loop
Processing one migration request at a time is very slow and processing
a batch of records can take longer than VacuumInterval. This causes
subsequent vacuum fetch requests to be dropped. The dropped records
can accumulate quickly and will cause the vacuum database traverse to
be quite expensive.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Dec 5 17:06:58 CET 2014 on sn-devel-104
(Imported from commit
959b9ea0ef85c57ffc84d66a6e5e855868943391)
Amitay Isaacs [Mon, 14 Apr 2014 04:53:25 +0000 (14:53 +1000)]
vacuum: Do not delete VACUUM MIGRATED records immediately
Such records should be processed by the local vacuuming daemon to ensure
that all the remote copies have been deleted first.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(Imported from commit
257311e337065f089df688cbf261d2577949203d)
Amitay Isaacs [Wed, 5 Nov 2014 22:33:50 +0000 (09:33 +1100)]
vacuum: Use non-blocking lock when traversing delete tree
This avoids vacuuming getting in the way of ctdb daemon to process
record requests.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(Imported from commit
dbb1958284657f26a868705e5f9612bc377fd5e0)
Amitay Isaacs [Mon, 14 Apr 2014 03:18:41 +0000 (13:18 +1000)]
vacuum: Use non-blocking lock when traversing delete queue
This avoids vacuuming getting in the way of ctdb daemon to process
record requests.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(Imported from commit
d35f512cd972ac1f732fe998b2179242d042082d)
Amitay Isaacs [Fri, 21 Feb 2014 03:58:00 +0000 (14:58 +1100)]
vacuum: Stagger vacuuming child processes
This prevents multiple child processes being forked at the same time
for vacuuming TDBs.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(Imported from commit
e4597f8771f42cf315bd163c18b2f27147d3de5f)
Amitay Isaacs [Tue, 11 Feb 2014 03:23:28 +0000 (14:23 +1100)]
vacuum: Track time for vacuuming in database statistics
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(Imported from commit
a0628e317df76c7c38a7cca9c3090077fa352899)
Martin Schwenke [Mon, 17 Nov 2014 03:15:14 +0000 (14:15 +1100)]
scripts: Fix stack dumping when debugging hung scripts
There are parentheses missing that stop the default pattern from
matching commands with trailing garbage (e.g. "exportfs.orig").
A careful check of POSIX (and running GNU sed with --posix) suggests
that "\|" isn't a supported way of specifying alternation in a regular
expression. Therefore, it is clearer to switch to extended regular
expressions so that this has a chance of being portable (even though
the point is to print /proc/<pid>/stack, which only works on Linux).
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Tue Nov 18 06:37:45 CET 2014 on sn-devel-104
(Imported from commit
7f377cf26ecec10cd77f28c1993c48337279892d)