Ronnie Sahlberg [Thu, 28 Oct 2010 02:43:57 +0000 (13:43 +1100)]
Both nfs and nfslock scripts can fail under redhat in very rare situations.
Ctdb can also be configured to ignore checking for knfsd and if it is alive.
In that situation, no attempt will be made to restart nfs, and sicne nfs is not running, lockd can not be restarted either.
To workaround this, everytime we try to restart the lockmanager, also try to restart nfsd
Ronnie Sahlberg [Thu, 28 Oct 2010 02:38:34 +0000 (13:38 +1100)]
during shutdown there is a window after we have stopped TCP and disconnected from all other nodes but before we have stopped all processing.
During this window we may still hit asynchronous events that will fail because we can not send/receive packets from other nodes.
These messages are logged as ... Transport is DOWN. To help indicate that they are benign messages related to the process of shutting down.
These messages spam the syslog during normal shutdown, so this patch will drop the loglevel of these messages to DEBUG, so that they will not appear in or spam the syslog.
Ronnie Sahlberg [Thu, 28 Oct 2010 02:36:24 +0000 (13:36 +1100)]
When shuttind down, we always unconditionally try to remove the natgw address
even if we are not currently the natgw master.
This adds extra reliability in case we have stopped previously without removing it proper,
but does add spam messages to syslog everytime we shutdowm.
Remove these spam messages from pulluting the syslog upon normal shutdown
Ronnie Sahlberg [Thu, 28 Oct 2010 02:34:33 +0000 (13:34 +1100)]
Redirect the output from 00.ctdb pfetch to stdout.
Normally, the config.tdb database would not exist, so we do not need
to spam syslog with a "config.tdb does not exist" message every time we start ctdb
Ronnie Sahlberg [Mon, 25 Oct 2010 00:31:12 +0000 (11:31 +1100)]
Add support to create TDB databases using the new jenkins hash.
SRVID for the control to attach to a database is used to pass
tdb flags from samba to ctdb when samba attached to a database.
This has been used earlier for TDB_NOSYNC flag.
Add TDB_INCOMPATIBLE_HASH as a supported tdb flag to store in the
SRVID field when attaching to a database.
This allows samba to control if ctdb should create databases using the
new jenkins hash, or using the old hash.
This only affects new databases when they are initially created.
Existing databases remain using the old hash when attached to.
Ronnie Sahlberg [Mon, 18 Oct 2010 04:58:03 +0000 (15:58 +1100)]
New version 1.2.7
- Dont monitor GPFS filesystems in 62.cnfs
- If tdb_open() fails, print errno to make troubleshooting easier
- Try restarting RPC.LOCKD if it failed to start
- Remove a dbug message
- Make sure the statd state directory exists before trying to touch files in
Ronnie Sahlberg [Mon, 11 Oct 2010 15:54:12 +0000 (02:54 +1100)]
New version 1.2.6
* Tue Oct 12 2010 : Version 1.2.6
- Move config.tdb handling into a function in 00.ctdb
- Latency counters min/max/avg for all latency statistics
- Update default hash size to be 100001
- Check all bond devices, dont exit after the first one
- Change to useing the Jenkins hash for LMASTER selection
- Sync with TDB from upstream samba
- idtree fix for AIX
- remove some log messages
- add rolling statistics
- libctdb updates
Ronnie Sahlberg [Sun, 10 Oct 2010 20:09:18 +0000 (07:09 +1100)]
Update the default hash size to be 100001 instead of 10000
This can sometimes improve performance for environments where very many
files are touched in rapid succession
Rusty Russell [Fri, 24 Sep 2010 06:15:11 +0000 (15:45 +0930)]
tdb: TDB_INCOMPATIBLE_HASH, to allow safe changing of default hash.
This flag to tdb_open/tdb_open_ex effects creation of a new database:
1) Uses the Jenkins lookup3 hash instead of the old gdbm hash if none is
specified,
2) Places a non-zero field in header->rwlocks, so older versions of TDB will
refuse to open it.
This means that the caller (ie Samba) can set this flag to safely
change the hash function. Versions of TDB from this one on will either
use the correct hash or refuse to open (if a different hash is specified).
Older TDB versions will see the nonzero rwlocks field and refuse to open
it under any conditions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The problem was tdb->name is NULL for TDB_INTERNAL databases, and
so it was crashing ...
#0 0xb76944f3 in strlen () from /lib/i686/cmov/libc.so.6
#1 0x0809862b in PyString_FromFormatV (format=0xb72b6a26 "Tdb('%s')", vargs=0xbfc26a94 "")
at ../Objects/stringobject.c:211
#2 0x08098888 in PyString_FromFormat (format=0xb72b6a26 "Tdb('%s')") at ../Objects/stringobject.c:358
#3 0xb72b65f2 in tdb_object_repr (self=0xb759e060) at ./pytdb.c:439
Rusty Russell [Mon, 13 Sep 2010 10:35:59 +0000 (20:05 +0930)]
tdb: put example hashes into header, so we notice incorrect hash_fn.
This is Stefan Metzmacher <metze@samba.org>'s patch with minor changes:
1) Use the TDB_MAGIC constant so both hashes aren't of strings.
2) Check the hash in tdb_check (paranoia, really).
3) Additional check in the (unlikely!) case where both examples hash to 0.
4) Cosmetic changes to var names and complaint message.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Tue, 5 Oct 2010 02:36:19 +0000 (13:06 +1030)]
idtree: fix right shift of signed ints, crash on large ids on AIX
Right-shifting signed integers in undefined; indeed it seems that on
AIX with their compiler, doing a 30-bit shift on (INT_MAX-200) gives
0, not 1 as we might expect.
The obvious fix is to make id and oid unsigned: l (level count) is also
logically unsigned.
(Note: Samba doesn't generally get to ids > 1 billion, but ctdb does)
Reported-by: Chris Cowan <cc@us.ibm.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Autobuild-User: Rusty Russell <rusty@samba.org>
Autobuild-Date: Wed Oct 6 08:31:09 UTC 2010 on sn-devel-104
Ronnie Sahlberg [Mon, 27 Sep 2010 22:58:03 +0000 (08:58 +1000)]
Add back monitoring for time skips, forward as well as backward.
This serviceability tool was lost during the migration from the old eventsystem to the tevent system.
Ronnie Sahlberg [Thu, 9 Sep 2010 06:34:00 +0000 (16:34 +1000)]
new version 1.2.5
- Suppress some VSFTPD warnings
- Make sure all STATD directories exist before we dereference them
- AIX socket fix
- Fix for a crash when we write a debug message after a memory allocation
fail. Fix the message and call ctdb_fatal() properly
- Move the state directory off /etc/ctdb and to /var/ctdb
- Natgw changes to allow "slave only" natgw members
- Fix "ctdb listnodes" so it works again.
Ronnie Sahlberg [Tue, 7 Sep 2010 23:16:42 +0000 (09:16 +1000)]
Change how NATGW is configured to allow special nodes that do not have
network connectivity outside of the cluster to still be able to
participate in a natgw group.
These nodes can not become natgw master since they lack external network
connectivity.
These nodes are configured just the same way as for any other node with
NATGW, with the following two exceptions :
* we do NOT set CTDB_NATGW_PUBLIC_IFACE at all on these nodes.
since these ndoes lack external network we should not check the interface
for link.
* we must set CTDB_NATGW_SLAVE_ONLY=yes to flag that this is a node that
can not become natgw master.
Ronnie Sahlberg [Mon, 30 Aug 2010 08:15:41 +0000 (18:15 +1000)]
remove the mention of a tickle and statd directory in shared storage now that we are removing these and migrating to store the data inside ctdbd or persistent databases
Ronnie Sahlberg [Mon, 30 Aug 2010 08:13:28 +0000 (18:13 +1000)]
Remove the dependency on the underlying cluster filesystem for handling
the clusterwide persistent data associated with the lock manager and
statd notifications.
Use persistent databases to store this data instead of a shared directory.
Ronnie Sahlberg [Mon, 30 Aug 2010 08:08:38 +0000 (18:08 +1000)]
Add a new event "ipreallocated"
This is called everytime a reallocation is performed.
While STARTRECOVERY/RECOVERED events are only called when
we do ipreallocation as part of a full database/cluster recovery,
this new event can be used to trigger on when we just do a light
failover due to a node becomming unhealthy.
I.e. situations where we do a failover but we do not perform a full
cluster recovery.
Use this to trigger for natgw so we select a new natgw master node
when failover happens and not just when cluster rebuilds happen.
Martin Schwenke [Thu, 26 Aug 2010 04:59:59 +0000 (14:59 +1000)]
NFS tickles: use addtickle/deltickle instead of shared tickle directory.
This adds a new function update_tickles() that tracks tickles for a
given port using the new ctdb addtickle/deltickle commands. This
function is used in events.d/60.nfs to handle NFS tickles.
events.d/61.nfstickle is removed. The
/proc/sys/net/ipv4/tcp_tw_recycle setup is also moved to
events.d/60.nfs.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Ronnie Sahlberg [Wed, 25 Aug 2010 01:37:32 +0000 (11:37 +1000)]
Add a configuration database, implemented as a persistent database.
This database can be used, as an option, to store
the public address assignment instead of editing the /etc/ctdb/public-addresses file manually.
This configuration is stored in one record per key, with a key-name of
public-addresses:node#<pnn>
where <pnn> is the node number.
The content of this record is the same syntax as the /etc/ctdb/public-addresses file.
When ctdbd starts, if this key exist and contains data. It is extracted from the database and compared with the normal file /etc/ctdb/public-addresses.
If the content differs, the config database "wins" and is used to overwrite/update the /etc/ctdb/public-addresses file, after which ctdbd is restarted.
The main benefit with this option is that it can be used to update the public address configuration for nodes that are offline/unreachable by updating their configuration in the persistent database.
Once the offline node is available again, it will resync its databases with the rest of the cluster, find out that the config has changed, apply the changes and restart ctdbd automatically.
The command to store the public address configuration for a node into the persistent database is :
where <pnn> is the node# we wish to update the config for, and <filename> is a file containing the new content for that nodes public address configuration.
Ronnie Sahlberg [Wed, 25 Aug 2010 01:37:32 +0000 (11:37 +1000)]
Add a configuration database, implemented as a persistent database.
This database can be used, as an option, to store
the public address assignment instead of editing the /etc/ctdb/public-addresses file manually.
This configuration is stored in one record per key, with a key-name of
public-addresses:node#<pnn>
where <pnn> is the node number.
The content of this record is the same syntax as the /etc/ctdb/public-addresses file.
When ctdbd starts, if this key exist and contains data. It is extracted from the database and compared with the normal file /etc/ctdb/public-addresses.
If the content differs, the config database "wins" and is used to overwrite/update the /etc/ctdb/public-addresses file, after which ctdbd is restarted.
The main benefit with this option is that it can be used to update the public address configuration for nodes that are offline/unreachable by updating their configuration in the persistent database.
Once the offline node is available again, it will resync its databases with the rest of the cluster, find out that the config has changed, apply the changes and restart ctdbd automatically.
The command to store the public address configuration for a node into the persistent database is :
where <pnn> is the node# we wish to update the config for, and <filename> is a file containing the new content for that nodes public address configuration.