7 * The default location of the ctdbd socket is now:
9 /var/run/ctdb/ctdbd.socket
11 If you currently set CTDB_SOCKET in configuration then unsetting it
12 will probably do what you want.
14 * The default location of CTDB TDB databases is now:
18 If you only set CTDB_DBDIR (to the old default of /var/ctdb) then
19 you probably want to move your databases to /var/lib/ctdb, drop your
20 setting of CTDB_DBDIR and just use the default.
22 To maintain the database files in /var/ctdb you will need to set
23 CTDB_DBDIR, CTDB_DBDIR_PERSISTENT and CTDB_DBDIR_STATE, since all of
26 * Use of CTDB_OPTIONS to set ctdbd command-line options is no longer
27 supported. Please use individual configuration variables instead.
29 * Obsolete tunables VacuumDefaultInterval, VacuumMinInterval and
30 VacuumMaxInterval have been removed. Setting them had no effect but
31 if you now try to set them in a configuration files via CTDB_SET_X=Y
32 then CTDB will not start.
34 * Much improved manual pages. Added new manpages ctdb(7),
35 ctdbd.conf(5), ctdb-tunables(7). Still some work to do.
37 * Most CTDB-specific configuration can now be set in
40 This avoids cluttering distribution-specific configuration files,
41 such as /etc/sysconfig/ctdb. It also means that we can say: see
42 ctdbd.conf(5) for more details. :-)
44 * Configuration variable NFS_SERVER_MODE is deprecated and has been
45 replaced by CTDB_NFS_SERVER_MODE. See ctdbd.conf(5) for more
48 * "ctdb reloadips" is much improved and should be used for reloading
49 the public IP configuration.
51 This commands attempts to yield much more predictable IP allocations
52 than using sequences of delip and addip commands. See ctdb(1) for
55 * Ability to pass comma-separated string to ctdb(1) tool commands via
56 the -n option is now documented and works for most commands. See
59 * "ctdb rebalancenode" is now a debugging command and should not be
60 used in normal operation. See ctdb(1) for details.
62 * "ctdb ban 0" is now invalid.
64 This was documented as causing a permanent ban. However, this was
65 not implemented and caused an "unban" instead. To avoid confusion,
66 0 is now an invalid ban duration. To administratively "ban" a node
67 use "ctdb stop" instead.
69 * The systemd configuration now puts the PID file in /run/ctdb (rather
70 than /run/ctdbd) for consistency with the initscript and other uses
76 * Traverse regression fixed.
78 * The default recovery method for persistent databases has been
79 changed to use database sequence numbers instead of doing
80 record-by-record recovery (using record sequence numbers). This
81 fixes issues including registry corruption.
83 * Banned nodes are no longer told to run the "ipreallocated" event
84 during a takeover run, when in fallback mode with nodes that don't
85 support the IPREALLOCATED control.
87 Important internal changes
88 --------------------------
90 * Persistent transactions are now compatible with Samba and work
93 * The recovery master role has been made more stable by resetting the
94 priority time each time a node becomes inactive. This means that
95 nodes that are active for a long time are more likely to retain the
98 * The incomplete libctdb library has been removed.
100 * Test suite now starts ctdbd with the --sloppy-start option to speed
101 up startup. However, this should not be done in production.
110 * A missing network interface now causes monitoring to fail and the
111 node to become unhealthy.
113 * Changed ctdb command's default control timeout from 3s to 10s.
115 * debug-hung-script.sh now includes the output of "ctdb scriptstatus"
116 to provide more information.
121 * Starting CTDB daemon by running ctdbd directly should not remove
122 existing unix socket unconditionally.
124 * ctdbd once again successfully kills client processes on releasing
125 public IPs. It was checking for them as tracked child processes
126 and not finding them, so wasn't killing them.
128 * ctdbd_wrapper now exports CTDB_SOCKET so that child processes of
129 ctdbd (such as uses of ctdb in eventscripts) use the correct socket.
131 * Always use Jenkins hash when creating volatile databases. There
132 were a few places where TDBs would be attached with the wrong flags.
134 * Vacuuming code fixes in CTDB 2.2 introduced bugs in the new code
135 which led to header corruption for empty records. This resulted
136 in inconsistent headers on two nodes and a request for such a record
137 keeps bouncing between nodes indefinitely and logs "High hopcount"
138 messages in the log. This also caused performance degradation.
140 * ctdbd was losing log messages at shutdown because they weren't being
141 given time to flush. ctdbd now sleeps for a second during shutdown
142 to allow time to flush log messages.
144 * Improved socket handling introduced in CTDB 2.2 caused ctdbd to
145 process a large number of packets available on single FD before
146 polling other FDs. Use fixed size queue buffers to allow fair
147 scheduling across multiple FDs.
149 Important internal changes
150 --------------------------
152 * A node that fails to take/release multiple IPs will only incur a
153 single banning credit. This makes a brief failure less likely to
154 cause node to be banned.
156 * ctdb killtcp has been changed to read connections from stdin and
157 10.interface now uses this feature to improve the time taken to kill
160 * Improvements to hot records statistics in ctdb dbstatistics.
162 * Recovery daemon now assembles up-to-date node flags information
163 from remote nodes before checking if any flags are inconsistent and
166 * ctdbd no longer creates multiple lock sub-processes for the same
167 key. This reduces the number of lock sub-processes substantially.
169 * Changed the nfsd RPC check failure policy to failover quickly
170 instead of trying to repair a node first by restarting NFS. Such
171 restarts would often hang if the cause of the RPC check failure was
172 the cluster filesystem or storage.
174 * Logging improvements relating to high hopcounts and sticky records.
176 * Make sure lower level tdb messages are logged correctly.
178 * CTDB commands disable/enable/stop/continue are now resilient to
179 individual control failures and retry in case of failures.
188 * 2 new configuration variables for 60.nfs eventscript:
190 - CTDB_MONITOR_NFS_THREAD_COUNT
191 - CTDB_NFS_DUMP_STUCK_THREADS
193 See ctdb.sysconfig for details.
195 * Removed DeadlockTimeout tunable. To enable debug of locking issues set
197 CTDB_DEBUG_LOCKS=/etc/ctdb/debug_locks.sh
199 * In overall statistics and database statistics, lock buckets have been
200 updated to use following timings:
202 < 1ms, < 10ms, < 100ms, < 1s, < 2s, < 4s, < 8s, < 16s, < 32s, < 64s, >= 64s
204 * Initscript is now simplified with most CTDB-specific functionality
205 split out to ctdbd_wrapper, which is used to start and stop ctdbd.
207 * Add systemd support.
209 * CTDB subprocesses are now given informative names to allow them to
210 be easily distinguished when using programs like "top" or "perf".
215 * ctdb tool should not exit from a retry loop if a control times out
216 (e.g. under high load). This simple fix will stop an exit from the
217 retry loop on any error.
219 * When updating flags on all nodes, use the correct updated flags. This
220 should avoid wrong flag change messages in the logs.
222 * The recovery daemon will not ban other nodes if the current node
225 * ctdb dbstatistics command now correctly outputs database statistics.
227 * Fixed a panic with overlapping shutdowns (regression in 2.2).
229 * Fixed 60.ganesha "monitor" event (regression in 2.2).
231 * Fixed a buffer overflow in the "reloadips" implementation.
233 * Fixed segmentation faults in ping_pong (called with incorrect
234 argument) and test binaries (called when ctdbd not running).
236 Important internal changes
237 --------------------------
239 * The recovery daemon on stopped or banned node will stop participating in any
242 * Improve cluster wide database traverse by sending the records directly from
243 traverse child process to requesting node.
245 * TDB checking and dropping of all IPs moved from initscript to "init"
248 * To avoid "rogue IPs" the release IP callback now fails if the
249 released IP is still present on an interface.
258 * The "stopped" event has been removed.
260 The "ipreallocated" event is now run when a node is stopped. Use
261 this instead of "stopped".
263 * New --pidfile option for ctdbd, used by initscript
265 * The 60.nfs eventscript now uses configuration files in
266 /etc/ctdb/nfs-rpc-checks.d/ for timeouts and actions instead of
267 hardcoding them into the script.
269 * Notification handler scripts can now be dropped into /etc/ctdb/notify.d/.
271 * The NoIPTakeoverOnDisabled tunable has been renamed to
272 NoIPHostOnAllDisabled and now works properly when set on individual
275 * New ctdb subcommand "runstate" prints the current internal runstate.
276 Runstates are used for serialising startup.
281 * The Unix domain socket is now set to non-blocking after the
282 connection succeeds. This avoids connections failing with EAGAIN
283 and not being retried.
285 * Fetching from the log ringbuffer now succeeds if the buffer is full.
287 * Fix a severe recovery bug that can lead to data corruption for SMB clients.
289 * The statd-callout script now runs as root via sudo.
291 * "ctdb delip" no longer fails if it is unable to move the IP.
293 * A race in the ctdb tool's ipreallocate code was fixed. This fixes
294 potential bugs in the "disable", "enable", "stop", "continue",
295 "ban", "unban", "ipreallocate" and "sync" commands.
297 * The monitor cancellation code could sometimes hang indefinitely.
298 This could cause "ctdb stop" and "ctdb shutdown" to fail.
300 Important internal changes
301 --------------------------
303 * The socket I/O handling has been optimised to improve performance.
305 * IPs will not be assigned to nodes during CTDB initialisation. They
306 will only be assigned to nodes that are in the "running" runstate.
308 * Improved database locking code. One improvement is to use a
309 standalone locking helper executable - the avoids creating many
310 forked copies of ctdbd and potentially running a node out of memory.
312 * New control CTDB_CONTROL_IPREALLOCATED is now used to generate
313 "ipreallocated" events.
315 * Message handlers are now indexed, providing a significant
316 performance improvement.