Michael Adam [Fri, 22 Feb 2013 15:12:17 +0000 (16:12 +0100)]
(TODO: cherry-pick-info)vacuum: Update (C)
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit
aa9723c0242effe94e6aeec07f6dd2e3f9d4b291)
Michael Adam [Sat, 29 Dec 2012 16:23:27 +0000 (17:23 +0100)]
(TODO: cherry-pick-info)vacuum: extend the header comment for ctdb_process_delete_list()
Describe the (new) process more precisely.
And mention that is the last step of the vacuuming process
that is performed on the lmaster.
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit
aef535cbbe1eece49611fc0f57409f4544b3d909)
Michael Adam [Sat, 5 Jan 2013 00:20:18 +0000 (01:20 +0100)]
(TODO: cherry-pick-info)vacuum: turn the vacuuming on lmaster into a three-phase process.
More precisely, before locally deleting an empty record, that has been
migrated with data and that we are dmaster and laster for, we now perform
the deletion on the other nodes in two steps instead of a single step.
- First send out the list of records to be deleted to all
other nodes with the new RECEIVE_RECORDS control to store
the lmaster's current empty copy.
- Then send those records that could be deleted on all nodes
to all nodes again with the TRY_DELETE_RECORDS control
as before for deletion.
- Finally delete those records locally that were successfully
deleted remotely in the previous step.
This fixes and old race where a recovery that hits the vacuum process
square between the eyes can create gaps in the record's history and
hence let the records resurrect. In the case of the locking.tdb,
that could mean that a file that was already closed, was recorded as
being open and locked again, so samba clients were locked out of that
file until samba was restarted.
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit
4fb86aa8410fb56f36462aa8ea9dfcddad1f53bf)
Michael Adam [Thu, 20 Dec 2012 23:24:47 +0000 (00:24 +0100)]
(TODO: cherry-pick-info)vacuum: introduce the RECEIVE_RECORDS control
This in preparation of turning the vacuming on the lmaster into
into a two phase process:
- First the node sends the list of records to be vacuumed
to all other nodes with this new RECEIVE_RECORDS control.
The remote nodes should store the lmaster's empty current copy.
- Only those records that could be stored on all other nodes
are processed further. They are send to all other nodes with
the TRY_DELETE_RECORDS control as before for deletion.
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX)
Conflicts:
include/ctdb_protocol.h
server/ctdb_control.c
Michael Adam [Sat, 29 Dec 2012 17:32:39 +0000 (18:32 +0100)]
(TODO: cherry-pick-info)vacuum: reorder some of ctdb_process_delete_list() more intuitively
Now that the nodemap and its talloc children don't hang off of the
delete_records_list talloc context, we can build the nodemap
and earlier, and move the construction of the delete_records_list
to where it is more obvious what it is used for.
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit
875f635c83adb523c7db24cd89e36b23bd7dbbfe)
Michael Adam [Sat, 29 Dec 2012 16:16:33 +0000 (17:16 +0100)]
(TODO: cherry-pick-info)vacuum: add explicit temporary memory context to ctdb_process_delete_list()
This removes the implicit artificial talloc hierarchy and makes the
code easier to understand.
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit
db7d7d36abd58ccc972a41e55d53b52771c6b9fc)
Michael Adam [Sat, 5 Jan 2013 00:19:06 +0000 (01:19 +0100)]
(TODO: cherry-pick-info)vacuum: fix indentation in ctdb_process_delete_list()
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX)
Michael Adam [Mon, 17 Dec 2012 16:31:55 +0000 (17:31 +0100)]
(TODO: cherry-pick-info)vacuum: free temporary allocated memory correctly in ctdb_process_delete_list().
Add a common exit point for cleanup.
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX)
Michael Adam [Mon, 17 Dec 2012 16:26:22 +0000 (17:26 +0100)]
(TODO: cherry-pick-info)vacuum: move variable into scope of use in ctdb_process_delete_list()
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX)
Michael Adam [Mon, 17 Dec 2012 12:07:21 +0000 (13:07 +0100)]
(TODO: cherry-pick-info)vacuum: move variable into scope of use in ctdb_process_delete_list()
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX)
Michael Adam [Mon, 17 Dec 2012 12:03:42 +0000 (13:03 +0100)]
(TODO: cherry-pick-info)vacuum: simplify ctdb_process_delete_list(): reduce indentation
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX)
Conflicts:
server/ctdb_vacuum.c
Michael Adam [Wed, 3 Apr 2013 12:12:27 +0000 (14:12 +0200)]
(TODO: cherry-pick-info)vacuum: add DEBUG to skip conditions in delete_record_traverse()
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit
eb7e0a316b27451d1375ea9d5649e49b5262ec59)
Conflicts:
server/ctdb_vacuum.c
Michael Adam [Wed, 3 Apr 2013 10:02:59 +0000 (12:02 +0200)]
(TODO: cherry-pick-info)ctdb_call: don't bump the rsn in ctdb_become_dmaster() any more
This is now done in ctdb_ltdb_store_server(), so this
extra bump can be spared.
Signed-off-by: Michael Adam <obnox@samba.org>
Michael Adam [Wed, 3 Apr 2013 09:40:25 +0000 (11:40 +0200)]
(TODO: cherry-pick-info)Fix a severe recovery bug that can lead to data corruption for SMB clients.
Problem:
Recovery can under certain circumstances lead to
old record copies resurrecting: Recovery selects the newest
record copy purely by RSN. At the end of the recovery,
the recovery master is the dmaster for all records in all
(non-persistent) databases. And the other nodes locally hold
the complete copy of the databases. The bug is that the recovery
process does not increment the RSN on the recovery master at
the end of the recovery. Now clients acting directly on the
Recovery master will directly change a record's content on the
recmaster without migration and hence without RSN bump.
So a subsequent recovery can not tell that the recmaster's copy
is newer than the copies on the other nodes, since their RSN
is the same. Hence, if the recmaster is not node 0 (or more precisely
not the active node with the lowest node number), the recovery
will choose copies from nodes with lower number and stick to these.
Here is how to reproduce:
- assume we have a cluster with at least 2 nodes
- ensure that the recmaster ist node node 0
(maybe ensure with "onnode 0 ctdb setrecmasterrole off")
say recmaster is node 1
- choose a new database name, say "test1.tdb"
(make sure it is not yet attached as persistent)
- choose a key name, say "key1"
- all clustere nodes should ok and no recovery running
- now do the following on node 1:
1. dbwrap_tool test1.tdb store key1 uint32 1
2. dbwrap_tool test1.tdb fetch key1 uint32
==> 1
3. ctdb recover
4. dbwrap_tool test1.tdb store key1 uint32 2
5. dbwrap_tool test1.tdb fetch key1 uint32
==> 2
4. ctdb recover
7. dbwrap_tool test1.tdb fetch key1 uint32
==> 1
==> BUG
This is a very severe bug, since when applied to Samba's locking.tdb
database, it means that for SMB clients on clustered Samba there is
the potential for locking out oneself from previously opened files
or even worse, data corruption:
Case 1: locking out
- client on recmaster opens file
- recovery propagates open file handle (entry in locking.tdb) to
other nodes
- client closes file
- client opens the same file
- recovery resurrects old copy of open file record in locking.tdb
from lower node
- client closes file but fails to delete entry in locking.tdb
- client tries to open same file again but fails, since
the old record locks it out (since the client is still connected)
Case 2: data corruption
- clien1 on recmaster opens file
- recovery propagates open file info to other nodes
- client1 closes the file and disconnects
- client2 opens the same file
- recovery resurrects old copy of locking.tdb record,
where client2 has no entry, but client1 has.
- but client2 believes it still has a handle
- client3 opens the file and succees without
conflicting with client2
(the detached entry for client1 is discarded because
the server does not exist any more).
=> both client2 and client3 believe they have exclusive
access to the file and writing creates data corruption
Fix:
When storing a record on the dmaster, bump its RSN.
The ctdb_ltdb_store_server() is the central function for storing
a record to a local tdb from the ctdbd server context.
So this is also the place where the RSN of the record to be stored
should be incremented, when storing on the dmaster.
For the case of the record migration, this is currently done in
ctdb_become_dmaster() in ctdb_call.c, but there are other places
such as in recovery, where we should bump the RSN, but currently
don't do it.
So moving the RSN incrementation into ctdb_ltdb_store_server fixes
the recovery-record-resurrection bug.
Signed-off-by: Michael Adam <obnox@samba.org>
Michael Adam [Wed, 3 Apr 2013 12:03:32 +0000 (14:03 +0200)]
(TODO: cherry-pick-info)ctdbd: unimplement the unused SET_DMASTER control
Signed-off-by: Michael Adam <obnox@samba.org>
Martin Schwenke [Wed, 9 Nov 2011 03:55:07 +0000 (14:55 +1100)]
Fix typo in ctdb_ltdb_store_server()
The if statement uses ret but means to use ret2.
Signed-off-by: Martin Schwenke <martin@meltin.net>
(cherry picked from commit
f40101a615f8b9826a484e4697bfea6ee2b9ba88)
Michael Adam [Tue, 20 Nov 2012 10:20:34 +0000 (11:20 +0100)]
ctdb:recover: fix a comment typo
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit
5067392d2e06795559f25828b65c129608b65c0b)
Volker Lendecke [Thu, 22 Nov 2012 14:27:51 +0000 (15:27 +0100)]
vacuum: Avoid some tallocs in ctdb recovery
In a heavily loaded and volatile database a lot of SCHEDULE_FOR_DELETION
requests can come in between fast vacuuming runs. This can lead to
significant ctdb cpu load due to the cost of doing talloc_free. This
reduces the number of objects a bit by coalescing the two objects
of delete_record_data into one. It will also avoid having to allocate
another talloc header for a SCHEDULE_FOR_DELETION key. Not the full fix
for this problem, but it might contribute a bit.
(cherry picked from commit
9a02f61547ddf74629aca21639d8fb61c1df7cbb)
Michael Adam [Fri, 16 Nov 2012 13:33:41 +0000 (14:33 +0100)]
vacuum: fix a comment typo
Pair-Programmed-With: Volker Lendecke <vl@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit
bab744e3c49efef2e05dc09e8ea9bd3e3fa58716)
Michael Adam [Mon, 7 May 2012 09:51:28 +0000 (11:51 +0200)]
New version 1.0.114.5.
Michael Adam [Wed, 23 Feb 2011 16:39:57 +0000 (17:39 +0100)]
recover: finish pending trans3 commits when a recovery is finished.
When the end_recovery control is received, pending trans3 commits are
finished. During the recovery, all the actions like persistent_callback
and persistent_store_timeout had been disabled to let the recovery do
its job. After the recover is completed, send the reply to the waiting
clients.
(cherry picked from commit
f7dfeb7143f574c2434f7dd16917380dfd1f4f64)
Signed-off-by: Michael Adam <obnox@samba.org>
Michael Adam [Wed, 23 Feb 2011 16:38:40 +0000 (17:38 +0100)]
persistent: add ctdb_persistent_finish_trans3_commits().
This function walks all databases and checks for running trans3 commits.
It sends replies to all of them (with error code) and ends them.
To be called when a recovery finishes.
(cherry picked from commit
70ba153b532528bdccea70c5ea28972257f384c1)
Signed-off-by: Michael Adam <obnox@samba.org>
Michael Adam [Wed, 23 Feb 2011 16:37:42 +0000 (17:37 +0100)]
daemon: correctly end a running trans3_commit if the client disconnects.
(cherry picked from commit
9e0898db6df52d9bc799dd87bfea8c72d5f70ba0)
Signed-off-by: Michael Adam <obnox@samba.org>
Michael Adam [Wed, 23 Feb 2011 16:35:27 +0000 (17:35 +0100)]
persistent: add a client context to the persistent_stat and track the db_id
The db_id is tracked in the client context as an indication that a
transaction commit is in progress. This is cleared in the persistent_state
talloc destructor.
This is in order to properly treat running trans3_commits if the client
disconnects.
(cherry picked from commit
e886ff24f4e3e250944289db95916b948893d26c)
Signed-off-by: Michael Adam <obnox@samba.org>
Michael Adam [Tue, 22 Feb 2011 23:03:07 +0000 (00:03 +0100)]
persistent: reject trans3_control when a commit is already active.
This should actually never happen.
(cherry picked from commit
f416e76838fe2adf629d4356d1cc87054b1af164)
Signed-off-by: Michael Adam <obnox@samba.org>
Michael Adam [Tue, 22 Feb 2011 23:01:13 +0000 (00:01 +0100)]
persistent: allocate the persistent state in the ctdb_db struct in trans3_commit
Make sure that ctdb_db->persistent_state is correctly NULL-ed when
the state is freed. This way, we can use ctdb_db->persistent_state
as an indication for whether a transaction commit is currently
running.
(cherry picked from commit
761cb235193564a0f337d0308f0a9e6de0ef2710)
Signed-off-by: Michael Adam <obnox@samba.org>
Michael Adam [Tue, 22 Feb 2011 23:23:18 +0000 (00:23 +0100)]
persistent: add a ctdb_db context to the ctdb_persistent_state struct.
(cherry picked from commit
a14917c983c3b9bbbf38f5ddeecdbbe5bde32364)
Signed-off-by: Michael Adam <obnox@samba.org>
Michael Adam [Tue, 22 Feb 2011 23:00:04 +0000 (00:00 +0100)]
persistent: add a ctdb_persistent_state member to the ctdb_db context.
To be used for tracking running transaction commits through recoveries.
(Backported from commit
1237e15df4af58a3d220eea42a4b75e21e65029f)
Michael Adam [Tue, 22 Feb 2011 21:49:52 +0000 (22:49 +0100)]
persistent_callback: print "no error message given" instead of "(null)"
(cherry picked from commit
d871a38978219e004833608c11aae98fe47614b9)
Signed-off-by: Michael Adam <obnox@samba.org>
Michael Adam [Tue, 22 Feb 2011 21:47:30 +0000 (22:47 +0100)]
persistent: reduce indentation for the finishing moves in ctdb_persistent_callback
(cherry picked from commit
2c2d1646eb753ea9561f085bcb101153267b052b)
Signed-off-by: Michael Adam <obnox@samba.org>
Michael Adam [Tue, 22 Feb 2011 21:44:16 +0000 (22:44 +0100)]
persistent: if a node failed to update_record, trigger a recovery
and stop processing of the update_record replies in order to let
the recovery finish the trans3_commit control.
(cherry picked from commit
cab95570dc1eefb08abbac5ae411c29f699b51cc)
Signed-off-by: Michael Adam <obnox@samba.org>
Michael Adam [Tue, 22 Feb 2011 21:24:50 +0000 (22:24 +0100)]
persistent_store_timout: do not really time out the trans3_commit control in recovery
If a recovery was started, then all further processing of the update_record
controls sent by the trans3_commit control and timing them out is disabled.
The recovery should trigger sending the reply for the update record control
when finished.
(cherry picked from commit
983c1ca2e18ecd60fca69bfe9e116125cc695857)
Signed-off-by: Michael Adam <obnox@samba.org>
Michael Adam [Tue, 22 Feb 2011 21:24:50 +0000 (22:24 +0100)]
persistent_callback: ignore the update-recordreturn code of remote node in recovery
If a recovery was started, then all further processing of the update_record
controls sent by the trans3_commit control is disabled. The recovery should
trigger sending the reply for the update record control when finished.
(cherry picked from commit
12cf0619255b12230843cd8bb49cbfdea376ca2f)
Signed-off-by: Michael Adam <obnox@samba.org>
Rusty Russell [Mon, 12 Jul 2010 05:41:42 +0000 (15:11 +0930)]
config: wrap iptables in flock to avoid concurrancy.
When doing a releaseip event, we do them in parallel for all the separate
IPs. This creates a problem for iptables, which isn't reentrant, giving
the strange message:
iptables encountered unknown error "
18446744073709551615" while initializing table "filter"
The worst possible symptom of this is that releaseip won't remove the rule
which prevents us listening to clients during releaseip, and the node will be
healthy but non-responsive.
The simple workaround is to flock-wrap iptables. Better would be to rework
the code so we didn't need to use iptables in these paths.
CQ:S1018353
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from commit
72d6914ee913272312d7b68f1be5ad05ad06587d)
Signed-off-by: Michael Adam <obnox@samba.org>
Michael Adam [Thu, 12 Jan 2012 13:47:50 +0000 (14:47 +0100)]
New version 1.0.114.4
Includes vacuuming fixes and cleanup.
Michael Adam [Thu, 12 Jan 2012 11:46:06 +0000 (12:46 +0100)]
ltdb_server: improve logging in ctdb_ltdb_store_server
When logging which key to store/delete, list the ctdb_hash
and the jenkins_hash. In this version, the databases still
use the the old hash (with lots of hash collisions) while
the internal lists (vacuuming, schedule_for_deletion, ...)
already use the jenkins hash.
Michael Adam [Tue, 13 Dec 2011 13:48:38 +0000 (14:48 +0100)]
doc:ctdb.1: add generated files after update
Michael Adam [Thu, 22 Dec 2011 12:05:26 +0000 (13:05 +0100)]
doc:ctdb.1: explain that "ctdb listvars" does not list obsolete tunables any more
(cherry picked from commit
4ed94279f243729d521d365a44607fd086ae7c41)
Michael Adam [Tue, 13 Dec 2011 13:47:42 +0000 (14:47 +0100)]
doc:ctdb.1: adapt example output of "ctdb listvars" to current state
Michael Adam [Thu, 22 Dec 2011 11:13:10 +0000 (12:13 +0100)]
ctdb-tool: adapt formatting of "listvars" output to lenght of longest name
(cherry picked from commit
295168d99183d2d21f27a8ac6825d421fa866b28)
Michael Adam [Thu, 22 Dec 2011 11:52:55 +0000 (12:52 +0100)]
tunables: don't list obsolete tunables in the list_tunables control
(cherry picked from commit
d8ab86f0eb11437e50d18183858dd3177a8f61e6)
Michael Adam [Wed, 11 Jan 2012 23:08:03 +0000 (00:08 +0100)]
tunables: add a bool obsolete flag to the tunable_map list
(cherry picked from commit
1a7d9b25fdcf7b59598618d406c2a681c90d9163)
Michael Adam [Fri, 23 Dec 2011 16:23:07 +0000 (17:23 +0100)]
vacuum: improve logging in remove_record_from_delete_queue()
(cherry picked from commit
809d1f3b8a9232fce0376cb47e4f633b49237529)
Michael Adam [Fri, 23 Dec 2011 16:22:46 +0000 (17:22 +0100)]
vacuum: improve logging in delete_record_traverse()
(cherry picked from commit
40ca781b3305d9f0bfee5b109d4eefde461a3f13)
Michael Adam [Fri, 23 Dec 2011 16:10:20 +0000 (17:10 +0100)]
vacuum: improve logging in insert_record_into_delete_queue()
(cherry picked from commit
db59ac2ee6a9f5f7f088ebbc5e2800d9e57aae17)
Michael Adam [Fri, 23 Dec 2011 16:08:28 +0000 (17:08 +0100)]
vacuum: improve debugging in delete_queue_traverse()
(cherry picked from commit
2c44c6dc583872c603a399afb01936abcbb05158)
Michael Adam [Fri, 23 Dec 2011 14:14:06 +0000 (15:14 +0100)]
vacuum: add debug message to insert_record_into_delete_queue() for failed insert
(cherry picked from commit
ae78e8d991636a3a79dd9fbcefc15032afb08f55)
Michael Adam [Fri, 23 Dec 2011 09:37:25 +0000 (10:37 +0100)]
vacuum: improve debugging in insert_record_into_delete_queue()
(cherry picked from commit
6e6aaba28ac560d51d7450ebcf99babf868d0aea)
Michael Adam [Thu, 22 Dec 2011 23:14:18 +0000 (00:14 +0100)]
vacuum: initialize (to zero) the vacuum fetch list in ctdb_vacuum_init_vacuum_data()
(cherry picked from commit
772e08c29ef70a4cc73f7887939e5071a90a98cb)
Michael Adam [Thu, 22 Dec 2011 14:46:49 +0000 (15:46 +0100)]
vacuum: add statistics output for processing of the delete list
(cherry picked from commit
bec2ff46402b6d90521c75e8d46b9f6f2f47239a)
Michael Adam [Tue, 20 Dec 2011 14:58:34 +0000 (15:58 +0100)]
vacuum: add a comment explaining a race in the vacuum code.
(cherry picked from commit
07f1ba54b1617fadd6fe83826dd8e76f37cea071)
Michael Adam [Tue, 20 Dec 2011 14:50:51 +0000 (15:50 +0100)]
vacuum: factor out initialization of the vacuum data from ctdb_vacuum_db()
for readability
(cherry picked from commit
1963e33859e9dc51e9e88664c6d95e62c7c8b8be)
Michael Adam [Fri, 16 Dec 2011 22:16:27 +0000 (23:16 +0100)]
vacuum: streamline ctdb_process_delete_list()
Reducing indentiation and improving use of talloc.
(cherry picked from commit
11f52ba18370f1dda983f2263f9af82045b4fdea)
Michael Adam [Fri, 16 Dec 2011 22:15:51 +0000 (23:15 +0100)]
vacuum: factor processing of the delete list out of ctdb_vacuum_db()
(cherry picked from commit
cf68b762432a18e4a3cda137591b3f005c196906)
Michael Adam [Fri, 16 Dec 2011 16:00:07 +0000 (17:00 +0100)]
vacuum: factor processing of vacuum fetch lists out of ctdb_vacuum_db()
(cherry picked from commit
905fc9cd7126ab84c23d4a803c59e32b9c54e6f0)
Michael Adam [Fri, 16 Dec 2011 10:04:13 +0000 (11:04 +0100)]
vacuum: factor out full vacuum run out of ctdb_vacuum_db() as ctdb_vacuum_db_full()
(cherry picked from commit
e31594e0c13899e88013ce7eb7e3dc434c2f0723)
Michael Adam [Fri, 16 Dec 2011 09:59:26 +0000 (10:59 +0100)]
vacuum: factor out the fast vacuuming run into ctdb_vacuum_db_fast()
for readability
(cherry picked from commit
6596f2c4a6b4783b6498502fbbf50f87d4f59085)
Michael Adam [Fri, 16 Dec 2011 09:53:26 +0000 (10:53 +0100)]
vacuum: rename delete_tree to delete_list
It is not important for the user that this is implemented as a tree.
(cherry picked from commit
f686bc440560074c1ce866f2037ae3671f182185)
Michael Adam [Fri, 16 Dec 2011 09:49:41 +0000 (10:49 +0100)]
vacuum: appropriately rename vdata.list to vdata.vacuum_fetch_list
(cherry picked from commit
09c55b76c96c4ab3fe812763007b81cf0a2ac86c)
Michael Adam [Fri, 16 Dec 2011 09:46:54 +0000 (10:46 +0100)]
vacuum: introduce a helper variable for the vacuum fetch list in ctdb_vacuum_db()
Michael Adam [Fri, 16 Dec 2011 09:43:06 +0000 (10:43 +0100)]
vacuum: introduce a helper variable in add_record_to_vacuum_fetch_list() to reduce verbosity
(cherry picked from commit
48fe56ea27e3649ae7a67257fdce54f973e7c3c5)
Michael Adam [Tue, 13 Dec 2011 16:43:58 +0000 (17:43 +0100)]
tools/ctdb: allow the "ctdb vacuum" stub to be run without the daemon running.
(cherry picked from commit
e71afbf731f06a3b89b64220c7fd1b1ea5492d4a)
Michael Adam [Tue, 13 Dec 2011 16:43:10 +0000 (17:43 +0100)]
tools/ctdb: remove functionality of "ctdb vacuum", just keeping a stub.
(cherry picked from commit
8fc4d5018ee6eee8856634095694d3ad0de5b4ae)
Michael Adam [Tue, 13 Dec 2011 16:32:45 +0000 (17:32 +0100)]
vacuum: locally delete the records in the delete_tree that have be deleted remotely
This way, especially records added in the fast vacuuming runs
are deleted direclty instead of being handed back to the
repack run which will probably not hit because of the vacuum limit.
(cherry picked from commit
ea3e640a28178ddcb85285f4efec62ccba2483d9)
Michael Adam [Tue, 13 Dec 2011 16:30:39 +0000 (17:30 +0100)]
vacuum: improve vacuuming statistics for handling the delete_tree.
Track the number of records failed to delete remotely with the
TRY_DELETE_RECORDS control.
And add a number of records left to delete locally.
(cherry picked from
536aad024d52f2c32dd397fc86294c963b91341b)
Michael Adam [Tue, 13 Dec 2011 15:05:05 +0000 (16:05 +0100)]
vacuum: fix and improve a comment
(cherry picked from commit
0ed349c24c7399b3b440c1f5d573db77ddb988b3)
Michael Adam [Tue, 13 Dec 2011 14:59:38 +0000 (15:59 +0100)]
vacuum: appropriately rename delete_traverse() to delete_marshall_traverse()
(cherry picked from commit
7c5b27bc8dce9e30f2aff05297113da981f7b9a3)
Michael Adam [Tue, 13 Dec 2011 14:58:52 +0000 (15:58 +0100)]
vacuum: initialize the delete_count in the vacuum data
(cherry picked from commit
970b8fc60857ad2f18f45312d355c571fb2f0246)
Michael Adam [Tue, 13 Dec 2011 13:21:04 +0000 (14:21 +0100)]
vacuum: fix the VacuumInterval tunable's type to uint32_t in get_vacuum_interval()
(cherry picked from commit
a5a29bf92b23e9b5a1523b06461eced422b22ffe)
Michael Adam [Tue, 13 Dec 2011 13:33:08 +0000 (14:33 +0100)]
vacuum: add new tunable VacuumInterval and mark Vacuum{Default,Min,Max}Interval obsolete
And use VacuumInterval instead of VacuumDefaultInterval in the code.
(cherry picked from commit
78530f40338f511a7cd1d33ada450905742bfa8f)
Michael Adam [Tue, 13 Dec 2011 13:36:35 +0000 (14:36 +0100)]
vacuum: remove struct vacuum_tuning_data (unused)
(cherry picked from commit
1f62afbe1ae0b3e7dbad4081d1b51e3ab6ee5c39)
Michael Adam [Tue, 13 Dec 2011 13:32:12 +0000 (14:32 +0100)]
vacuum: use get_vacuum_interval() in ctdb_vacuum_event()
(instead of using the tunable directly)
(cherry picked from commit
e40fa9efd1c77aa35c917aba6e6e4d21d9c5cdea)
Michael Adam [Tue, 13 Dec 2011 13:16:23 +0000 (14:16 +0100)]
vacuum: change get_vacuum_interval() to simply return the VacuumDefaultInterval tunable
(cherry picked from commit
621c3ce957cb946ca05a2d527c037746ae2f01d3)
Michael Adam [Tue, 13 Dec 2011 13:06:09 +0000 (14:06 +0100)]
vacuum: remove update_tuning_db()
(cherry picked from commit
d93750ea3f5cb854e5893e53795fd08656e7c12c)
Michael Adam [Thu, 7 Apr 2011 10:17:42 +0000 (12:17 +0200)]
ctdb_ltdb_store_server: when storing a record that is not to be scheduled for deletion, remove it from the delete queue
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit
489148e465e2b8aed87ea836e3518f43490671ca)
Michael Adam [Thu, 7 Apr 2011 10:17:16 +0000 (12:17 +0200)]
vacuum: add ctdb_local_remove_from_delete_queue()
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit
a5065b42a98c709173503e02d217f97792878625)
Michael Adam [Thu, 22 Dec 2011 17:09:36 +0000 (18:09 +0100)]
rb_tree: fix possible access-after-free-error in trbt_traversearray32_node
When the traverse callback frees the current node, the traverse of the
rbtree can fail (the next node->right fails since node is not there any more...).
This is fixed by introducing variables to store the right (and left)
pointers before the callback is called.
(cherry picked from commit
8b0caaeed154d26c67a73659d3bbbdd63b21be11)
Michael Adam [Thu, 22 Dec 2011 16:18:38 +0000 (17:18 +0100)]
fix zero-initialization of header in _ctdbd_allocate_pkt to the correct size
(cherry picked from commit
6c4d1b768ccade344a7462ecad7cf5cbbf3873d4)
Ronnie Sahlberg [Wed, 14 Dec 2011 01:52:35 +0000 (12:52 +1100)]
Michael Adam [Sat, 3 Dec 2011 01:15:30 +0000 (02:15 +0100)]
traverse: fix traversing with empty records by adding a new (internal) control CTDB_CONTROL_TRAVERSE_START_EXT
By this, the original CTDB_CONTROL_TRAVERSE_START control that is
used by e.g. samba's smbstatus, is not changed, so that samba
continues working without code change.
The CTDB_CONTROL_TRAVERSE_START currently just adds the "withemptyrecords"
flag to the state and processon on as CTDB_CONTROL_TRAVERSE_START_EXT.
(cherry picked from
8281bb210858ed04992eacea7f6d02261e0fc1b1)
Michael Adam [Tue, 10 Jan 2012 13:15:26 +0000 (14:15 +0100)]
doc: commit generated ctdb.1 manpages after xml change
Michael Adam [Tue, 29 Nov 2011 12:41:16 +0000 (13:41 +0100)]
doc: document the "--print-recordflags" parameter to "ctdb"
(cherry picked from commit
5b527e6127a649fa80dcf9a7599b22bcb7cd3640)
Michael Adam [Tue, 29 Nov 2011 09:24:52 +0000 (10:24 +0100)]
ctdb: add an option --print-recordflags to trigger printing record flags in catdb and dumpdbbackup
This changes the default behaviour to not print record flags.
(cherry picked from commit
2d2ce07c51055d9400b22cd3c1fd682597cb921c)
Michael Adam [Tue, 29 Nov 2011 12:39:08 +0000 (13:39 +0100)]
doc: document the "--print-hash" parameter to "ctdb".
(cherry picked from commit
998b7f4450026051867525d91c6d8dcbd2326ab5)
Michael Adam [Mon, 3 Jan 2011 12:15:34 +0000 (13:15 +0100)]
ctdb: add output of jenkins hash to ctdb catdb (with --print-hash)
Michael Adam [Mon, 28 Nov 2011 23:56:23 +0000 (00:56 +0100)]
ctdb: add an option --print-hash to enable printing of record hashes when dumping dbs
(cherry picked from commit
efc033c28ade97f9884794256d59a4553e052d5f)
Michael Adam [Tue, 29 Nov 2011 12:36:38 +0000 (13:36 +0100)]
doc: document the "--print-lmaster" parameter to "ctdb"
(cherry picked from commit
7e7d86ac9b05f8b67414200adc8ac87cfccf26cf)
Michael Adam [Mon, 28 Nov 2011 16:36:03 +0000 (17:36 +0100)]
ctdb: add an option --print-lmaster to enable printing of lmaster in "ctdb catdb"
(cherry picked from commit
326f88ef622620cb9e0569c4497bc0e86124beaa)
Michael Adam [Tue, 29 Nov 2011 12:31:11 +0000 (13:31 +0100)]
doc: document the "--print-datasize" parameter to "ctdb"
(cherry picked from commit
9e8045a5b0d8bd9ab2e0343b329de9f45b883531)
Michael Adam [Mon, 28 Nov 2011 16:19:03 +0000 (17:19 +0100)]
ctdb: add an option --print-datasize to only print datasize instead of dumping data in db dumps
Used in catdb and dumpdbbackup.
(cherry picked from commit
dd866116041e71cbf91e7fd91edcc9501634051d)
Michael Adam [Tue, 29 Nov 2011 12:28:35 +0000 (13:28 +0100)]
doc: document the --print-emptyrecords switch in the "ctdb" manpage.
(cherry picked from commit
7b4663dd2454b798841815044f898dead9b734e2)
Michael Adam [Mon, 28 Nov 2011 16:11:16 +0000 (17:11 +0100)]
ctdb: add an option --print-emptyrecords to enable printing of empty records in dumping databases
this option is used with the commands catdb and dumpdbbackup.
(cherry picked from commit
6ec68a2e667f66d2b194fe48cb75229a2777842e)
Michael Adam [Wed, 22 Dec 2010 11:45:06 +0000 (12:45 +0100)]
client: add version ctdb_traverse_ext() of ctdb_traverse() that can list empty records.
(cherry picked from commit
4ada4bfc4510886c5c7fcf49e09711b9d2dcb75d)
Michael Adam [Sun, 27 Nov 2011 22:16:33 +0000 (23:16 +0100)]
traverse: add a flag to enable transferring empty records in cluster wide traverse
This will be useful for also printing information about empty/deleted
records in "ctdb catdb", e.g. for debugging vacuuming issues.
(cherry-picked from commit
ddc5da3a0df7701934404192a0a0aa659a806acb)
Michael Adam [Sat, 26 Nov 2011 00:06:13 +0000 (01:06 +0100)]
vacuum: update (C) from vacuum rewrite
(cherry picked from commit
a95b524e57b861416f65baef7a0235bf7e394cef)
Michael Adam [Fri, 25 Nov 2011 12:55:20 +0000 (13:55 +0100)]
vacuum: write a big and up-to-date explaining comment for ctdb_vacuum_db()
(cherry picked from commit
6b9d308985ef2363c62fde6340ad37d56f6d7702)
Michael Adam [Fri, 25 Nov 2011 12:54:48 +0000 (13:54 +0100)]
vacuum: fix a comment typo in vacuum_traverse()
(cherry picked from commit
88ad1e75c686cd47aec18b0a8eb93c3b125f0e97)
Michael Adam [Fri, 25 Nov 2011 12:35:05 +0000 (13:35 +0100)]
vacuum: fix horrible indentation/whitespacing for ctdb_vacuum_event()
(cherry picked from commit
0f070a65a322bd6d3240ca96a9e5c3633ef647bf)
Michael Adam [Fri, 25 Nov 2011 12:34:15 +0000 (13:34 +0100)]
vacuum: fix a comment typo in vacuum_traverse()
(cherry picked from commit
39a5c0d8cca4ce5e8bc7055739a6377fb2dc36e0)
Michael Adam [Fri, 25 Nov 2011 12:25:27 +0000 (13:25 +0100)]
vacuum: fix an abundance of indentation white spaces in ctdb_vacuum_db()
(cherry picked from commit
4ae564e415560cde7e4cd4c5a1d7699f3d49621b)