git.samba.org - obnox/glusterfs.git/log

fuse: forbid only access to security.selinux xattr if not mounted with 'selinux'

Originally, all selinux.* xattrs were forbidden, causing
for example Samba's acl_xattr module which uses security.NTACL
to fail without the 'selinux' mount option, which is confusing
at least. This change specializes the check to the security.selinux
attribute, so other selinux.* attributes work with or without the option.

Change-Id: I9d3083123efbf403f20572cfb325a300ce2e90d9
Signed-off-by: Michael Adam <obnox@samba.org>

tests: Introduce a Vagrant VM based make test

This is still a work-in-progress.
Comments welcome.

Have disabled regression tests.

Change-Id: Ic87801172c8b614cdecbdf2a765e1b3370a5faf7
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>

cli/geo-rep : remove unused code

Change-Id: I8a8e27b4d6c35ea5e57bd0b556fd2c6ab7b496ab
BUG: 1285968
Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com>
Reviewed-on: http://review.gluster.org/12771
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

features/shard: Set ctime to 0 in fsync callback

... to indicate to md-cache that it should not be caching
file attributes.

Change-Id: Iaef9bf7fec8008ca47d682b4b15984f26421bcd6
BUG: 1285660
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/12759
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>

dht : changing variable type to avoid overflow

For layout computation we find total size of the cluster
and store it in an unsigned 32 bit variable. For large
clusters this value may overflow which leads to wrong
computations and for some bricks the layout may overflow.
Hence using unsigned 64 bit to handle large values.

Change-Id: I7c3ba26ea2c4158065ea9e74705a7ede1b6759c7
BUG: 1282751
Signed-off-by: Sakshi Bansal <sabansal@redhat.com>
Reviewed-on: http://review.gluster.org/12597
Reviewed-by: Susant Palai <spalai@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>

features/bit-rot: Fix NULL dereference

Problem:
By the time br_stub_worker is accessing this->private in it's
thread, 'init' may not have set 'this->private = priv'. This
leads to NULL dereference leading to brick crash.

Fix:
Set this->private before launching these threads.

Change-Id: Ic797eb195fdd0c70d19f28d0b97bc0181fd3dd2f
BUG: 1285616
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/12754
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

glusterd: glusterfsd to support volfile-server-transport type "unix"

If glusterfsd uses transport type "unix", 'addrstr' can be empty. So,
it may fail to fetch the trusted volfile in this case. This patch allows
to fetch volfile even if addrstr is empty.

Change-Id: I831c5cd0b07b90ae843ec63f0fad9241d9407f6b
BUG: 1279484
Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com>
Signed-off-by: Mohamed Ashiq <mliyazud@redhat.com>
Reviewed-on: http://review.gluster.org/12757
Reviewed-by: Vijaikumar Mallikarjuna <vmallika@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>

glusterd/tier: Reset to reconfigured values after detach-tier

After detach-tier commit , we have to reset the option
reconfigured for tier volume

Change-Id: Iae0210259720d6ac14ccc0cc339dc9f54a0c4571
BUG: 1285046
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: http://review.gluster.org/12736
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Tested-by: Dan Lambright <dlambrig@redhat.com>

cluster/afr: change data self-heal size check for arbiter

Size mismatch should consider that arbiter brick will have zero size file to
prevent data self-heal to spuriously trigger/assuming need of self-heals.

Change-Id: I179775d604236b9c8abfa360657abbb36abae829
BUG: 1285634
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/12755
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>

Tiering: change of error message for "v tier <vname> detach status"

If the volume was not a tiered volume then empty status was being
printed instead of an error message.

Change-Id: I13ccb16e1562966976a48d9365ced4c8a124de59
BUG: 1284357
Signed-off-by: hari gowtham <hgowtham@redhat.com>
Reviewed-on: http://review.gluster.org/12713
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: Dan Lambright <dlambrig@redhat.com>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>

cluster/afr: Remember flags sent by create fop

Problem:
In create fop, afr doesn't remember the flags. When afr has to perform
fixing of the fd that needs to be opened on other bricks because the brick
was down at the time of create, the flags with which it needs to send open
are not present.

Fix:
Remember the flags in the fd_ctx.

Thanks to Nitya for showing us the problem in re-open with the flags.

Change-Id: I8ce1eb50c35fc0722cfc25cb4b6d234ef56180e5
BUG: 1285173
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/12739
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>

cli: Indicate which brick of the replica is the arbiter

Enhances the cli output for arbiter volumes as requested in the BZ.
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Change-Id: I28cc34d7d19def043d54291cede25a58dbcc5051
BUG: 1285288
Reviewed-on: http://review.gluster.org/12747
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>

glusterd: copy snapshot object during duplication of volfile

When creating duplicate volfile for hot/cold tier, we need to
copy the snapshot object in to volfile as it requires to generate
snapshot brick volfile.

Change-Id: I39ccfa20cd1c16ef2801901e3cd3a31c76f8995d
BUG: 1284789
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: http://review.gluster.org/12734
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>

tier/test: Fixing tests/basic/tier/legacy-many.t

Change-Id: Icbd83afdeac053aec5b3b8fa19665a2908e87d8e
BUG: 1285483
Signed-off-by: Joseph Fernandes <josferna@redhat.com>
Reviewed-on: http://review.gluster.org/12751
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Tested-by: Dan Lambright <dlambrig@redhat.com>

tier/ctr: Correcting rename logic

Problem: When a file with old_file_name and GFID_1 is renamed with a new_file_name
which already exists and with GFID_2, this is what happens in linux internaly.
   a. "new_file_name" is unlinked for GFID_2
   b. a hardlink "new_file_name" is created to GFID_1
   c. "old_file_name" hardlink is unlinked for GFID_2.
Well this is all internal to linux, and gluster just issues a rename system call
at POSIX layer. But CTR Xlator doesn't delete the entries corresponding to the
"new_file_name" and GFID_2. Thus leaving the stale entry in the DB.
The following are the implications.
a. Promotion are tried on these stale entries which will fail and show
false results in the status of migration,
b. GFID_2 Files with 2 hardlinks, which will have only one hardlink
after the rename will not be promoted or demoted as the DB shows 2 entries.

Solution: Delete the older database entry for the replaced hardlink

Change-Id: I4eafa0872253e29ff1f0bec4283bcfc579ecf0e2
BUG: 1284090
Signed-off-by: Joseph Fernandes <josferna@redhat.com>
Reviewed-on: http://review.gluster.org/12711
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Tested-by: Dan Lambright <dlambrig@redhat.com>

glusterd: Change volume start into v3 framework

As part of volume start, if the volume is of tier type
then we need to start tiering daemon also. But before
starting tier daemon all the bricks should be started.

So by changing volume start into v3 framework, we can
do tier start in post validate phase

Change-Id: If921067f4739e6b9a3239fc5717696eaf382c22a
BUG: 1284372
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: http://review.gluster.org/12718
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>

tools/glusterfind: add query command to list files

When session information is maintained outside Gluster, there needs to
be some mechanism to list files starting from a time-stamp. This patch
implements the feature via the "query" command-line option.

The only caveat is that the first time the query command is run for the
volume, it will likely report that "historical changelogs are not
available". This is due to the fact that changelogs had not been turned
on for the volume so far. So the volume options need to be turned on
outside glusterfind or the since_time need to be greater than the
current time when the query command is run for the very first time for
the volume. The query command turns on the required volume options for
collecting changelogs.

Change-Id: I6cb7a57a5ecd166210e2eb4deede06d40ccfa996
BUG: 1272006
Signed-off-by: Milind Changire <mchangir@redhat.com>
Reviewed-on: http://review.gluster.org/12362
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>

features/index : Readdirp performance improvement

Things done :
Index now maintains two different directories :
1) for pending xattrs (trusted.afr.volname-client-n,
trusted.ec.dirty)

2) for on going xattrs (trusted.afr.dirty)

Based on the xattr, index decides the directory
to add/del entry in/from.

Change-Id: Ie04c02ea2d862cf80426a871a9a1e80b0773d9fd
BUG: 1250803
Signed-off-by: Anuradha Talur <atalur@redhat.com>
Reviewed-on: http://review.gluster.org/12468
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>

glusterd/bitrot : Integration of bad files from bitd with scrub status command

Currently scrub status command is not displaying list of all the bad files. All
the bad files are avaliable in the bitd daemon.

With this patch it will dispaly list of all the bad file's in the scrub
status command.

Change-Id: If09babafaf5d7cf158fa79119abbf5b986027748
BUG: 1207627
Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com>
Reviewed-on: http://review.gluster.org/12720
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>

features/shard: Eliminate extra update to postbuf in writev

After an extending write is complete, shard translator updates postbuf
at two places:
1. shard_update_file_size_cbk(), and
2. shard_post_update_size_writev_handler().

This can lead to unexpected behavior if md-cache is part of the client
stack and caches and serves values returned by shard translator in
postbuf. This patch eliminates the update to postbuf in
shard_post_update_size_writev_handler().

Change-Id: I9d107bf57baad66886eebec14aa369b6a3c88c49
BUG: 1284365
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/12717
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>

quota: fix backward compatibility of quota xattr version

quota-version features is implemented for 3.7.6
please see below patch for more details:
http://review.gluster.org/#/c/12386

Problem is when quota is already enabled,
we suffix 0 to contri xattr key. for backward compatibility
don't add suffix if quota-version is 0

Change-Id: Id7d713b18d989e4e86019969eb511617848127f2
BUG: 1283567
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/12660
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>

afr:vol heal info fails when transport.socket.bind-address is set in glusterd

Problem: If glusterd's bind address is specified using transport.socket.bind- address
option to a specific IP, the heal command 'gluster volume heal <VOLNAME> info' fails
with error '<vol_name>: Not able to fetch volfile from glusterd.Volume heal failed.'
This patch uses Unix domain socket transport type of libglfapi to get the volfile.

Change-Id: Ic4239b339f0246d77f1622c07d5f4708129d30c7
BUG: 1277997
Signed-off-by: Mohamed Ashiq <mliyazud@redhat.com>
Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com>
Reviewed-on: http://review.gluster.org/12508
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Anuradha Talur <atalur@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>

geo-rep: Fix syntax errors in GsyncdError

%s was not replaced by actual values in GsyncdError

BUG: 1279921
Change-Id: I3c0a10f07383ca72844a46f930b4aa3d3c29f568
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/12566
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>

marker : fixing memory leak issue in marker

In marker_readdirp_cbk, variable resolvedpath is not properly freed
and because of which there was a memory leak. The patch fixes it.

Change-Id: I9d80f72e3551aa912369257da3e8e2b261a2067f
BUG: 1284419
Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com>
Reviewed-on: http://review.gluster.org/12719
Reviewed-by: Vijaikumar Mallikarjuna <vmallika@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>

geo-rep: Fix Geo-rep logging to log datetime in GMT/UTC

Geo-rep is logging in Local time, all other Gluster logs are in
GMT/UTC. It is very difficult to co-relate Geo-rep logs with
other Gluster logs.

BUG: 1282331
Change-Id: Ieae8bda7e4788e587cf4595e21e0e772c210cfbb
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/12583
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>

tools/glusterfind: StrictHostKeyChecking=no for ssh/scp verification

Also do not use scp command in case copy file from local
node.

Change-Id: Ie78c77eb0252945867173937391b82001f29c3b0
Signed-off-by: Aravinda VK <avishwan@redhat.com>
BUG: 1260918
Reviewed-on: http://review.gluster.org/12124
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>

cluster/tier: readdirp to cold tier only

It is possible a file would get migrated in the middle
of a readdir operation. If there are four subvolumes A,B,C,D,
and if readdir reads them in order and reaches subvol B,
then, if a file is moved from D to A, it will not be included
in the readdir output.

This phenonema has pre-existed in DHT migration but is more
apparent in tiering.

When a file is moved off the hashed subvolume a T file is created.
For tiering, we will make the cold subvolume the hashed subvolume.
This will ensure the creation of a T file. Readdir will not skip T
files in the tier translator.

Making the cold subvolume the hashed subvolume ensures the T
files created on promotions or creates will be less likely to
fill the volume.

Creates still put the data on the hot subvolume.

Change-Id: Ifde557d3d0e94a4570ca9f115adee3db2ee75407
BUG: 1281598
Signed-off-by: Dan Lambright <dlambrig@redhat.com>
Reviewed-on: http://review.gluster.org/12530
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: N Balachandran <nbalacha@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>

features/bit-rot: scrubber changes for getting the list of bad objects from stub

Change-Id: I62885e4aba4a9b345db3c78c3291d563ff3d3567
BUG: 1207627
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/12654
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

features/changelog: Capture FXATTROP and XATTROP in changelog

GEO-REP INTEROP WITH SHARD FEATURE

shard xlator updates size of the file using FXATTROP
or XATTROP. Hence record the same in changelog.

Change-Id: Ie0c21e9326da05ea78dc1ef3fd32a90ef38b4bb9
BUG: 1265148
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/12225
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

features/bit-rot: stub changes for showing bad objects in the status

Change-Id: If905132f6f1df4aebd9ab255e1e8c59902f84fe5
BUG: 1207627
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/12503
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

glusterfsd: To support volfile-server-transport type "unix"

glusterfsd fails if the glusterd is bind to specific-IP address.
This patch helps glusterfsd to get the volfile using Unix domain socket.
glusterfs -s <unix socket path> --volfile-server-transport unix
--volfile-id <volume-name> <mount-point>
The patch checks if the volfile-server-transport is of type "unix",
If It is then uses rpc_transport_unix_options_build to get the volfile.

Change-Id: I81b881e7ac5a3a4f2ac83c789c385cf547f0d53e
BUG: 1279484
Signed-off-by: Mohamed Ashiq <mliyazud@redhat.com>
Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com>
Reviewed-on: http://review.gluster.org/12556
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>

glusterd: cli command implementation for bitrot scrub status

CLI command for bitrot scrub status will be :

gluster volume bitrot <volname> scrub status

Above command will show the statistics of bitrot scrubber.

Upon execution of this command it will show some common
scrubber tunable value of volume <VOLNAME> followed by
statistics of scrubber statistics of individual nodes.

sample ouput for single node:

Volume name : <VOLNAME>

State of scrub: Active

Scrub frequency: biweekly

Bitrot error log location: /var/log/glusterfs/bitd.log

Scrubber error log location: /var/log/glusterfs/scrub.log

=========================================================

Node name:

Number of Scrubbed files:

Number of Unsigned files:

Last completed scrub time:

Duration of last scrub:

Error count:

=========================================================

This is just infrastructure. list of bad file, last scrub
time, error count value will be taken care by
http://review.gluster.org/#/c/12503/ and
http://review.gluster.org/#/c/12654/ patches.

Change-Id: I3ed3c7057c9d0c894233f4079a7f185d90c202d1
BUG: 1207627
Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com>
Reviewed-on: http://review.gluster.org/10231
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>

Tiering: Adding space between error meassge for detach-tier

Change-Id: I730cf7fa6fbfb3842d337cd3d7b8394b9c3876d8
BUG: 1283488
Signed-off-by: hari gowtham <hgowtham@redhat.com>
Reviewed-on: http://review.gluster.org/12657
Reviewed-by: mohammed rafi kc <rkavunga@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Tested-by: Dan Lambright <dlambrig@redhat.com>

core : Use correct path in dlopen for socket.so

This patch fixes the path for socket.so file while loading the so dynamically.
Also for config.memory-accounting & config.transport voltype is changed to
glusterd to fix the warning message coming from xlator_volopt_dynload

Change-Id: I0f7964814586f2018d4922b23c683f4e1eb3098e
BUG: 1283485
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/12656
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>

cluster/ec: Mark internal fops appropriately

1) Mark read fops in read-modify-write by EC as internal.
2) Handle uid/gid set/reset correctly

BUG: 1282761
Change-Id: I5c1ce0cd6213367eaead5fed33aa2397c4e46df7
Signed-off-by: Xavier Hernandez <xhernandez@datalab.es>
Reviewed-on: http://review.gluster.org/12599
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>

mgmt/gluster: Handle tier brick volgen

Index xlator watches only some xattrs based on type of volume. i.e.
disperse/afr. When the volume becomes tiered then index is not adding these
options in the volfile leading to no maintenance of indices. Thus no
proactive self-heals. With this fix, we write brick volfiles considering
the type of volume they belong to.

BUG: 1250803
Change-Id: Ibe8f2d4ad5cb350306ab7ca0753e0f9a40b96a26
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/12595
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaushal M <kaushal@redhat.com>

tools/glusterfind: Handle Keyboard interrupt

Do not print Python traceback when glusterfind command is interrupted

Change-Id: I67383534f965e410fef7ce09798e9d435ef738ae
Signed-off-by: Aravinda VK <avishwan@redhat.com>
BUG: 1243812
Reviewed-on: http://review.gluster.org/11698
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

geo-rep: Fix FD leak from Active Geo-rep worker

Active worker tries to acquire lock in each iteration. On every successfull
lock acqusition it was not closing previously opened lock fd.

To see the leak, get the PID of worker,
ps -ax | grep feedback-fd
watch ls /proc/$pid/fd

BUG: 1225566
Change-Id: Ic476c24c306e7ab372c5560fbb80ef39f4fb31af
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/12332
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

geo-rep: Don't log geo-rep safe errors in mount logs

ENOENT is a safe error for geo-replication in case of
rm -rf. RMDIR is recorded in changelog of each brick,
geo-rep processes all changelogs among which one will
succeed and rest will get ENOENT which can be ignored.
Similarly ENOENT can also be ignored in case of all
unlink operation during changelog replay that can
happen when worker goes down and comes back.

Change-Id: I6756f8f4c3fce7a159751a2bfce891ff16ad31a4
BUG: 1250009
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/11833
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

afr: Drop compatibility lock for data self-heal

In glusterfs 3.4 and older, AFR did not take locks in self-heal domain
during data self-heal. So this compat lock in data domain was added to prevent
older clients from trying to heal a file while an existing self-heal was going
on by a newer client. But the side effect was that all appending writes (which
take full locks in data domain) from mounts would be stalled until self-heal
was complete.

Since glusterfs 3.4 is not supported anymore, remove the compat lock.

Change-Id: I31c8e4d7f3364f769a14eec295154e3c40d9f78e
BUG: 1283032
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/12602
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>

glusterd/geo-rep: Adding ssh-port option for geo-rep create

Geo-replication uses default ssh port 22 for setup.
i.e., to distribute ssh keys to slaves. In container
environments, custom port number might be used.
Hence to support custom port number for ssh, option
is provided in geo-rep create command to take the
same.

Change-Id: I0fb61959b1c085342b8e4c21ac4e076fba5462f1
BUG: 1276028
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/12504
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

cluster/tier: Do not delete linkto file on demotion

The current DHT migration code will always delete the
src linkto file after migration as dht always moves
files to the hashed subvol. This is not the case in tiering.
The lack of linkto files causes rename to fail leaving 2 files
with the same name but different gfids on the volume.
Modified to leave the linkto file behind if the source
volume is the hashed subvolume.

Change-Id: I2b99f7d34b4b719aee6232dc40c6a8f8ba88225d
BUG: 1279376
Signed-off-by: N Balachandran <nbalacha@redhat.com>
Reviewed-on: http://review.gluster.org/12551
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Tested-by: Dan Lambright <dlambrig@redhat.com>

glusterd/afr : Readdirp performance improvement

Add xlator options to index xlator with xattrs that
it needs to keep track of.

Change-Id: If818673be5e626f77e65cc3a340f8cdd624179c2
BUG: 1250803
Signed-off-by: Anuradha Talur <atalur@redhat.com>
Reviewed-on: http://review.gluster.org/12467
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>

cluster/ec: Mark self-heal fops as internal

Change-Id: I8ae7af266d3e00460f0cfdc9389a926e5f2fee36
BUG: 1282761
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/12598
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>

build: add libglusterfs/src/defaults.c to .gitignore

defaults.c is auto-generated since commit
fb2a511493868a9ff0c2926537a4d1d23821ce38. Hence adding it to .gitignore
to stop git from complaining when switching branches.

Change-Id: I586e20f3fe27124c0dc6ba7d151f957d3662a7d0
BUG: 1271325
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/12603
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>

core: Add experimental xlator directory

Added an experimental xlator directory under ./xlators/

The intent of this directory is presented in the README.md
that accompanies this commit.

This directory can be disabled from being compiled using,
- configure --disable-experimental

Change-Id: I047f380c91a082d111432f8bbdbd4d7bdcbaa809
Signed-off-by: Shyam <srangana@redhat.com>
Reviewed-on: http://review.gluster.org/12321
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>

glusterd.service: Ensure rpcbind is started before glusterd

Having the rpcbind.service under the `After` option only ensures that
glusterd.service is started after rpcbind.service if both are
enabled/started at the same time. It doesn't ensure that starting
glusterd.service will start rpcbind.service.

The systemd.unit(5) man page suggests to use both the `Requires` and
`After` options to ensure that rpcbind is started before glusterd,
whenever glusterd is started.

BUG: 1282915
Change-Id: Iee69965486be08711299aba235f7b00c3e2fe7e9
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/12605
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Anand Nekkunti <anekkunt@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>

protocol/client: prevent use-after-free of frame->root

A regression failure generated a coredump on the glusterfs-client side:

  (gdb) f 0
  #0  0x00007fba6cd76432 in client_submit_request (this=0x7fba68006fc0,
                  req=0x7fba6579aa70, frame=0x7fba5c0058cc,
                  prog=0x7fba6cfb53c0 <clnt3_3_fop_prog>, procnum=41,
                  cbkfn=0x7fba6cd9206d <client3_3_release_cbk>,
                  iobref=0x0, rsphdr=0x0, rsphdr_count=0,
                  rsp_payload=0x0, rsp_payload_count=0, rsp_iobref=0x0,
                  xdrproc=0x7fba79801075 <xdr_gfs3_release_req>) at
  /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/xlators/protocol/client/src/client.c:324
  324                   frame->root->ngrps = ngroups;
  (gdb) l
  319                   gf_msg_debug (this->name, 0, "rpc_clnt_submit failed");
  320           }
  321
  322           if (!conf->send_gids) {
  323                   /* restore previous values */
  324                   frame->root->ngrps = ngroups;
  325                   if (ngroups <= SMALL_GROUP_COUNT)
  326                           frame->root->groups_small[0] = gid;
  327           }
  328
  (gdb) p *frame->root
  Cannot access memory at address 0x64185df000000000

After looking at this in more detail, the flow is like this:

  client_submit_request()
    |
    '- rpc_clnt_submit() // on line 314
         |
         '- cbkfn() // = client3_3_release_cbk
              |
              :- STACK_DESTROY (frame->root);
         .----'
    .----'
    |
    :- frame->root->ngrps = ngroups; // on line 324
    '

So, there is a use-after-free, and it is not needed to restore the
previous groups in frame->root.

Change-Id: I9e7d712183692ed92cfc2f75cd3c2781a9db20e2
BUG: 128128
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/12575
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>

libgfapi: To support set_volfile-server-transport type "unix"

This patch helps libgfapi to get the volfile using Unix domain socket.
run the attachment file in the bug to test.
The patch checks if the glfs_set_volfile_server transport is of type "unix",
If It is then uses rpc_transport_unix_options_build to get the volfile.

Change-Id: Ifd5d1e7c0d8cc9a906c3c3355b8977141e892a2f
BUG: 1279739
Signed-off-by: Mohamed Ashiq <mliyazud@redhat.com>
Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com>
Reviewed-on: http://review.gluster.org/12563
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Poornima G <pgurusid@redhat.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>

geo-rep: Allow setting config remote_gsyncd

Restrictive ssh is not used in containerized environment
where networking configuration is "net=host". SSH Pem keys
pushed to the slave without gsyncd path in it. (Patch #12459)

Actual remote_gsyncd path need to be set to actual path of gsyncd.
With this patch, remote_gsyncd is removed from reserved option list.

Change-Id: Ia2063e4654e378b62b2414bdad21143c86ad1b9a
Signed-off-by: Aravinda VK <avishwan@redhat.com>
BUG: 1276028
Reviewed-on: http://review.gluster.org/12472
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

geo-rep: New Config option for ssh_port

If different port used for SSH instead of 22, Geo-replication
was failing to establish SSH connection.

ssh_port option can be added using config:ssh_command and
config:ssh_command_tar, but user has to remember complete
ssh command used with parameter to add/modify ssh port.

This patch adds new config option for ssh_port,

gluster volume geo-replication <MASTERVOL> <SLAVEHOST::<SLAVEVOL> \
config ssh_port 52022

Change-Id: I7753a09485f0b1f49d2b2a80b962c720817c96f4
Signed-off-by: Aravinda VK <avishwan@redhat.com>
BUG: 1276028
Reviewed-on: http://review.gluster.org/12444
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

tests: spurious failure fix for bug-948686.t

Ensured import volume and volume start doesn't race in volinfo by refcounting
volinfo.

Change-Id: I7467eccaba9a00fd63ba0121d8157df24d1c00a6
BUG: 1258714
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/12329
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>

core: use syscall wrappers instead of direct syscalls - regression test

various xlators and other components are invoking system calls
directly instead of using the libglusterfs/syscall.[ch] wrappers.

If not using the system call wrappers there should be a comment
in the source explaining why the wrapper isn't used.

Change-Id: Id2207deb81a75e1af6f34bf857e74725f8bb532f
BUG: 1267967
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/12410
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>

cluster/tier make cache mode default for tiered volumes

The default mode for tiered volumes must be cache. The current
test mode was for engineering and should ordinarily not be used
by customers.

Change-Id: I20583f54a9269ce75daade645be18ab8575b0b9b
BUG: 1282076
Signed-off-by: Dan Lambright <dlambrig@redhat.com>
Reviewed-on: http://review.gluster.org/12581
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: mohammed rafi kc <rkavunga@redhat.com>

quota: fix spurious failure

Change-Id: I5d18533d66df3175752a73430f680dcdfdb3c12a
BUG: 1278689
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/12546
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>

marker : handle variable `ret` properly in marker_unlink_cbk

While compiling new source, the following issue got hit :
marker.c: In function 'marker_unlink_cbk':
marker.c:994:29: warning: variable 'ret' set but not used
int32_t ret = 0;
This patch will fix the same.

Change-Id: I0de60bed3351b3aa1cc80f52d178e447826210e7
BUG: 1257694
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
Reviewed-on: http://review.gluster.org/12584
Reviewed-by: Vijaikumar Mallikarjuna <vmallika@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>

dht: loc should store proper gfid

Change-Id: Ic1393d44a9ed4aaba23d7c9ddea45977b9dae5e4
BUG: 1281265
Signed-off-by: Sakshi Bansal <sabansal@redhat.com>
Reviewed-on: http://review.gluster.org/12574
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>

dht: set proper errno when hashed subvol is not found

Change-Id: I0c4c72e2f5a9f8a7c60ef65251c596b54de89479
BUG: 1279705
Signed-off-by: Sakshi Bansal <sabansal@redhat.com>
Reviewed-on: http://review.gluster.org/12559
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Susant Palai <spalai@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>

core: use syscall wrappers instead of direct syscalls - tail(2)

tail, as in dog chasing its tail. These are the unwrapped
syscalls that have crept in (or were missed) in the previous
patches.

various xlators and other components are invoking system calls
directly instead of using the libglusterfs/syscall.[ch] wrappers.

If not using the system call wrappers there should be a comment
in the source explaining why the wrapper isn't used.

Change-Id: I6015f97109c4250a5a049496a214d19835e4ae19
BUG: 1267967
Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/12590
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>

tier/ctr : MArking ./tests/basic/tier/record-metadata-heat.t as bad

./tests/basic/tier/record-metadata-heat.t as bad

Change-Id: I92e1f9f3103e557e99af627d25700946fcb0b7a7
BUG: 1282673
Signed-off-by: Joseph Fernandes <josferna@redhat.com>
Reviewed-on: http://review.gluster.org/12591
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Tested-by: Dan Lambright <dlambrig@redhat.com>

core: use syscall wrappers instead of direct syscalls - tail

tail, as in dog chasing its tail. These are the unwrapped
syscalls that have crept in (or were missed) in the previous
patches.

various xlators and other components are invoking system calls
directly instead of using the libglusterfs/syscall.[ch] wrappers.

If not using the system call wrappers there should be a comment
in the source explaining why the wrapper isn't used.

Change-Id: If183487de92fc7cbc47d4c5aa3f3e80eae50b84f
BUG: 1267967
Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/12589
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>

build: fix ecdh.h and dh.h deps

openssl.ecdh.h and openssl/dh.h are not available on all platforms.
This patch adds check to autoconf and updates relevant source files.
Add missing #include "config.h" to socket.c to make HAVE_OPENSSL_DH_H
and HAVE_OPENSSL_ECDH_H macros available.

Definitions for UTIME_OMIT and UTIME_NOW in
contrib/qemu/util/oslib-posix.c have been selected from
/usr/include/bits/stat.h on Fedora 22

SSL context options SSL_OP_NO_TICKET and SSL_OP_NO_COMPRESSION are now
conditionally set by testing their presence.

glusterfs.spec.in file now adds CFLAGS=-DUSE_INSECURE_OPENSSL for
RHEL < 6 in the %build section.

Change-Id: Ie32a950dad77bb0f09b4ba53edb3e1f3147056f3
BUG: 1258883
Signed-off-by: Milind Changire <mchangir@redhat.com>
Reviewed-on: http://review.gluster.org/12517
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>

core: use syscall wrappers instead of direct syscalls - libglusterfs

various xlators and other components are invoking system calls
directly instead of using the libglusterfs/syscall.[ch] wrappers.

If not using the system call wrappers there should be a comment
in the source explaining why the wrapper isn't used.

Change-Id: Ieeca2d36adbc884e4cfa0026dba40df70310d40b
BUG: 1267967
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/12275
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>

afr: replica pair going offline does not require CHILD_MODIFIED event

As a part of CHILD_MODIFIED event DHT forgets the current layout and
performs fresh lookup. However this is not required when a replica pair
goes offline as the xattrs can be read from other replica pairs. Hence
setting different event to handle replica pair going down.

Change-Id: I5ede2a6398e63f34f89f9d3c9bc30598974402e3
BUG: 1281230
Signed-off-by: Sakshi Bansal <sabansal@redhat.com>
Reviewed-on: http://review.gluster.org/12573
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-by: Susant Palai <spalai@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>

extras/hooks: Fix parsing of args in S30samba-set.sh

Change-Id: I75756f44757a144b0ed229fcc0e29a273fc75886
BUG: 1242892
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/11668
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Niels de Vos <ndevos@redhat.com>

cluster/tier: Disallow detach commit when detach in progress

1. Check if detach is running, disallow detach commit if so.
2. Cleanup shutdown of tier daemon on detach: do not rerun fix-layout,
do not send incorrect status back to glusterd.

Change-Id: I97202f748773c1176396a4ffd32a4c7fa9b9c1bc
BUG: 1279637
Signed-off-by: Dan Lambright <dlambrig@redhat.com>
Reviewed-on: http://review.gluster.org/12560
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>

glusterd: brick failed to start

brick volfiles are generated in post validate, if
it is running version higher than GLUSTER_3_7_5,
else will be running in syncop.

If the code fall back to syncop, and volume is stopped
then we were returning the operation with out generating
volfiles.

Change-Id: I3b16ee29de19c5d34e45d77d6b7e4b665c2a4653
BUG: 1282322
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: http://review.gluster.org/12552
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>

build: install and package header files more conventionally

The current way we install and package header files for the -devel
package is a hack. This patch uses more conventional autoconf, libtool,
and rpmbuild idioms to package -devel headers and libraries.

Change-Id: I63ffb3460f5c12b6b355493bd00824ac9e5354c5
BUG: 1271907
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/12360
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>

packaging: gfind_missing_files not in geo-rep %if ... %endif conditional

if geo-rep is disabled in the build then the gfind_missing_files
are erroneously included in the ganesha sub-package

Change-Id: I440df7e8283bcc8a80aa057c1c9d6f8034ca98d2
BUG: 1281892
Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/12577
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>

cluster/afr : Remove stale indices

Change-Id: Iba23338a452b49dc9fe6ae7b4ca108ebc377fe42
BUG: 1270668
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
Reviewed-on: http://review.gluster.org/12336
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>

tier/ctr: Providing option to record or ignore metadata heat

Currently we heat up a file for both data and metadata write.
Here we provide a ctr xlator option called "ctr-record-metadata-heat"
were the admin can decide on recording metadata heat i.e heatup a
file on metadata writes or not.
Metadata data operation are
a. setattr: explicit changing of atime/mtime using utimes,
changing of posix permissions of the file
b. rename: Renaming a file,
c. unlink, link: adding or deleting hardlinks
d. xattrs: setting or removal of xattrs.

NOTE: atime, mtime and ctime change through writev, readv, truncate, mknod
and create will not be considered here as these fops are data and primary
metadata fops.

Defaultly "ctr-record-metadata-heat" is off. Admin can
switch it on using gluster volume set command.

Change-Id: I91157509255dd5cb429cda2b6d4f64582e155e7b
BUG: 1279166
Signed-off-by: Joseph Fernandes <josferna@redhat.com>
Reviewed-on: http://review.gluster.org/12540
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Tested-by: Dan Lambright <dlambrig@redhat.com>

docs: move contributor docs to the glusterdocs repository

*  Moved few files to glusterdocs repo
*  Files were moved in pull requests given below:
    https://github.com/gluster/glusterdocs/pull/60
    https://github.com/gluster/glusterdocs/pull/62
* This patch removes file from this repo which were
   moved to glusterdocs repo

Change-Id: I6e529911e0be66b261961c44bcdbe361aafa2886
BUG: 1206539
Signed-off-by: Bipin Kunal <bkunal@redhat.com>
Reviewed-on: http://review.gluster.org/12576
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>

build: `make distclean` doesn't clean all it ought to, needs to

Debian builds fail to "build twice in a row" due to left over files
remaining after running `make distclean`

The main culprits are files created from *.in files during ./configure.
In particular these are ./glusterfs-api.pc, ./libgfchangelog.pc,
./libgfdb.pc, and ./tests/env.rc.

The strange one is contrib/umountd/Makefile{,.in}. While these are
created by ./configure - for reasons I don't quite fathom, perhaps
because contrib/umountd is included in EXTRA_DIST - after that
make and make distclean don't descend into the directory to build or
clean it (because it's not needed for Linux, and not built.)

Also removing the `find . -name Makefile -exec rm -f {}\;` from the
gitclean target, it's not needed (redundant) as its prereq distclean
target will have already removed them.

Change-Id: Ifc45b71d852052ef4b92f356279ce2ba2bbc5bb6
BUG: 1279836
Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/12579
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Tested-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>

tests: fix spurious error in fops-during-migration-pause.t

The test did not spend long enough time moving the file for the
pause to occur simultaneously, leading to failure. Solution is
to elongate that time by increasing the file size.

Change-Id: I1727fa9e3f7a987dfa07dd5da44c68d3f17218d9
BUG: 1280428
Signed-off-by: Dan Lambright <dlambrig@redhat.com>
Reviewed-on: http://review.gluster.org/12570
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Joseph Fernandes

tier/libgfdb: Extending log level flexibity in libgfdb

Extending log level flexibity to relevant fops and operations
This is an extension to
http://review.gluster.org/#/c/12491/

Change-Id: I33b2f7732f89f52569fb99baa692c7e3bb4c7ab1
BUG: 1277352
Signed-off-by: Joseph Fernandes <josferna@redhat.com>
Reviewed-on: http://review.gluster.org/12567
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Tested-by: Dan Lambright <dlambrig@redhat.com>

cluster/ec: fix bug in update_good

Problem:
Bricks that didn't participate in the fops are considered to be good. This is happening two fold.
Examples:
Case-1:
1) 2+1 volume. 'd1' directory on Brick-0 is bad.
2) readdir takes locks and lock->good_mask is '7'
3) readdir does xattrop and fop->mask is '6'.
4) because fop->expected is '1' lock->good_mask remains '7'

Case-2:
1) when all the bricks are up, it does lock + xattrop before op and figures out
   all the bricks are good.
2) By the time second operation starts brick-0 is down. Now lock->good_mask
   will always have the '0' bit set as long as the operations are happening on it.
   because: "lock->good_mask &= ~fop->mask | fop->remaining" fop->mask doesn't
   have '0' th bit.
3) When it comes time to perform the final xattrop in update_size_version
   brick-0 comes online because of which it gives the same version to brick-0
   as well thinking it has participated in all the transactions till then, even
   when it didn't participate in the transactions.

Fix:
Case-1's fix: Update lock->good_mask in ec_prepare_update_cbk with latest
good/bad bricks
Case-2's fix: Consider non-participating brick as bad.

Change-Id: Ic01a733f8180131ded6a3cc784fcb1960758cf23
BUG: 1276989
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/12561
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>

tier/ctr: ignoring bitrot scrubber fops

We had missed adding GF_CLIENT_PID_SCRUB to the internal fops
list of bitrot. Doing that in this fix

Change-Id: I0c7de37e2bf625fb577a32a599a885ee95f5d3bd
BUG: 1278326
Signed-off-by: Joseph Fernandes <josferna@redhat.com>
Reviewed-on: http://review.gluster.org/12555
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Tested-by: Dan Lambright <dlambrig@redhat.com>

Remove selinux mount option from "man mount.glusterfs"

Gluster doesn't SELinux mount option, so it is removed.

Change-Id: I471ad98a24b5f5d64279c805b3243cb1168dd3d1
BUG: 1274626
Signed-off-by: hari gowtham <hgowtham@redhat.com>
Reviewed-on: http://review.gluster.org/12422
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>

build: Remove systemtap references from source

We don't actually support systemtap. And so it is confusing
to have --enable-systemtap configure option in configure.ac.
This patch removes the systemtap option and corresponding
references from other files which include INSTALL, spec.in,
docs etc.

Change-Id: I719c22ca36f41a2faca156f8e41daea44e64b65e
BUG: 1198849
Signed-off-by: Anoop C S <anoopcs@redhat.com>
Reviewed-on: http://review.gluster.org/12565
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>

features/gfid-access: Fix entry creation via setxattr for geo-rep

GEO-REP INTEROP WITH SHARD FEATURE

Problem:
    Geo-replication uses setxattr interface of gfid-access
    xlator to create entries and send explicit setattr
    after entry creation to set uid and gid. But between
    entry creation and setattr, the inode would not be
    linked. Hence operation which accesses inode structure
    during setattr by any the below xlator fails.

Solution:
    Linking inode would seem the obvious solution but,
    gfid-access xlator cannot link inodes and maintain
    it as it would result in same inode pointing
    to two different paths one being virtual .gfid/<gfid>
    path and other being actual path.

    The solution is to set uid and gid in frame->root->uid
    and frame->root->gid respectively from which posix
    extracts and sets.

Change-Id: Ic0749ee471432caeb8ded3152a07de6e64d8538d
BUG: 1265148
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/12206
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

build: MKDIR_P is not defined for Makefiles, use mkdir_p instead

Change-Id: Id6d5263eb7b1c53e72a7668e716e9cc4e34b82cd
Reported-by: Milind Changire <mchangir@redhat.com>
BUG: 1198849
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/12553
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>

extras: Exit with SUCCESS if no processes to stop

This script might be executed even when there are no
valid processes running to be stopped. In this scenario,
the script should return with SUCCESS

Change-Id: Ia293214a4b5052bc4bef9769f197f7b05c55ffe9
BUG: 1277533
Signed-off-by: Shubhendu Tripathi <shtripat@redhat.com>
Reviewed-on: http://review.gluster.org/11739
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>

tests: make mount-nfs-auth.t more stable

mount-nfs-auth.t has a funky way of restarting the Gluster/NFS service.
It is a little racy and does not always work. Disabling and enabling the
nfs.disable volume option triggers a restart of the Gluster/NFS service
too, and is much simpler.

Also adding a little more EXPECT_WITHIN statements to prevent the
occasional failures.

Change-Id: I6765e9f021abbe995dfac00fbfc67298e2ec769c
BUG: 1278476
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/12542
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>

snapshot: Remove bug-1275616.t from bad tests

Remove bug-1275616.t from bad tests list as
commit 466f56a91265ac928759642a866bdc27e850654f
has fixed the spurious failure.

Change-Id: I0d3d86e4e98e3ca810692835ca0a7f119b759620
BUG: 1279297
Signed-off-by: N Balachandran <nbalacha@redhat.com>
Reviewed-on: http://review.gluster.org/12544
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>

geo-rep: Make restrictive ssh keys optional

In containerized environment where networking
configuration is "net=host", both host and
containers use the same IP. The validations
gsyncd shell and rsync to be the siblings
fails. Hence, for now, creating restrictive
ssh keys is made optional as follows.

If the argument 'container' is passed, it
will create non restrictive ssh keys else
restrictive ssh keys.

e.g.,
gluster system:: execute gsec_create container
Creates non restrictive ssh keys.
gluster system:: execute gsec_create
Creates restrictive ssh keys.

Change-Id: Ibed362f64b9b4c9931207f863a2da944c6bd1d66
BUG: 1276028
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/12459
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>

marker: do remove xattr only for last link

With unlink, rename, rmdir, contribution xattrs
are removed. If the file is a last link
then remove_xattr will fail with ENOENT.

So it better to perform remove_xattr
only if there are more links to the file

Change-Id: Ifc1e7fda4d310fd87f6f28a635c9ea78b8f3929d
BUG: 1257694
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/12033
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>

md-cache: Remove readdirp fop for md-cache

readdirp call will return inode for each entry and
will share this nodeid with kernal, also md-cache
will cache this gfid and base name. So when a lookup
operation is perfromed on such an inode, md-cache
will wind the call, that prevents populating inode
ctx for other lower layer xlators.

Change-Id: I43c768703a3cc66d05b1c32909d1a2781001cb49
BUG: 1236032
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: http://review.gluster.org/11894
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>

gfapi: function exit should use __GLFS_EXIT_FS

glfs api functions must be using __GLFS_EXIT_FS to exit from functions

Change-Id: If6b71d8182f3dffe15a3606f31798904cb67be1b
BUG: 1263056
Signed-off-by: Milind Changire <mchangir@redhat.com>
Reviewed-on: http://review.gluster.org/12462
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>

geo-rep: Kill Geo-rep Worker when Agent process dies

When Changelog agent process dies, Geo-replication fails to detect
and worker will run without respective Changelog agent. Status shows
Active/Passive without any progress.

With this patch, Worker process gets killed whenever Changelog
agent dies.

Change-Id: I30b4cc77f924f7e8174b8bfe415ac17f0b3851b4
Signed-off-by: Aravinda VK <avishwan@redhat.com>
BUG: 1277076
Reviewed-on: http://review.gluster.org/12485
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>

build: Fix autoconf warnings

This patch avoids the following warnings on running autogen script

. . .
Running autoconf...
configure.ac:896: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in
body
../../lib/autoconf/lang.m4:193: AC_LANG_CONFTEST is expanded from...
configure.ac:896: the top level
. . .

This change uses AC_LINK_IFELSE for checking the atomic built-in function
support. Since AC_COMPILE_IFELSE checks for syntactical errors only, we need
to use AC_LINK_IFELSE to achieve the same which is more appropriate.

Reference links:
[1] https://autotools.io/forwardporting/autoconf.html
[2] http://www.gnu.org/software/autoconf/manual/autoconf.html#Generating-Sources
[3] http://www.gnu.org/software/autoconf/manual/autoconf.html#Running-the-Compiler

Change-Id: I4597f2976623496745b66f98bb78a0c9f1b07f79
BUG: 1198849
Signed-off-by: Anoop C S <anoopcs@redhat.com>
Reviewed-on: http://review.gluster.org/12351
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>

dht: heal directory path if the directory is not present

After a successful nameless lookup if the directory is not
present on any of the subvol, then we will get the path of
the directory and will recursively send a named lookp on
each parent directory.

This will help particularly for the scenarios like add brick
and attach-tier.

Change-Id: I64c2118a5ab03bbaa59b0dfc62babdf4472a92a3
BUG: 1272949
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: http://review.gluster.org/12376
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: N Balachandran <nbalacha@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>