Tag Archives: filesystem

Setting up a 2-node GlusterFS filesystem

This will be a quick howto on how you would set up a 2-node GlusterFS filesystem. You may look up more information at http://www.gluster.org/.

Volume types for GlusterFS

– Distributed. Distributed volumes distributes files throughout the bricks in the volume
– Replicated. Replicated volumes replicates files across bricks in the volume
– Striped. Striped volumes stripes data across bricks in the volume
– Distributed Striped. Distributed striped volumes stripe data across two or more nodes in the cluster
– Distributed Replicated. Distributed replicated volumes distributes files across replicated bricks in the volume
– Distributed Striped Replicated. Distributed striped replicated volumes distributes striped data across replicated bricks in the cluster
– Striped Replicated. Striped replicated volumes stripes data across replicated bricks in the cluster

The high level overview of how the process will be is as follows

  • Installing the required software
  • Disable or add proper firewall rules
  • Adding nodes into the cluster
  • Preparing “bricks” for use on each server
  • Creating and starting the actual GlusterFS volume
  • Mounting the GlusterFS volume
  • Installing the required software

    I will be providing examples for CentOS, Fedora, Debian and Arch Linux. The examples for CentOS will work for RHEL and Scientific Linux as well.
    CentOS
    The following command will install all dependencies.

    # yum install glusterfs

    Fedora
    The following command will install all dependencies.

    # yum install glusterfs-server

    Debian
    The following command will install all dependencies.

    # apt-get install glusterfs-server

    Arch Linux
    The following command will install all dependencies.

    # pacman -S glusterfs

    Disable or add proper firewall rules

    You will need to open the following ports for GlusterFS.

    24007 – GlusterFS Daemon
    24008 – Management
    24009 - Each brick for every volume on your host requires it’s own port. For every new brick, one new port will be used starting at 24009. (For GlusterFS versions earlier than 3.4)
    49152 - Each brick for every volume on your host requires it’s own port. For every new brick, one new port will be used starting at 49152 (GlusterFS 3.4 and later)
    38465:38467 - This is required if you use the GlusterFS NFS service.
    

    CentOS
    Disabling the default firewall

    # chkconfig iptables off
    # service stop iptables

    Fedora

    systemctl disable firewalld
    systemctl stop firewalld

    Debian
    There are no default firewall installed on Debian.
    Arch Linux
    There are no default firewall installed on Arch Linux.

    Adding nodes into the cluster

    This is incredibly easy. You may do the following command from either server. In my example I am on server1. If you don’t have a solid DNS you should add each server to each others hosts file.

    # gluster peer probe server2
    Probe successful

    Preparing “bricks” for use on each server

    Nothing fanzy, you just need to create folders. It’s also important to note that you will need to use a folder, even if you intended to use a single disk.
    Execute the following on both of your servers

    # mkdir -p /data/brick>

    Creating and starting the actual GlusterFS volume

    Creating the GlusterFS volume
    Syntax:

    gluster volume create NEW-VOLNAME [replica COUNT] [transport [tcp | rdma | tcp,rdma]] NEW-BRICK...

    Example:

    # gluster volume create test-volume replica 2 transport tcp server1:/data/brick server2:/data/brick
    Creation of test-volume has been successful
    Please start the volume to access data.
    

    Starting the GlusterFS volume

    # gluster volume start test-volume

    Mounting the GlusterFS volume

    It’s important to note that you will need to mount the GlusterFS to use it. WARNING: Adding files directly to a brick will not be included in a GlusterFS volume.
    Syntax:

    # mount.glusterfs servername:volumename /mnt/mountpoint

    Examples:

    # mount.glusterfs server1:test-volume /mnt/glusterfs/

    OR

    # mount -t glusterfs server1:test-volume /mnt/glusterfs/

    References

    http://www.gluster.org/wp-content/uploads/2012/05/Gluster_File_System-3.3.0-Administration_Guide-en-US.pdf
    http://gluster.org/community/documentation/index.php/QuickStart

    // CrashMAG

    Disable the filesystem check (fsck) at boot time

    There’s several ways of accomplishing this. I will list all the methods beneath, just pick the one that fits the situation/you.

    • Filesystem tunable
    • Grub boot parameter
    • Placing command files on your root device
    • Active reboot without FSCK

    Filesystem tunable

    Use the tune2fs command to tell your filesystem to have a max count of mounts before a check to 0 to disable it.

    # tune2fs -c 0 /dev/sda1

    Parameter reference:

    -c max-mount-counts
     Adjust the number of mounts after which the filesystem will be  checked  by  e2fsck(8).   If max-mount-counts  is  0  or -1, the number of times the filesystem is mounted will be disregarded by e2fsck(8) and the kernel.
    

    Grub boot parameter

    Add the following at the end of your grub boot linux line.

    fastboot

    This can be done by editing “grub.conf” or by editing the boot command via the grub menu at boot.

    Placing command files on your root device

    To disable the filesystem check on boot.

    # touch /fastboot

    To enable a filesystem check on boot.

    # touch /forcefsck

    Active reboot without FSCK

    # shutdown -rf

    Parameter reference:

    -r     Reboot after shutdown.
    -f     Skip fsck on reboot.
    

    // CrashMAG

    Linux ACL

    An access control list (ACL), with respect to a computer file system, is a list of permissions attached to an object. ACL allows you to grant or deny permissions for any user or group on a filesystem resource.

    Enabling ACL

    To enable ACL, edit your /etc/fstab file as such:

    /dev/VolGroup00/LogVol00 /                       ext3    defaults,acl        1 1

    Note: Moderm Redhat distributions enable ACL by default for the root filesystem.

    Set ACL

    To modify ACL use setfacl command. To add permissions use setfacl -m.

    Add permissions to some user:

    # setfacl -m "u:username:permissions"

    or

    # setfacl -m "u:uid:permissions"

    Add permissions to some group:

    # setfacl -m "g:groupname:permissions"

    or

    # setfacl -m "g:gid:permissions"

    Add default ACL:

    # setfacl -d -m "u:uid:permissions"

    Remove all permissions:

    # setfacl -b

    Remove each entry:

    # setfacl -x "entry"

    To check permissions use:

    # getfacl filename

    Examples

    Set read,write and execute permissions for user “johndoe” on the file named “abc”.

    # setfacl -m "u:johndoe:rwx" abc

    Check permissions.

    # getfacl abc
    # file: abc
    # owner: someone
    # group: someone
    user::rw-
    user:johny:rwx
    group::r--
    mask::rwx
    other::r--

    Change permissions for user “johndoe”.

    # setfacl -m "u:johndoe:rw-" abc

    Check permissions.

    # getfacl abc
    # file: abc
    # owner: someone
    # group: someone
    user::rw-
    user:johndoe:rw-
    group::r--
    mask::r-x
    other::r--

    Remove all extended ACL entries.

    # setfacl -b abc

    Check permissions.

    # getfacl abc
    # file: abc
    # owner: someone
    # group: someone
    user::rw-
    group::r--
    other::r--

    Additional Resources

    man getfacl
    man setfacl

    If you weren’t using these already, you should.

    // CrashMAG