Tag Archives: howto

Setting up a 2-node GlusterFS filesystem

This will be a quick howto on how you would set up a 2-node GlusterFS filesystem. You may look up more information at http://www.gluster.org/.

Volume types for GlusterFS

– Distributed. Distributed volumes distributes files throughout the bricks in the volume
– Replicated. Replicated volumes replicates files across bricks in the volume
– Striped. Striped volumes stripes data across bricks in the volume
– Distributed Striped. Distributed striped volumes stripe data across two or more nodes in the cluster
– Distributed Replicated. Distributed replicated volumes distributes files across replicated bricks in the volume
– Distributed Striped Replicated. Distributed striped replicated volumes distributes striped data across replicated bricks in the cluster
– Striped Replicated. Striped replicated volumes stripes data across replicated bricks in the cluster

The high level overview of how the process will be is as follows

  • Installing the required software
  • Disable or add proper firewall rules
  • Adding nodes into the cluster
  • Preparing “bricks” for use on each server
  • Creating and starting the actual GlusterFS volume
  • Mounting the GlusterFS volume
  • Installing the required software

    I will be providing examples for CentOS, Fedora, Debian and Arch Linux. The examples for CentOS will work for RHEL and Scientific Linux as well.
    The following command will install all dependencies.

    # yum install glusterfs

    The following command will install all dependencies.

    # yum install glusterfs-server

    The following command will install all dependencies.

    # apt-get install glusterfs-server

    Arch Linux
    The following command will install all dependencies.

    # pacman -S glusterfs

    Disable or add proper firewall rules

    You will need to open the following ports for GlusterFS.

    24007 โ€“ GlusterFS Daemon
    24008 โ€“ Management
    24009 - Each brick for every volume on your host requires itโ€™s own port. For every new brick, one new port will be used starting at 24009. (For GlusterFS versions earlier than 3.4)
    49152 - Each brick for every volume on your host requires itโ€™s own port. For every new brick, one new port will be used starting at 49152 (GlusterFS 3.4 and later)
    38465:38467 - This is required if you use the GlusterFS NFS service.

    Disabling the default firewall

    # chkconfig iptables off
    # service stop iptables


    systemctl disable firewalld
    systemctl stop firewalld

    There are no default firewall installed on Debian.
    Arch Linux
    There are no default firewall installed on Arch Linux.

    Adding nodes into the cluster

    This is incredibly easy. You may do the following command from either server. In my example I am on server1. If you don’t have a solid DNS you should add each server to each others hosts file.

    # gluster peer probe server2
    Probe successful

    Preparing “bricks” for use on each server

    Nothing fanzy, you just need to create folders. It’s also important to note that you will need to use a folder, even if you intended to use a single disk.
    Execute the following on both of your servers

    # mkdir -p /data/brick>

    Creating and starting the actual GlusterFS volume

    Creating the GlusterFS volume

    gluster volume create NEW-VOLNAME [replica COUNT] [transport [tcp | rdma | tcp,rdma]] NEW-BRICK...


    # gluster volume create test-volume replica 2 transport tcp server1:/data/brick server2:/data/brick
    Creation of test-volume has been successful
    Please start the volume to access data.

    Starting the GlusterFS volume

    # gluster volume start test-volume

    Mounting the GlusterFS volume

    It’s important to note that you will need to mount the GlusterFS to use it. WARNING: Adding files directly to a brick will not be included in a GlusterFS volume.

    # mount.glusterfs servername:volumename /mnt/mountpoint


    # mount.glusterfs server1:test-volume /mnt/glusterfs/


    # mount -t glusterfs server1:test-volume /mnt/glusterfs/



    // CrashMAG

    Change the default SSH port and alter SELinux context to match

    Security through obscurity is not something one would generally recommend. But to thwart the effort of automated scanners changing the default OpenSSH port will yield you less pain in every day life. This will not fend off directed attacks or nullify vulnerabilities or bad security design.

    Should you see an error message such as

    shd[14221]: error: Bind to port 9898 on failed: Permission denied

    it indicates that the system prevented the daemon to bind that port. Most likely SELinux.

    The instructions provided will be valid on Fedora 14/15, CentOS 6, RHEL 6, Scientific Linux 6 and newer versions.

    To change the default SSH port you need to do the following.

    • Stop the SSH daemon
    • Alter the /etc/ssh/sshd_config with your new port
    • Alter the SELinux context with semanage
    • Start the SSH daemon

    Stop the SSH daemon

    # service sshd stop

    Alter the /etc/ssh/sshd_config with your new port

    Alter the configuration file with your favorite editor, in my case “nano”.

    # nano /etc/ssh/sshd_config

    Alter the port configuration parameter change the following line

    Port 22


    Port 9898

    Alter the SELinux context with semanage

    # semanage port -a -t ssh_port_t -p tcp 9898

    Initially you would think the following would work. But it will not. For it to work you would have to alter the policy in the selinux-policy package, rebuild and install it. So skip it, but now you know why.

    # semanage port -d -t ssh_port_t -p tcp 22

    Start the SSH daemon

    # service sshd start

    // CrashMAG

    View information about your BIOS from Linux using dmidecode

    To get at this information we will use a utility called “dmidecode”. dmidecode is a tool for dumping a computer’s DMI (some say SMBIOS) table contents in a human-readable format.

    On CentOS/RHEL/Fedora you may run the following to install it.

    # yum install dmidecode

    On Arch Linux you may run

    # pacman -S dmidecode

    The following examples will allow you to see a few important parts of information such as;

    • The manufacturer of your motherboard
    • What type of motherboard you have
    • The version of the BIOS running on your motherboard

    To view the manufacturer and what type of motherboard you have, run the following

    dmidecode --type system


    # dmidecode 2.11
    SMBIOS 2.4 present.
    Handle 0x0001, DMI type 1, 27 bytes
    System Information
            Manufacturer: Gigabyte Technology Co., Ltd.
            Product Name: GA-MA78G-DS3H
            Serial Number:
            UUID: 4E2F4100-0000-0000-0000-0000FFFFFFFF
            Wake-up Type: Power Switch
            SKU Number:
    Handle 0x0034, DMI type 32, 11 bytes
    System Boot Information
            Status: No errors detected

    To view the version of your BIOS you may run the following

    #dmidecode --type bios


    # dmidecode 2.11
    SMBIOS 2.4 present.
    Handle 0x0000, DMI type 0, 24 bytes
    BIOS Information
            Vendor: Award Software International, Inc.
            Version: FA
            Release Date: 09/19/2008
            Address: 0xE0000
            Runtime Size: 128 kB
            ROM Size: 1024 kB
                    ISA is supported
                    PCI is supported
                    PNP is supported
                    APM is supported
                    BIOS is upgradeable
                    BIOS shadowing is allowed
                    Boot from CD is supported
                    Selectable boot is supported
                    BIOS ROM is socketed
                    EDD is supported
                    5.25"/360 kB floppy services are supported (int 13h)
                    5.25"/1.2 MB floppy services are supported (int 13h)
                    3.5"/720 kB floppy services are supported (int 13h)
                    3.5"/2.88 MB floppy services are supported (int 13h)
                    Print screen service is supported (int 5h)
                    8042 keyboard services are supported (int 9h)
                    Serial services are supported (int 14h)
                    Printer services are supported (int 17h)
                    CGA/mono video services are supported (int 10h)
                    ACPI is supported
                    USB legacy is supported
                    AGP is supported
                    LS-120 boot is supported
                    ATAPI Zip drive boot is supported
                    BIOS boot specification is supported
                    Targeted content distribution is supported
    Handle 0x0029, DMI type 13, 22 bytes
    BIOS Language Information
            Language Description Format: Long
            Installable Languages: 3
            Currently Installed Language: n|US|iso8859-1

    There’s also additional options to use with dmidecode. You probably also want to try the following to get an idea of what type of information you can get your hands on.

    #dmidecode --type keyword
    Valid type keywords are:

    // CrashMAG

    How you tell Firefox 4 to open links in a new tab instead of a new window

    There’s so little useful information on the matter so I’ve decided to post about it. And the default option under Preferences -> Tabs called “Open new windows in a new tab instead” does not work. I have no idea why, and I’m embarrassed on behalf of Mozilla that it doesn’t. However here’s how you fix it.

    This is what you have to do to have Firefox 4 open your links in a new tab instead of a new window.

    1. In your URL bar enter “about:config”.
    2. Accept the prompt.
    3. Search up the line


    4. Change the default value “2” to “0”.

    Once you’ve set it to “0” it will immediately work.

    Further refrence can be found here http://kb.mozillazine.org/About:config_entries

    // CrashMAG

    Resetting the root/postgres password for PostgreSQL

    The following is required to reset the root/postgres user password for PostgreSQL. The distribution used in my example is CentOS 5.5 and PostgreSQL 8.4.

    Note: By default there’s no password for the postgres user.

    In step 2 and 5 you will most likely not be using “ident” but rather “password” or “md5”.

    1. Shut down PostgreSQL

    # service postgresql stop

    2. Reset the authentication mechanism (assuming defaults are already being used)

    Edit the /usr/lib/pgsql/data/pg_hba.conf file

    # nano /usr/lib/pgsql/data/pg_hba.conf

    Navigate down to the line that says

    local all all ident

    Edit it to

    local all postgres trust

    And now save the file.

    3. Start PostgreSQL

    # service postgresql start

    4. Log in and change the password

    # su - postgres
    $ psql -d template1 -U postgres
    alter user postgres with password 'new_password';

    Or alternatively do it all in one go with the following command

    > psql -U postgres template1 -c "alter user postgres with password 'new_password';"

    5. Reverse the actions you did in step 2

    Edit the /usr/lib/pgsql/data/pg_hba.conf file

    # nano /usr/lib/pgsql/data/pg_hba.conf

    Navigate down to the line that says

    local all all trust

    Edit it to

    local all postgres ident

    And now save the file.

    6. Start PostgreSQL

    # service postgresql start


    // CrashMAG

    Changing the default PostgreSQL data folder (PGDATA)

    Installing the PostgreSQL server on RHEL, CentOS, Scientific Linux or Fedora installs the PostgreSQL databases and configuration files in “/var/lib/pgsql/data”.

    This may or may not be desirable. Let’s assume for a moment you have a separately crafted partition for PostgreSQL to use, let’s say a RAID10 volume. You’d want to change this.

    Change the defaults

    Use your favorite text editor, in my case nano to create the following file (must be the same as the name of the service)

    # nano /etc/sysconfig/pgsql/postgresql

    Add the following


    Optionally you can also add the following to change the default port (example is the default port)


    Adjusting SELinux to permit the new data folder (pgdata) location

    Should the following command output “Permissive” or “Disabled” then you may skip the details for SELinux.

    # getenforce

    Run the semanage command to add a context mapping for /opt/postgresql and any other directories/files within it.

    # semanage fcontext -a -t postgresql_db_t "/postgresql/data(/.*)?"

    Now use the restorecon command to apply this context mapping to the running system

    # restorecon -Rv /postgresql/data

    Starting PostgreSQL

    # chkconfig --levels 345 postgresql on
    # service postgresql initdb
    # service postgresql start

    You’re all set to go! Keep in mind that PostgreSQL listens to ‘localhost’ by default. To change this you need to alter the “listen_address” parameter in “/var/lib/pgsql/data/postgresql.conf” (change will require restart).

    // CrashMAG

    RHEL/Centos 5 minimal installation

    There’s no option during the CentOS 5 install, for a minimal installation. The purpose is quite simple, to keep the attack surface as small as possible.

    A minimal installation is performed by doing the following

    • During the category/task selection, deselect all package categories, and choose the “Customize now” option at the bottom of screen.
    • During the customized package selection, deselect everything ( including the Base group ).

    This will yield you 234 packages with the Centos 5.5 installation media. I’ve attached a .txt file containing all the packages for your leisure.

    Link: installed-packages

    // CrashMAG

    Guide and hardning tips for RHEL/CentOS 5 from NSA

    As I was looking to see if NSA had updated their guides for RHEL 6 and it turns out they haven’t. I decided it would be a good idea to post about them to give them some better coverage.

    This is just a small tip of free and useful information in regards to securing your RHEL/CentOS installation. A lot of the information is general in nature and can therefore be applied to any Linux distribution. It’s definitely worth your time.

    I take no credit, the credit goes to NSA for creating the documents to begin with.

    Guide to the Secure Configuration of Red Hat Enterprise Linux 5

    Red Hat Linux 5 Hardening Tips

    I just love how just about every section starts with “Disable ‘insert your service here’ if possible…” ๐Ÿ˜‰

    // CrashMAG

    Setting up sSMTP with GMail

    Let me introduce you to the “extremely simple MTA to get mail off the system to a mailhub”. Particularly useful when you don’t want systems to have a full blown MTA installed. Such as Postfix, Exim or Sendmail. I find ssmtp extremely helpful on standalone servers that use Logwatch.

    Getting this up and running requires 4 steps.

    • Installing SSMTP
    • Configuring SSMTP
    • Changing the MTA on your system
    • Testing

    Installing the daemon, ssmtp.

    Use your favorite package manager, in my example I’ll be using YUM. (Fedora/CentOS/RHEL/Scientific Linux). For Centos/RHEL/Scientific Linux 5.5 or 5.6 you need access to the EPEL repository to install sSMTP. Add EPEL to your system using the following command.

    rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm

    You can find eventual new links from http://download.fedora.redhat.com/pub/epel/5/i386/repoview/epel-release.html

    yum install ssmtp

    Configuring SSMTP

    Edit /etc/ssmtp/ssmtp.conf with your favorite text editor. I’ll be using nano.

    nano /etc/ssmtp/ssmtp.conf

    Remove all the entries and replace it with the ones beneath.

    root=insert_your_email_address here

    Changing the MTA

    For CentOS/Fedora/RHEL

    alternatives --config mta

    Press the number that equals /usr/sbin/sendmail.ssmtp and you’re done.


    I’m testing this using the verbose mode just to be able to see the dialogue with the Google SMTP server.

    cat random_file | sendmail -v your_email_address

    // CrashMAG

    Managing /etc with etckeeper and git

    The following was done on Fedora 14. Keep in mind that the Etckeeper and git specific actions will be similar on whatever platform you’re on.

    Simply put, Etckeeper automatically revisions your /etc folder. Allows you to compare, commit and revert the changes that have been made. It’ll also allow you to restore files, should you be unlucky and delete them. Once etckeeper is installed, it will work together with your package manager and cron to do its work. To manage all this you’ll use the commands that your chosen VCS (Version Control System).

    Etckeeper supports Git, Bazaar, Darcs and Mercurial.

    Use of Etckeeper


    yum install etckeeper


    etckeeper init

    Initial commit

    etckeeper commit "initial commit"

    Once this is done, etckeeper will make sure that every time you use the package manager (YUM) changes will be recorded. There are however a few git related commands you should be aware of.

    Useful and necessary commands

    Note: All of these commands assumes your current path is /etc

    Viewing the Git log

    git log

    Check if there’s any modified files

    git status

    Complete status overview

    git log --stat --summary

    Revert a change

    git revert 

    View changes you haven’t commited yet

    git diff

    List different commits, each on one line.

    git log --pretty=oneline

    Revert to latest change-set, discarding changes

    git reset --hard

    Re-enter commit message

    git commit --amend

    Have at it folks!

    // CrashMAG