Tag Archives: fedora

Setting up a 2-node GlusterFS filesystem

This will be a quick howto on how you would set up a 2-node GlusterFS filesystem. You may look up more information at http://www.gluster.org/.

Volume types for GlusterFS

– Distributed. Distributed volumes distributes files throughout the bricks in the volume
– Replicated. Replicated volumes replicates files across bricks in the volume
– Striped. Striped volumes stripes data across bricks in the volume
– Distributed Striped. Distributed striped volumes stripe data across two or more nodes in the cluster
– Distributed Replicated. Distributed replicated volumes distributes files across replicated bricks in the volume
– Distributed Striped Replicated. Distributed striped replicated volumes distributes striped data across replicated bricks in the cluster
– Striped Replicated. Striped replicated volumes stripes data across replicated bricks in the cluster

The high level overview of how the process will be is as follows

  • Installing the required software
  • Disable or add proper firewall rules
  • Adding nodes into the cluster
  • Preparing “bricks” for use on each server
  • Creating and starting the actual GlusterFS volume
  • Mounting the GlusterFS volume
  • Installing the required software

    I will be providing examples for CentOS, Fedora, Debian and Arch Linux. The examples for CentOS will work for RHEL and Scientific Linux as well.
    CentOS
    The following command will install all dependencies.

    # yum install glusterfs

    Fedora
    The following command will install all dependencies.

    # yum install glusterfs-server

    Debian
    The following command will install all dependencies.

    # apt-get install glusterfs-server

    Arch Linux
    The following command will install all dependencies.

    # pacman -S glusterfs

    Disable or add proper firewall rules

    You will need to open the following ports for GlusterFS.

    24007 – GlusterFS Daemon
    24008 – Management
    24009 - Each brick for every volume on your host requires it’s own port. For every new brick, one new port will be used starting at 24009. (For GlusterFS versions earlier than 3.4)
    49152 - Each brick for every volume on your host requires it’s own port. For every new brick, one new port will be used starting at 49152 (GlusterFS 3.4 and later)
    38465:38467 - This is required if you use the GlusterFS NFS service.
    

    CentOS
    Disabling the default firewall

    # chkconfig iptables off
    # service stop iptables

    Fedora

    systemctl disable firewalld
    systemctl stop firewalld

    Debian
    There are no default firewall installed on Debian.
    Arch Linux
    There are no default firewall installed on Arch Linux.

    Adding nodes into the cluster

    This is incredibly easy. You may do the following command from either server. In my example I am on server1. If you don’t have a solid DNS you should add each server to each others hosts file.

    # gluster peer probe server2
    Probe successful

    Preparing “bricks” for use on each server

    Nothing fanzy, you just need to create folders. It’s also important to note that you will need to use a folder, even if you intended to use a single disk.
    Execute the following on both of your servers

    # mkdir -p /data/brick>

    Creating and starting the actual GlusterFS volume

    Creating the GlusterFS volume
    Syntax:

    gluster volume create NEW-VOLNAME [replica COUNT] [transport [tcp | rdma | tcp,rdma]] NEW-BRICK...

    Example:

    # gluster volume create test-volume replica 2 transport tcp server1:/data/brick server2:/data/brick
    Creation of test-volume has been successful
    Please start the volume to access data.
    

    Starting the GlusterFS volume

    # gluster volume start test-volume

    Mounting the GlusterFS volume

    It’s important to note that you will need to mount the GlusterFS to use it. WARNING: Adding files directly to a brick will not be included in a GlusterFS volume.
    Syntax:

    # mount.glusterfs servername:volumename /mnt/mountpoint

    Examples:

    # mount.glusterfs server1:test-volume /mnt/glusterfs/

    OR

    # mount -t glusterfs server1:test-volume /mnt/glusterfs/

    References

    http://www.gluster.org/wp-content/uploads/2012/05/Gluster_File_System-3.3.0-Administration_Guide-en-US.pdf
    http://gluster.org/community/documentation/index.php/QuickStart

    // CrashMAG

    Change the default SSH port and alter SELinux context to match

    Security through obscurity is not something one would generally recommend. But to thwart the effort of automated scanners changing the default OpenSSH port will yield you less pain in every day life. This will not fend off directed attacks or nullify vulnerabilities or bad security design.

    Should you see an error message such as

    shd[14221]: error: Bind to port 9898 on 192.168.0.50 failed: Permission denied

    it indicates that the system prevented the daemon to bind that port. Most likely SELinux.

    The instructions provided will be valid on Fedora 14/15, CentOS 6, RHEL 6, Scientific Linux 6 and newer versions.

    To change the default SSH port you need to do the following.

    • Stop the SSH daemon
    • Alter the /etc/ssh/sshd_config with your new port
    • Alter the SELinux context with semanage
    • Start the SSH daemon

    Stop the SSH daemon

    # service sshd stop

    Alter the /etc/ssh/sshd_config with your new port

    Alter the configuration file with your favorite editor, in my case “nano”.

    # nano /etc/ssh/sshd_config

    Alter the port configuration parameter change the following line

    Port 22

    to

    Port 9898

    Alter the SELinux context with semanage

    # semanage port -a -t ssh_port_t -p tcp 9898

    Initially you would think the following would work. But it will not. For it to work you would have to alter the policy in the selinux-policy package, rebuild and install it. So skip it, but now you know why.

    # semanage port -d -t ssh_port_t -p tcp 22

    Start the SSH daemon

    # service sshd start

    // CrashMAG

    Change the default MySQL data directory with SELinux enabled

    This is a short article that explains how you change the default MySQL data directory and adjust SELinux to account for the changes. The article assumes that you’re running either RHEL, CentOS, Scientific Linux or Fedora with SELinux enabled. This works with the most recent EL (6.2) version.

    We’ll be doing this in the following order.

    • Stopping the MySQL server
    • Create a new data directory and move the content from the old data directory
    • Correct the MySQL configuration file
    • Adjust SELinux parameters to accept our new change
    • Starting the MySQL server

    Stopping the MySQL server

    # service mysqld stop

    Create a new data diretory and move the content from the old one

    Creating a new data directory

    # mkdir /srv/mysql/
    # chown mysql:mysql /srv/mysql

    Moving the original data files

     # mv /var/lib/mysql/* /srv/mysql/

    Correct the MySQL configuration file

    Edit the my.cnf file for your distribution. In my example it’s located in the /etc/mysql/ directory. RHEL/CentOS/Scientific Linux put the my.cnf file directly in /etc by default.

    # nano /etc/mysql/my.cnf

    Change

    datadir=/var/lib/mysql

    to

    datadir=/srv/mysql

    and

    socket=/var/lib/mysql/mysql.sock

    to

    socket=/srv/mysql/mysql.sock

    and save the file.

    Adjust SELinux parameters to accept our new change

    Should the following command output “Permissive” or “Disabled” then you may skip the details for SELinux.

    # getenforce

    Run the semanage command to add a context mapping for /srv/mysql.

    # semanage fcontext -a -t mysqld_db_t "/srv/mysql(/.*)?"

    Now use the restorecon command to apply this context mapping to the running system.

    # restorecon -Rv /srv/mysql

    Starting the MySQL server

    # service mysqld start

    Verifying access and connectivity

    $ mysql -u root -p
    mysql> show databases;

    If this is working, you’re up and running. Should you get a message that says

    ERROR 2002 (HY000): Can’t connect to local MySQL server through socket ‘/var/lib/mysql/mysql.sock’

    then add the following to your /etc/my.cnf

    [client]
    socket = /srv/mysql/mysql.sock
    

    Optionally you can just use

    $ mysql -u root -p --protocol tcp

    to avoid connecting via the socket.

    // CrashMAG

    Useful SystemD commands

    List all running services

    # systemctl

    Start/stop or enable/disable services

    Activates a service immediately:

    # systemctl start foo.service

    Deactivates a service immediately:

    # systemctl stop foo.service

    Restarts a service:

    # systemctl restart foo.service

    Shows status of a service including whether it is running or not:

    # systemctl status foo.service

    Enables a service to be started on bootup:

    # systemctl enable foo.service

    Disables a service to not start during bootup:

    # systemctl disable foo.service

    Check whether a service is already enabled or not:

    # systemctl is-enabled foo.service; echo $?

    0 indicates that it is enabled. 1 indicates that it is disabled

    How do I change the runlevel?

    systemd has the concept of targets which is a more flexible replacement for runlevels in sysvinit.

    Run level 3 is emulated by multi-user.target. Run level 5 is emulated by graphical.target. runlevel3.target is a symbolic link to multi-user.target and runlevel5.target is a symbolic link to graphical.target.

    You can switch to ‘runlevel 3’ by running

    # systemctl isolate multi-user.target (or) systemctl isolate runlevel3.target

    You can switch to ‘runlevel 5’ by running

    # systemctl isolate graphical.target (or) systemctl isolate runlevel5.target

    How do I change the default runlevel?

    systemd uses symlinks to point to the default runlevel. You have to delete the existing symlink first before creating a new one

    # rm /etc/systemd/system/default.target

    Switch to runlevel 3 by default

    # ln -sf /lib/systemd/system/multi-user.target /etc/systemd/system/default.target

    Switch to runlevel 5 by default

    # ln -sf /lib/systemd/system/graphical.target /etc/systemd/system/default.target

    systemd does not use /etc/inittab file.

    List the current run level

    runlevel command still works with systemd. You can continue using that however runlevels is a legacy concept in systemd and is emulated via ‘targets’ and multiple targets can be active at the same time. So the equivalent in systemd terms is

    # systemctl list-units --type=target

    Powering off the machine

    You can use

    # poweroff

    Some more possibilities are: halt -p, init 0, shutdown -P now

    Note that halt used to work the same as poweroff in previous Fedora releases, but systemd distinguishes between the two, so halt without parameters now does exactly what it says – it merely stops the system without turning it off.

     

    Service vs. systemd

    # service NetworkManager stop

    (or)

    # systemctl stop NetworkManager.service

    Chkconfig vs. systemd

    # chkconfig NetworkManager off

    (or)

    # systemctl disable NetworkManager.service

    Readahead

    systemd has a built-in readahead implementation is not enabled on upgrades. It should improve bootup speed but your mileage may vary depending on your hardware. To enable it:

    # systemctl enable systemd-readahead-collect.service
    # systemctl enable systemd-readahead-replay.service

    SystemD cheatsheet

    service foobar start systemctl start foobar.service Used to start a service (not reboot persistent)
    service foobar stop systemctl stop foobar.service Used to stop a service (not reboot persistent)
    service foobar restart systemctl restart foobar.service Used to stop and then start a service
    service foobar reload systemctl reload foobar.service When supported, reloads the config file without interrupting pending operations.
    service foobar condrestart systemctl condrestart foobar.service Restarts if the service is already running.
    service foobar status systemctl status foobar.service Tells whether a service is currently running.
    ls /etc/rc.d/init.d/ ls /lib/systemd/system/*.service /etc/systemd/system/*.service Used to list the services that can be started or stopped
    chkconfig foobar on systemctl enable foobar.service Turn the service on, for start at next boot, or other trigger.
    chkconfig foobar off systemctl disable foobar.service Turn the service off for the next reboot, or any other trigger.
    chkconfig foobar systemctl is-enabled foobar.service Used to check whether a service is configured to start or not in the current environment.
    chkconfig foobar –list ls /etc/systemd/system/*.wants/foobar.service Used to list what levels this service is configured on or off
    chkconfig foobar –add Not needed, no equivalent.

    References

    fedoraproject.org/wiki/Systemd
    fedoraproject.org/wiki/SysVinit_to_Systemd_Cheatsheet

    Distribution Documentation

    Gentoo
    Arch
    Ubuntu
    Debian

    // CrashMAG

    Self-signed certificate for Apache

    These instructions are distribution agnostic. However I used CentOS during my tests, so file paths will match that of CentOS, RHEL, Scientific Linux and Fedora. For any other distribution you’ll have to look that up yourself.

    The tools required are OpenSSL, Apache and mod_ssl for Apache. To accomplish this I had to run

    # yum install mod_ssl

    on my CentOS 5.6 box. Which already had Apache up and running.

    Setting up a self-signed certificate using certificate and key

    Generate your key and certificate

    Most of these parameters explain themselves, see beneath for those who do not.

    openssl req -new -newkey rsa:2048 -days 365 -nodes -x509 -keyout website.key -out website.crt

    -nodes
    don’t encrypt the output key
    -x509
    output a x509 structure instead of a cert. req.

    Copy the key and certificate

    # cp website.key website.crt /etc/httpd/conf/

    Set permissions and ownership on your key and certificate

    This way nobody except root has read access.

    chmod 440 /etc/httpd/conf/website.key /etc/httpd/conf/website.crt
    chown root:root /etc/httpd/conf/website.key /etc/httpd/conf/website.crt

    Alter the apache configuration file, also known as httpd.conf

    Edit /etc/httpd/conf/httpd.conf with your favorite text editor, in my case, nano. Add the following text at the bottom of the file.

          <VirtualHost *:443>
            SSLEngine on
            # Change the next two lines according to where you've actually
            # stored the certificate and key files.
            SSLCertificateFile /etc/httpd/conf/website.crt
    	SSLCertificateKeyFile /etc/httpd/conf/apache2/website.key
    
            ServerName domain.tld
            SSLOptions StrictRequire
            SSLProtocol all -SSLv2
    
            DocumentRoot /path/to/ssl/enabled/site
            <Directory /path/to/ssl/enabled/site/>
              SSLRequireSSL
              Order Deny,Allow
              Allow from All
            </Directory>
          </VirtualHost>
    

    StrictRequire
    This forces forbidden access when SSLRequireSSL or SSLRequire successfully decided that access should be forbidden. Usually the default is that in the case where a “Satisfy any” directive is used, and other access restrictions are passed, denial of access due to SSLRequireSSL or SSLRequire is overridden (because that’s how the Apache Satisfy mechanism should work.) But for strict access restriction you can use SSLRequireSSL and/or SSLRequire in combination with an “SSLOptions +StrictRequire”. Then an additional “Satisfy Any” has no chance once mod_ssl has decided to deny access.

    Enable SSLv3 and TLSv1, but not SSLv2
    SSLProtocol all -SSLv2

    Setting up a self-signed certificate with the certificate and key in one file

    Generate your key and certificate

    Most of these parameters explain themselves, see beneath for those who do not.

    openssl req -new -newkey rsa:2048 -days 365 -nodes -x509 -keyout website.pem -out website.pem

    -nodes
    don’t encrypt the output key
    -x509
    output a x509 structure instead of a cert. req.

    Copy the key and certificate

    # cp website.pem  /etc/httpd/conf/

    Set permissions and ownership on your key and certificate

    This way nobody except root has read access.

    chmod 440 /etc/httpd/conf/website.pem
    chown root:root /etc/httpd/conf/website.pem

    Alter the apache configuration file, also known as httpd.conf

    Edit /etc/httpd/conf/httpd.conf with your favorite text editor, in my case, nano. Add the following text at the bottom of the file.

          <VirtualHost *:443>
            SSLEngine on
            # Change the next line according to where you've actually
            # stored the certificate and key file.
            SSLCertificateFile /etc/httpd/conf/website.pem
    
            ServerName domain.tld
            SSLOptions StrictRequire
            SSLProtocol all -SSLv2
    
            DocumentRoot /path/to/ssl/enabled/site
            <Directory /path/to/ssl/enabled/site/>
              SSLRequireSSL
              Order Deny,Allow
              Allow from All
            </Directory>
          </VirtualHost>
    

    StrictRequire
    This forces forbidden access when SSLRequireSSL or SSLRequire successfully decided that access should be forbidden. Usually the default is that in the case where a “Satisfy any” directive is used, and other access restrictions are passed, denial of access due to SSLRequireSSL or SSLRequire is overridden (because that’s how the Apache Satisfy mechanism should work.) But for strict access restriction you can use SSLRequireSSL and/or SSLRequire in combination with an “SSLOptions +StrictRequire”. Then an additional “Satisfy Any” has no chance once mod_ssl has decided to deny access.

    Enable SSLv3 and TLSv1, but not SSLv2
    SSLProtocol all -SSLv2

    // CrashMAG

    View information about your BIOS from Linux using dmidecode

    To get at this information we will use a utility called “dmidecode”. dmidecode is a tool for dumping a computer’s DMI (some say SMBIOS) table contents in a human-readable format.

    On CentOS/RHEL/Fedora you may run the following to install it.

    # yum install dmidecode

    On Arch Linux you may run

    # pacman -S dmidecode

    The following examples will allow you to see a few important parts of information such as;

    • The manufacturer of your motherboard
    • What type of motherboard you have
    • The version of the BIOS running on your motherboard

    To view the manufacturer and what type of motherboard you have, run the following

    dmidecode --type system

    Example

    # dmidecode 2.11
    SMBIOS 2.4 present.
    
    Handle 0x0001, DMI type 1, 27 bytes
    System Information
            Manufacturer: Gigabyte Technology Co., Ltd.
            Product Name: GA-MA78G-DS3H
            Version:
            Serial Number:
            UUID: 4E2F4100-0000-0000-0000-0000FFFFFFFF
            Wake-up Type: Power Switch
            SKU Number:
            Family:
    
    Handle 0x0034, DMI type 32, 11 bytes
    System Boot Information
            Status: No errors detected

    To view the version of your BIOS you may run the following

    #dmidecode --type bios

    Example

    # dmidecode 2.11
    SMBIOS 2.4 present.
    
    Handle 0x0000, DMI type 0, 24 bytes
    BIOS Information
            Vendor: Award Software International, Inc.
            Version: FA
            Release Date: 09/19/2008
            Address: 0xE0000
            Runtime Size: 128 kB
            ROM Size: 1024 kB
            Characteristics:
                    ISA is supported
                    PCI is supported
                    PNP is supported
                    APM is supported
                    BIOS is upgradeable
                    BIOS shadowing is allowed
                    Boot from CD is supported
                    Selectable boot is supported
                    BIOS ROM is socketed
                    EDD is supported
                    5.25"/360 kB floppy services are supported (int 13h)
                    5.25"/1.2 MB floppy services are supported (int 13h)
                    3.5"/720 kB floppy services are supported (int 13h)
                    3.5"/2.88 MB floppy services are supported (int 13h)
                    Print screen service is supported (int 5h)
                    8042 keyboard services are supported (int 9h)
                    Serial services are supported (int 14h)
                    Printer services are supported (int 17h)
                    CGA/mono video services are supported (int 10h)
                    ACPI is supported
                    USB legacy is supported
                    AGP is supported
                    LS-120 boot is supported
                    ATAPI Zip drive boot is supported
                    BIOS boot specification is supported
                    Targeted content distribution is supported
    
    Handle 0x0029, DMI type 13, 22 bytes
    BIOS Language Information
            Language Description Format: Long
            Installable Languages: 3
                    n|US|iso8859-1
                    n|US|iso8859-1
                    r|CA|iso8859-1
            Currently Installed Language: n|US|iso8859-1

    There’s also additional options to use with dmidecode. You probably also want to try the following to get an idea of what type of information you can get your hands on.

    #dmidecode --type keyword
    Valid type keywords are:
      bios
      system
      baseboard
      chassis
      processor
      memory
      cache
      connector
      slot

    // CrashMAG

    Changing the default PostgreSQL data folder (PGDATA)

    Installing the PostgreSQL server on RHEL, CentOS, Scientific Linux or Fedora installs the PostgreSQL databases and configuration files in “/var/lib/pgsql/data”.

    This may or may not be desirable. Let’s assume for a moment you have a separately crafted partition for PostgreSQL to use, let’s say a RAID10 volume. You’d want to change this.

    Change the defaults

    Use your favorite text editor, in my case nano to create the following file (must be the same as the name of the service)

    # nano /etc/sysconfig/pgsql/postgresql

    Add the following

    PGDATA=/postgresql/data

    Optionally you can also add the following to change the default port (example is the default port)

    PGPORT=5432

    Adjusting SELinux to permit the new data folder (pgdata) location

    Should the following command output “Permissive” or “Disabled” then you may skip the details for SELinux.

    # getenforce

    Run the semanage command to add a context mapping for /opt/postgresql and any other directories/files within it.

    # semanage fcontext -a -t postgresql_db_t "/postgresql/data(/.*)?"

    Now use the restorecon command to apply this context mapping to the running system

    # restorecon -Rv /postgresql/data

    Starting PostgreSQL

    # chkconfig --levels 345 postgresql on
    # service postgresql initdb
    # service postgresql start

    You’re all set to go! Keep in mind that PostgreSQL listens to ‘localhost’ by default. To change this you need to alter the “listen_address” parameter in “/var/lib/pgsql/data/postgresql.conf” (change will require restart).

    // CrashMAG

    Managing /etc with etckeeper and git

    The following was done on Fedora 14. Keep in mind that the Etckeeper and git specific actions will be similar on whatever platform you’re on.

    Simply put, Etckeeper automatically revisions your /etc folder. Allows you to compare, commit and revert the changes that have been made. It’ll also allow you to restore files, should you be unlucky and delete them. Once etckeeper is installed, it will work together with your package manager and cron to do its work. To manage all this you’ll use the commands that your chosen VCS (Version Control System).

    Etckeeper supports Git, Bazaar, Darcs and Mercurial.

    Use of Etckeeper

    Installation

    yum install etckeeper

    Initialization

    etckeeper init

    Initial commit

    etckeeper commit "initial commit"

    Once this is done, etckeeper will make sure that every time you use the package manager (YUM) changes will be recorded. There are however a few git related commands you should be aware of.

    Useful and necessary commands

    Note: All of these commands assumes your current path is /etc

    Viewing the Git log

    git log

    Check if there’s any modified files

    git status

    Complete status overview

    git log --stat --summary

    Revert a change

    git revert 

    View changes you haven’t commited yet

    git diff

    List different commits, each on one line.

    git log --pretty=oneline

    Revert to latest change-set, discarding changes

    git reset --hard

    Re-enter commit message

    git commit --amend

    Have at it folks!

    // CrashMAG

    Disable IPv6 lookups with Bind on RHEL or CentOS

    Discovered during a recent project. Bind / Named was constantly spamming the logs about it being unable to reach root servers. The logs revealed that we were talking IPv6 addresses. Which was assumed to be disabled.

    The less cool part was that in “/etc/named.conf” the following was commented out.

    //      listen-on-v6 port 53 { ::1; };

    It turns out that to disable the IPv6 lookups you have to edit “/etc/sysconfig/named” and set

    OPTIONS="-4"

    The option does the following

    Use IPv4 only even if the host machine is capable of IPv6. -4 and -6 are mutually exclusive.

    You then run

    service named restart

    This serves the very practical purpose of not spamming the logs. My ISP has yet to enable IPv6 so it does me no good.

    // CrashMAG

    How to configure the networking in Fedora 14 when you used a minimal install

    Using the minimal Fedora 14 install presented two small challenges.

    1. No networking except for loopback / 127.0.0.1
    2. No nano to edit the relevant configuration files.

    In a nutshell, it’s a paradox. I’d like to get nano to edit configuration files. But to do that I need network access. Turned out that I had to use “vi” which I never do to edit the networking files. What a pain. I personally can’t stress how retarded it is with an editor, that requires you to enter text, to be able to enter text.

    So the following was done to remedy the matter.

    Edit the networking configuration using vi

    # vi /etc/sysconfig/networking-scripts/ifcfg-eth0

    Used the arrows to navigate to the end of the “ONBOOT=no” line.

    1. Pressed i to enter insert mode.
    2. Modified “ONBOOT=no” to “ONBOOT=yes”.
    3. Pressed ESC to exit insert mode.
    4. Pressed o to add a new line.
    5. Press i to enter insert mode.
    6. Added “BOOTPROTO=dhcp”
    7. Pressed ESC to exit insert mode.
    8. Typed in :wq to exit and save the file.

    Or for a static IP

    1. Press i to enter insert mode.
    2. Modify “ONBOOT=no” to “ONBOOT=yes”.
    3. Press ESC to exit insert mode.
    4. Press o to add a new line.
    5. Press i to enter insert mode.
    6. Added “BOOTPROTO=static”
    7. Press o to add a new line.
    8. Add IPADDR=X.X.X.X
    9. Press o to add a new line.
    10. Add NETMASK=X.X.X.X
    11. Press ESC to exit insert mode.
    12. Type in :wq to exit and save the file.

    Restart the networking service

    # service network restart

    Done!

    Install nano

    # yum install nano

    Voila! This way one can edit text files easily, without having to enter text to enter text like in vi. (Made my dizzy just typing it)

    // CrashMAG