Tag Archives: centos

Setting up a 2-node GlusterFS filesystem

This will be a quick howto on how you would set up a 2-node GlusterFS filesystem. You may look up more information at http://www.gluster.org/.

Volume types for GlusterFS

– Distributed. Distributed volumes distributes files throughout the bricks in the volume
– Replicated. Replicated volumes replicates files across bricks in the volume
– Striped. Striped volumes stripes data across bricks in the volume
– Distributed Striped. Distributed striped volumes stripe data across two or more nodes in the cluster
– Distributed Replicated. Distributed replicated volumes distributes files across replicated bricks in the volume
– Distributed Striped Replicated. Distributed striped replicated volumes distributes striped data across replicated bricks in the cluster
– Striped Replicated. Striped replicated volumes stripes data across replicated bricks in the cluster

The high level overview of how the process will be is as follows

  • Installing the required software
  • Disable or add proper firewall rules
  • Adding nodes into the cluster
  • Preparing “bricks” for use on each server
  • Creating and starting the actual GlusterFS volume
  • Mounting the GlusterFS volume
  • Installing the required software

    I will be providing examples for CentOS, Fedora, Debian and Arch Linux. The examples for CentOS will work for RHEL and Scientific Linux as well.
    CentOS
    The following command will install all dependencies.

    # yum install glusterfs

    Fedora
    The following command will install all dependencies.

    # yum install glusterfs-server

    Debian
    The following command will install all dependencies.

    # apt-get install glusterfs-server

    Arch Linux
    The following command will install all dependencies.

    # pacman -S glusterfs

    Disable or add proper firewall rules

    You will need to open the following ports for GlusterFS.

    24007 – GlusterFS Daemon
    24008 – Management
    24009 - Each brick for every volume on your host requires it’s own port. For every new brick, one new port will be used starting at 24009. (For GlusterFS versions earlier than 3.4)
    49152 - Each brick for every volume on your host requires it’s own port. For every new brick, one new port will be used starting at 49152 (GlusterFS 3.4 and later)
    38465:38467 - This is required if you use the GlusterFS NFS service.
    

    CentOS
    Disabling the default firewall

    # chkconfig iptables off
    # service stop iptables

    Fedora

    systemctl disable firewalld
    systemctl stop firewalld

    Debian
    There are no default firewall installed on Debian.
    Arch Linux
    There are no default firewall installed on Arch Linux.

    Adding nodes into the cluster

    This is incredibly easy. You may do the following command from either server. In my example I am on server1. If you don’t have a solid DNS you should add each server to each others hosts file.

    # gluster peer probe server2
    Probe successful

    Preparing “bricks” for use on each server

    Nothing fanzy, you just need to create folders. It’s also important to note that you will need to use a folder, even if you intended to use a single disk.
    Execute the following on both of your servers

    # mkdir -p /data/brick>

    Creating and starting the actual GlusterFS volume

    Creating the GlusterFS volume
    Syntax:

    gluster volume create NEW-VOLNAME [replica COUNT] [transport [tcp | rdma | tcp,rdma]] NEW-BRICK...

    Example:

    # gluster volume create test-volume replica 2 transport tcp server1:/data/brick server2:/data/brick
    Creation of test-volume has been successful
    Please start the volume to access data.
    

    Starting the GlusterFS volume

    # gluster volume start test-volume

    Mounting the GlusterFS volume

    It’s important to note that you will need to mount the GlusterFS to use it. WARNING: Adding files directly to a brick will not be included in a GlusterFS volume.
    Syntax:

    # mount.glusterfs servername:volumename /mnt/mountpoint

    Examples:

    # mount.glusterfs server1:test-volume /mnt/glusterfs/

    OR

    # mount -t glusterfs server1:test-volume /mnt/glusterfs/

    References

    http://www.gluster.org/wp-content/uploads/2012/05/Gluster_File_System-3.3.0-Administration_Guide-en-US.pdf
    http://gluster.org/community/documentation/index.php/QuickStart

    // CrashMAG

    Disable the filesystem check (fsck) at boot time

    There’s several ways of accomplishing this. I will list all the methods beneath, just pick the one that fits the situation/you.

    • Filesystem tunable
    • Grub boot parameter
    • Placing command files on your root device
    • Active reboot without FSCK

    Filesystem tunable

    Use the tune2fs command to tell your filesystem to have a max count of mounts before a check to 0 to disable it.

    # tune2fs -c 0 /dev/sda1

    Parameter reference:

    -c max-mount-counts
     Adjust the number of mounts after which the filesystem will be  checked  by  e2fsck(8).   If max-mount-counts  is  0  or -1, the number of times the filesystem is mounted will be disregarded by e2fsck(8) and the kernel.
    

    Grub boot parameter

    Add the following at the end of your grub boot linux line.

    fastboot

    This can be done by editing “grub.conf” or by editing the boot command via the grub menu at boot.

    Placing command files on your root device

    To disable the filesystem check on boot.

    # touch /fastboot

    To enable a filesystem check on boot.

    # touch /forcefsck

    Active reboot without FSCK

    # shutdown -rf

    Parameter reference:

    -r     Reboot after shutdown.
    -f     Skip fsck on reboot.
    

    // CrashMAG

    Linux ACL

    An access control list (ACL), with respect to a computer file system, is a list of permissions attached to an object. ACL allows you to grant or deny permissions for any user or group on a filesystem resource.

    Enabling ACL

    To enable ACL, edit your /etc/fstab file as such:

    /dev/VolGroup00/LogVol00 /                       ext3    defaults,acl        1 1

    Note: Moderm Redhat distributions enable ACL by default for the root filesystem.

    Set ACL

    To modify ACL use setfacl command. To add permissions use setfacl -m.

    Add permissions to some user:

    # setfacl -m "u:username:permissions"

    or

    # setfacl -m "u:uid:permissions"

    Add permissions to some group:

    # setfacl -m "g:groupname:permissions"

    or

    # setfacl -m "g:gid:permissions"

    Add default ACL:

    # setfacl -d -m "u:uid:permissions"

    Remove all permissions:

    # setfacl -b

    Remove each entry:

    # setfacl -x "entry"

    To check permissions use:

    # getfacl filename

    Examples

    Set read,write and execute permissions for user “johndoe” on the file named “abc”.

    # setfacl -m "u:johndoe:rwx" abc

    Check permissions.

    # getfacl abc
    # file: abc
    # owner: someone
    # group: someone
    user::rw-
    user:johny:rwx
    group::r--
    mask::rwx
    other::r--

    Change permissions for user “johndoe”.

    # setfacl -m "u:johndoe:rw-" abc

    Check permissions.

    # getfacl abc
    # file: abc
    # owner: someone
    # group: someone
    user::rw-
    user:johndoe:rw-
    group::r--
    mask::r-x
    other::r--

    Remove all extended ACL entries.

    # setfacl -b abc

    Check permissions.

    # getfacl abc
    # file: abc
    # owner: someone
    # group: someone
    user::rw-
    group::r--
    other::r--

    Additional Resources

    man getfacl
    man setfacl

    If you weren’t using these already, you should.

    // CrashMAG

    Change the default SSH port and alter SELinux context to match

    Security through obscurity is not something one would generally recommend. But to thwart the effort of automated scanners changing the default OpenSSH port will yield you less pain in every day life. This will not fend off directed attacks or nullify vulnerabilities or bad security design.

    Should you see an error message such as

    shd[14221]: error: Bind to port 9898 on 192.168.0.50 failed: Permission denied

    it indicates that the system prevented the daemon to bind that port. Most likely SELinux.

    The instructions provided will be valid on Fedora 14/15, CentOS 6, RHEL 6, Scientific Linux 6 and newer versions.

    To change the default SSH port you need to do the following.

    • Stop the SSH daemon
    • Alter the /etc/ssh/sshd_config with your new port
    • Alter the SELinux context with semanage
    • Start the SSH daemon

    Stop the SSH daemon

    # service sshd stop

    Alter the /etc/ssh/sshd_config with your new port

    Alter the configuration file with your favorite editor, in my case “nano”.

    # nano /etc/ssh/sshd_config

    Alter the port configuration parameter change the following line

    Port 22

    to

    Port 9898

    Alter the SELinux context with semanage

    # semanage port -a -t ssh_port_t -p tcp 9898

    Initially you would think the following would work. But it will not. For it to work you would have to alter the policy in the selinux-policy package, rebuild and install it. So skip it, but now you know why.

    # semanage port -d -t ssh_port_t -p tcp 22

    Start the SSH daemon

    # service sshd start

    // CrashMAG

    Change the default MySQL data directory with SELinux enabled

    This is a short article that explains how you change the default MySQL data directory and adjust SELinux to account for the changes. The article assumes that you’re running either RHEL, CentOS, Scientific Linux or Fedora with SELinux enabled. This works with the most recent EL (6.2) version.

    We’ll be doing this in the following order.

    • Stopping the MySQL server
    • Create a new data directory and move the content from the old data directory
    • Correct the MySQL configuration file
    • Adjust SELinux parameters to accept our new change
    • Starting the MySQL server

    Stopping the MySQL server

    # service mysqld stop

    Create a new data diretory and move the content from the old one

    Creating a new data directory

    # mkdir /srv/mysql/
    # chown mysql:mysql /srv/mysql

    Moving the original data files

     # mv /var/lib/mysql/* /srv/mysql/

    Correct the MySQL configuration file

    Edit the my.cnf file for your distribution. In my example it’s located in the /etc/mysql/ directory. RHEL/CentOS/Scientific Linux put the my.cnf file directly in /etc by default.

    # nano /etc/mysql/my.cnf

    Change

    datadir=/var/lib/mysql

    to

    datadir=/srv/mysql

    and

    socket=/var/lib/mysql/mysql.sock

    to

    socket=/srv/mysql/mysql.sock

    and save the file.

    Adjust SELinux parameters to accept our new change

    Should the following command output “Permissive” or “Disabled” then you may skip the details for SELinux.

    # getenforce

    Run the semanage command to add a context mapping for /srv/mysql.

    # semanage fcontext -a -t mysqld_db_t "/srv/mysql(/.*)?"

    Now use the restorecon command to apply this context mapping to the running system.

    # restorecon -Rv /srv/mysql

    Starting the MySQL server

    # service mysqld start

    Verifying access and connectivity

    $ mysql -u root -p
    mysql> show databases;

    If this is working, you’re up and running. Should you get a message that says

    ERROR 2002 (HY000): Can’t connect to local MySQL server through socket ‘/var/lib/mysql/mysql.sock’

    then add the following to your /etc/my.cnf

    [client]
    socket = /srv/mysql/mysql.sock
    

    Optionally you can just use

    $ mysql -u root -p --protocol tcp

    to avoid connecting via the socket.

    // CrashMAG

    Self-signed certificate for Apache

    These instructions are distribution agnostic. However I used CentOS during my tests, so file paths will match that of CentOS, RHEL, Scientific Linux and Fedora. For any other distribution you’ll have to look that up yourself.

    The tools required are OpenSSL, Apache and mod_ssl for Apache. To accomplish this I had to run

    # yum install mod_ssl

    on my CentOS 5.6 box. Which already had Apache up and running.

    Setting up a self-signed certificate using certificate and key

    Generate your key and certificate

    Most of these parameters explain themselves, see beneath for those who do not.

    openssl req -new -newkey rsa:2048 -days 365 -nodes -x509 -keyout website.key -out website.crt

    -nodes
    don’t encrypt the output key
    -x509
    output a x509 structure instead of a cert. req.

    Copy the key and certificate

    # cp website.key website.crt /etc/httpd/conf/

    Set permissions and ownership on your key and certificate

    This way nobody except root has read access.

    chmod 440 /etc/httpd/conf/website.key /etc/httpd/conf/website.crt
    chown root:root /etc/httpd/conf/website.key /etc/httpd/conf/website.crt

    Alter the apache configuration file, also known as httpd.conf

    Edit /etc/httpd/conf/httpd.conf with your favorite text editor, in my case, nano. Add the following text at the bottom of the file.

          <VirtualHost *:443>
            SSLEngine on
            # Change the next two lines according to where you've actually
            # stored the certificate and key files.
            SSLCertificateFile /etc/httpd/conf/website.crt
    	SSLCertificateKeyFile /etc/httpd/conf/apache2/website.key
    
            ServerName domain.tld
            SSLOptions StrictRequire
            SSLProtocol all -SSLv2
    
            DocumentRoot /path/to/ssl/enabled/site
            <Directory /path/to/ssl/enabled/site/>
              SSLRequireSSL
              Order Deny,Allow
              Allow from All
            </Directory>
          </VirtualHost>
    

    StrictRequire
    This forces forbidden access when SSLRequireSSL or SSLRequire successfully decided that access should be forbidden. Usually the default is that in the case where a “Satisfy any” directive is used, and other access restrictions are passed, denial of access due to SSLRequireSSL or SSLRequire is overridden (because that’s how the Apache Satisfy mechanism should work.) But for strict access restriction you can use SSLRequireSSL and/or SSLRequire in combination with an “SSLOptions +StrictRequire”. Then an additional “Satisfy Any” has no chance once mod_ssl has decided to deny access.

    Enable SSLv3 and TLSv1, but not SSLv2
    SSLProtocol all -SSLv2

    Setting up a self-signed certificate with the certificate and key in one file

    Generate your key and certificate

    Most of these parameters explain themselves, see beneath for those who do not.

    openssl req -new -newkey rsa:2048 -days 365 -nodes -x509 -keyout website.pem -out website.pem

    -nodes
    don’t encrypt the output key
    -x509
    output a x509 structure instead of a cert. req.

    Copy the key and certificate

    # cp website.pem  /etc/httpd/conf/

    Set permissions and ownership on your key and certificate

    This way nobody except root has read access.

    chmod 440 /etc/httpd/conf/website.pem
    chown root:root /etc/httpd/conf/website.pem

    Alter the apache configuration file, also known as httpd.conf

    Edit /etc/httpd/conf/httpd.conf with your favorite text editor, in my case, nano. Add the following text at the bottom of the file.

          <VirtualHost *:443>
            SSLEngine on
            # Change the next line according to where you've actually
            # stored the certificate and key file.
            SSLCertificateFile /etc/httpd/conf/website.pem
    
            ServerName domain.tld
            SSLOptions StrictRequire
            SSLProtocol all -SSLv2
    
            DocumentRoot /path/to/ssl/enabled/site
            <Directory /path/to/ssl/enabled/site/>
              SSLRequireSSL
              Order Deny,Allow
              Allow from All
            </Directory>
          </VirtualHost>
    

    StrictRequire
    This forces forbidden access when SSLRequireSSL or SSLRequire successfully decided that access should be forbidden. Usually the default is that in the case where a “Satisfy any” directive is used, and other access restrictions are passed, denial of access due to SSLRequireSSL or SSLRequire is overridden (because that’s how the Apache Satisfy mechanism should work.) But for strict access restriction you can use SSLRequireSSL and/or SSLRequire in combination with an “SSLOptions +StrictRequire”. Then an additional “Satisfy Any” has no chance once mod_ssl has decided to deny access.

    Enable SSLv3 and TLSv1, but not SSLv2
    SSLProtocol all -SSLv2

    // CrashMAG

    View information about your BIOS from Linux using dmidecode

    To get at this information we will use a utility called “dmidecode”. dmidecode is a tool for dumping a computer’s DMI (some say SMBIOS) table contents in a human-readable format.

    On CentOS/RHEL/Fedora you may run the following to install it.

    # yum install dmidecode

    On Arch Linux you may run

    # pacman -S dmidecode

    The following examples will allow you to see a few important parts of information such as;

    • The manufacturer of your motherboard
    • What type of motherboard you have
    • The version of the BIOS running on your motherboard

    To view the manufacturer and what type of motherboard you have, run the following

    dmidecode --type system

    Example

    # dmidecode 2.11
    SMBIOS 2.4 present.
    
    Handle 0x0001, DMI type 1, 27 bytes
    System Information
            Manufacturer: Gigabyte Technology Co., Ltd.
            Product Name: GA-MA78G-DS3H
            Version:
            Serial Number:
            UUID: 4E2F4100-0000-0000-0000-0000FFFFFFFF
            Wake-up Type: Power Switch
            SKU Number:
            Family:
    
    Handle 0x0034, DMI type 32, 11 bytes
    System Boot Information
            Status: No errors detected

    To view the version of your BIOS you may run the following

    #dmidecode --type bios

    Example

    # dmidecode 2.11
    SMBIOS 2.4 present.
    
    Handle 0x0000, DMI type 0, 24 bytes
    BIOS Information
            Vendor: Award Software International, Inc.
            Version: FA
            Release Date: 09/19/2008
            Address: 0xE0000
            Runtime Size: 128 kB
            ROM Size: 1024 kB
            Characteristics:
                    ISA is supported
                    PCI is supported
                    PNP is supported
                    APM is supported
                    BIOS is upgradeable
                    BIOS shadowing is allowed
                    Boot from CD is supported
                    Selectable boot is supported
                    BIOS ROM is socketed
                    EDD is supported
                    5.25"/360 kB floppy services are supported (int 13h)
                    5.25"/1.2 MB floppy services are supported (int 13h)
                    3.5"/720 kB floppy services are supported (int 13h)
                    3.5"/2.88 MB floppy services are supported (int 13h)
                    Print screen service is supported (int 5h)
                    8042 keyboard services are supported (int 9h)
                    Serial services are supported (int 14h)
                    Printer services are supported (int 17h)
                    CGA/mono video services are supported (int 10h)
                    ACPI is supported
                    USB legacy is supported
                    AGP is supported
                    LS-120 boot is supported
                    ATAPI Zip drive boot is supported
                    BIOS boot specification is supported
                    Targeted content distribution is supported
    
    Handle 0x0029, DMI type 13, 22 bytes
    BIOS Language Information
            Language Description Format: Long
            Installable Languages: 3
                    n|US|iso8859-1
                    n|US|iso8859-1
                    r|CA|iso8859-1
            Currently Installed Language: n|US|iso8859-1

    There’s also additional options to use with dmidecode. You probably also want to try the following to get an idea of what type of information you can get your hands on.

    #dmidecode --type keyword
    Valid type keywords are:
      bios
      system
      baseboard
      chassis
      processor
      memory
      cache
      connector
      slot

    // CrashMAG

    Resetting the root/postgres password for PostgreSQL

    The following is required to reset the root/postgres user password for PostgreSQL. The distribution used in my example is CentOS 5.5 and PostgreSQL 8.4.

    Note: By default there’s no password for the postgres user.

    In step 2 and 5 you will most likely not be using “ident” but rather “password” or “md5”.

    1. Shut down PostgreSQL

    # service postgresql stop

    2. Reset the authentication mechanism (assuming defaults are already being used)

    Edit the /usr/lib/pgsql/data/pg_hba.conf file

    # nano /usr/lib/pgsql/data/pg_hba.conf

    Navigate down to the line that says

    local all all ident

    Edit it to

    local all postgres trust

    And now save the file.

    3. Start PostgreSQL

    # service postgresql start

    4. Log in and change the password

    # su - postgres
    $ psql -d template1 -U postgres
    alter user postgres with password 'new_password';

    Or alternatively do it all in one go with the following command

    > psql -U postgres template1 -c "alter user postgres with password 'new_password';"

    5. Reverse the actions you did in step 2

    Edit the /usr/lib/pgsql/data/pg_hba.conf file

    # nano /usr/lib/pgsql/data/pg_hba.conf

    Navigate down to the line that says

    local all all trust

    Edit it to

    local all postgres ident

    And now save the file.

    6. Start PostgreSQL

    # service postgresql start

    Success!

    // CrashMAG

    Changing the default PostgreSQL data folder (PGDATA)

    Installing the PostgreSQL server on RHEL, CentOS, Scientific Linux or Fedora installs the PostgreSQL databases and configuration files in “/var/lib/pgsql/data”.

    This may or may not be desirable. Let’s assume for a moment you have a separately crafted partition for PostgreSQL to use, let’s say a RAID10 volume. You’d want to change this.

    Change the defaults

    Use your favorite text editor, in my case nano to create the following file (must be the same as the name of the service)

    # nano /etc/sysconfig/pgsql/postgresql

    Add the following

    PGDATA=/postgresql/data

    Optionally you can also add the following to change the default port (example is the default port)

    PGPORT=5432

    Adjusting SELinux to permit the new data folder (pgdata) location

    Should the following command output “Permissive” or “Disabled” then you may skip the details for SELinux.

    # getenforce

    Run the semanage command to add a context mapping for /opt/postgresql and any other directories/files within it.

    # semanage fcontext -a -t postgresql_db_t "/postgresql/data(/.*)?"

    Now use the restorecon command to apply this context mapping to the running system

    # restorecon -Rv /postgresql/data

    Starting PostgreSQL

    # chkconfig --levels 345 postgresql on
    # service postgresql initdb
    # service postgresql start

    You’re all set to go! Keep in mind that PostgreSQL listens to ‘localhost’ by default. To change this you need to alter the “listen_address” parameter in “/var/lib/pgsql/data/postgresql.conf” (change will require restart).

    // CrashMAG

    RHEL/Centos 5 minimal installation

    There’s no option during the CentOS 5 install, for a minimal installation. The purpose is quite simple, to keep the attack surface as small as possible.

    A minimal installation is performed by doing the following

    • During the category/task selection, deselect all package categories, and choose the “Customize now” option at the bottom of screen.
    • During the customized package selection, deselect everything ( including the Base group ).

    This will yield you 234 packages with the Centos 5.5 installation media. I’ve attached a .txt file containing all the packages for your leisure.

    Link: installed-packages

    // CrashMAG