Understanding cron environment variables

27 June 2015System Admincentos, cron, linux, scriptingHazrul Nizam Sidek

Introduction

I wanted to automate a certain task on a CentOS server, so I wrote a bash script to perform this task, with the intention of using cron to provide the scheduling.

The script worked beautifully when invoked from the SSH bash shell. However, when cron calls the script, the task failed.

Looking at the error logs, it seems like some line of commands failed with “command not found” error. This usually means that the program called in those lines are nowhere to be found in the current $PATH environment variable.

I have always assumed that when a cron job executes, it inherits the same environment variables as when you log in directly to the shell via SSH. The above error proves that this is not the case.

Finding the difference

In order to modify the script so that it will work with cron, I needed to know what the environment looks like when cron calls a script. Thankfully, a quick Google search revealed this post on stackoverflow.

First, we add the following line to the crontab. This will save the output of the env command, which is the list of all current environment variables, into a file once every minute. We just need it to run once, therefore remove the line from crontab once the first minute mark has passed.

* * * * * env > ~/env.out

The contents of this output file reveal that the environment variables loaded when cron runs a script is a very small subset of the environment variables loaded when you get to the shell prompt via SSH login.

Load the environment and test

Now we can use this output file to load the same environment variables as when cron executes a script, and modify the script we wish to run via cron until it works.

# env - `cat ~/env.out` /bin/sh
# ./script.sh

Renewing SSL certificates

16 February 2015System Admincert, certificate, openssl, renew, sslHazrul Nizam Sidek

It is that time of the year when I needed to renew all of my SSL certificates for my web servers and email servers. This post is a reminder for myself on the steps and commands needed to succesfully renew the certificates, for future reference.

First, check the current certificates and key files. For CentOS 6, this is located at /etc/pki/tls/private and /etc/pki/tls/certs.

[root@localhost ~]# cd /etc/pki/tls/private
[root@localhost private]# ls -lart
-rw-------. 1 root root 1704 Feb 19  2014 localhost.key
-rw-r--r--. 1 root root 1033 Feb 20  2014 localhost.csr
drwxr-xr-x. 5 root root 4096 Jan 29 15:33 ..
drwxr-xr-x. 2 root root 4096 Feb 16 09:06 .
[root@localhost private]# cd ../certs
[root@localhost certs]# ls -lart
-rw-r--r--. 1 root root    1805 Feb 20  2014 localhost.crt
-rw-r--r--. 1 root root    1757 Feb 20  2014 localhost-chain.crt
-rw-r--r--. 1 root root    1521 Feb 20  2014 localhost-root.crt
-rw-r--r--. 1 root root    5083 Feb 20  2014 localhost-postfix.crt
-rw-r--r--. 1 root root 1005005 Jul 14  2014 ca-bundle.trust.crt
-rw-r--r--. 1 root root  786601 Jul 14  2014 ca-bundle.crt
-rwxr-xr-x. 1 root root     829 Jan 21 01:32 renew-dummy-cert
-rw-r--r--. 1 root root    2242 Jan 21 01:32 Makefile
-rwxr-xr-x. 1 root root     610 Jan 21 01:32 make-dummy-cert
drwxr-xr-x. 5 root root    4096 Jan 29 15:33 ..
drwxr-xr-x. 2 root root    4096 Feb 16 09:09 .

Next create the new certificate signing request (CSR). Take this chance to create a new private key, with perhaps a higher number of bits and a better algorithm. This is done using the -newkey, -sha256, and -keyout flags. The -nodes flag is used to create the new private key without a passphrase (so that the services using the private key can start without needing input from a human).

[root@localhost ~]# cd /etc/pki/tls/private/
[root@localhost private]# openssl req -nodes -newkey rsa:4096 -sha256 -keyout localhost-new.key -out localhost-new.csr
Generating a 4096 bit RSA private key
................................................++
......................++
writing new private key to 'localhost-new.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:MY
State or Province Name (full name) []:Kuala Lumpur
Locality Name (eg, city) [Default City]:Kuala Lumpur
Organization Name (eg, company) [Default Company Ltd]:hazrulnizam.com
Organizational Unit Name (eg, section) []:localhost
Common Name (eg, your name or your server's hostname) []:localhost.hazrulnizam.com
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
[root@localhost private]# ls -lart
-rw-------. 1 root root 1704 Feb 19  2014 localhost.key
-rw-r--r--. 1 root root 1033 Feb 20  2014 localhost.csr
drwxr-xr-x. 5 root root 4096 Jan 29 15:33 ..
-rw-r--r--. 1 root root 3272 Feb 16 10:05 localhost-new.key
-rw-r--r--. 1 root root 1748 Feb 16 10:05 localhost-new.csr
drwxr-xr-x. 2 root root 4096 Feb 16 10:05 .

Send the CSR file (localhost-new.csr) to the certificate authority, and wait for them to issue the certificate for your new key. Put the certificate files (including the CA root and chain) in the /etc/pki/tls/certs directory. Do not overwrite the old certificates yet! I named them localhost-new.crt, localhost-chain-new.crt and localhost-root-new.crt

Now we need to stop all the services using the old certificates.

[root@localhost ~]# service httpd stop
Stopping httpd:                                            [  OK  ]
[root@localhost ~]# service dovecot stop
Stopping Dovecot Imap:                                     [  OK  ]
[root@localhost ~]# service postfix stop
Shutting down postfix:                                     [  OK  ]

Now rename the old certificates for backup.

[root@localhost ~]# cd /etc/pki/tls/certs
[root@localhost certs]# mv localhost.crt localhost-old.crt
[root@localhost certs]# mv localhost-chain.crt localhost-chain-old.crt
[root@localhost certs]# mv localhost-root.crt localhost-root-old.crt

[root@localhost certs]# mv localhost-postfix.crt localhost-postfix-old.crt

Then it is time to rename the new certificates to take the old certificates’ names.

[root@localhost certs]# mv localhost-new.crt localhost.crt
[root@localhost certs]# mv localhost-chain-new.crt localhost-chain.crt
[root@localhost certs]# mv localhost-root-new.crt localhost-root.crt

For postfix, all the root, chain and host certificates need to be in one file. Create that file, taking the old file name.

[root@localhost certs]# cat localhost.crt >> localhost-postfix.crt
[root@localhost certs]# cat localhost-chain.crt >> localhost-postfix.crt
[root@localhost certs]# cat localhost-root.crt >> localhost-postfix.crt

Start all the services that was stopped.

[root@localhost ~]# service postfix start
Starting postfix:                                          [  OK  ]
[root@localhost ~]# service dovecot start
Starting Dovecot Imap:                                     [  OK  ]
[root@localhost ~]# service httpd start
Starting httpd:                                            [  OK  ]

If everything is done correctly, this should be the end of the exercise. Test all the services and make sure everything is working as before.

Install Oracle Enterprise Manager Cloud Control 12c Release 3 on Oracle Linux 6

21 May 2014Database, System Admin12c, cloud control, enterpise manager, install, linux, oracleHazrul Nizam Sidek

Introduction

This post is intended to document the process of installing Oracle Enterprise Manager Cloud Control 12c Release 3 on Oracle Linux 6.

Licensing

OEM Cloud Control is used as a centralised management point for Oracle Database installations in your environment.

You can install this for free and use the base functionalities for free as long as you have an active support subscription with Oracle for any targets (database, hosts, etc.) you want to manage.

The list of the base functionalities you can use for free can be reviewed on the OEM Cloud Control Licensing Guide.

Server setup

First, we start with a minimal installation of Oracle Linux 6 with 8GB of RAM and 100GB hard disk. A swap space of 8GB is configured.

Install Oracle Database

OEM Cloud Control will use an Oracle Database to store OEM data, and this database will have to have been installed when you run the OEM Cloud Control installer. The OEM Cloud Control installer will not install the Oracle Database as part of the installation.

You may use an existing Oracle Database for this purpose or you can install a separate database specifically for it.

From the Licensing Guide:

Enterprise Manager includes a restricted-use license of the Oracle Database for use only with the Oracle Management Repository or other complementary repositories included with Enterprise Manager (such as, Ops Center, Real User Experience Insight, Load Testing, and Test Manager).

Additional database options or additional servers for disaster recovery require separate licensing. Customers receive one single-instance database with the Cloud Control, or RMAN, repository. To protect the repository with Data Guard, customers need to purchase a license for the standby site. To protect the repository with Oracle Real Application Clusters, customers must license the second node for the database, and both nodes require an Oracle Real Application Clusters license.

As long as you use a single-instance installation of Oracle Database and do not use this database for anything other than OEM, the database is considered properly licensed.

Prep the server for Oracle Database installation

Funnily enough, at the time of writing, the highest version of Oracle Database certified to be used as the OEM repository is 11gR2. 12c is not certified.

You can check the certifications via My Oracle Support by following the steps in the Installation Guide.

Now we install the 11gR2 pre-requisite RPM to prepare the server for Oracle Database installation.

[root@server ~]# yum install oracle-rdbms-server-11gR2-preinstall

The preinstall RPM will have added the oracle OS user, as well as the oinstall and dba OS groups.

Next, we create the directory structure to hold the Oracle Database files.

[root@server ~]# mkdir -p /u01/app/oracle
[root@server ~]# chown -R oracle:oinstall /u01

Get the Oracle Database installer from OTN, and copy the files into the server. Finally, unzip the installation files.

Perform Oracle Database Installation

Set up X forwarding, and run the installer as the oracle user.

[oracle@server ~]$ cd database
[oracle@server database]$ ./runInstaller

Choose to configure a database during installation.

Choose a server class installation.

Choose single instance installation.

Choose Enterprise Edition, and de-select all options except Partitioning, which is required by OEM Cloud Control 12c. We remove all unnecessary options to avoid future trouble with Oracle Licensing.

Proceed with the rest of the installation as usual. Just accept all the defaults for the database configuration, as we will remove this database later and create a new one.

Create repository instance

After the installation has completed, run the Database Configuration Assistant (DBCA) and remove the instance configured during the database installation.

Then use DBCA to create a new instance from scratch. Using any of the seed databases supplied by Oracle will not do as the database for the OEM repository should not have the SYSMAN schema present.

The settings for the new database should match your expected deployment size. Refer to the Advanced Installation and Configuration Guide for sizing guide. For this article, I will be using the SMALL deployment size, which have the following settings for the repository database:

Parameter	Minimum Value
processes	300
pga_aggregate_target*	1024 MB
sga_target*	2 GB
redo log file size	300 MB
shared_pool_size	600 MB
*memory_target of 3 GB can be used in place of sga_target and pga_aggregate_target

We can now begin creating the database using DBCA.

Choose to create a database.

Choose to create a custom database.

Uncheck the Configure Enterprise Manager and Enable automatic maintenance tasks checkboxes.

Remove all the optional database components

Check the Use Automatic Memory Management checkbox and make sure the memory size is more than 3 GB.

Set processes to 300.

Change all three redo log file sizes to 300 MB.

Complete the database creation.

Install Oracle Enterprise Manager Cloud Control 12c Release 3

With the repository database ready, it is time to install the OEM proper.

Review package requirements.

First, review the operating system package requirements from the Installation Guide.

For my installation, since the oracle-rdbms-server-11gR2-preinstall package was already installed, the only missing package is the glibc-devel 32-bit package. So, that will now be installed using yum.

[root@server ~]# yum install glibc-devel.i686

Next, get the installer from OTN.

Unzip the files, and run the installer as the oracle user.

[oracle@server oem]$ ./runInstaller

Configure the email and software updates if you want to, and click Next.

The pre-requisite checks will be performed, and in my case, all the tests were successful.

Choose to create a new Advanced Enterprise Manager System.

Next, specify the location to install the Middleware and Agent components of OEM Cloud Control. I chose to use the ORACLE_BASE of the database installation as the base directory, with directories for the Middleware and Agent components located in this base.

The next page we will be able to choose which management plugins we wish to install. For my install, I will only use OEM Cloud Control to manage Oracle Databases. Therefore, I will not install any additional plugins.

Next, provide passwords WebLogic and Node Manager, and specify installation directory for OMS. Again, I used the same directory structure as the Oracle Database installation.

Step 8: Provide the details of the repository database. Here we insert the details of the repository database which we have installed earlier, and select the corresponding deployment size.

When I clicked Next, an error popped out saying that a CBO statistics collection job must be disabled. The installer offered to fix this automatically, and I accept.

After clicking Yes, the installer presented me with several warnings regarding database parameter mis-matches that will affect the performance of the OEM Cloud Control. I can choose to fix them now, or proceed with the installation and fix them later.

The first warning is about the memory_target parameter. However, checking the parameter shows that the parameter is set up correctly for the SMALL deployment size.

SQL> show parameter memory_target

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
memory_target                        big integer 3200M

The second warning is for the session_cached_cursors parameter. Change this to the recommended value. This parameter is not changeable online, so a database restart is required.

SQL> show parameter session_cached_cursors

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
session_cached_cursors               integer     50

SQL> alter system set session_cached_cursors=500 scope=SPFILE;

System altered.

SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup
ORACLE instance started.

Total System Global Area 3340451840 bytes
Fixed Size                  2257840 bytes
Variable Size            1811942480 bytes
Database Buffers         1509949440 bytes
Redo Buffers               16302080 bytes
Database mounted.
Database opened.
SQL> show parameter session_cached_cursors;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
session_cached_cursors               integer     500

The third and final change is to the shared_pool_size parameter.

SQL> show parameter shared_pool_size

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
shared_pool_size                     big integer 0
SQL> alter system set shared_pool_size=600M scope=both;

System altered.

SQL> show parameter shared_pool_size

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
shared_pool_size                     big integer 608M

Now, we return to the installer and click Cancel and re-run the database check. Now, only one warning appears, which I guess is an installer glitch in reading the memory parameter. Click Ok to resume with the installation.

Step 9: On this page of the installer, we specify the passwords for SYSMAN and registration. This SYSMAN password will be used in the initial login to the OEM Cloud Control. You can change the default locations of the tablespaces to be created, and also the location of the software library to be used by OEM Cloud Control.

Step 10: Review the port assignments for OEM Cloud Control, or change them if you wish.

Step 11: Finally, we get to the review page. Click Install to begin OEM Cloud Control installation.

The installer will prompt for execution of root scripts. Run the script.

[root@server ~]# /u01/app/oracle/middleware/oms/allroot.sh

Starting to execute allroot.sh .........

Starting to execute /u01/app/oracle/middleware/oms/root.sh ......
Running Oracle 11g root.sh script...

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/oracle/middleware/oms

Enter the full pathname of the local bin directory: [/usr/local/bin]: <Press Enter>
The file "dbhome" already exists in /usr/local/bin.  Overwrite it? (y/n)
[n]:<Press Enter>
The file "oraenv" already exists in /usr/local/bin.  Overwrite it? (y/n)
[n]:<Press Enter>
The file "coraenv" already exists in /usr/local/bin.  Overwrite it? (y/n)
[n]:<Press Enter>

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
/etc exist

Creating /etc/oragchomelist file...
/u01/app/oracle/middleware/oms
Finished execution of  /u01/app/oracle/middleware/oms/root.sh ......


Starting to execute /u01/app/oracle/agent/core/12.1.0.3.0/root.sh ......
Finished product-specific root actions.
/etc exist
Finished execution of  /u01/app/oracle/agent/core/12.1.0.3.0/root.sh ......

The installation is now completed.

Configure firewall for OEM Cloud Control

Now we we will need to configure the firewall to enable OEM Cloud Control to work properly. Of course, you can disable the firewall instead should you choose to.

Check the Port List

First, check the list of ports used by your installation. This list can be located inside your middleware installation directory <path_to_middleware>/oms/install/portlist.ini.

[root@server ~]# cat /u01/app/oracle/middleware/oms/install/portlist.ini
Enterprise Manager Upload Http Port=4889
Enterprise Manager Upload Http SSL Port=4903
Enterprise Manager Central Console Http SSL Port=7802
Node Manager Http SSL Port=7403
Managed Server Http Port=7202
Enterprise Manager Central Console Http Port=7788
Oracle Management Agent Port=3872
Admin Server Http SSL Port=7102
Managed Server Http SSL Port=7301

Configure Firewall

Next, configure the firewall to allow incoming connections from all the ports listed above.

[root@server ~]# iptables -I INPUT 5 -m state --state NEW -p tcp --dport 4889 -j ACCEPT
[root@server ~]# iptables -I INPUT 6 -m state --state NEW -p tcp --dport 4903 -j ACCEPT
[root@server ~]# iptables -I INPUT 7 -m state --state NEW -p tcp --dport 7802 -j ACCEPT
[root@server ~]# iptables -I INPUT 8 -m state --state NEW -p tcp --dport 7403 -j ACCEPT
[root@server ~]# iptables -I INPUT 9 -m state --state NEW -p tcp --dport 7202 -j ACCEPT
[root@server ~]# iptables -I INPUT 10 -m state --state NEW -p tcp --dport 7788 -j ACCEPT
[root@server ~]# iptables -I INPUT 11 -m state --state NEW -p tcp --dport 3872 -j ACCEPT
[root@server ~]# iptables -I INPUT 12 -m state --state NEW -p tcp --dport 7102 -j ACCEPT
[root@server ~]# iptables -I INPUT 13 -m state --state NEW -p tcp --dport 7301 -j ACCEPT
[root@server ~]# service iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables:[  OK  ]

Login to OEM Cloud Control

Now, we are ready to login for the first time to the OEM Cloud Control installation. Open up a browser and go to the following URL:

https://server:<port>/em

The port should be the port listed for Enterprise Manager Central Console Http SSL Port in the portlist.ini file that we checked earlier.

You will be presented with a License Agreement Page.

Read the license agreement and click I Accept.

You will now be presented with the Welcome Page. You can now start using OEM Cloud Control 12c.

Upgrade Oracle APEX from 4.2.4 to 4.2.5

25 April 2014Database, System Adminapex, centos, glassfish, oracle, oracle xe, upgradeHazrul Nizam Sidek

Introduction

Oracle recently released version 4.2.5 of its Application Express (APEX) software. This post is to document how to upgrade Oracle APEX from 4.2.4 to 4.2.5.

The Oracle APEX installation to be upgraded has the following specification:

The operating system of the server is CentOS 6.5.
The Oracle Database version is Oracle XE 11gR2.
Oracle APEX is served using the APEX Listener, via GlassFish 4.

Download and read the Patch Set Notes

First things first, read up on the patch set notes here.

Download and unzip the Patch Set

Next, the patch set is downloaded via Oracle Support.

The downloaded patch set is then extracted.

[oracle@server ~]$ unzip p17966818_425_Generic.zip

Backup database

The database is backed up as a precaution if the patch is unsuccessful.

I prefer a cold and consistent backup to ease the recovery process.

[oracle@server ~]$ rman target / nocatalog

Recovery Manager: Release 11.2.0.2.0 - Production on Fri Apr 25 09:38:09 2014

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.

connected to target database: XE (DBID=2709648321)
using target database control file instead of recovery catalog

RMAN> shutdown immediate

database closed
database dismounted
Oracle instance shut down

RMAN> startup mount

connected to target database (not started)
Oracle instance started
database mounted

RMAN> backup database plus archivelog;

RMAN> alter database open;

database opened
RMAN> exit

Recovery Manager complete.

Prevent access to the APEX instance

No users should be using the APEX installation while it is being upgraded. Access to the instance should be limited. In this particular case, since APEX is served by the APEX Listener which is hosted by a GlassFish instance, user access is prevented by shutting down the GlassFish instance.

[oracle@server ~]$ sudo service glassfish4 stop
Shutting down glassfish4: Waiting for the domain to stop ...
Command stop-domain executed successfully.
                                                           [  OK  ]

The above command is used to control the GlassFish installation only if the corresponding service init script has already been correctly created. An example on how this is done can be found here.

Apply the patch set

Now, set the current directory to the patch directory, and apply the patch.

[oracle@server ~]$ cd patch
[oracle@server patch]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.2.0 Production on Fri Apr 25 09:38:45 2014

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production

SQL> @apxpatch.sql

The patch will now be applied, and the process will take a few minutes.

Copy patch directory contents into APEX directory

After the patch process is completed, it is time to copy the contents of the patch directory into the original APEX installation directory.

[oracle@server patch]$ cp -R * /path/to/apex/

This will include the images directory as well, and if the path to the APEX directory is the same one used to originally create the images WAR file, the WAR file need not be recreated.

Restart GlassFish

The final step is to restart the GlassFish instance.

[oracle@server ~]$ sudo service glassfish4 start
Starting glassfish4: Waiting for domain1 to start .....................................................
Successfully started the domain : domain1
domain  Location: /home/gfish/glassfish4/glassfish/domains/domain1
Log File: /home/gfish/glassfish4/glassfish/domains/domain1/logs/server.log
Admin Port: 4848
Command start-domain executed successfully.
                                                           [  OK  ]

Now the APEX installation is successfully updated to version 4.2.5, and is ready to be accessed.

Upgrade Oracle APEX from 4.2.3 to 4.2.4

7 January 2014Database, System Adminapex, centos, glassfish, oracle, oracle xe, upgradeHazrul Nizam Sidek

Introduction

Oracle recently released version 4.2.4 of its Application Express (APEX) software. This post is to document how to upgrade Oracle APEX from 4.2.3 to 4.2.4.

The Oracle APEX installation to be upgraded has the following specification:

The operating system of the server is CentOS 6.5.
The Oracle Database version is Oracle XE 11gR2.
Oracle APEX is served using the APEX Listener, via GlassFish 4.

Download and read the Patch Set Notes

First things first, read up on the patch set notes here.

Download and unzip the Patch Set

Next, the patch set is downloaded via Oracle Support.

The downloaded patch set is then extracted.

[oracle@server ~]$ unzip p17607802_424_Generic.zip

Backup database

The database is backed up as a precaution if the patch is unsuccessful.

I prefer a cold and consistent backup to ease the recovery process.

[oracle@server ~]$ rman target / nocatalog

Recovery Manager: Release 11.2.0.2.0 - Production on Tue Jan 7 12:33:41 2014

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.

connected to target database: XE (DBID=2709648321)
using target database control file instead of recovery catalog

RMAN> shutdown immediate

database closed
database dismounted
Oracle instance shut down

RMAN> startup mount

connected to target database (not started)
Oracle instance started
database mounted

RMAN> backup database plus archivelog;

RMAN> alter database open;

database opened
RMAN> exit

Recovery Manager complete.

Prevent access to the APEX instance

[oracle@server ~]$ sudo service glassfish4 stop
Shutting down glassfish4: Waiting for the domain to stop ...
Command stop-domain executed successfully.
                                                           [  OK  ]

The above command is used to control the GlassFish installation only if the corresponding service init script has already been correctly created. An example on how this is done can be found here.

Apply the patch set

Now, set the current directory to the patch directory, and apply the patch.

[oracle@server ~]$ cd patch
[oracle@server patch]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.2.0 Production on Tue Jan 7 16:07:39 2014

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production

SQL> @apxpatch.sql

The patch will now be applied, and the process will take a few minutes.

Copy patch directory contents into APEX directory

After the patch process is completed, it is time to copy the contents of the patch directory into the original APEX installation directory.

[oracle@server patch]$ cp -R * /path/to/apex/

This will include the images directory as well, and if the path to the APEX directory is the same one used to originally create the images WAR file, the WAR file need not be recreated.

Restart GlassFish

The final step is to restart the GlassFish instance.

[oracle@server ~]$ sudo service glassfish4 start
Starting glassfish4: Waiting for domain1 to start .....................................................
Successfully started the domain : domain1
domain  Location: /home/gfish/glassfish4/glassfish/domains/domain1
Log File: /home/gfish/glassfish4/glassfish/domains/domain1/logs/server.log
Admin Port: 4848
Command start-domain executed successfully.
                                                           [  OK  ]

Now the APEX installation is successfully updated to version 4.2.4, and is ready to be accessed.

Set up FTP server on CentOS 6

19 December 2013System Admincentos, ftp, install, setup, vsftpdHazrul Nizam Sidek

Introduction

The purpose of this post is to document the steps to set up FTP server on CentOS 6. The FTP server software used is the default FTP server daemon for CentOS 6 at the time of writing, which is vsftpd. The FTP server is run behind an iptables firewall and SELinux, so this post will include iptables and SELinux settings that are needed.

Install FTP server software

To start things off, the FTP server software is installed.

[hazrul@server ~]$ sudo yum grouplist | grep -i ftp
   FTP server
[hazrul@server ~]$ sudo yum groupinstall "FTP server"

Check FTP server configuration file

The vsftpd config files are located in /etc/vsftpd. The main configuration file vsftpd.conf is edited to check the configuration, and modified where necessary.

[hazrul@server ~]$ sudo vi /etc/vsftpd/vsftpd.conf

# Example config file /etc/vsftpd/vsftpd.conf
#
# The default compiled in settings are fairly paranoid. This sample file
# loosens things up a bit, to make the ftp daemon more usable.
# Please see vsftpd.conf.5 for all compiled in defaults.
#
# READ THIS: This example file is NOT an exhaustive list of vsftpd options.
# Please read the vsftpd.conf.5 manual page to get a full idea of vsftpd's
# capabilities.
#
# Allow anonymous FTP? (Beware - allowed by default if you comment this out).
#anonymous_enable=YES
anonymous_enable=NO
#
# Uncomment this to allow local users to log in.
local_enable=YES
#
# Uncomment this to enable any form of FTP write command.
write_enable=YES
#
# Default umask for local users is 077. You may wish to change this to 022,
# if your users expect that (022 is used by most other ftpd's)
local_umask=022
#
# Uncomment this to allow the anonymous FTP user to upload files. This only
# has an effect if the above global write enable is activated. Also, you will
# obviously need to create a directory writable by the FTP user.
#anon_upload_enable=YES
#
# Uncomment this if you want the anonymous FTP user to be able to create
# new directories.
#anon_mkdir_write_enable=YES
#
# Activate directory messages - messages given to remote users when they
# go into a certain directory.
dirmessage_enable=YES
#
# The target log file can be vsftpd_log_file or xferlog_file.
# This depends on setting xferlog_std_format parameter
xferlog_enable=YES
#
# Make sure PORT transfer connections originate from port 20 (ftp-data).
connect_from_port_20=YES
#
# If you want, you can arrange for uploaded anonymous files to be owned by
# a different user. Note! Using "root" for uploaded files is not
# recommended!
#chown_uploads=YES
#chown_username=whoever
#
# The name of log file when xferlog_enable=YES and xferlog_std_format=YES
# WARNING - changing this filename affects /etc/logrotate.d/vsftpd.log
#xferlog_file=/var/log/xferlog
#
# Switches between logging into vsftpd_log_file and xferlog_file files.
# NO writes to vsftpd_log_file, YES to xferlog_file
xferlog_std_format=YES
#
# You may change the default value for timing out an idle session.
#idle_session_timeout=600
#
# You may change the default value for timing out a data connection.
#data_connection_timeout=120
#
# It is recommended that you define on your system a unique user which the
# ftp server can use as a totally isolated and unprivileged user.
#nopriv_user=ftpsecure
#
# Enable this and the server will recognise asynchronous ABOR requests. Not
# recommended for security (the code is non-trivial). Not enabling it,
# however, may confuse older FTP clients.
#async_abor_enable=YES
#
# By default the server will pretend to allow ASCII mode but in fact ignore
# the request. Turn on the below options to have the server actually do ASCII
# mangling on files when in ASCII mode.
# Beware that on some FTP servers, ASCII support allows a denial of service
# attack (DoS) via the command "SIZE /big/file" in ASCII mode. vsftpd
# predicted this attack and has always been safe, reporting the size of the
# raw file.
# ASCII mangling is a horrible feature of the protocol.
#ascii_upload_enable=YES
#ascii_download_enable=YES
#
# You may fully customise the login banner string:
#ftpd_banner=Welcome to blah FTP service.
#
# You may specify a file of disallowed anonymous e-mail addresses. Apparently
# useful for combatting certain DoS attacks.
#deny_email_enable=YES
# (default follows)
#banned_email_file=/etc/vsftpd/banned_emails
#
# You may specify an explicit list of local users to chroot() to their home
# directory. If chroot_local_user is YES, then this list becomes a list of
# users to NOT chroot().
#chroot_local_user=YES
chroot_local_user=YES
#chroot_list_enable=YES
# (default follows)
#chroot_list_file=/etc/vsftpd/chroot_list
#
# You may activate the "-R" option to the builtin ls. This is disabled by
# default to avoid remote users being able to cause excessive I/O on large
# sites. However, some broken FTP clients such as "ncftp" and "mirror" assume
# the presence of the "-R" option, so there is a strong case for enabling it.
#ls_recurse_enable=YES
#
# When "listen" directive is enabled, vsftpd runs in standalone mode and
# listens on IPv4 sockets. This directive cannot be used in conjunction
# with the listen_ipv6 directive.
listen=YES
#
# This directive enables listening on IPv6 sockets. To listen on IPv4 and IPv6
# sockets, you must run two copies of vsftpd with two configuration files.
# Make sure, that one of the listen options is commented !!
#listen_ipv6=YES

pam_service_name=vsftpd
userlist_enable=YES
userlist_DENY=NO
tcp_wrappers=YES

The default configuration provided by CentOS 6 allows the following users to log in:-

Anonymous users – Read-only access to /var/ftp and its subdirectories.
Local users – Read-write access to the whole filesystem, governed only by file permission.

However, if SELinux is turned on, the default SELinux configuration will stop any writes from happening.

The default vsftpd configuration is a bit loose for my liking. I prefer not to have anonymous users logging into the server, and I prefer to have the local users confined to their home directories. Therefore, the configuration was modified as highlighted above.

To deny anonymous users, anonymous_enable is set to NO.

To confine local users to their home directories, chroot_local_user is set to YES.

An extra line is also added to the configuration file which is userlist_deny=NO. By default, vsftpd will check for the file /etc/vsftpd/user_list, and deny access to the users listed in that file. By setting userlist_deny=NO, ONLY users listed in that file is allowed access.

Therefore, the initial user list file will have to be changed.

[hazrul@server ~]$ sudo cat /etc/vsftpd/user_list
# vsftpd userlist
# If userlist_deny=NO, only allow users in this file
# If userlist_deny=YES (default), never allow users in this file, and
# do not even prompt for a password.
# Note that the default vsftpd pam config also checks /etc/vsftpd/ftpusers
# for users that are denied.
root
bin
daemon
adm
lp
sync
shutdown
halt
mail
news
uucp
operator
games
nobody

[hazrul@server ~]$ sudo mv /etc/vsftpd/user_list /etc/vsftpd/user_list.bak
[hazrul@server ~]$ sudo touch /etc/vsftpd/user_list

The user list is now empty, and must be later filled with the usernames of the users that are to be allowed FTP access.

Configure SELinux to work with vsftpd

SELinux controls vsftpd on a tight leash. To make the FTP server work as intended, at least one SELinux boolean needs to be modified. First, all the booleans related to FTP is listed.

[hazrul@server ~]$ sudo getsebool -a | grep -i ftp
allow_ftpd_anon_write --> off
allow_ftpd_full_access --> off
allow_ftpd_use_cifs --> off
allow_ftpd_use_nfs --> off
ftp_home_dir --> off
ftpd_connect_db --> off
ftpd_use_fusefs --> off
ftpd_use_passive_mode --> off
httpd_enable_ftp_server --> off
tftp_anon_write --> off
tftp_use_cifs --> off
tftp_use_nfs --> off

In this particular setup, the boolean that needs to be changed is ftp_home_dir. This will allow users to read and write to their home directories located in /home/username.

[hazrul@server ~]$ sudo setsebool -P ftp_home_dir on
[hazrul@server ~]$ sudo getsebool ftp_home_dir
ftp_home_dir --> on

Start and configure the vsftpd service

Next, the FTP server is started, and configured to start at boot.

[hazrul@server ~]$ sudo service vsftpd start
Starting vsftpd for vsftpd:                                [  OK  ]
[hazrul@server ~]$ sudo chkconfig vsftpd on

Configure iptables firewall to allow FTP

FTP is an interesting protocol; it requires two TCP connections: one for command, and another for data.

In active mode, the client initiates the command TCP connection on port 21 of the FTP server while providing a port number to the server for the data connection. The FTP server then connects to the provided port from port 20 of the server.

In passive mode, similarly the client initiates the command TCP connection on port 21 of the server. However, instead of providing a port number, the client then sends a PASV command indicating the intention to use passive mode. The server receives the PASV command and opens up an unregistered port for the client to connect to for the data connection. The server then sends the port number to the client for the client to initiate the connection.

A more detailed explanation of this can be found here.

To configure the iptables firewall for FTP is quite straight-forward for active mode. Just open up incoming connections on port 21, and allow outgoing connections on port 20.

Passive mode is a bit tricky. For the command connection, similarly port 21 is opened for incoming connections. However, for the data connection, the port number that is needed to be opened changes from one session to another. To open up all the unregistered port is not an acceptable solution security-wise as that is virtually one step below disabling the firewall altogether.

Fortunately, there exists an iptables module called ip_conntrack_ftp that uses the connection tracking feature of iptables to automatically open up ports for passive mode. All that is needed to be done is to load this module when iptables is started.

To recap, the following configuration needs to be performed on iptables to allow vsftpd to work properly:

Open up incoming port 21.
Open up outgoing port 20 (unnecessary if using the default iptables rules provided by CentOS which already opened all outgoing ports).
Load the ip_conntrack_ftp module.

To open port 21, the following command is used:

[hazrul@server ~]$ sudo iptables -I INPUT 5 -m state --state NEW -p tcp --dport 21 -j ACCEPT
[hazrul@server ~]$ sudo service iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables:[  OK  ]

To load the ip_conntrack_ftp module, the iptables configuration file is modified to include the module, and the iptables service is restarted.

[hazrul@server ~]$ sudo vi /etc/sysconfig/iptables-config

# Load additional iptables modules (nat helpers)
#   Default: -none-
# Space separated list of nat helpers (e.g. 'ip_nat_ftp ip_nat_irc'), which
# are loaded after the firewall rules are applied. Options for the helpers are
# stored in /etc/modprobe.conf.
IPTABLES_MODULES="ip_conntrack_ftp"

[hazrul@server ~]$ sudo service iptables restart
iptables: Setting chains to policy ACCEPT: filter          [  OK  ]
iptables: Flushing firewall rules:                         [  OK  ]
iptables: Unloading modules:                               [  OK  ]
iptables: Applying firewall rules:                         [  OK  ]
iptables: Loading additional modules: ip_conntrack_ftp     [  OK  ]

Create a user and test FTP access

The FTP server is now properly set up and can be tested.

Begin by creating a new user.

[hazrul@server ~]$ sudo useradd ftpuser
[hazrul@server ~]$ sudo passwd ftpuser
Changing password for user ftpuser.
New password:<password>
Retype new password:<password>
passwd: all authentication tokens updated successfully.

Add the user to the vsftpd user list.

[hazrul@server ~]$ sudo bash -c "echo ftpuser >> /etc/vsftpd/user_list"

Connect to the server using an FTP client. The connection should succeed, and transfer of files to and from the server should now work as expected.

That concludes this post on how to set up FTP server on CentOS 6.

Create init script in CentOS 6

3 December 2013System Admincentos, init, script, serviceHazrul Nizam Sidek

The purpose of this post is to document how to create init script in CentOS 6 to be used with the usual service management commands such as service and chkconfig.

Introduction

After being somewhat dissatisfied with the service script provided by GlassFish, I decided to learn how to create my own service script to start, stop, and view the status of non-packaged applications.

Learn Shell Scripting

I began by looking at other service scripts that has been already provided by CentOS such as /etc/init.d/httpd and /etc/init.d/iptables for ideas.

To help understand the scripts, I had to start learning shell scripting. A very good tutorial can be found here.

Writing the script

The skeleton script

There is a file inside CentOS that explains a bit about how to write init scripts. The path to the file is /usr/share/doc/initscripts-*/sysvinitfiles. You can open this file in a text editor, and have a read.

Included in this file is a base outline of an init script that we can base our script on. The following is the content of the outline:

#!/bin/bash
#
#       /etc/rc.d/init.d/<servicename>
#
#       <description of the *service*>
#       <any general comments about this init script>
#
# <tags -- see below for tag definitions.  *Every line* from the top
#  of the file to the end of the tags section must begin with a #
#  character.  After the tags section, there should be a blank line.
#  This keeps normal comments in the rest of the file from being
#  mistaken for tags, should they happen to fit the pattern.>

# Source function library.
. /etc/init.d/functions

<define any local shell functions used by the code that follows>

start() {
        echo -n "Starting <servicename>: "
        <start daemons, perhaps with the daemon function>
        touch /var/lock/subsys/<servicename>
        return <return code of starting daemon>
}

stop() {
        echo -n "Shutting down <servicename>: "
        <stop daemons, perhaps with the killproc function>
        rm -f /var/lock/subsys/<servicename>
        return <return code of stopping daemon>
}

case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    status)
        <report the status of the daemons in free-form format,
        perhaps with the status function>
        ;;
    restart)
        stop
        start
        ;;
    reload)
        <cause the service configuration to be reread, either with
        kill -HUP or by restarting the daemons, in a manner similar
        to restart above>
        ;;
    condrestart)
        <Restarts the servce if it is already running. For example:>
        [ -f /var/lock/subsys/<service> ] && restart || :
    probe)
        <optional.  If it exists, then it should determine whether
        or not the service needs to be restarted or reloaded (or
        whatever) in order to activate any changes in the configuration
        scripts.  It should print out a list of commands to give to
        $0; see the description under the probe tag below.>
        ;;
    *)
        echo "Usage: <servicename> {start|stop|status|reload|restart[|probe]"
        exit 1
        ;;
esac
exit $?

Create init script by modifying the skeleton script

Using the skeleton script as a base, and comparing it with other established init scripts, I have come up with the following for GlassFish 4:

#!/bin/bash
#
# glassfish4    GlassFish Server Open Source Edition 4.0
#
# chkconfig: 345 70 30
# description: GlassFish Server is a Java EE Application Server Platform
# processname: glassfish4

# Source function library.
. /etc/init.d/functions

RETVAL=0
prog="glassfish4"
LOCKFILE=/var/lock/subsys/$prog

# Declare variables for GlassFish Server
GLASSFISH_DIR=/home/gfish/glassfish4
GLASSFISH_USER=gfish
ASADMIN=$GLASSFISH_DIR/bin/asadmin
DOMAIN=domain1

start() {
        echo -n "Starting $prog: "
        daemon --user $GLASSFISH_USER $ASADMIN start-domain $DOMAIN
        RETVAL=$?
        [ $RETVAL -eq 0 ] && touch $LOCKFILE
        echo
        return $RETVAL
}

stop() {
        echo -n "Shutting down $prog: "
        $ASADMIN stop-domain domain1 && success || failure
        RETVAL=$?
        [ $RETVAL -eq 0 ] && rm -f $LOCKFILE
        echo
        return $RETVAL
}

status() {
        echo -n "Checking $prog status: "
        $ASADMIN list-domains | grep $DOMAIN
        RETVAL=$?
        return $RETVAL
}

case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    status)
        status
        ;;
    restart)
        stop
        start
        ;;
    *)
        echo "Usage: $prog {start|stop|status|restart}"
        exit 1
        ;;
esac
exit $RETVAL

Using the script

Add the script to chkconfig

The script is placed inside the /etc/init.d/ directory, and given the name glassfish4.

Then, the script is added to the list using the chkconfig command.

[root@server ~]# chkconfig --add glassfish4

[root@server ~]# chkconfig --list glassfish4
glassfish4      0:off   1:off   2:off   3:on    4:on    5:on    6:off

Use the start operand

[root@server ~]# service glassfish4 start
Starting glassfish4: Waiting for domain1 to start .........................
Successfully started the domain : domain1
domain  Location: /home/gfish/glassfish4/glassfish/domains/domain1
Log File: /home/gfish/glassfish4/glassfish/domains/domain1/logs/server.log
Admin Port: 4848
Command start-domain executed successfully.
                                                           [  OK  ]

Use the stop operand

[root@server ~]# service glassfish4 stop
Shutting down glassfish4: Waiting for the domain to stop ......
Command stop-domain executed successfully.
                                                           [  OK  ]

Use the restart operand

[root@server ~]# service glassfish4 restart
Shutting down glassfish4: Waiting for the domain to stop ..
Command stop-domain executed successfully.
                                                           [  OK  ]
Starting glassfish4: Waiting for domain1 to start ......................
Successfully started the domain : domain1
domain  Location: /home/gfish/glassfish4/glassfish/domains/domain1
Log File: /home/gfish/glassfish4/glassfish/domains/domain1/logs/server.log
Admin Port: 4848
Command start-domain executed successfully.
                                                           [  OK  ]

Use the status operand

[root@server ~]# service glassfish4 status
Checking glassfish4 status: domain1 running

Use any other operands

[root@server ~]# service glassfish4 lalala
Usage: glassfish4 {start|stop|status|restart}