Production Deployment

Overview

Starexec can be directly pulled down from subversion, be built and deployed all in one action. We use a bash script to accomplish most of the file operations such as creating a temporary directory and pulling the repository down, copying to the production folder, etc. The actual compilation and auto-configuration is accomplished by ANT. ANT fiddles with starexec's files (to ensure it the project is ready for production), compiles the code and packages it into a WAR (WebARchive) file. WAR files are like JAR files except they contain entire web applications all bundled up into one file which Tomcat can understand and explode (unzip). The WAR file needs to be placed in /project/tomcat-webapps/webapps/ to be recognized as a web-app. Tomcat scans that directory for war files and explodes them if it encounters one.

First deployment

The star_report user needs to be created in MySQL and given rights to write results back to the DB from the epilog (src/org/starexec/config/sge/epilog).

CREATE USER 'star_report'@'%.star.cs.uiowa.edu' identified by 'password';  
GRANT EXECUTE on starexec.* to 'star_report'@'%.star.cs.uiowa.edu';

Deployment Process

Lead developers deploy to stardev while CSG deploy to starexec.  Here are the actions that take place to get the application from the repository to a live application running on the server.  This process is done by a stardeploy script (see below):

1. The latest latest revision is checked out from Git into /home/starexec/deployed/
2. The /home/starexec/production.properties file is copied to deployed/local.properties, for the benefit of the next step.
3. ANT is called from the project directory ant -buildfile build.xml  
   1. A build.properties file is generated containing information about the build
   2. log4j is configured to not output debug messages to the log
   3. prolog and epilog scripts are copied to /home/starexec/sgescripts
   4. Compile the source code of the application
   5. org.starexec.config package is copied into the compiled version
   6. The compiled version is packaged into a war file
4. The existing WAR file is removed, which prompts Tomcat to delete the old version of starexec
5. The new WAR file is moved into the /project/tomcat-webapps/webapps directory
6. Tomcat is restarted so it recognizes the new WAR file
7. Tomcat recognizes the WAR file and explodes it, producing a /starexec directory which is the live web application
8. If necessary, you update the SQL procedures and functions.  This can be done by running the script updateSQLProcedures in /home/starexec/scripts.  

Auto-Deploy

We have a handy script that does everything mentioned above. It is called stardeploy which is located in /home/starexec/scripts/scripts-common/stardeploy.sh. Simply call stardeploy from that directory and watch starexec get pulled down, configured, compiled, and deployed to production. You can specify which branch for stardeploy to use by specifying the desired branch as a flag.  Currently, it is recommended that you manually remove existing WAR file first.  The WAR file is in /project/tomcat-webapps/webapps/.  For some time, all development has occured on fb1.

stardeploy -trunk
stardeploy -fb1
stardeploy -fb2

Production Environment

Tomcat

Tomcat was described previously as the web application "container" (manager) we're using that basically runs an application server. Tomcat wasn't originally installed on the StarExec head node, so the Computer Support Group installed it. They maintain it by patching, updating, etc. so we never have to touch it as developers.

NFS Shares

The head node shares certain directories with all of its worker nodes. These are /home /cluster and /project. This share is used by worker nodes to access common files that the head node has. This was set up by CSG. It is important to note that /home/starexec is shared between all nodes. This directory is where worker nodes need to pull jobs from. Each worker node has local workspace that jobs need to be copied to /export/starexec before execution to prevent delays in reading jobs from the /home/starexec shared space.

Oracle Grid Engine

The grid engine was installed, maintained and configured by CSG (on all worker nodes and the head node). There are certain environmental variables that need to be set up to hook into grid engine from the Java bindings (drmaa.jar). Tomcat passes environmental variables to all of its webapps in a certain configuration file. By default, it does not have the proper variables set. These variables are outlined in the official Grid Engine documentation.