Java
StarExec is written almost exclusively in Java aside from the actual distribution and running of jobs which is mostly done with scripting on the Linux operating system. We have a wide variety of Java classes that represent certain objects in the system such as solvers and benchmarks. We also have Java classes that are responsible for such things as submitting jobs and managing the database. Advanced knowledge of Java is required for this application, specifically thread-safe programming.
Java Server Pages
StarExec utilizes the Java Server Pages platform (JSP) as its user interface (front-end) technology. This allows us to easily present things to the user that exist as Java objects in the core application, as well as process incoming information and take appropriate action with certain Java classes. Another advantage is the ability to utilize an extensive and powerful collection of tools and libraries that Java provides.
Apache Tomcat
Tomcat is the web application container we use which "hosts" the StarExec web application. You can think of Tomcat as a manager that is responsible for redirecting requests to certain webapps (in our case we only have one, but you could have multiple) and provides important functionality like restricting access to content, etc. As a developer you won't have to worry much about Tomcat since it is maintained by CSG. Currently the only time we delve into Tomcat is to change security configurations.
Expression Language
Traditionally in older versions of JSP you had to embed Java into your page which is messy and ugly. Essentially you're combining business logic with layout which is always a bad idea since it makes a simple layout harder to maintain. To get around this problem, Expression Language (EL) was introduced to make it easier to insert dynamic content into a page.
For example, this is how a user's name would be inserted into a JSP the old way:
<p>Hello, <%= (User)(session.getAttribute("user")).getName()%></p>
and this is the simplified way with EL:
<p>Hello, ${user.name}</p>
Expression Language is very simple and intended for web designers to use so as to avoid learning programming. However it is still almost an entire other language on its own so it may require some reading up on to fully understand.
Java Standard Tag Library
Another problem with the old version of JSP is control flow. It was messy to implement loops and conditionals with Java and mix it with HTML. For this reason, the Java Standard Tag Library (JSTL) was created and designed to work with EL. For example say we wanted to display an html element based on some condition. First at the top of the page we must declared that we intend to use JSTL.
<%@taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
<c:if test="${user.role == 'admin'">
<p>This is only displayed if the condition is true!</p>
</c:if>
The taglib directive allows us to specify which tag libraries to use. The most standard one is the JSTL core library which is usually prefixed with c. The if tag is one of many special tags that JSTL provides for us. Another useful JSTL library is the function library.
Here's an example that utilizes a simple function:
<%@taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
<%@ taglib prefix="fn" uri="http://java.sun.com/jsp/jstl/functions" %>
<c:out value="Username in CAPS: ${fn:toUpperCase(user.name)"/>
Custom Tags
On top of the JSTL core and function libraries, you can also create your custom tags that you can specify the behaviour of. We utilize this to provide site templating. For instance we have a template.tag file that specifies the layout of all the common elements of a page (headers, footers, menus, logos, etc.) Then we use that tag in other pages where we can just specify the main content.
For example, here's a custom tag that represents the site's template (we'll call this template.tag):
<html>
<body>
<div id="content" class="round">
<jsp:doBody/>
</div>
</body>
</html>
Now here is how we would use that template in some welcome page:
<%@taglib prefix="star" tagdir="/WEB-INF/tags" %> <star:template> <h1>Welcome!</h1> <p>This content is inserted in place of jsp:doBody</p> </star:template>
Oracle Grid Engine
The Oracle Grid Engine is a system used to distribute jobs to worker nodes and manage the queue/workload of the jobs in the system. For us, this means the Grid Engine will distribute solvers and benchmarks to nodes to be solved locally. This takes the responsibility of managing a scheduling and queuing system off our hands.
We will be interfacing with the grid engine through a package Oracle provides called DRMAA (pronounced "Drama") or more formally the Distributed Resource Management Application API which allows us to hook into the Grid Engine from Java. The official Oracle overview should be included along with this documentation. The full javadoc for this API is located on StarExec under /cluster/sge-6.2u5/doc/javadoc.
TODO: This may change? We may be using a daemon or chron job to do this.
HTML5
Starexec utilizes HTML 5 and CSS3. What this means is we use a simplified DOM and advanced selectors for styling pages. We also utilize open source scripts and style sheets to make StarExec compatible with non-html5 browsers. More information can be obtained here.
jQuery
As with any web project, all of the client-side scripting will be done with Javascript. To cope with the rather ugly syntax and quirks of javascript between different browsers, we will be using the jQuery javascript library. jQuery a lightweight, CSS3 compliant and cross-browser javascript library that allows many common tasks to be accomplished with ease. For example it is extremely easy in jQuery to animate DOM objects and send AJAX requests. To put it simply, jQuery gives us a nice, concise and easy to use interface to javascript.
AJAX
AJAX (asynchronous javascript and XML) is a set of technologies that allows us to process requests for data behind the scenes and update the results locally on the browser. This means we don't have to make a page request to the server and wait for the server to process the request and send a response. The main advantage of AJAX is the fact that less information is sent back and forth between the server and client because full HTML pages don't have to be sent back and forth. It is also better from a user interface standpoint because we can indicate through javascript that something is loading, or that some work is being done. When the work is done we can update the results on the page without the page flickering that an HTML get or post causes. Although it won't be possible to handle everything with AJAX, it will be a goal of StarExec to make as much functionality as possible be handled via AJAX.
Here is an example of how to make an AJAX post request with jQuery. The method below can also be called with $.get
$.post(
"/starexec/services/users/1",
function(returnData){
alert(responseText);
},
"json"
);
If you are using a $.get and the return type is JSON, you can shorthand this as follows:
$.getJSON(
"/starexec/services/users?id=1",
function(returnData){
alert(returnData);
}
);
Unfortunately, a similar method does not exist if you are using a $.post.
Google Gson
To send data between the client and server with AJAX, both sides must know how the data is structured. Traditionally this is done with XML, but the disadvantage is it takes up a lot of space due to the structure of XML itself. For instance every tag has a start and end tag which basically doubles the size of the XML to send. A newer standard has emerged called JSON (JavaScript Object Notation) which allows you to express "Objects" in an object oriented sense which have fields of data which may or may not contain more objects and so on. JSON allows a more compact representation of information that is sent as a string between the client and server. JSON is strings are comprised of nothing but name value pairs that may or may not be nested. Here is an example of a JSON string representing a user:
{
"firstName": "Tyler",
"lastName": "Jensen",
"age": 22,
"address":
{
"streetAddress": "14 Maclean St",
"city": "Iowa City",
"state": "IA",
"postalCode": "52240"
},
"phoneNumber":
[
{
"type": "home",
"number": "123 456-7890"
},
{
"type": "fax",
"number": "098 765-4321"
}
]
}
Google-gson is a Java library that allows us to serialize any Java object into a JSON string that can be sent back to the client to process. This way, we won't have to manually build a JSON string to send back ourselves. The gson library is released under the Apache License 2.0. For example if we use an AJAX call to get all the names in our database, and the Java servlet that is handling the request retrieves all the people in the database as "Person" objects then we will need to be able to convert this into a single string to send back to the client. gson will create this string for us which sends back the JSON representation of the list of people, and jQuery on the client side allows us to easily retrieve the information in the JSON string.
Sometimes you don't want to serialize an entire object to be sent to the client. For example we certainly don't want to be sending a password or confidential information to the client. To control which information is serialized into Json we have one of two options. Gson will ignore null fields by default, so we can null out a field to prevent it from being sent (obviously if a field doesn't have a value it can't be serialized!). Another option is to use gson's built in @Expose annotation. We place this on a field to explicitly allow it to be serialized. All other fields are ignored. When in our serializer we have to explicitly say to ignore all fields that aren't exposed.
Example:
import com.google.gson.annotations.Expose;
public class Foo {
@Expose
private String baz = "star";
priate String bat = "exec";
}
Gson gson = new GsonBuilder().excludeFieldsWithoutExposeAnnotation().create(); system.out.println(gson.toJson(new Foo())); // Will only print "baz":"star"
RESTEasy
RESTEasy is a Java framework for handling restful web services. Restful web services (representational state transfer) are services that are invoked via http requests with existing HTTP methods such as POST, GET, PUT or DELETE. A service is requested via a standard service URL, and the data is returned in a known format as an HTTP response (in our case, JSON). All services are exposed in the StarExec service URL at starexec.cs.uiowa.edu/starexec/services/
Requests come into to the com.starexec.app.RESTServices servlet and are processed in methods decorated with special annotations that RESTEasy recognizes. Here is an example of a RESTEasy web service method:
@GET
@Path("/users")
@Produces("application/json")
public String getUserInfo(@QueryParam("id") int id) {
return new Gson().toJson(database.getUser(id));
}
@GET indicates the method accepts only HTTP GET requests. @Path denotes the URL (relative to the service url seen above) to invoke the service, and @Produces represents the type of the data being returned to the service caller which as we've stated will always be JSON for our purpose. The @QueryParam annotation represents a querystring parameter. This example method takes an id of a user to get information for through a query param. So to get the user with id 5, we would call the service like so:
starexec.cs.uiowa.edu/starexec/services/users?id=5
RESTEasy also supports an alternative way of passing information besides query params. They can also be passed directly as part of the URL. This is the preferred method however query params may need to be used in special cases such as when using third party jQuery plugins on the client. Here is an example of a RESTEasy method that supports sending an ID as part of the path. Notice we now use @PathParam instead of @QueryParam and we use curly brackets in the path to denote the corresponding parameter.
@POST
@Path("/jobs/pairs/{id")
@Produces("application/json")
public String getJobPairs(@PathParam("id") int id) {
return new Gson().toJson(database.getJobPairs(id));
}
Now if we wanted all job pairs of job number 4, we would call the service like so:
starexec.cs.uiowa.edu/starexec/services/jobs/pairs/4
MySQL
All backend data is stored in a MySQL database (the version changes as new updates are pushed out from Oracle). We make extensive use of stored procedures for efficiency and performance. We use JDBC to make calls to the database from Java.