Web programming, an example

Type classification: this is an article resource.

Subject classification: this is an information technology resource.

Abstract

Although a plethora of "Hello World" examples can be found on the web, examples of itermediate complexity are more difficult to come by. When organization or vendor sites provide examples, these are almost always restricted to the products they support, even though a great way to develop web projects is by putting together components from many resources, all of them available from open source projects.

This post makes available source code and a high level description of a web programming project of intermediate complexity - a web alert service that allows registered users to schedule email to distribution lists. It includes both client and server side code. The high level description provided here serves as a guide to the code, but would probably not be an adequate substitute for actually reading the code. The reader should already have some knowledge of Servlet, Java Server Page (JSP), Java Database Connection (JDBC), and JavaScript technologies.

The Web Alert Service makes extensive use of open source components. The server side implementation consists of Servlets and Java Server Pages (JSP), and uses:

The Apache Derby Java database to store user data.

The Javamail API to send alert messages.

The Cron4j Scheduler to manage schedules.

Google Gson to manage JavaScript Object Notation (JSON) data exchange with browser clients.

The Servlets and JSP are hosted by an Apache Web Server with a Tomcat Servlet Container, running on Ubuntu.

The client side includes two distinct implementations:

A pure HTTP client with very simple forms, appropriate for browsers with limited capabilities on handheld devices.

A JavaScript based client using the jQuery JavaScript library, appropriate for full featured browsers on desktop devices.

Last but not least, the Eclipse Integrated Development Environment (IDE) is a valuable tool to facilitate and accelerate development.

Web Alert Service

Design and Implementation

The main purpose of the Web Alert Service is to provide an example, at an intermediate complexity level, of both client and server side web programming. The service may also be useful in its own right. There are many existing ways to schedule email alerts that can readily be found on the web. These applications fall into three major types:

Plug-ins for commercial email clients. These plug-ins allow the email client (for example Microsoft Outlook on your personal computer) to schedule messages to be sent by their email server. They rely on the client's existing email service (for example Microsoft Exchange supported by an IT department at the workplace).
Browser plug-ins for web mail services. At present, freely available web mail service provide no scheduling capability. These browser plug-ins use the web browser as the graphical interface for scheduling messages. The plug-in relays the message to a third party server, which takes responsibility for sending the message at the correct time.
Commercial mass mailing web sites where a user account can be created. After logging in, the site's web interface (usually a series of forms) allows the user to write and schedule messages to a list of email addresses. Trial accounts are often available for free, with a limit on the number of messages that could be sent per month.

Our Web Alert Service most resembles type 3 above, but the main purpose is educational not commercial. It follows the (passive) Model View Controller pattern. The user data model is represented by tables in an Apache Derby database. Data Access Objects (DAO) interact with the database via Structured Query Language (SQL) statements. Servlets provide user account authentication and transaction control logic, as well as the capability to run jobs at future times (via Cron4j) and to send email messages (via Javamail). For HTTP-only browsers, a series of JSP forms provide simple (if somewhat tedious) views of the data model. For JavaScript capable browsers, a more integrated view is available (via jQuery) where the user can make a series of fairly complex changes to the data before saving, a.k.a. committing the transaction.

Data Model

This section describes the data model, i.e. the Derby tables and associated DAO java classes.

Database Tables

The diagram below shows all Alert Service tables.

The Users table stores the login (UserName), password, and GMT offset for each registered user. Its primary key is an integer UserId. Business rules (see section on Controller) enforce uniqueness for UserName. The password is used for authentication, and the GMT offset is used to translate between the user's time and server time.

The Groups, Addresses, Messages, and Jobs tables store user data for scheduling and sending alert messages. They each also have integer primary keys and unique name fields. Groups are aliases for distribution lists, i.e. lists of addresses. Messages include the subject and body text to be sent. Jobs describe when to send a message, and to which group. Jobs (schedules) have a repeat attribute - when repeat is "None" the job will be run only once, at the designated time. Repeat can also be "Daily", "Weekly", or "Monthly". Jobs can be active or inactive. Only active jobs will be launched at the designated time (inactive jobs are dormant entries).

Relationship tables UserAddresses, UserMessages, UserJobs contain pairs of integer Ids - a UserId paired with the key into the Addresses, Messages, or Jobs table (a.k.a foreign keys). These relationships are used to partition user data (so that each user's data is visible only to its owner) and to cascade changes to all relevant tables when a user is modified or deleted.

Likewise, relationship tables GroupAddress, GroupJobs, and MessageJobs allow changes to be propagated to dependent tables when a group, address, or job is modified or deleted.

Data Access Objects

Our implementation uses two Java classes to support each database table. For example classes User and Users support the Users table. Class User is a simple representation of a single user, with get and set methods for its instance variables. User instance variables are named to match fields in the Users table. Class Users is the actual DAO for the Users table. It includes Java Database Connectivity (JDBC) methods to add, delete, and modify users. Also included are some support methods, to authenticate a user and to pretty-print the Users table.

Similarly, classes Group/Groups, Address/Addresses, and Job/Jobs support the Groups, Addresses, and Jobs tables. The relationship tables are supported by two classes, Links and Db. These classes do not follow the same pattern as the previously described DAO classes. Both provide methods to make sure that when a table is modified, changes are cascaded to all dependent tables. Class Db also supports basic database functionality (e.g. createConnection, getConnection, and shutdown), as well as some utilities to format data into HTML elements for GUI display.

Access to the database occurs only through the classes described above (User/Users, Group/Groups, Address/Addresses, Job/Jobs, Links, and Db). Please see the Abstract for a link to the source code.

View, HTTP-Only Browsers

As mentioned in the Abstract, there are two distinct implementations of the client, offering different views of the data. This section describes the HTTP-only thin client, which provides a table-by-table view in a series of GUI screens. Each GUI screen is kept simple so as to be easily navigable on hand-held devices. The implementation consists only of Java Server Pages (JSP).

The thin client consists of five static JSPs index, login, tables, actions, newuser, and two JSP pages dynamically generated by methods generateForm and generateActivationForm in the AlertServlet class. This is one of the main classes in our Alert Service implementation, and will be described in more detail in the Controllers section below.

Static pages

index.jsp - the top level page. It links to pages for logging in or for requesting a new login, and to the Quick Start guide.
login.jsp - the login page.
tables.jsp - allows the user to select the type of data to manage (groups, addresses, messages, or schedules).
actions.jsp - allows the user to select the action to take (add, delete, modify, or view all) on the previously chosen data table.

Dynamic pages

AlertServlet.generateForm() - collect all data relevant to the chosen user action and display them in an HTML form. For example if the user previously chose Groups/View All, this method will generate an HTML table containing all of the user's groups and comments and send it back to the user's browser in a HTTP GET response message. If the user chose Address/Add, this method generates and sends an HTML form to the browser. The form will include fields for the address name, comments, as well as a drop down list (HTML select element) of all the user's groups.
AlertServelet.generateActivationForm() - if the user chose Schedule/Activate, generate a form containing a set of HTML radio buttions including all of the user's schedules (jobs) that could be activated, and send it back to the user's browser in a HTTP GET response. The user can then choose which schedule to activate.

View, Javascript Capable Browsers

The richer jQuery client interacts primarily with the JSON Exchange servlet, class SideDoor. It consists of one static JSP and one library of JavaScript functions built on jQuery and jQuery UI libraries. Below is a screen capture of the client GUI running on the Mozilla Firefox browser (the user name is blotted out).

Instead of multiple pages showing one data table each, the rich client shows all tables as tabs (Groups, Addresses, Messages, and Schedules). Each tab includes a table displaying already present data, and a set of buttons to Add, Delete, or Modify data. These behave like radio buttons. Clicking Add, Delete, or Modify brings up a form appropriate for the current tab and action. The user can make multiple changes and revisions, which are not committed until the Save Changes button is clicked. The Discard Changes button allows the user to cancel all changes since the previous save.

Additionally, the Account tab allows the user to modify the password and GMT offset, and to delete the account (i.e. unsubscribe from the service). Arguably, this is a more user friendly and better integrated view of the data. It comes at the cost of more complexity on the client side, requiring the use of JavaScript.

JSP

Static page desktop.jsp implements part of the rich client. The Id of the main division is "root". It contains the "tabs" division, which in turn contains divisions "groupstab", "addressestab", "messagestab", "schdulestab", and "accounttab". These correspond to the five tabs on the GUI. Division and other HTML element Ids act ask keys (a.k.a tags) for jQuery, triggering JavaScript function calls.

JavaScript

Script gui.js implements the remaining part of the rich client. All code in this script is enclosed in a $(document).readdy() statement, to ensure that user data from the server has been completely transferred and ready for display before making the page visible in the browser.

The server URL is stored in global variable url. After sucessful authentication the client "root" division becomes ready. The $("#root").ready() section invokes function reloadUserData() which makes an HTTP GET request to the server. The server sends all data belonging to the current login (as a JSON string) in an HTTP GET respsonse to the client browser. The JSON object is stored in global object userData.

As previously mentioned, clicking Add, Delete, or Modify brings up a form where the user types in or selects values for a group, address, message, or schedule. These input values are stored in global variables, needed by many functions at various stages. Global variables group, address, message, schedule, and account store temporary working values for the corresponding tabs.

Script gui.js also includes a set of plain JavaScript, globally available utility functions to support manipulating user data and the corresponding HTML elements to display the data. The remainder of code in this script consists of jQuerey sections that are triggered to execute when user selects the corresponding element on the GUI. In particular, clicking on Save Changes trigger section $("#commit-radio").button().click(), which invokes commitUserData(). This gathers all of the current user data, packages it into a JSON string, and sends it in an HTTP POST request to the server.

Controllers

This section describes the two servlets, as well as the supporting tools, which make up the server side of this implementation. The two servlets share supporting back-end classes and libraries, but differ in the client they support. The AlertServlet servlet supports the thin HTTP-only client, and the SideDoor servlet supports the JavaScript client. Both functionalities could have been combined into a single servlet class, but we preferred to separate them for clarity. The thin client is not capable of transaction management so that logic must be pushed into to the servlet, making it more complex. Conversely, since the rich client can manage the transaction, it manipulates all user data for the current transaction and the servlet merely acts as an interface to the database. The server side logic therefore is simplified.

JSON Exchange Servlet for the JavaScript Client

We will describe the simpler servlet first. Class SideDoor implements the servlet for exchanging JSON strings with the rich client.

After the user logs in on the client browser, the client sends an HTTP GET request to the server. SideDoor.doGet() intercepts this request and invokes getUserJsonString(). This method formats the current user's data from the Derby database into a JSON string and puts it into the GET response and sends it to the client. Inner classes Group, Message, Address, Schedule, Account, and UserJson are skeleton classes to support JSON, with instance attributes that map exactly to JSON variable names.

When the user clicks "Save Changes" on the client, it sends an HTTP POST request to the server. SideDoor.doPost() intercepts this request and invokes updateUserData(). Method updateUSerData() unpacks the JSON string from the client and updates Derby database table.

The above approach allows good transaction control. Data on the client side is treated as a working copy. User the various GUI tabs and dialogs, the user can continue to manipulate the data until satisfied. The transaction is committed only when the user clicks Save Chances. Prior to clicking Save Changes, the user can rollback the transaction by clicking Cancel Changes (this is not a true transaction rollback since it can only occur before the commit).

AlertServlet for the Thin Client

Compared to the SideDoor servlet, the AlertServlet supporting the thin HTTP-only client is more complex, since it must incorporate transaction control logic. It does so with information in HttpSession, obtained from the client HTTP GET and POST requests. Additionally it uses class State to store transaction information. To illustrate the flow of information, consider the scenario of a user (already logged in) adding a new group.

When the client first connects to the server, the HttpServlet infrastructure assigns a valid HttpSession Id to the connection. This session Id persists until the user has been inactive (has not typed anything into the client) for thirty minutes.

After successfully logging in, State instance attributes userName and password shoud be correctly populated, and the client is presented with the form (tables.jsp) to select a data table. Expected values for the remaining State instance variables at this point are:

table	oper	status	stage
NONE	SELECT	START	SEL_TABLE

The user now selects the Groups radio button and clicks the Select button, triggering an HTTP POST request to AlertServlet (it may be useful to refer to screen shots in the Quick Start guide for the rest of this description). In AlertServlet.doPost(), execution drops into the if (formName.equals(TABLES)) {} block where State.table is set to GROUP and the client is redirected to the Actions form (actions.jsp).

The rest of the transaction is similarly managed: when the user submits a form, doPost() processes the request; when the user selects a link, doGet() processes the request. State attributes table, oper, status, and stage determine which block in doGet() and doPost() does the processing and therefore controls the transaction.

Please review the code for further details.

Supporting Tools

As previously mentioned, classes Db and Links support Derby database access. Classes Log, Test, Tools, and Util provide other support functions. Of these classes, Util and Tools are by far the most important.

Log.java - provides logging and traffic monitoring counters.
Test.java - provides a simple unit test harness.
Tools.java - provides HTML element manipulation and interfaces with the scheduler.
Util.java - jobs (a.k.a. schedules) are stored in GMT in the Derby database, and presented to the user in the user's own time zone. The server time zone must be taken into account when launching these jobs. Class Util provides the date, calendar, and time zone support for these translations.

Parting Thoughts

Hopefully this has been a useful web programming example with source code and a high level description, beyond the more available "Hello World" samples. On the server side our example includes JDBC, servlets, and JSP. On the client side we touched on JavaScript and the jQuery libraries. In our implementation we also made use of open source components to schedule jobs, send Email, and manipulate JSON objects. We rely heavily on Eclipse in our development and testing environments, and on the Apache web server and Tomcat servlet container in our runtime environment. These will be described in future posts.

The table below lists the specific version of components used to develop and run the web alert service:

Component	Version	Description
Eclipse	Helios V2	Integrated Development Environment
Apache	2.2	Web Server
Tomcat	6.0	Servlet Container
Derby	10.8.2.2	Relational Database
Cron4j	2.2.5	Scheduler
JavaMail	1.4.5	Java Email Library
Gson	2.2.1	Java JSON Library
jQuery	1.7.2	Client Side JavaScript Library
jQuery UI	1.8.21	Client GUI JavaScript Library