Soaplab2: How to build it

Soaplab is installed and built by Ant.

Ant is a very powerful tool that can do almost everything. It can be also widely configured, by setting various properties, in order to customise user's actions. Which means that its usage can sometimes become a bit confusing. Therefore, this documents starts by explaining few basic rules how to use Ant, and only the next chapter describes how to use it for building Soaplab2.

General rules for using Ant
Installing Soaplab2
Soaplab2 building tasks Notes for Eclipse users

General rules for using Ant

The Ant has a help for the command-line details (ant -help). The list of available tasks can be obtained by:

ant -projecthelp

It displays something like this:

Main targets:

 all             Clean, compile, etc..
 axis1deploy     Deploy (copy) an Axis-1 war file to a Tomcat server
 axis1dist       Populate Axis-1 services distribution space
 axis1war        Create a Axis-1 war file for services distribution
 bootstrap       Fetch Maven library - needed only the first time.
 clean           Remove all generated files.
 cleanxml        Remove all generated service metadata
 compile         Compile all source code (default)
 config          Create all run-time scripts and configuration files.
 docs            Generate documentation.
 gen             Generate service metadata from all ACD files
 genebi          Generate XML metadata for EBI's sample services.
 genemboss       Generate XML metadata for EMBOSS (needs EMBOSS installed).
 gengowlab       Generate XML metadata for Gowlab services.
 gensowa         Generate XML metadata for Sowa services.
 info-list       Show a list of available services
 initeclipse     For Eclipse users... create pom files in the right place
 install         Install and build Soaplab2
 install-clients Install and build Soaplab2 clients
 jars            Create all jar files related to this project.
 jaxdeploy       Deploy (copy) a JAX-WS war file to a Tomcat server
 jaxdist         Populate JAX-WS services distribution space
 jaxwar          Create a JAX-WS war file for services distribution
 test-junit      Run the junit test cases

Soaplab2 uses Ant version 1.6.5 and later. Check your version. For example:

ant -version
Apache Ant version 1.7.0 compiled on December 13 2006

The Ant can be (and, in some point, should be) customised by defining some properties in the file build.properties (located in the Soaplab2 directory). Such file does not come automatically with the Soaplab distribution (because its contents is specific for your installation and environment). However, there is a template of such file build.properties.template. Make a copy of this template, and un-comment and edit properties that you wish/need to change.

If you wish to use the same properties for more projects, you can also put the build.properties file into your home directory. Such property file is then shared and re-used by every project. The question is, however, whether it is a good practise. You can easily forget about it and have a hard time to find why something is not working as expected.

What is the format of the build.properties file? Simply speaking, it follows the same syntax as required by java.util.Property.load() method. The gory details can be found here. For example, here are few, randomly picked-up, properties:

compile.deprecation = true
compile.warnings = true
tomcat.home = /home/senger/Software/jakarta-catalina
emboss.home = /home/senger/Software/emboss

The usual problem with properties could be to mix together properties used by Ant and properties used by Soaplab2. The properties used by Ant are usually referred to as build-time properties, while the Soaplab2 properties, used in the run-time by Soaplab2 services, are called run-time properties.

About Soaplab2 run-time properties, you will learn all details later, in the configuration guide. They use a similar syntax (similar, not the same, because the run-time properties allow more features than the build-time properties) but they are used in different time - when Soaplab2 services are running and accessed by users.

What adds to the confusion is also a fact that Ant helps to build run-time property files for Soaplab2 by re-using (propagating) some build-time properties to the run-time properties. Sorry about it...

The Ant properties can be also specified directly on the command line by using -D option (or more of them, if you wish to specify more properties). The properties on the command line has precedence over properties defined in the property files. For example:

ant -Dcompile.warnings=true compile
ant -Dcompile.warnings=true -Dcompile.debug=false compile

The whole construct with the -D option must constitute a single command-line parameter. Therefore, this is wrong:

ant -D compile.warnings=true compile

From the same reason, if the property value contains a space, the whole construct must be quoted:

ant "-Dbig.apple=New York"

Some build-time properties

Of course, the most important is to know what properties can be used. Fortunately, they are not many of them because most of them have reasonable default values. The ones you have to specified are those defining something specific to your machine. At the moment, it is only one such property - defining the local path to your Tomcat:

tomcat.home = /home/senger/Software/jakarta-catalina

Other properties that you may use are described later in this document, when individual Ant's targets (tasks) are explained. The best way (but not always an easy) way to find what properties can be used, is to look into build.xml file (and its include files in the xmls directory).

For enabling typed interface for your Soaplab Web Services please check typed interface help page.

A note for Eclipse users

If you are a Java developer, you may be using Eclipse. Be aware, that some tasks defined in the Soaplab2's Ant cannot be done (or not easily done) directly by Eclipse itself. You still should make them by calling the Ant. But you can do it from Eclipse (see there is a difference: by Eclipse and from Eclipse). Use a double click in the Ant's view.

This is especially true for the config task that creates important files in the build/classes directory, the files needed for proper run of Soaplab. But Eclipse has a tendency (especially after refreshing by F5 key) to remove these files. The config task brings them back. (When using Ant from the command-line you usually do not call this task directly, it is called by other tasks.)

Ant and Maven

Soaplab2 is built by Ant but under the hood it uses nice features of another famous tool - Maven. Mainly because it nicely handles Soaplab2's dependencies on the third-party libraries. You do not need to know details about it - but you should know what is going on on your computer. Here it is:

Ant calls Maven whenever you build Soaplab2 to find whether there are some unresolved third-party dependencies. If there are - and definitely there are the first time you build Soaplab2 - Maven downloads them to your computer. It creates a local Maven repository - which is a directory in your home directory (the name depends on the platform: on Linux, its name is .m2). This repository can be used by more projects using Maven. Important is to know that:

Something is created outside of your Soaplab2 directory, and
The first time, it can take some time, and you need to be on-line.

There is an article describing details how the Ant-Maven marriage is handled. It is from another project, but the principles (and files) are very similar to Soaplab2 (because both Soaplab2 and the article share the same author ). The article also explains what to do if you wish to have the local Maven repository created elsewhere - which you may need if your home directory has some space limits.

Installing Soaplab2

Once you have Soaplab2 in your directory, by default named soaplab2, you can install it by calling:

ant install

The installation is nothing more than calling several Ant's task in a row.

There is also a simplified version of this installation task, the install-clients task. You can use it when you wish to install only Soaplab2 clients.

First, it downloads one special library that glues Maven with Ant. Without this library, the Ant would complain that it cannot work. Internally (and anytime repeatable), it calls ant bootstrap.

Next, it checks that you have installed Perl. This is a legacy issue and will, sooner or later, disappear. The goal is to have Soaplab2 pure Java. The only remaining non-Java part is the converter between ACD and XML files (more about ACD files in the metadata guide).

After that, it reminds you to create the build.properties file and put there properties locating the place where you have installed Tomcat. These properties are not used yet, but ultimately they will be needed when you deploy your Soaplab2 services (see more about that step in the deployment guide document).

Then the real installation starts. In a series of steps, Ant downloads many third-party libraries, builds the whole Soaplab2, generates XML files for several testing services, and as a jewel at the end, it runs one testing service, the HelloWorld service.

The HelloWorld service call may fail if your machine has Perl program installed elsewhere than in /usr/bin/perl. It will almost definitely fail on Windows. If you still wish to see this service succeeded please correct the testing scripts in he run directory.

Once installation finished, it suggests to run the deployment task:

ant jaxdeploy

This will deploy few testing services into your Tomcat and you can start using them (see the command-line client document how to use your services from the command-line):

build/run/run-cmdline-client -name classic.helloworld -w -r

Or, perhaps even easier, you can open your browser and point it to your Tomcat and use the Spinet client. Unless you run Tomcat on a different port, or unless you deployed your services into a different context, the URL to open should be this: http://localhost:8080/soaplab2/.

The installation task can be repeated anytime without any harm.

Soaplab2 building tasks

This chapter lists various Ant tasks, explaining what they do and how you can customise them. For better understanding, it helps if you read first the general rules for using Ant.

More Ant tasks (the more specialised ones) are described in separate documents. The deployment tasks:

 axis1deploy  Deploy (copy) an Axis-1 war file to a Tomcat server
 axis1dist    Populate Axis-1 services distribution space
 axis1war     Create a Axis-1 war file for services distribution
 jaxdeploy    Deploy (copy) a JAX-WS war file to a Tomcat server
 jaxdist      Populate JAX-WS services distribution space
 jaxwar       Create a JAX-WS war file for services distribution

are in the deployment guide, and the metadata tasks:

 cleanxml     Remove all generated service metadata
 gen          Generate service metadata from all ACD files
 genebi       Generate XML metadata for EBI's sample services.
 genemboss    Generate XML metadata for EMBOSS (needs EMBOSS installed).
 gengowlab    Generate XML metadata for Gowlab services.
 gensowa      Generate XML metadata for Sowa services.

are in the service metadata guide.

The remaining tasks are:

Ant task: all

This is simply an aggregation of several other tasks. It cleans the building space and fills it again with a fresh build.

Most of the new created files (those that can be re-generated anytime later) are generated in the build directory.

Ant task: bootstrap

An important task - but done only once, at the very beginning. It fetches a Maven library.

Note that other libraries can be (and are) fetched later by Maven, but Maven itself needs a library which obviously "cannot be fetched before it is fetched". That's why we have here a separate task for it. Probably we should call it better a Catch-22 task .

Ant task: clean

It removes all generated files, and the build directory itself. Whenever you encountered a strange problem, the first try is to clean and build again:

ant clean compile

Ant task: compile

It compiles all source code. It is also a default task - the one called when you simply call ant.

You can influence how the Java compiled code is produced by setting compilation properties. The default values are:

compile.debug = true
compile.verbose = false
compile.deprecation = false
compile.optimize = false
compile.warnings = false

Ant task: config

It creates all run-time scripts and configuration files. This is usually not called directly (the other tasks call it) but it is an important task.

Generally, it transforms template files (sitting mostly in the directory src/etc/config) into real configuration files and puts them into build/classes directory where they are used by Soaplab services.

The template files do not contain any local dependencies (such as a full path to your soaplab2 directory). The config task injects the "local knowledge" into them during the copying. The "injection" is done on places marked in the templates by pairs of the @ characters. You do not need to know it - unless you wish to write your own template file. Please check the template what injection points are used. Here is an example of two pairs of replacement tokens (from the soaplab.properties.template:

base.dir = @PROJECT_HOME@
metadata.dir = @METADATA_DIR@
...

If you have your own template file, you need to let Ant know about it by setting a build-time property pointing to your template file. For example, for the major Soaplab2 configuration file, you set the property my.soaplab.properties:

my.soaplab.properties = testing.soaplab.properties

Other build-time properties for setting other configuration files are:

log4j.configuration = testing.log4j.properties
my.soaplab.client.properties = testing.soaplab.client.properties

Examples and details about configuration files are in the configuration guide.

The config task also generates run-time scripts into build/run directory. The scripts can be used to start various Soaplab2 client and administrative command-line programs (as shown in the command-line client and administrative tools documents).

The run-time scripts are also a good place where you can see names and location of all third-party libraries Soaplab2 is using.

Ant task: docs

It generates documentation, mostly the javadoc API.

The Soaplab2 is distributed together with the full documentation. In includes also hyper-links to the API documentation - which must be first created by this task. Whenever you see a dead link in the Soaplab pages, a chance is that it may be revived by running this task.

Ant task: info-list

This is an interesting task: it actually runs some Soaplab2 code (the same the Soaplab2 services run), reading your configuration files, and showing the current list of Soaplab2 services on your machine. The services may be not yet available to all clients (they may not be yet deployed; even the Tomcat does not need to be yet started). For example:

  [service] alignment.wu_blastn_raw
  [service] alignment.wu_blastn_xml
  [service] alignment.wu_blastp_raw
  [service] alignment.wu_blastp_xml
  [service] alignment.wu_blastx_raw
  [service] alignment.wu_blastx_xml
  [service] alignment.wu_tblastn_raw
  [service] alignment.wu_tblastn_xml
  [service] alignment.wu_tblastx_raw
  [service] alignment.wu_tblastx_xml
  [service] alignment_consensus.cons
  [service] alignment_consensus.megamerger
  [service] alignment_consensus.merger
  [service] alignment_differences.diffseq
  [service] alignment_dot_plots.dotmatcher
  [service] alignment_dot_plots.dotpath
  [service] alignment_dot_plots.dottup
  [service] alignment_dot_plots.polydot
  [service] alignment_global.est2genome
  ...

It also creates a file services.list. Its name can be changed by the property services.names.file:

ant -Dservices.names.file=my.services info-list

The file contains nothing more than the service names, each one on a separate line.

Ant task: jars

It creates all jar files related to this project. Well, what else to say...

Ant task: test-junit

It runs the junit test cases.

Before running them, it compiles them first (the testing classes are not part of the normal compilation). Running this task may give you the following error:

================================= WARNING ================================
Junit library is not present in a correct directory. Tests not executed.
The library (junit.jar) is, however, in your local Maven repository.
Therefore:
    Put it in your ${ANT_HOME}/lib directory.
    Or, put in your ${HOME}/.ant/lib directory.
    Or start ant with -lib <your junit library>.
==========================================================================

Ant task: initeclipse and Notes for Eclipse users

Many of the building tasks can be accomplished also from Eclipse. But not all. For example, as already mentioned above, the Ant's config task must be run by Ant. Another example is the docs task (generating javadoc API documentation). There are, however, few issues that, when not known and solved properly, can be annoying:

Eclipse automatically creates and uses two configuration files (both are created in the main soaplab2 directory): the .project and .classpath. For building purposes, the latter one is more important. It contains information about what to compile (source paths) and where to put it (output paths). You should make sure that both Eclipse and Ant use the same locations. The best way to do so is to use the provided .classpath.template file and to make its copy (named .classpath) and only then start Eclipse.
Whenever (well, not always; I am not sure exactly when it happens) you refresh Eclipse Soaplab2 project, it removes Soaplab2's configuration files from the build/classes. You need to run the Ant's config task to re-create them.
If anybody can tell me how to do it better I will appreciate it very much. Thanks.
Maven and Eclipse is another issue. All Soaplab2 dependencies are defined in the xmls/project.pom. You obviously want to share this dependencies in Eclipse, as well.
First, you need to install a Maven plug-in into your Eclipse. Then, you need to create several pom.xml files - the files where the Eclipse Maven plug-in looks for the dependencies. These files are created by running the Ant's initeclipse task (which is often invoked from other tasks automatically, but can be run also separately). Make sure that you enable Maven plug-in only after the pom.xml files have been created.
This is the sequence of steps I usually follow when Eclipse is not happy (which happens when Soaplab2's dependencies or Soaplab2's version change):
1. Disable Maven in Eclipse.
2. Manually remove pom.xml and src/main/pom.xml files.
  This should be done automatically by Ant but I have not yet figured why it does not do it always.
3. Run ant config.
4. Enable again Maven in Eclipse.
There is also an article about the above. It is, again, an article from another project, but similar to Soaplab2, regarding the usage of Ant and maven.
Until the article is updated, it explains situation with the Maven plug-in version 1. With the version 2 or later (available now) it may be easier.

Last modified: Tue Aug 10 14:28:54 2010