Batch-Test Client

Soaplab2 services can be tested by calling many of them, or even all of them, from a single client, in a batch mode, using a pre-defined set of input data. They can be executed sequentially, or in parallel. Obviously, this client hardly provide any scientifically valuable results but can be used by service providers, or even by Soaplab2 plug-in developers, to see that things have not changed to the worse.

In some scenarios, it can be, however, also useful for running repeating jobs, because it can keep the results even after the batch test client finished.


Structure of the testing input files
Run-time properties
How to run it
What is reported

Structure of the testing input files

The batch test client can accept one or more testing input files. Their format is a properties file format. The property names are service names (with a prefix sn.) and property values are the input data, specified in the same way as in the Soaplab's main command-line client. For example (taken from the testing file for EMBOSS, the full file is distributed with Soaplab2 in data/embossTests.cfg): 
sn.alignment_consensus.cons=sequence_direct_data :${emboss.home}/share/EMBOSS/test/data/dna.msf
sn.alignment_consensus.megamerger=asequence_usa embl:AC073385 bsequence_usa embl:AP006749
sn.alignment_consensus.merger=asequence_usa embl:eclacy bsequence_usa embl:eclaca
sn.alignment_differences.diffseq=asequence_usa embl:ap000504 bsequence_usa embl:af129756
The same service can be called several time, each time with different input data (or even without any data, assuming there are some default values). For example: 
sn.classic.helloworld
sn.classic.helloworld = greeting 'Ciao\, mundi'
The example above also shows that commas in the input data must be escaped - otherwise the value will be interpreted as two invocations of the same service.

The examples also show that the property values can contain another properties (e.g. emboss.home) that will be substituted for their real value before the property is used. Of course, the referenced properties must be defined somewhere (either in the same testing file, or, usually, in the soaplab.client.properties file).

Please notice that here, as in the Soaplab's main command-line client, one can use the ":" character to indicate that the real input will be actually read from the given file, and the "::" characters similarly to indicate a file of file names.

Run-time properties

The batch test client can be influenced by setting several run-time properties in file soaplab.client.properties. Such file must then appear in any directory on your CLASSPATH. Don't worry if you do not know what the CLASSPATH is for. Just do one of the two things (the second one is more recommended):

The properties used by the batch test client:

batch.test.file
A file name with the testing input data. There can be more such properties, defining more testing input files. Usually, these properties use a reference to another property (e.g. base.dir): 
test.data.dir = ${base.dir}/data
batch.test.file = ${base.dir}/data/defaultTests.cfg
batch.test.file = ${base.dir}/data/embossTests.cfg
batch.test.file = ${base.dir}/data/embassyTests.cfg
test.data.dir
Even though, generally, it is not a mandatory property, it is used in the distributed testing files (files data/*Tests.cfg). Therefore, if you wish to use them, you need to set this property. Its value is a full path to a directory data in the Soaplab project directory.

batch.max.threads
An integer defining maximum number of services invoked in parallel. Its default value is 25. The too high number may cause problem in your Tomcat server (the server has also its own maximum number of threads which may be too low).

A value 1 causes that services will be called sequentially. A value 0 causes that all services will be called at once, in parallel.

batch.keep.results
A boolean property. Its true value causes that the results from tested services will not be removed after the batch test client finishes. In this case, the batch test client listing will also include the Job IDs of the individual tested services that can be used to retrieve the results (usually by the Soaplab's main command-line client). As shown here: 
[COMPLETED]   7_classic.helloworld [classic.helloworld]43ab9197.117d16aa957.-7ffd
[COMPLETED]  14_geocoding.geonames [geocoding.geonames]43ab9197.117d16aa957.-7fff
[COMPLETED]  13_geocoding.geonames [geocoding.geonames]43ab9197.117d16aa957.-8000
Default value is false.

batch.report.table
A boolean property. Its true value causes to list every tested service with its elapsed time. See the reports below what else can be shown.

Default value is false (but the default template file src/etc/config/soaplab.client.properties.template sets it to true).

How to run it

Quick start...

Without setting any run-time properties, without installing EMBOSS and EMBASSY, just with all default settings, you can run it after you deploy default Soaplab2 services by typing: 
build/run/run-batch-client

...and the rest

The batch test client is a command-line client. Its help page can be displayed with the -help option: 
build/run/run-batch-client -help

BatchTestClient
===============

Usage:
   build/run/run-batch-client -h[elp]
   build/run/run-batch-client  [options]

where:

  -h[elp]  ... print this help
  -q       ... quiet mode (useful when you want to redirect
               a result into a file)
  -stack   ... in case of any error it print the stact trace
               (without this option, it prints it only when a
                serious runtime error occures)

where  defines where to find all tested services
and what protocol to use to access them (some of these parameters may
also be specified in the Soaplab client configuration property file -
by default named soaplab.client.properties and put on the CLASSPATH):

   -protocol jaxws   ... service will be accessed as a Web Service,
                         using JAX-WS toolkit (default protocol)
   -protocol local   ... service will be accessed as local class
                         (no remote server or any Web Service used)
   -protocol axis1   ... service will be accessed as a Web Service,
                         using old Axis 1 toolkit (such protocol may
                         not be supported by the given Soaplab server)

   -host       ... host name (default: localhost)
   -port       ... port number (defaullt: 8080)
   -ctx     ... context part of the URL (default: soaplab2)

   -e           ... a full service URL (an endpoint);
                         if given it has precedence over -host, -port,
                         -ctx, and -name;
                         default is created from other parameters as:
                         http://://services/

   -lname   ... a name of a list service to be called
                         (default   'AnalysisFactory' for axis1 protocol,
                          otherwise 'list')

   -el          ... a full List service URL (an endpoint);
                         if given it has precedence over -host, -port, -ctx,
                         and -lname;
                         default is created from other parameters as:
                         http://://services/

and where options are (most of these options can be specified as properties
                       in the soaplab.client.properties file - but the
                       command-lien options take precedence):

   -batchfile  ... a file name with a list of services to be tested
                         (e.g. look into data/defaultTests.txt);

                         Default is taken from properties: batch.test.file

   -maxthreads  ... how many threads to se when testing more services:
                         0   ... as many as we have tested services,
                         1   ... test all services sequentially,
                         > 1 ... number of maximum tested services at once;

                         Default is taken from property batch.max.threads
                         (which is, by default, 25).

   -keep             ... do not remove jobs after they finished
                         (in this case, the job IDs are included in the
                          reports, so one can use them to retrieve results
                          using the regular Soaplab2 command-line client);

                         Default is taken from property batch.keep.results
                         (which is, by deafult, false).

   -table            ... produce a tabular output of all tests;

                         Default is taken from property batch.report.table
                         (which is, by deafult, true).
You can see that the command-line options can overwrite the run-time properties described above.

Soaplab2 comes with several predefined sets of testing input data. They can be found in the data directory:

defaultTests.cfg is for default, more or less testing, services,
embossTests.cfg is for those installing the EMBOSS package, and
embassyTests.cfg is for those installing the EMBASSY package.

What is reported

The batch test client produces up to four sections of reports, in this order:

Progress

       
[Start ]                 39_alignment_local.water
[Start ]                 40_alignment_local.wordmatch
[COMPLETED]               8_classic.helloworld
[Start ]                128_alignment_multiple.emma
[COMPLETED]              11_testing.streams
[COMPLETED]               7_classic.helloworld
[COMPLETED]              23_alignment_consensus.cons
[COMPLETED]              12_testing.files
[COMPLETED]              15_graphics.dot
[COMPLETED]              30_alignment_dot_plots.polydot
[TERMINATED_BY_ERROR]   128_alignment_multiple.emma
[Start ]                142_alignment_multiple.infoalign
[Start ]                143_alignment_multiple.plotcon
[Start ]                144_alignment_multiple.prettyplot
[Start ]                145_alignment_multiple.showalign
[Start ]                146_alignment_multiple.tranalign
[Start ]                147_display.cirdna
[Start ]                148_display.lindna
[COMPLETED]             142_alignment_multiple.infoalign
It reports when each service call starts and when it finishes. The starts and completions are mixed together because the services are called in parallel (in this example).

This section can be avoid by the command-line option -q (as "quiet").

All tests

       
All tests
---------
classic.helloworld      0:00:00.443 COMPLETED
testing.sleep           0:00:02.431 COMPLETED
testing.sleep           0:00:03.581 COMPLETED
testing.streams         0:00:00.496 COMPLETED
testing.files           0:00:01.027 COMPLETED
geocoding.geonames      0:00:02.100 COMPLETED
classic.helloworld      0:00:00.250 COMPLETED
geocoding.geonames      0:00:02.289 COMPLETED
graphics.dot            0:00:01.101 COMPLETED
It reports the elapsed time and completion status of all tested services. It is created only if the command-line option -table is present or if the run-time property batch.report.table is set to true.

Errors

       
------------------------------------------------------------------------------
ERROR: classic.helloworld
------------------------------------------------------------------------------
Unrecognized inputs: greetingXX Ciao, mundi
        Input:  greetingXX 'Ciao, mundi'
        Status: NOT STARTED
This section is always produced (unless there are no errors) to the standard error stream (so it can be redirected by using various shell tools). It contains errors found either by Soaplab2 (as the example above), or by the started underlying application itself.

Summary

       
Summary
-------
Successfully:  8
Erroneously:   1
Not available: 0
It is produced always to the standard output. The Not available contains the number of services that appeared in the testing input files but that are not available from the given Soaplab server, at least not at the moment of running the batch test client.

Last modified: Mon Feb 4 17:20:23 2008