Longbow

The changes for this release are summarised below:

1. Fix for new gromacs packaging where the gmx mdrun CLI would not be
recognised.

2. Fix for case in gromacs where if input file -s and -deffnm would be
provided together and a file was a global file in a replicate job that
all files would be set to output global.

3. Can now use -deffnm with gromacs on replicate jobs with global files
without having to also set -s.

4. Fix for missing space in sge replicate script generator

5. Added the ability for emails to be sent to the user, these are
invoked by providing the email address in a submit file by
'email-address = blahblah.com' and also to set the email flags, these
flags should be the same ones and same format you use in your submit
script for example 'email-flags = -M' or 'email-flags = ib'.

6. New common executable naming schemes added for things like cuda.

7. Fixed some missing newline characters from the job submit script
generator.

8. Recovery system broken by renaming the method to same as a variable,
this has now been resolved.

9. Rewrites to the applications.py module, it was too unwield to add to.
This has now been split down to allow future expansion in a much easier
fashion.

The changes in this release are summarised below:

1. For machines running SGE a new parameter to control the flag used on
the -pe directive has been implemented. This parameter is "sge-peflag"
and has a default value of "mpi", if your cluster requires something
different then use this parameter in a configuration file to set it.

2. Some cases on SGE clusters, it can be set that even if a job needs
only 1 core that the -pe mpi cores must be set. A new flag called
"sge-peoverride" will make this happen, just provide "sge-peoverride =
true" in a configuration file to enable.

3. Fixed a problem where Longbow would exit when no jobs are running,
this was fixed in PBS but not in other schedulers. It is now fixed in
all supported schedulers.

4. The import statements that looked pretty nasty across the library
have been changed for nicer more pythonic ones.

5. Some refactoring to get rid of pylint warnings about not using lazy
logging.

6. Ability to add script calls into the job submission script that is
created by Longbow. The parameter "scripts" should be used, for multiple
script references then a comma separated list should be provided. Just
add something like this to your configuration file "scripts = source
/some/file" or "scripts = source /first/file, source /second/file"

7. Fix a problem when using job configuration files and not referencing
a resource, what should have happened is that Longbow should choose the
default (top one in hosts.conf). But it crashed with a KeyError
exception, this is now fixed.

The changes in this release are summarised below:

1. Fix for recovery file bug where the file name was mangled by
misplaced comma in os.path.join()

2. Fix for bug where the recovery file was not being recognised from the
command-line

3. Hydra MPI support was added to the LSF plugin, to get this to work a
user needs to supply mpiexec.hydra as the handler parameter in
hosts.conf

4. Some bad file keywords have been removed from the NAMD plugin, more
files have been added to the list and some corrections to bad case in
list.

5. Fix for required parameters for staging not being written into the
recovery file.

6. GROMACS files added to NAMD plugin

7. A disconnect feature has been implemented, users supplying
--disconnect on the command-line will have Longbow disconnect after
submitting jobs, these can be reconnected by using the recovery file.

8. Small change to the executable, all code moved from the if __name__
== "__main__": to main() and all code previously in main() moved to
longbowmain() so unit tests can be made for this part of the
application.

The changes in this release are summarised below:

1. pip now complains if http:// is not on the front of a URL, the setup.py has been fixed for this.

2. recovery files now have time stamps in their name so it is possible to select a recovery file if running multiple concurrent instances of Longbow.

The changes in this release are summarised below:

1. Simplification of the whole Longbow library. This means lots of changes have
been made to the source code and thus developers making use of Longbow in
their code might be affected, those that simply wrap the executable will be
largely unaffected by this, but may be affected by the below changes. The
main change as part of the API re-write is that the hosts and jobs
structures have simply been merged into a single structure called jobs, the
library is now much more simple to use since all parameters are passed
around in this single structure, so no more checking which data structure a
parameter belongs to.

2. Extend informative error messages to all schedulers. This is issue is all
about trying to get information from the scheduler as to why a job
submission has failed.

Error messages from PBS/Torque now passed to the Longbow logging system.
Error messages from LSF now passed to the Longbow logging system.
Error messages from SGE now passed to the Longbow logging system.
Error messages from slurm now passed to the Longbow logging system.
Error messages from SoGE now passed to the Longbow logging system.
Fixed vague error that occurs during staging if bad path is used.

3. Longbow can now detect queue size limits dynamically. Machines such as
ARCHER that only allow a certain number of jobs in the queued state at any
one time would mean users have to use multiple Longbow sessions to do larger
numbers of jobs. This improvement now means that a single Longbow instance
can now submit a number of jobs larger than this limit and Longbow will hold
back any that cannot get onto the queue in the first submit instance, as
jobs complete, further jobs will be submitted.

4. A new recovery mode has been introduced that can recover a Longbow session
if for some reason it gets disconnected. See documentation for more details
on this.
5. Removal of the logging module and explicit error handling configuration
within the top level of the library, this issue has ramifications mainly for
developers using Longbow within their applications, users of Longbow won't
be affected much.

The console output messages have been aligned to 80 character widths for
portability (users might notice this).
Removal of the internal logging module from within Longbow (affects
developers).

Logging for Longbow is now configured and instantiated at the entry point
of the application (affects developers).
For developers using Longbow as a library now have the ability to specify
how logging happens themselves each Longbow module logs to its own log,
these can then be controlled by the developer allowing for deeper
integration (affects developers).

The top level longbow method main() has had all unnecessary code and
parameters stripped out, this has all been moved to the application entry
point. This now provides developers with a very high level integration point
for simple integrations (affects developers).

6. Bad comment on the configuration method fixed (developers only).

7. API comments have been updated to be more accurate with latest changes.

8. Fix for unicode/byte string problem between Python2.x and Python3.x which
would garble outputs from subprocess.

9. Rename the scheduler plugin "sge_arc2" to "soge" (Son of Grid Engine).

10. All Longbow command-line arguments now support both GNU standard and non
standard forms for convenience (eg. --version and -version would be valid),
although future versions might phase out the non-standard -param in favour
of --param but keeping the single letter -h, -V etc as these are part of
the standard.

The changes in this release are summarised below:

1. Longbow would not install under python 3 using either pip or by running the
setup script manually. This has now been fixed by using print statements of
the format print("text"), this form will work in all versions of python.

2. Fix for user reported annoyance of installing the hosts.conf in secret, and
also leaving the archive that is downloaded in $home. This has now been
replaced with a warning during install, and also the file is created locally
and not downloaded.

3. Removed padding from version numbers so v1.01.005 > v1.1.5.

4. Changed the README.md from markdown format to reStructured text, so now the
README is README.rst, this is to make Longbow more pip friendly whilst still
being bitbucket and possibly github (in case in future we move).

5. setup.py modified to eliminate the python 3 issues, and also the long
description is now the README.rst, this means that users will see something
standard between the pip page and the bitbucket page, this will also tie in
with new pages on the HECBioSim website which will push pip as the preferred
way to install more prominently.

6. hosts.conf file has been removed from the development version since this is
now created on install for new users using pip, this will be marked as
deprecated on the website for a while so users on old versions can still get
it. Documentation will need to be updated to reflect this change and provide
the quickstart example that users can grab a copy of if they are doing a more
manual install. However pip will be the encouraged way to install.

7. Parameters specified within the configuration files had to be specified in a
very strict format (param = value), users that omitted the spaces would find
that Longbow would crash. The code has now been fixed to use regular
expression to read in and parse for the equals sign, this can now handle
cases where users specify cases such as:

param1 = value1
param2 =value2
param3= value3
param4=value4

8. Added python version to logging.

9. Added longbow version to logging.

10. Moved version to the top of the longbow file.

11. Added the paper citation to the logging and readme.