Altocumulus
Command line utilities for running workflows on Terra or Cromwell including:
Run a Terra method, and bulk add/delete methods on Terra.
Submit WDL workflow jobs to a sever running Cromwell, as well as check jobs’ status, abort jobs, and get logs.
Replace local file paths with remote Cloud (Google Cloud or Amazon AWS) bucket URIs, and automatically upload referenced files to Cloud buckets.
Parse monitoring log files to determine optimal instance type and disk space.
Important tools used by Altocumulus:
FireCloud Service Selector (FISS). In particular, fiss/firecloud/api.py.
Version 2.2.0 November 4, 2022
In
cromwell run
command, automatically create zip file on dependency WDLs for a local WDL workflow.Remove
query
command.Bug fix.
Version 2.1.2 August 15, 2022
Version 2.1.1 August 12,2022
Add
--type
option toquery
command to specify query type.
Version 2.1.0 August 4, 2022
Altocumulus now only works with Python 3.8+.
Improve FASTQ file uploading. [PR #28]
Add
query
command to query project metadata from a LIMS (Laboratory Information Management System) via RESTful APIs.
Version 2.0.3 May 24, 2022
Support uploading only the FASTQ files with filename prefix specified within the source folder, instead of the whole folder, to the Cloud. [PR #24]
In
cromwell list_jobs
command, add-n
option to show only top n jobs. [PR #21]Bug fix:
Make all the temporary files with filenames unique per process, and remove them even when submission fails.
Fix the issue in
cromwell list_jobs
command when workflows’ names are not returned by Cromwell API. [PR #20 by Asma Bankapur]Fix the issue in
cromwell get_logs
command when no subworkflow exists in a WDL subtask call. [PR #22]
Version 2.0.2 March 16, 2022
Fix the issue when submitting jobs using Dockstore workflow without specifying version (i.e. implicitly using default version):
Dockstore API points to an incorrect path in the top-level
workflow_path
value.So always search through all versions to use the corresponding
workflow_path
inside the default version entry.
Version 2.0.1 March 1, 2022
Add
--profile
option to allow use a specific AWS profile when dealing with AWS backend:In terra command:
run
andget_logs
sub-commands.In upload command.
In cromwell
run
sub-command:Add
-d
option to allow attach dependency WDL files along with the main workflow WDL file specified in-m
option.Fix the issue on processing floating numbers in workflow input JSON files.
Version 2.0.0 January 12, 2022
Make method-related commands in legacy version as sub-commands under terra command, including:
run
,add_method
,remove_method
,storage_estimate
.
Create sub-commands under cromwell command for interaction between users and Cromwell server, including:
run
,check_status
,abort
,get_metadata
,get_logs
,list_jobs
.
Make uploading local data to Cloud buckets a separate command upload.
Add parse_monitoring_log command to extract computing resource usage info from monitoring logs generated by Cumulus WDL workflows.
Version 1.1.1 September 3, 2021
Legacy version:
Make sure that float values would look the same as in JSON input. For example, if
0.00005
is given, altocumulus should pass0.00005
instead of5e-05
to Terra.