| Badges | |
|---|---|
| Fairness |  |
| Packages and Releases |   |
| Build Status |   |
| Documentation | |
| DOI |  |
| License |  |
APE is a command line tool and Java API for the automated exploration of possible computational pipelines (scientific workflows) from large collections of computational tools. Generated workflows can be exported in CWL format, as well as in graphical (PNG, SVG) formats.
APE relies on a semantic domain model that includes tool and type taxonomies as controlled vocabularies for the description of computational tools, and functional tool annotations (inputs, outputs, operations performed) using terms from these taxonomies. Based on this domain model and a specification of the available workflow inputs, the intended workflow outputs and possibly additional constraints, APE then computes possible workflows.
Internally, APE uses a component-based program synthesis approach. It translates the domain knowledge and workflow specification into logical formulas that are then fed to a SAT solver to compute satisfying instances. These solutions are then translated into the actual candidate workflows. For a detailed description, we refer to [1].
For our paper at ICCS 2020 [2] we created a video that explains APE in 5 minutes:
For detailed information please visit our page.
To run APE locally you need to have Java 1.8 (or higher) installed on your system (use the command $ java -version to check your local version).
To build APE from source, Maven 3.3+ has to be installed as well (use the command $ mvn -version to check your local version).
Note: Building APE from source is not required to run it, as the latest stable version is available at maven repository.
To add a dependency on APE using Maven, use the following:
<!-- https://mvnrepository.com/artifact/io.github.sanctuuary/APE -->
<dependency>
<groupId>io.github.sanctuuary</groupId>
<artifactId>APE</artifactId>
<version>2.x.x</version>
</dependency>For information regarding Gradle, Ivy, etc. we refer to the APE mvn repository.
| Date | Version | Download |
|---|---|---|
| 15-07-2020 | 1.0.1 | jar, executable, javadoc, sources |
| 02-05-2021 | 1.1.7 | jar, executable, javadoc, sources |
| 20-12-2021 | 1.1.12 | jar, executable, javadoc, sources |
| 17-05-2022 | 2.0.0 | jar, executable, javadoc, sources |
| 19-02-2024 | 2.3.0 | jar, executable, javadoc, sources |
From the project root, simply launch
$ mvn -DskipTests=true installto build the APE modules from the source tree and the built files will be generated under the /target directory. All the dependencies will be gathered by Maven and the following stand-alone module will be generated: APE-[latest]-executable.jar
Automated workflow composition with APE can be performed through its command line interface (CLI) or its application programming interface (API). While the CLI provides a simple means to interact and experiment with the system, the API provides more flexibility and control over the synthesis process. It can be used to integrate APE’s functionality into other systems.
APE-[latest]-executable.jar is available in maven repository.
When running APE-[latest]-executable.jar from the command line, it requires a JSON configuration file given as a parameter and executes the automated workflow composition process accordingly:
java -jar APE-[latest]-executable.jar [path-to-ape-configuration]The configuration file (see APE cofiguration example and APE configuration documentation) provides references to all therefor required information:
- Domain model - classification of the types and operations in the domain in form of an ontology (see ontology example in OWL) and a tool annotation file (see tool annotations example in JSON).
- Workflow specification - including a list of workflow inputs/outputs and template-based (see constraint templates) workflow constraints (see workflow constraints example)
- Parameters for the synthesis execution, such as the number of desired solutions, output directory, system configurations, etc. (see APE configuration documentation).
git clone [email protected]:sanctuuary/APE_UseCases.gitor
git clone https://github.com/sanctuuary/APE_UseCases.gitDownload the latest version of APE-[latest]-executable.jar and add it to the APE_UseCases directory (~/git/APE_UseCases)
cd ~/git/APE_UseCases
java -jar APE-[latest]-executable.jar ImageMagick/Example1/config.jsonSee ImageMagick: Example 1 for more information about the results and on how to execute the composed workflow.
Use cases contains all the details and examples regarding the composition setup and the composition execution using the existing use cases (such as composition of ImageMagick operations).
Like the CLI, the APE API relies on a configuration file that references the domain ontology, tool annotations, workflow specification and execution parameters:
// set up the framework
APE ape = new APE("path/to/setup-configuration.json");
// run the synthesis
SATsolutionsList solutions = ape.runSynthesis("path/to/run-configuration.json");
// write the solutions for the file system
APE.writeSolutionToFile(solutions);
APE.writeDataFlowGraphs(solutions, RankDir.TOP_TO_BOTTOM);
APE.writeExecutableWorkflows(solutions);However, the API allows to generate and edit the configuration file programmatically:
// set up the framework
APECoreConfig coreConfig = new APECoreConfig(...);
APE ape = new APE(coreConfig);
// run the synthesis
APERunConfig runConfig = APERunConfig.builder().withSolutionMinLength(1).withSolutionMaxLength(10)
.withMaxNoSolutions(100).withApeDomainSetup(ape.getDomainSetup())
.build();
SATsolutionsList solutions1 = ape.runSynthesis(runConfig);
// run the synthesis again with altered parameters
runConfig.setUseWorkflowInput(ConfigEnum.ONE);
SATsolutionsList solutions2 = ape.runSynthesis(runConfig);For more information see APE javadoc.io page.
The architecture of the APE v2 library is presented in the following figure. Components coloured light blue extend existing components in the APE v1 framework; dark blue components are new modules.
Graphical Web Interface for the APE library is available at APE Web.
Our use cases are motivated by practical problems in various domains (e.g. bioinformatics, GIS [3]). Different examples are available at the APE Use Cases Repository.
- Vedran Kasalica (v.kasalica[at]esciencecenter.nl), lead developer
- Maurin Voshol, student developer
- Koen Haverkort, student developer
- Anna-Lena Lamprecht (anna-lena.lamprecht[at]uni-potsdam.de), project initiator and principal investigator
For any questions concerning APE please get in touch with Vedran Kasalica (v.kasalica[at]esciencecenter.nl.nl).
We welcome all contributions (bug reports, bug fixes, feature requests, extensions, use cases, etc.) to APE. Please get in touch with Vedran Kasalica (v.kasalica[at]esciencecenter.nl.nl) to coordinate your contribution. We expect all contributors to follow our Code of Conduct.
In case you have a specific request, want to report a bug or suggest a new constraint template please make an issue here.
APE has been inspired by the Loose Programming framework PROPHETS. It uses similar mechanisms for semantic domain modelling, workflow specification and synthesis, but strives to provide the automated exploration and composition functionality independent from a concrete workflow system.
We thank our brave first-generation users for their patience and constructive feedback that helped us to get APE into shape.
APE is licensed under the Apache 2.0 license.
- OWL API - LGPL or Apache 2.0
- SAT4J - EPL or GNu LGPL
- apache-common-lang - Apache 2.0
- graphviz-java - Apache 2.0
- org.json - JSON license
[1] Kasalica, V., & Lamprecht, A.-L. (2020). Workflow Discovery with Semantic Constraints: The SAT-Based Implementation of APE. Electronic Communications of the EASST, 78(0). https://doi.org/10.14279/tuj.eceasst.78.1092
[2] Kasalica V., Lamprecht AL. (2020) APE: A Command-Line Tool and API for Automated Workflow Composition. ICCS 2020. ICCS 2020. Lecture Notes in Computer Science, vol 12143. Springer, https://doi.org/10.1007/978-3-030-50436-6_34
[3] Kasalica, V., & Lamprecht, A.-L. (2019). Workflow discovery through semantic constraints: A geovisualization case study. In Computational science and its applications – ICCSA 2019 (pp. 473–488), Springer International Publishing, https://doi.org/10.1007/978-3-030-24302-9_53