Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CWL Model, Encoding & Decoding #421

Merged
merged 49 commits into from
Oct 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
081ca04
Add cwl model
caroott Jul 19, 2024
ef04da1
Add cwl decode functions
caroott Jul 19, 2024
8cfb04c
Add testing project
caroott Jul 19, 2024
63f8cdc
fix references in decode
caroott Jul 19, 2024
cd3cdee
fix cwl type naming
caroott Jul 19, 2024
3da6974
change naming of CWL to CWLProcessingUnits
caroott Jul 19, 2024
9c36d2d
fix test fsproj
caroott Jul 19, 2024
1932c44
add test strings
caroott Jul 19, 2024
6ae8e47
Add cwl test
caroott Sep 15, 2024
bbc5a42
fix optional fields
caroott Sep 16, 2024
a9e64d3
rewrite tests to be more clear
caroott Sep 16, 2024
a59d834
reorganize tests
caroott Sep 16, 2024
3c5e2f8
dix dockerFile requirement
caroott Sep 16, 2024
86913f6
fix EnvVarRequirement
caroott Sep 17, 2024
27c8037
fix SchemaDefRequirement
caroott Sep 17, 2024
f4a662d
fix SoftwareRequirements
caroott Sep 17, 2024
e44e736
fix workdirReq and resourceReq
caroott Sep 17, 2024
3c6c0a3
start separating processunits for better modularity / edge case handling
caroott Sep 22, 2024
13d8a9c
reorder and specify wf specific types
caroott Sep 22, 2024
c7a0082
change in-/outputs to dynamicObj for more flexibility
caroott Sep 22, 2024
9930238
add workflow steps
caroott Oct 6, 2024
0d049b1
add workflow description type
caroott Oct 6, 2024
ccefbff
add steps decoding
caroott Oct 6, 2024
01a3c04
fix direct map decode
caroott Oct 6, 2024
3db45c6
add WorkflowSteps tests
caroott Oct 6, 2024
95dab47
support direct type mappings for inputs and outputs
caroott Oct 6, 2024
f53e73f
remove try with for dirent initial workdir
caroott Oct 6, 2024
0fd5be9
add overflow decoder
caroott Oct 6, 2024
7007a0b
add cmd tool metadata and tests
caroott Oct 8, 2024
17fabe8
fix for new dynamic obj version
caroott Oct 8, 2024
3fca0c1
update fsproj
caroott Oct 8, 2024
4ff39f9
Update YAMLicious to version 0.0.2
caroott Oct 8, 2024
ff5af9c
rename Type to Type_ for fable
caroott Oct 8, 2024
379efb4
add failcase to requirements
caroott Oct 8, 2024
44112a8
add wf requirements to decoder
caroott Oct 8, 2024
548b2b7
add metadata and dynobj testing (fix test runs)
caroott Oct 11, 2024
8b6963e
trim strings for python
caroott Oct 11, 2024
7728ebb
update yamlicious version
caroott Oct 15, 2024
1222f01
add workflow decoding
caroott Oct 15, 2024
83cad3e
add cwl to all test list
caroott Oct 15, 2024
f452622
rename types and tests
caroott Oct 17, 2024
e6d9902
add attachmembers attribute
caroott Oct 18, 2024
070ca96
switch to resizearray and add some comments
caroott Oct 18, 2024
090a214
add optional field to inputs
caroott Oct 18, 2024
10e40ba
separate processing units and remove module
caroott Oct 21, 2024
de6ef63
change version and class to optional on tool and wf descriptions
caroott Oct 21, 2024
b783782
remove type CLWClass :(
caroott Oct 21, 2024
fedc6d4
reorganize modules
caroott Oct 21, 2024
844ca8b
remove underlying modules
caroott Oct 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions ARCtrl.sln
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,8 @@ Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "ARCtrl.Yaml.Tests", "tests\
EndProject
Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "ARCtrl.Contract.Tests", "tests\Contract\ARCtrl.Contract.Tests.fsproj", "{D10D12C7-B877-423B-867D-161D99E673C9}"
EndProject
Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "ARCtrl.CWL.Tests", "tests\CWL\ARCtrl.CWL.Tests.fsproj", "{0F2188D3-144C-41BF-89F6-AA85883AE0D3}"
EndProject
Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "ARCtrl.ROCrate", "src\ROCrate\ARCtrl.ROCrate.fsproj", "{658BF141-B4B5-4B90-891D-AC36A3FD7574}"
EndProject
Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "ARCtrl.ROCrate.Tests", "tests\ROCrate\ARCtrl.ROCrate.Tests.fsproj", "{212A1C64-02FC-465A-B0FA-F69735F37ACC}"
Expand Down Expand Up @@ -180,6 +182,10 @@ Global
{D10D12C7-B877-423B-867D-161D99E673C9}.Debug|Any CPU.Build.0 = Debug|Any CPU
{D10D12C7-B877-423B-867D-161D99E673C9}.Release|Any CPU.ActiveCfg = Release|Any CPU
{D10D12C7-B877-423B-867D-161D99E673C9}.Release|Any CPU.Build.0 = Release|Any CPU
{0F2188D3-144C-41BF-89F6-AA85883AE0D3}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{0F2188D3-144C-41BF-89F6-AA85883AE0D3}.Debug|Any CPU.Build.0 = Debug|Any CPU
{0F2188D3-144C-41BF-89F6-AA85883AE0D3}.Release|Any CPU.ActiveCfg = Release|Any CPU
{0F2188D3-144C-41BF-89F6-AA85883AE0D3}.Release|Any CPU.Build.0 = Release|Any CPU
{658BF141-B4B5-4B90-891D-AC36A3FD7574}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{658BF141-B4B5-4B90-891D-AC36A3FD7574}.Debug|Any CPU.Build.0 = Debug|Any CPU
{658BF141-B4B5-4B90-891D-AC36A3FD7574}.Release|Any CPU.ActiveCfg = Release|Any CPU
Expand Down Expand Up @@ -218,6 +224,7 @@ Global
{1CA11165-4B70-41D2-A846-50374E85385E} = {64B34A6E-318D-4E6E-9262-CE52C9B85A38}
{5810EF87-4F85-4B4C-98E3-833AE914C628} = {64B34A6E-318D-4E6E-9262-CE52C9B85A38}
{D10D12C7-B877-423B-867D-161D99E673C9} = {64B34A6E-318D-4E6E-9262-CE52C9B85A38}
{0F2188D3-144C-41BF-89F6-AA85883AE0D3} = {64B34A6E-318D-4E6E-9262-CE52C9B85A38}
{658BF141-B4B5-4B90-891D-AC36A3FD7574} = {6DA2330B-D407-4FB1-AF05-B0184034EC44}
{212A1C64-02FC-465A-B0FA-F69735F37ACC} = {64B34A6E-318D-4E6E-9262-CE52C9B85A38}
{243ACD5F-10AD-4BE6-9932-829667BE2053} = {64B34A6E-318D-4E6E-9262-CE52C9B85A38}
Expand Down
2 changes: 1 addition & 1 deletion Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
<PackageVersion Include="Thoth.Json.Python" Version="0.4.0" />
<PackageVersion Include="Thoth.Json.Newtonsoft" Version="0.2.0" />
<PackageVersion Include="FsSpreadsheet" Version="6.3.0-alpha.4" />
<PackageVersion Include="YAMLicious" Version="0.0.1" />
<PackageVersion Include="YAMLicious" Version="0.0.3" />
<PackageVersion Include="DynamicObj" Version="4.0.3" />
<PackageVersion Include="Fable.SimpleHttp" Version="3.5.0" />
<PackageVersion Include="Fable.Fetch" Version="2.6.0" />
Expand Down
4 changes: 2 additions & 2 deletions src/ARCtrl/ARC.fs
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ module ARCAux =
|> FileSystem.create
fs.Union(tree)

let updateFSByCWL (cwl : CWL.CWL option) (fs : FileSystem) =
let updateFSByCWL (cwl : unit option) (fs : FileSystem) =
let workflows = FileSystemTree.createWorkflowsFolder [||]
let runs = FileSystemTree.createRunsFolder [||]
let tree =
Expand All @@ -65,7 +65,7 @@ module ARCAux =


[<AttachMembers>]
type ARC(?isa : ArcInvestigation, ?cwl : CWL.CWL, ?fs : FileSystem.FileSystem) =
type ARC(?isa : ArcInvestigation, ?cwl : unit, ?fs : FileSystem.FileSystem) =

let mutable _isa = isa
let mutable _cwl = cwl
Expand Down
14 changes: 13 additions & 1 deletion src/CWL/ARCtrl.CWL.fsproj
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,18 @@
<Import Project="../Package.Metadata.props" />

<ItemGroup>
<Compile Include="Library.fs" />
<Compile Include="CWLTypes.fs" />
<Compile Include="Requirements.fs" />
<Compile Include="Outputs.fs" />
<Compile Include="Inputs.fs" />
<Compile Include="WorkflowSteps.fs" />
<Compile Include="ToolDescription.fs" />
<Compile Include="WorkflowDescription.fs" />
<Compile Include="Decode.fs" />
<Compile Include="Encode.fs" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="YAMLicious" />
<PackageReference Include="DynamicObj" />
</ItemGroup>
</Project>
56 changes: 56 additions & 0 deletions src/CWL/CWLTypes.fs
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
namespace ARCtrl.CWL

open DynamicObj

type FileInstance () =
inherit DynamicObj ()

type DirectoryInstance () =
inherit DynamicObj ()

type DirentInstance = {
// can be string or expression, but expression is string as well
Entry: string
Entryname: string option
Writable: bool option
}

/// Primitive types with the concept of a file and directory as a builtin type.
type CWLType =
/// Represents a file (or group of files when secondaryFiles is provided)
| File of FileInstance
/// Represents a directory to present to a command line tool.
/// Directories are represented as objects with class of Directory. Directory objects have a number of properties that provide metadata about the directory.
| Directory of DirectoryInstance
/// Define a file or subdirectory that must be placed in the designated output directory prior to executing the command line tool.
/// May be the result of executing an expression, such as building a configuration file from a template.
| Dirent of DirentInstance
| String
| Int
| Long
| Float
| Double
| Boolean
| Stdout
| Null
| Array of CWLType

type InputRecordSchema () =
inherit DynamicObj ()

type InputEnumSchema () =
inherit DynamicObj ()

type InputArraySchema () =
inherit DynamicObj ()

type SchemaDefRequirementType (types, definitions) as this =
inherit DynamicObj ()
do
DynObj.setProperty (nameof types) definitions this

type SoftwarePackage = {
Package: string
Version: ResizeArray<string> option
Specs: ResizeArray<string> option
}
Loading
Loading