Seed Integration Testing

• Input testing
• Output testing

Create a Seed job-type to perform basic input / output. I'm going to do this by updating the read-bytes job. Sample read-bytes job-type definition:

{
    "docker_image": "geoint/read-bytes",
    "interface": {
  "seedVersion": "1.0.0",
  "job": {
    "name": "read-bytes",
    "jobVersion": "1.0.0",
    "packageVersion": "1.0.0",
    "title": "Read Bytes",
    "description": "Reads x bytes of an input file and writes to output dir",
    "maintainer": {
      "name": "Jonathan Meyer",
      "organization": "Applied Information Sciences",
      "email": "[email protected]"
    },
    "timeout": 3600,
    "interface": {
      "command": "1024 ${INPUT_FILE} ${OUTPUT_DIR}",
      "inputs": {
        "files": [
          {
            "name": "INPUT_FILE"
          }
        ]
      },
      "outputs": {
        "files": [
          {
            "name": "OUTPUT_FILE",
            "pattern": "output_file/*"
          }
        ]
      }
    }
  }
}
}

This job can be queued up to test with the following job queue API call:

POST scale/api/v5/workspaces/

{"name":"host-etc","title":"Host etc dir","description":"/etc directory from slave hosts","json_config":{"broker":{"type":"host","nfs_path":"","host_path":"/etc/"}}}

Then Scan for files:

POST scale/api/v5/scans/

{
	"name": "scan-etc",
	"title": "My Scan Process",
	"description": "This is my Scan process for detecting my favorite files!",
	"configuration": {
		"version": "1.0",
		"workspace": "host-etc",
		"scanner": {
			"type": "dir"
		},
		"recursive": true,
		"files_to_ingest": [{
			"filename_regex": "group"
		}]
	}
}

Kick off scan:

POST scale/api/v5/scans/1/process/?ingest=true

Find file id to queue job:

GET scale/api/v5/sources/

Queue job:

POST scale/api/v5/queue/new-job/

{
    "job_type_id": 10, 
    "job_data": {
        "version": "2.0",
        "input_data": {
"files":[
            {
                "name": "INPUT_FILE",
                "file_ids": [1]
            }]
        }
    }
}

For super boss quick code changes:

#!/bin/bash

dcos marathon app stop /scale-webserver
docker pull $1
docker create --name scale-temp $1
docker cp $2 scale-temp:/opt/$2
docker commit -m "Shim image with code updates from $2" scale-temp $1
docker rm scale-temp
docker push $1
dcos marathon app start /scale-webserver