Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publishable write #101

Merged
merged 11 commits into from
Jun 28, 2024
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,14 +43,19 @@ cp app/.env.example app/.env.local
cp service/.env.example service/.env
```

Make any changes to point to the measure repository service, Mongo database, and optionally the VSAC API. `0.0.0.0` may be a more appropriate database address than `localhost` for certain environment setups.
Make any changes to point to the measure repository service, Mongo database, and optionally the VSAC API. `0.0.0.0` may be a more appropriate database address than `localhost` for certain environment setups.
Additionally, some versions of tooling may have issues with running `next dev` within workspaces. Disabling telemetry can prevent the disallowed npm command from running under the hood.
```bash
npx next telemetry disable
```


### Mongo Replica Set Setup

Use the mongodb configuration file to configure the single node replica set. For more information about the configuration file and system location, see the mongodb [configuration file documentation](https://www.mongodb.com/docs/manual/reference/configuration-options/).

1. First shutdown any currently running mongodb standalone instances: `brew services stop mongodb-community`.
2. Locate your [Mongo Configuration File](https://www.mongodb.com/docs/compass/current/settings/config-file/#:~:text=For%20macOS%20and%20Linux%2C%20the,%5Cmongodb%2Dcompass.). _System dependent but may be found at `/usr/local/etc/mongod.conf`_.
2. Locate your [Mongo Configuration File](https://www.mongodb.com/docs/manual/reference/configuration-options/). _System dependent but may be found at `/opt/homebrew/etc/mongod.conf`_.
3. Add this replication set configuration to the mongo configuration file:

```
Expand Down
7 changes: 7 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
"test:service": "npm run test --workspace=service"
},
"devDependencies": {
"@types/lodash": "^4.17.4",
"concurrently": "^7.6.0"
}
}
3 changes: 2 additions & 1 deletion service/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,5 @@
PORT=3000
HOST='localhost'
DATABASE_URL='mongodb://localhost:27017/measure-repository?replicaSet=rs0'
VSAC_API_KEY="<your-api-key>" # Add if you plan on using the `include-terminology` query param
VSAC_API_KEY="<your-api-key>" # Add if you plan on using the `include-terminology` query param
AUTHORING=true # Make false if this is running as a publishable repository instead
1 change: 1 addition & 0 deletions service/.env.test
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
VSAC_API_KEY="example-api-key"
AUTHORING=true
24 changes: 23 additions & 1 deletion service/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,12 +95,34 @@ When sending requests, ensure that the `"Content-type": "application/json+fhir"`

### CRUD Operations

This server can be configured as a [Publishable Measure Repository](https://build.fhir.org/ig/HL7/cqf-measures/measure-repository-service.html#publishable-measure-repository) or an [Authoring Measure Repository](https://build.fhir.org/ig/HL7/cqf-measures/measure-repository-service.html#authoring-measure-repository) using the `AUTHORING` environment variable. The minimum write capabilities for these repositories are described further in the [CRMI Publishable Artifact Repository](https://hl7.org/fhir/uv/crmi/1.0.0-snapshot/artifact-repository-service.html#publishable-artifact-repository) and [CRMI Authoring Artifact Repository](https://hl7.org/fhir/uv/crmi/1.0.0-snapshot/artifact-repository-service.html#authoring-artifact-repository) specifications, respectively. The write capabilities implemented in this server are further detailed for the create, update, and delete operations described below.

This server currently supports the following CRUD operations:

- Read by ID with `GET` to endpoint: `4_0_1/<resourceType>/<resourceId>`
- Create resource (Library or Measure) with `POST` to endpoint: `4_0_1/<resourceType>`
- Publishable:
- Supports the _Publishable_ minimum write capability _publish_
- Artifact must be in active status and conform to appropriate shareable and publishable profiles
- Authoring:
- Supports the additional _Authoring_ capability _submit_
- Artifact must be in draft status

- Update resource (Library or Measure) with `PUT` to endpoint: `4_0_1/<resourceType>/<resourceId>`
_More functionality coming soon!_
- Publishable:
- Supports the _Publishable_ minimum write capability _retire_
- Artifact must be in active status and may only change the status to retired and update the date (and other metadata appropriate to indicate retired status)
- Authoring:
- Supports the additional _Authoring_ capability _revise_
- Artifact must be in (and remain in) draft status

- Delete resource (Library or Measure) with `DELETE` to endpoint: `4_0_1/<resourceType>/<resourceId>`
- Publishable:
- Supports the _Publishable_ minimum write capability _archive_
- Artifact must be in retired status
- Authoring:
- Supports the additional _Authoring_ capability _withdraw_
- Artifact must be in draft status

### Search

Expand Down
1 change: 1 addition & 0 deletions service/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
"db:reset": "ts-node ./scripts/dbSetup.ts reset",
"db:setup": "ts-node ./scripts/dbSetup.ts create",
"db:loadBundle": "ts-node ./scripts/dbSetup.ts loadBundle",
"db:postBundle": "ts-node ./scripts/dbSetup.ts postBundle",
"lint": "eslint \"./src/**/*.{js,ts}\"",
"lint:fix": "eslint \"./src/**/*.{js,ts}\" --fix",
"prettier": "prettier --check \"./src/**/*.{js,ts}\"",
Expand Down
186 changes: 137 additions & 49 deletions service/scripts/dbSetup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import * as dotenv from 'dotenv';
import { MongoError } from 'mongodb';
import { v4 as uuidv4 } from 'uuid';
import path from 'path';
import { DetailedEntry, addIsOwnedExtension, addLibraryIsOwned } from '../src/util/baseUtils';
import { addIsOwnedExtension, addLibraryIsOwned } from '../src/util/baseUtils';
dotenv.config();

const DB_URL = process.env.DATABASE_URL || 'mongodb://localhost:27017/measure-repository';
Expand Down Expand Up @@ -39,10 +39,10 @@ async function deleteCollections() {
/*
* Gathers necessary file path(s) for bundle(s) to upload, then uploads all measure and
* library resources found in the bundle(s).
* If a connectionURL is provided, then posts the resources to the server at the
* connectionURL (as a transaction bundle), otherwise, loads the resources directly to the database
*/
async function loadBundle(fileOrDirectoryPath: string) {
await Connection.connect(DB_URL);
console.log(`Connected to ${DB_URL}`);
async function loadBundle(fileOrDirectoryPath: string, connectionURL?: string) {
lmd59 marked this conversation as resolved.
Show resolved Hide resolved
const status = fs.statSync(fileOrDirectoryPath);
if (status.isDirectory()) {
const filePaths: string[] = [];
Expand All @@ -60,15 +60,72 @@ async function loadBundle(fileOrDirectoryPath: string) {
});

for (const filePath of filePaths) {
await uploadBundleResources(filePath);
await (connectionURL ? postBundleResources(filePath, connectionURL) : uploadBundleResources(filePath));
}
} else {
await uploadBundleResources(fileOrDirectoryPath);
await (connectionURL
? postBundleResources(fileOrDirectoryPath, connectionURL)
: uploadBundleResources(fileOrDirectoryPath));
}
}

/*
* POSTs a transaction bundle to url
*/
async function transactBundle(bundle: fhir4.Bundle, url: string) {
if (bundle.entry) {
// only upload Measures and Libraries
bundle.entry = bundle.entry.filter(
e => e.resource?.resourceType === 'Measure' || e.resource?.resourceType === 'Library'
);
for (const entry of bundle.entry) {
if (entry.request?.method === 'POST') {
entry.request.method = 'PUT';
}
}
}

try {
console.log(` POST ${url}`);

const resp = await fetch(`${url}`, {
method: 'POST',
body: JSON.stringify(bundle),
headers: {
'Content-Type': 'application/json+fhir'
}
});
console.log(` ${resp.status}`);
if (resp.status !== 200) {
console.log(`${JSON.stringify(await resp.json())}`);
}
} catch (e) {
console.error(e);
}
}

/*
* Loads all resources found in the bundle at filePath, by POSTing them to the provided url
*/
async function postBundleResources(filePath: string, url: string) {
console.log(`Loading bundle from path ${filePath}`);

const data = fs.readFileSync(filePath, 'utf8');
if (data) {
console.log(`POSTing ${filePath.split('/').slice(-1)}...`);
const bundle: fhir4.Bundle = JSON.parse(data);
const entries = bundle.entry as fhir4.BundleEntry<fhir4.FhirResource>[];
// modify bundles before posting
if (entries) {
const modifiedEntries = modifyEntriesForUpload(entries);
bundle.entry = modifiedEntries;
}
await transactBundle(bundle, url);
}
}

/*
* Loads all measure or library resources found in the bundle located at param filePath
* Loads all resources found in the bundle at filePath, directly to the database
*/
async function uploadBundleResources(filePath: string) {
console.log(`Loading bundle from path ${filePath}`);
Expand All @@ -77,56 +134,21 @@ async function uploadBundleResources(filePath: string) {
if (data) {
console.log(`Uploading ${filePath.split('/').slice(-1)}...`);
const bundle: fhir4.Bundle = JSON.parse(data);
const entries = bundle.entry as DetailedEntry[];
const entries = bundle.entry as fhir4.BundleEntry<fhir4.FhirResource>[];
// retrieve each resource and insert into database
if (entries) {
await Connection.connect(DB_URL);
console.log(`Connected to ${DB_URL}`);
let resourcesUploaded = 0;
let notUploaded = 0;
// pre-process to find owned relationships
const ownedUrls: string[] = [];
const modifiedEntries = entries.map(ent => {
// if the artifact is a Measure, get the main Library from the Measure and add the is owned extension on
// that library's entry in the relatedArtifacts of the measure
const { modifiedEntry, url } = addIsOwnedExtension(ent);
if (url) ownedUrls.push(url);
// check if there are other isOwned urls but already in the relatedArtifacts
if (ent.resource?.resourceType === 'Measure' || ent.resource?.resourceType === 'Library') {
ent.resource.relatedArtifact?.forEach(ra => {
if (ra.type === 'composed-of') {
if (
ra.extension?.some(
e => e.url === 'http://hl7.org/fhir/StructureDefinition/artifact-isOwned' && e.valueBoolean === true
)
) {
if (ra.resource) {
ownedUrls.push(ra.resource);
}
}
}
});
}
return modifiedEntry;
});
const modifiedEntries = modifyEntriesForUpload(entries);
const uploads = modifiedEntries.map(async entry => {
// add Library owned extension
entry = addLibraryIsOwned(entry, ownedUrls);
if (
entry.resource?.resourceType &&
(entry.resource?.resourceType === 'Library' || entry.resource?.resourceType === 'Measure')
) {
if (entry.resource?.resourceType === 'Library' || entry.resource?.resourceType === 'Measure') {
// Only upload Library or Measure resources
try {
if (!entry.resource.id) {
entry.resource.id = uuidv4();
}
if (entry.resource?.status != 'active') {
entry.resource.status = 'active';
console.warn(
`Resource ${entry?.resource?.resourceType}/${entry.resource.id} status has been coerced to 'active'.`
);
}
const collection = Connection.db.collection<fhir4.FhirResource>(entry.resource.resourceType);
console.log(`Inserting ${entry?.resource?.resourceType}/${entry.resource.id} into database`);
console.log(`Inserting ${entry.resource.resourceType}/${entry.resource.id} into database`);
await collection.insertOne(entry.resource);
resourcesUploaded += 1;
} catch (e) {
Expand All @@ -140,7 +162,7 @@ async function uploadBundleResources(filePath: string) {
}
}
} else {
if (entry?.resource?.resourceType) {
if (entry.resource?.resourceType) {
notUploaded += 1;
} else {
console.log('Resource or resource type undefined');
Expand All @@ -156,6 +178,56 @@ async function uploadBundleResources(filePath: string) {
}
}

/*
* Convenience modification of an array of entries to create isOwned relationships and coerce to status active.
* This let's us massage existing data that may not have the appropriate properties needed for a Publishable Measure Repository
lmd59 marked this conversation as resolved.
Show resolved Hide resolved
*/
function modifyEntriesForUpload(entries: fhir4.BundleEntry<fhir4.FhirResource>[]) {
// pre-process to find owned relationships
const ownedUrls: string[] = [];
const modifiedEntries = entries.map(ent => {
// if the artifact is a Measure, get the main Library from the Measure and add the is owned extension on
// that library's entry in the relatedArtifacts of the measure
const { modifiedEntry, url } = addIsOwnedExtension(ent);
if (url) ownedUrls.push(url);
// check if there are other isOwned urls but already in the relatedArtifacts
if (ent.resource?.resourceType === 'Measure' || ent.resource?.resourceType === 'Library') {
ent.resource.relatedArtifact?.forEach(ra => {
if (ra.type === 'composed-of') {
if (
ra.extension?.some(
e => e.url === 'http://hl7.org/fhir/StructureDefinition/artifact-isOwned' && e.valueBoolean === true
)
) {
if (ra.resource) {
ownedUrls.push(ra.resource);
}
}
}
});
}
return modifiedEntry;
});
const updatedEntries = modifiedEntries.map(entry => {
// add Library owned extension
const updatedEntry = addLibraryIsOwned(entry, ownedUrls);
if (updatedEntry.resource?.resourceType === 'Library' || updatedEntry.resource?.resourceType === 'Measure') {
// Only upload Library or Measure resources
if (!updatedEntry.resource.id) {
updatedEntry.resource.id = uuidv4();
}
if (updatedEntry.resource.status != 'active') {
updatedEntry.resource.status = 'active';
console.warn(
`Resource ${updatedEntry.resource.resourceType}/${updatedEntry.resource.id} status has been coerced to 'active'.`
);
}
}
return updatedEntry;
});
return updatedEntries;
}

/*
* Inserts the FHIR ModelInfo library into the database
*/
Expand Down Expand Up @@ -216,6 +288,22 @@ if (process.argv[2] === 'delete') {
.finally(() => {
Connection.connection?.close();
});
} else if (process.argv[2] === 'postBundle') {
if (process.argv.length < 4) {
throw new Error('Filename argument required.');
}
let url = 'http://localhost:3000/4_0_1';
if (process.argv.length < 5) {
console.log('Given only filename input. Defaulting service url to http://localhost:3000/4_0_1');
} else {
url = process.argv[4];
}

loadBundle(process.argv[3], url)
.then(() => {
console.log('Done');
})
.catch(console.error);
} else {
console.log('Usage: ts-node src/scripts/dbSetup.ts <create|delete|reset>');
}
20 changes: 20 additions & 0 deletions service/src/config/capabilityStatementResources.json
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,16 @@
],
"code": "update",
"documentation": "Update allows authoring workflows to update existing libraries in _draft_ (**revise**) status, add comments to existing libraries (**review** and **approve**), and **release** or **retire** a library."
},
{
"extension" : [
{
"url" : "http://hl7.org/fhir/StructureDefinition/capabilitystatement-expectation",
"valueCode" : "SHALL"
}
],
"code" : "delete",
"documentation" : "Delete allows authoring workflows to **withdraw** _draft_ libraries or **archive** _retired_ libraries."
}
],
"searchParam": [
Expand Down Expand Up @@ -207,6 +217,16 @@
],
"code": "update",
"documentation": "Update allows authoring workflows to update existing measures in _draft_ (**revise**) status, add comments to existing measures (**review** and **approve**), and **release** or **retire** a measure."
},
{
"extension" : [
{
"url" : "http://hl7.org/fhir/StructureDefinition/capabilitystatement-expectation",
"valueCode" : "SHALL"
}
],
"code" : "delete",
"documentation" : "Delete allows authoring workflows to **withdraw** _draft_ measures or **archive** _retired_ measures."
}
],
"searchParam": [
Expand Down
Loading
Loading