FML transform performance - excessive creation of FHIRPathEngine in new Properties

[mrunibe]

When using FML transform with large input/output data, a lot of runtime is spent in creating new Properties. Looks like this is caused by:

• In TransformSupportServices.createType()

  org.hl7.fhir.core/org.hl7.fhir.validation/src/main/java/org/hl7/fhir/validation/TransformSupportServices.java

  Lines 37 to 40 in 1ad319e

  	public Base createType(Object appInfo, String name) throws FHIRException {
  	StructureDefinition sd = context.fetchResource(StructureDefinition.class, name);
  	return Manager.build(context, sd);
  	}

• for every new Property, a new ProfileUtilities is instantiated:

  org.hl7.fhir.core/org.hl7.fhir.r5/src/main/java/org/hl7/fhir/r5/elementmodel/Property.java

  Lines 89 to 91 in 1ad319e

  	public Property(IWorkerContext context, ElementDefinition definition, StructureDefinition structure) {
  	this(context, definition, structure, new ProfileUtilities(context, null, null), new ContextUtilities(context));
  	}

• and therefore also a somewhat heavyweight FHIRPathEngine object is created:

  org.hl7.fhir.core/org.hl7.fhir.r5/src/main/java/org/hl7/fhir/r5/conformance/profile/ProfileUtilities.java

  Lines 458 to 460 in 1ad319e

  	if (context != null) {
  	this.fpe = new FHIRPathEngine(context, this);
  	}

Possible solution: if ProfileUtilities/FHIRPathEngine object is be re-used from StructureMapUtilities, the transform is substantially faster, especially for larger input data:

# Elements	Time [s]	Time with Workaround [s] Skip numberChildren #1699	Time with Tuning [s] Re-use FHIRPathEngine mrunibe@0c62b94
10	0.2	0.2	0.1
100	0.5	0.5	0.1
1'000	2	2	0.2
10'000	64	11	0.5
20'000	250	23	0.8
100'000	5316	126	2.4
200'000	-	240	3.6

It also significantly reduces the memory footprint (case with 200'000 elements now runs with 3.5gb heap, before required ~7gb).

The issue/numbers can be reproduced with the attached testcase.zip:

java org.hl7.fhir.validation.ValidatorCli testcase/bundle-200000.json -transform http://test.ch/DummyBundleToBundle -version 4.0 -ig testcase/map -output result-200000.json

CC @oliveregger

[oliveregger]

thanks @mrunibe, added comment in PR directly #1704