-
Notifications
You must be signed in to change notification settings - Fork 646
Flexporter
The Synthea Flexible Exporter a.k.a. "Flexporter" is a utility designed to make it easy to add new fields, values, and resource types to Synthea-generated FHIR, without having to modify the Synthea engine. Primary users of this feature are expected to be Implementation Guide authors and users, electronic clinical quality measure developers and testers, and anyone who needs just a slight tweak to the data exported by Synthea.
As of 2022-07-28, this feature is only available on the flexporter
branch: https://github.com/synthetichealth/synthea/tree/flexporter
The basic idea of the Flexporter is that users will define a series of transformations to apply to Synthea data. A predefined set of transformations, such as "set field X to value Y" or "create a new resource" is intended to cover the majority of use cases while still remaining easy to use.
./run_synthea -fm mapping_file [-ig ig_folder]
The Flexporter can also run standalone to post-process FHIR Bundles.
./run_flexporter -fm mapping_file -s source_fhir [-ig ig_folder]
https://github.com/synthetichealth/synthea/blob/flexporter/src/main/java/RunFlexporter.java
The format of the mapping file is as follows:
---
# name is just a friendly name for this mapping
name: Flexporter Mapping
# applicability determines whether this mapping applies to a given file.
# for now the assumption is 1 file = 1 synthea patient bundle.
# NOT YET IMPLEMENTED
# Eventually this should be a FHIRPath which should return a "truthy" value if the mapping applies
applicability: true
# actions is a list of Action objects. see below for details
actions:
- name: Apply Profiles
...
Actions are driven by a key property name on the object. (technical note: this is because the YML library doesn't support discriminators like GSON does for JSON, think like how state types get mapped into a different class by type)
Field Name | Type | Required | Description | |
---|---|---|---|---|
🗝️ | profiles | List | Yes | List of profile/applicability pairs |
profile | URL | Yes | Profile URL to be added to resources | |
applicability | FHIRPath | Yes | FHIRPath to select resources that the profile should be added to |
Example:
This example will apply the qicore-patient profile to all Patient resources, and qicore-encounter to all Encounters.
- name: Apply Profiles
profiles:
- profile: http://hl7.org/fhir/us/qicore/StructureDefinition/qicore-patient
applicability: Patient
- profile: http://hl7.org/fhir/us/qicore/StructureDefinition/qicore-encounter
applicability: Encounter
Field Name | Type | Required | Description | |
---|---|---|---|---|
🗝️ | set_values | List | Yes | List of profile/applicability pairs |
applicability | FHIRPath | Yes | FHIRPath to select resources to set fields on | |
fields | List | Yes | List of location/value pairs to set | |
location | FHIRPath | Yes | FHIRPath to the field to set the value on | |
value | raw value, Value Function, object | Yes | The value to set on the target, or the function to fetch a value to set on the target | |
transform | Value Transformation | No | Transformation to apply to the value before setting on the target |
Example 1:
This example will read the value from Immunization.occurence[x] and set it on the Immunization.recorded field, for all Immunization resources.
- name: testSetValues_getField
set_values:
- applicability: Immunization
fields:
- location: Immunization.recorded
value: $getField([Immunization.occurrence])
Example 2:
This example will set a full coding, with code/system/display, to Patient.maritalStatus.coding.
- name: testSetValues_object
set_values:
- applicability: Patient
fields:
- location: Patient.maritalStatus.coding
value:
system: http://snomedct.io
code: "36629006"
display: "Legally married (finding)"
Field Name | Type | Required | Description | |
---|---|---|---|---|
🗝️ | create_resource | List | Yes | List of resource definitions to create |
resourceType | FHIR resource type | Yes | Resource type to create (Patient, Encounter, etc) | |
based_on | FHIRPath | No | FHIRPath to select resources to base the new created resource off of. (Example use case, create an Appointment based on every Encounter) | |
fields | List | Yes | List of location/value pairs to set on the created resource | |
writeback | List | No | List of location/value pairs to set on the selected based_on resource | |
location | FHIRPath | Yes | FHIRPath to the field to set the value on | |
value | raw value, Value Function, object | Yes | The value to set on the target, or the function to fetch a value to set on the target | |
transform | Value Transformation | No | Transformation to apply to the value before setting on the target |
Example:
This example will create an instance of ServiceRequest for every instance of Procedure, with the following:
- ServiceRequest.intent set to "plan"
- ServiceRequest.encounter.reference set to the same as the source Procedure.encounter.reference
- ServiceRequest.subject.reference set to a reference to the Patient resource
- Procedure.basedOn.reference set to a reference to the newly created ServiceRequest
- name: testCreateResources_createBasedOn
create_resource:
- resourceType: ServiceRequest
based_on: Procedure
fields:
- location: ServiceRequest.intent
value: plan
- location: ServiceRequest.encounter.reference
value: $getField([Procedure.encounter.reference])
- location: ServiceRequest.subject.reference
value: $findRef([Patient])
writeback:
- location: Procedure.basedOn.reference
value: $setRef([ServiceRequest])
Field Name | Type | Required | Description | |
---|---|---|---|---|
🗝️ | keep_resources | List<FHIRPath> | Yes | List of FHIRPath to select resources that should be retained in the bundle, anything not selected by a rule will be removed |
Example:
This example will keep only the Patient, Encounter, and Condition resources. All other resources will be removed.
- name: testKeepResources
keep_resources:
- Patient
- Encounter
- Condition
Field Name | Type | Required | Description | |
---|---|---|---|---|
🗝️ | delete_resources | List<FHIRPath> | Yes | List of FHIRPath to select resources that should be deleted in the bundle, anything selected by a rule will be removed |
Example:
This example deletes the Provenance resource:
- name: testDeleteResources
delete_resources:
- Provenance
Function | Arguments | Description |
---|---|---|
getField |
FHIRPath pointing to field | Get value from a field from current or based-on resource |
findValue |
FHIRPath pointing to resource/field | Find a value from another resource within the entire Bundle |
setRef |
Create a reference to the current resource | |
findRef |
FHIRPath pointing to resource | Find a reference to another resource in the bundle |
getAttribute |
Attribute key | Read value from a Synthea attribute. Note this can only be used when running in Synthea, not standalone. |
randomCode |
ValueSet URL | Pick a random code from the target ValueSet |
Basic naming scheme here is "set" and "get" refer to a particular resource, (ie, the current one, or the resource a new one is being based on) and "find" refers to searching the entire bundle.
Function | Description |
---|---|
toInstant |
Convert the given date/time into an instant: YYYY-MM-DDThh:mm:ss.sss+zz:zz . If the given item doesn't have that level of precision, random values will populate the missing pieces. |
toDate |
Convert the given date/time into a date: YYYY-MM-DD . If the given item doesn't have that level of precision, random values will populate the missing pieces. |
toDateTime |
Convert the given date/time into a dateTime: YYYY-MM-DDThh:mm:ss+zz:zz . If the given item doesn't have that level of precision, random values will populate the missing pieces. |
toTime |
Convert the given date/time into a time: hh:mm:ss . If the given item doesn't have that level of precision, random values will populate the missing pieces. |
actions:
- name: Apply Profiles
profiles:
- profile: http://hl7.org/fhir/us/mcode/StructureDefinition/mcode-cancer-patient
applicability: Patient
- profile: http://example.com/bloodpressure
applicability: Observation.code.coding.where($this.code = '85354-9')
- profile: http://hl7.org/fhir/us/mcode/StructureDefinition/mcode-tnm-clinical-stage-group
applicability: Observation.code.coding.where($this.code = '21908-9')
- profile: http://hl7.org/fhir/us/mcode/StructureDefinition/mcode-tnm-clinical-primary-tumor-category
applicability: Observation.code.coding.where($this.code = '21905-5')
- profile: http://hl7.org/fhir/us/mcode/StructureDefinition/mcode-tnm-clinical-regional-nodes-category
applicability: Observation.code.coding.where($this.code = '21906-3')
- profile: http://hl7.org/fhir/us/mcode/StructureDefinition/mcode-tnm-clinical-distant-metastases-category
applicability: Observation.code.coding.where($this.code = '21907-1')
- name: Create TNM References
set_values:
- applicability: Observation.code.coding.where($this.code = '21908-9')
fields:
- location: Observation.hasMember.where(display='Tumor Category').reference
value: $findRef([Observation.code.coding.where($this.code = '21905-5')])
- location: Observation.hasMember.where(display='Nodes Category').reference
value: $findRef([Observation.code.coding.where($this.code = '21906-3')])
- location: Observation.hasMember.where(display='Metastases Category').reference
value: $findRef([Observation.code.coding.where($this.code = '21907-1')])
- ValueSets can only be loaded from the IG when they are fully defined and expanded within the IG. The existing Synthea terminology support works, but you have to provide your own terminology server.
- Resources removed by keep_resources or delete_resources will still be referenced by other resources. Need to define the best approach for what to do here. (Maybe options? "Cascade" = delete resources that reference deleted resources, "dangle" = leave dangling references, "prune field" = remove the reference field but leave the rest of the resource)
- Copying objects from one field to another doesn't currently work, only primitives. The workaround for now is to list out all the individual sub-fields, but this doesn't work well when the fields that are populated might change. One hack I tried to implement was to deconstruct the provided object into its component fields, but that didn't work because HAPI doesn't provide a way to serialize generic FHIR objects, only Resources. (Ex, you can't serialize a CodeableConcept)
At the core of the flexporter is the CustomFHIRPathResourceGeneratorR4 class - this is based on a community contribution within HAPI which we've added some additional functionality to. It does some magic using HAPI's internal reflection (on top of Java reflection), so it can be a bear to wrap your head around. Debugging and stepping through examples makes it easier. https://github.com/synthetichealth/synthea/blob/flexporter/src/main/java/org/mitre/synthea/export/flexporter/CustomFHIRPathResourceGeneratorR4.java