Questionnaire to CIM Conversion


In order to translate from a questionnaire instance to CIM instances we need to have at least captured the information that is required to be a valid CIM instance. Of course there will be lots of additional optional CIM information that will be useful too but capturing this information (or not) in the questionnaire is a scientific decision. All the questions that the questionnaire asks can be represented in the CIM (otherwise why are people filling them in). This ticket is looking at the technical issues of translating from questionnaire instances to CIM instances.

The current version of the Questionnaire (that is being worked on) is release 1.0. The current version of the CIM that is being targeted is 1.5. The plan is for the release version of 1.5 to be able to capture all the concepts in the questionnaire.

To Do

  1. Move bindings from the component document to the simulation document
  2. remove any "not in use" simulation bindings (ticket #948)
  3. typo (Ying) in grids values (ticket #951)
  4. remove other in grids? (emailed MP)

Questionnaire CIM export Changes since Release 1.0


  1. removed gmd:CI_ResponsibleParty element output as the latest version of the CIM does not use this
  2. removed gmd:CI_Citation element output as the latest version of the CIM does not use this
  3. added project@value='CMIP5' element to simulation to specify what the document relates to
  4. added authorsList/list element to simulation to store the questionnaire authorList information. This closes ticket 831
  5. changed default CV server to trunk (was previously branches/vn1.0_devel) as we are currently working on the trunk
  6. re-instated the open attribute for controlled vocabulary terms as the latest version of the CIM requires this


  1. grid: horizontal_properties/gridMnemonic now output to gridTile or GridMosaic? (for composite grid) as mnemonic
  2. grid: any grid references on top level page are now output in esmModelGrid/referenceList
  3. grid: isLeaf, numTiles and numMosaics attributes now output (these are inferred)
  4. grid: gridTile attribute now output correctly for mosaics (as specified by Marie-Pierre)


  1. component : removed empty numericalProperties, scientificProperties, componentProperties elements as these are now optional in the CIM
  2. grid: horizontal extent now output for non-composite grid
  3. grid: references now output correctly (bug fix)
  4. shortName and longName no longer output for gridMosaic and gridTile
  5. horizontal and vertical descriptions are output together in gridTile/description
  6. refinement details now output as part of gridTile/horizontalResolution/@description
  7. compositeGrid field output in gridMosaic/description


  1. grid: references now output correctly - after a change to the CIM
  2. grid: horizontal extent now output for composite grid


  1. cimDocumentSet replaces cimRecordSet
  2. cimRecord nesting removed
  3. namespace changed to
  4. experiment requiredDuration removed
  5. experiment duration information added inside spatiotemporalconstraint.
  6. model previousVersion information stored in CIM documentGenealogy at the end of a component now called previousVersionOf
  7. CIM documentGenealogy now has a direction attribute "toTarget" to explain which model "previousVersionOf" is referring to.


  1. gridSpec@gml:id added and set to uri. Only added as it is required by the CIM.
  2. gridSpec/esmModelGrid@id added and set to empty string. Only added as it is required by the CIM.
  3. gridSpec/mnemonic added when a mnemonic is provided on the grid page
  4. all grid information now output
  5. rip values now output
  6. conformances updated to new format


  1. referencing of requirement options now fully supported
  2. CIM documents now output e.g. export on component page (previously a bug)

Issues (depending on others)

  1. Implement Mark E's cardinality changes [ Gerry ticket 663 ]
  2. q2cim coupling does not match (transformations) temporal add order [Bryan ticket 665]
  3. file dataObject@dataStatus is mandatory. Do I put complete here?
  4. file distribution@distributionFormat has the same value as storage/ipStorage@dataFormat in the questionnaire. Is that right? What is the difference?
  5. file CIM has a standardName (used in dataObject/content/topic) but the name you supply (CF in our case) is not part of an enumerated list.
  6. Simulation duration period should be integer, or perhaps date offset? [ Bryan ticket 819 ]
  7. CIM1.5 needs to be frozen [ Allyn ticket 822 ]
  8. add refinement scheme to gridpackage [ Allyn ticket 890 ]
  9. add a type to the confomance [ Allyn ticket 893 ]
  10. making a load of stuff optional in the CIM grid package [ Allyn ticket 912 ]
  11. making grid CV consistent [ MP ticket 913 ]
  12. what is mandatory in the grid questionnaire section and what is optional? [ MP ticket 914 ]


  1. I've added the centre information as a responsible party to the simulation CIMObject and to the component/model CIMObject
  2. The questionnaire default responsible party "Unknown" is not output in the CIM
  3. The questionnaire responsible party URI is output in the CIM as a comment in case it is needed in the future
  4. Unlike the questionnaire, responsible parties will be added in-place in the CIM as there is no concept of a responsible party CIMObject that can be referenced. By adding the uri we can always remove redundancy in the future and others can avoid redundancy.
  5. The CIM output from the questionnaire will not have any information about simulation control runs so the control attribute will always be set to false.

Q2CIM Translation

Text in bold in the tables indicates an unresolved issue


Experiments are not editable in the questionnaire, however their information is loaded and stored internally. This allows experiment information to be viewed and for simulations and ensembles to be associated with experiments. This internal information is used to create CIM Experiment records.

grid translation

gridSpecCIM top level container
gridSpec[@gml:id='0']CIM requires a gml id so use URI
gridSpec/esmModelGrid[@id='']CIM requires an id but this can be empty
grid/mnemonic1gridSpec/esmModelGrid/mnemonicoutput if mnemonic value is supplied
grid/references/referencesee references table
grid/horizontalGrid/descriptiongridSpec/esmModelGridgridTile/descriptionnote: we prepend "Horizontal Properties: " to description
grid/horizontalGrid/coordinateSystem/discretization [if discretization != composite,N/A]1gridSpec/esmModelGrid/gridTile/@discretization
grid/horizontalGrid/coordinateSystem/discretization [if discretization == N/A]n/aDo not provide the gridSpec/esmModelGrid/gridTile/@discretization property in this case.
[if discretization==composite] grid/horizontalGrid/coordinateSystem/compositeDiscretizationgridSpec/esmModelGrid/gridTile/@discretizationcompositeDiscretization is multi-valued so create a separate gridTile for each discretization entry
grid/horizontalGrid/coordinateSystem/resolution1gridSpec/esmModelGridgridTile/horizontalResolutionIf we create a gridMosaic then we will replicate this information in each gridTile.
grid/horizontalGrid/coordinateSystem/refinementScheme1gridSpec/esmModelGridgridTile/horizontalResolutionIf we create a gridMosaic then we will replicate this information in each gridTile. Append "Refinement details : text" to horizontalResolution'
grid/horizontalGrid/coordinateSystem/type1gridSpec/esmModelGrid/@gridTypeValid CV values depend on choice of horizontal discretization.
[if discretization==composite] grid/horizontalGrid/coordinateSystem/compositeGridgridSpec/esmModelGrid/gridMosaic/description
[if grid/horizontalGrid/coordinateSystem/type is *]gridSpec/esmModelGridgridTile/horizontalResolution/property[@name=""][@value=""]Note, I will place the different values in the different tiles for a composite.
grid/horizontalGrid/extent1gridSpec/esmModelGridgridTile/extentShould this be part of gridMosaic (instead or as well as gridTile) for composite models?
grid/verticalGrid/descriptiongridSpec/esmModelGridgridTile/descriptionRF: prepend "Vertical Properties :" to description and append this to any existing (horizontalGrid description)
grid/verticalGrid/coordinateSystem/type1????Is type actually equivalent to discretisation (in horizontal grid terms)? If that is the case I think this should be added in the same place. Proposal : add the following gridSpec/esmModelGrid/gridTile/@verticalDiscretization and change the existing gridType attribute to horizontalDiscretization. Also I suggest that we change the mindmap names to be consistent.
grid/verticalGrid/coordinateSystem/coordinate1????Is coordinate actually equivalent to type (in horizontal grid terms)? If so I think this property helps define the grid. Proposal : add an additional attributes like gridSpec/esmModelGrid/@verticalGridType and change the existing gridType attribute to horizontalGridType. Also I suggest that we change the mindmap names to be consistent.
grid/verticalGrid/coordinateSystem/surfaceReference, partialSteps or hybridization????Proposal : make a verticalResolution container which can contain 1 to n name value pairs which conform to this CV. So gridSpec/esmModelGrid/gridTile/horizontalResolution/property/@name="xx"/"yy". We could then make the existing verticalResolution value an attribute. Note, I would replicate the same vertical values in the different tiles for a composite.'
grid/verticalGrid/AtmosphericLevels or OceanicLevels????Why are we associating levels with components here? I would (naively) say that a grid should not be associated with a component; rather a component is associated with a grid. If this is really the way these things are distinguished then we could add an associated id or a type. My first reaction is that I would prefer to just add levels and have a load of associated properties. Proposal : add this information by modifying the verticalResolution element in the same way as the horizontalResolution element would be modified. The fact properties are associated with an ocean grid or atmos grid can be inferred by the component referencing the grid.
extent?1All atmos or all ocean values are mandatory

The top level grid component is ignored by the Q2CIM translator as it is going to be removed

ensembles translation

ensemble characteristics1ensemblea new ensemble document is created and added to the CIMRecordSet
ensemble/ shortNamewe use the associated simulation shortName
ensemble/ longNamewe use the associated simulation longName
notes0..1ensemble/ description
ensemble/ supports/ reference (see reference table)reference to an experiment. This is already set up in the questionnaire
members2..nensemble/ ensembleMember
1ensemble/ ensembleMember/ simulation/ referencereference to the associated simulation. In the questionnaire this is the same simulation for all ensemble members but that does not have to be the case in the CIM.
member model mod0..1ensemble/ ensembleMember/ simulation/ reference/ change
member input mod0..1ensemble/ ensembleMember/ simulation/ reference/ change
member drs name1ensemble/ ensembleMember/ externalID/ name
1ensemble/ ensembleMember/ externalID/ standard[@value='DRS']Specify that this is a name from the DRS vocabulary
ensemble/ documentIDGenerated by the questionnaire
ensemble/ documentVersionGenerated by the questionnaire
ensemble/ documentAuthorThe Metafor questionnaire is set as the author
ensemble/ documentCreationDateGenerated by the questionnaire

references translation

QuestionnaireCardinalityCIM1.5notestds tag or attribute
citation/gdm:CI_Citation/gmd:dateI leave this mandatory element empty
0citation/gmd:CI_Citation/gmd:presentationForm/gmd:CI_PresentationFormCode@codeListValueI leave the @codeList empty [ME - no date on interface]

file translation

QuestionnaireCardinalityCIM1.5notestds tag or attribute
dataObject@dataStatus='complete'I just set this to complete at the moment. I'm not sure what it means.
file/abbrev1dataObject/acronymCurrently optional in the questionnaire
dataObject/storage/ipStorage@dataLocation='''''This is currently required. Just set this to empty stringfile_id
dataObject/storage/ipStorage/dataSize='0'Required in the CIM but not supplied in the Q
dataObject/storage/ipStorage/protocolRequired in the CIM but not supplied in the Q
dataObject/storage/ipStorage/hostRequired in the CIM but not supplied in the Q
dataObject/distributionRequired in the CIM
dataObject/distribution@distributionAccess='OnlineFileHTTP'Required in the CIM but not supplied in the Q
file/formatdataObject/distribution/distributionFormat@valueUsing the same value here as for storage
dataObject/distribution/responsiblePartyRequired in the CIM but not supplied in the Q
file/reference0...1dataObject/citationSee references translation table
file/variabledataObject/contentOne content element per file/variable
dataObject/content/aggregationRequired in the CIM but not supplied in the Q
dataObject/content/frequencyRequired in the CIM but not supplied in the Q
file/variable/reference0...1dataObject/content/citationSee references translation table

platform translation

platform/ shortNamewe use a concatenation of machine name + compiler name here as a platform is a description of these two things. If compiler name is not provided then we use the string "CompilerUnspecified" in its stead
platform/ longNamewe use "Machine "+machine name+" and Compiler "+compiler name, unless compiler name is not provided; in that case we use the string "Machine "+machine name+" and an unspecified compiler"
platform/ extra information0..1platform/ description
machine name1platform/machine/machineName
contact (responsible party)0...1platform/contact (see responsible party table)
Maximum Processors0...1platform/machine/machineMaximumProcessors
Cores Per Processor0...1platform/machine/machineCoresPerProcessor
Processor type0...1platform/machine/machineProcessorType[@value]
Interconnect type0...1platform/machine/machineInterconnect[@value]'
Hardware (type)0...1platform/machine/machineSystem
Operating System1platform/machine/machineOperatingSystem[@value]
Compiler Version1platform/compiler/compilerVersion

responsibleParty translation

QuestionnaireCardinalityCIM1.5Notestds tag or attribute
Responsible PartiesresponsibleParty
type(Contact,Lead Author,Funder,...)responsibleParty/gmd:CI_ResponsibleParty/gmd:role/gmd:CI_RoleCode@codeListcould be used: used:
type(Contact,Lead Author,Funder,...)responsibleParty/gmd:CI_ResponsibleParty/gmd:role/gmd:CI_RoleCode@codeListValuevalue examples: distributor, originator, pointOfContact,principalInvestigator,... from:
uriresponsibleParty/CommentOutput questionnaire uri as a comment in case it is of benefit in the future.
Name [if isOrganisation==false]1responsibleParty/gmd:CI_ResponsibleParty/!gmd:individualName/!gco:CharacterStringUse individual name if isOrganisation is falseinstitute or institute_id
Name [if isOrganisation==true]1responsibleParty/gmd:CI_ResponsibleParty/gmd:organisationName/gco:CharacterStringUse organisation name if isOrganisation is trueinstitution
Abbreviation1responsibleParty/abbreviation[ME - this is a mandatory field in the Q form - I think the system should derive the abbreviation to keep it simple]

simulation translation

shortName1simulation/shortNameif ensemble members >1 we append "BaseSimulation" to the shortName and the shortName is used as the ensemble short name
longName1simulation/longNameif ensemble members >1 we prepend "Base Simulation of Ensemble " to the longName and the longName is used as the ensemble short name
Model1simulation/model/reference (see reference table)reference to a model component
model modsimulation/model/reference/changea changeset is added here for each model mod that is associated with this simulation. This is irrespective of whether the model mod is used for a conformance or not. Therefore all model mods associated with a conformance will also appear here as well as the ones that do not. This means that model mod information is replicated.
Platform1simulation/deployment/reference (see reference table)reference to a deployment
Ensemble Members ==11There is no ensemble
Ensemble Members > 11An ensemble document will be created in which each of its members (equal to the number of members specified here) will reference this simulation. See ensemble table. Also note associated changes to the shortName and longName in this case.
Author1simulation/commentNot sure how to handle this [ME - Q allows three parties to be defined - contact is the key one; I'm not even sure what an author is in the context of a simulation]
simulation/supports/reference (see reference table)reference to an experiment. This is already set up in the questionnaire
Conformance0...nconformance (see conformance table)a conformance to a numerical requirement
simulation/ calendarcalendar information taken from the associated experiment.
Duration/endsimulation/endPointOne of end or length must be chosen but not both
Duration/lengthsimulation/durationThe value of length and the units of length are combined to make an xsi duration type e.g. 1 year is D1Y. We still have the problem that the value is a float and I need an integer.
Related Simulation [value = extends,fixedVersionOf,other]0..1simulation/ documentGenealogy/ relationship/ simulationRelationship/ @type="value"
Related Simulation/ detailssimulation/ documentGenealogy/ relationship/ simulationRelationship/ description
Related Simulation/ simulationsimulation/ documentGenealogy/ relationship/ simulationRelationship/ reference (see reference table)
Related Simulation [value = usesSpinup]simulation/ spinupSimulation
Related Simulation [value = usesSpinup]/ detailssimulation/ spinupSimulation/ reference/ details
Related Simulation/ simulationsimulation/ spinupSimulation/ reference (see reference table)
Related Simulation [value = hasControlSimulation]simulation/ controlSimulation
Related Simulation [value=hasControlSimulation]/ detailssimulation/ controlSimulation/ reference/ details
Related Simulation [value = hasControlSimulation]/ simulationsimulation/ controlSimulation/ reference (see reference table)

composition translation

component/componentProperties/componentPropertyAdd a new component property to the appropriate input component
binding/temporalTransformation==Exact or LastAvailable? or Other or TimeInterpolation1composition/coupling/comment

conformance translation

HowConformed?=="Not Conformant"1simulationRun/conformance@conformant=falseHowConformed? is one of : Not Conformant, Standard Configuration, Via Input Mods, Via Model Mods, Via Combination
HowConformed?!="Not Conformant"1simulationRun/conformance@conformant=true
simulationRun/conformance/requirement/referenceThis questionnaire is set up so that a conformance is associated with a numerical requirement of an experiment so we can work out what it is referencing.
Model Modification0...nsimulationRun/conformance/source/referencereference the component associated with this modification
Model Modification/typesimulationRun/conformance/source/reference/change/detail@typeDoes not currently conform to the CIM enumerations.
Model Modification/mnemonicsimulationRun/conformance/source/reference/change/detail/comment
Model Modification/descriptionsimulationRun/conformance/source/reference/change/detail/description
Input Bindings0...nsimulationRun/conformance/source/referencereference the component property associated with this input binding. Input bindings have no model modification details

component translation

Questionnaire datatypeCardinalityCIM1.5CIM
notes + questions
"Special" ComponentmodelComponent/ componentProperties componentProperty'To allow for a hierarchy of properties, the questionnaire allows certain tagged properties to be treated as restricted components. These components can only have a short, a long name and a description.
"Special" Component/ ShortNamemodelComponent/ componentProperties componentProperty/ shortName
"Special" Component/ LongNamemodelComponent/ componentProperties componentProperty/ longName
"Special" Component/ AdditionalInformationmodelComponent/ componentProperties componentProperty/ description
Componentn/a0..nmodelComponent0..nAll questionnaire components are assumed to be model components i.e. none are processor components.
Component/ ShortNamestring1modelComponent/ shortName1
Component/ LongNamestring1modelComponent/ longName1
Component/ TypeCV or User Defined string1modelComponent/ type/ @value for existing components and modelComponent/ type/ [@value='Other'] for user defined components1..2All existing components have a metafor type. New components have their own type. Realm components, in addition, have a DRS type, hence the cardinality of 2, as we output these as well in the CIM.
modelComponent/ type/ vocabularyServer/ vocabularyNameAdd in a vocabulary name of metafor for metafor CV, DRS for realm names and user_defined for user defined names
Component/ Implementedboolean1n/an/aIndicates whether the component exists in the coupled model (or not). If not then all it's descendents also do not exist.
Component/ Responsible parties0..3see responsible parties table
n/an/aCIMRecord/ modelComponent/ childComponent0..nGenerated from questionnaire component hierarchy.
Component/ Attribute Container (paramgroup) name != "General Attributes"n/aCIMRecord/ modelComponent/ componentProperties/ componentProperty @name=paramgroupTreat attribute containers as CIM properties if they are not the default container. Any parameters within these containers will be contained within the associated CIM property, leading to complex properties
Component/ Attribute Container (paramgroup) name=="General Attributes"n/an/aIgnore the default attribute container i.e. if the name is "General Attributes", as this is a convenience for the questionnaire.
Component/ Attribute Container/ Constraintn/a0..1n/aconformance to these constraints will be checked in the validation stage
Component/ Attribute Container/ Attributen/a0..nCIMRecord/ modelComponent/ componentProperties componentProperty/All attributes are assumed to be componentProperties. I do not make use of scientificProperties or numericProperties.
Component/ Attribute Container/ Attribute/ NameCV or User Defined string1CIMRecord/ modelComponent/ componentProperties componentProperty/ shortName & longNameMapping name to both shortName and longName.
Component/ Attribute Container/ Attribute/ ValueCV or User Defined string1..nCIMRecord/ modelComponent/ componentProperties componentProperty/ valueA questionnaire value may contain more than one value in a "|" separated list. If there is more than one value then it has separate value elements in the CIM.
Component/ Attribute Container/ Attribute/ Value== N/An/an/aDo not output a property if its value is N/A
Component/ Genealogyn/a0..1n/a
Component/ Genealogy/ YearReleasedstring0..1modelComponent/releaseDateUse validator to make sure it is a datetime, restrict to a datetime in questionnaire, or relax the CIM rules?
Component/ Genealogy/ PreviousVersionstring0..1modelComponent/ documentGenealogy/ relationship/ documentRelationship[@type=''previousVersion"]/ target/ reference/ nameThe name previousVersion is not a supported type in the CIM
Component/ Genealogy/ Improvementsstring0..1modelComponent/ documentGenealogy/ relationship/ documentRelationship/ description
Component/ AdditionalInformationstring0..1CIMRecord/ modelComponent/ description
Component/ References0..nsee references table0..n
n/aCIMRecord/ modelComponent@ documentVersionGenerated by the Questionnaire
n/aCIMRecord/ modelComponent/ documentCreationDateGenerated by the Questionnaire when the CIM document is created
n/aCIMRecord/ modelComponent/ documentIDGenerated by the questionnaire (using uuid for the model object)
n/aCIMRecord/ modelComponent @CIMVersionAdd current CIM version
Component/ InputRequirements0..nsee composition table
grid0..1see grid tablecomponent will reference the associated grid.