Telco 147 10th May 2012

Attending: Sarah, Eric, Hans, Gerry, Charlotte, Sylvia

Apologies: Mark M

Next telco:
to be arranged in 2 weeks time Doodle poll at

New Actions

  • Sarah - send full BAMS and Clivar papers and citations to the list.
  • Gerry - produce new questionnaire page with additional questions.

Previous Actions

  • Bryan - come up with a proposal for cover page portal - ongoing
  • Mark (ES Doc) - come up with long term roadmap for new portal - ongoing - Bryan in charge of this
  • Sebastien - email the Ensembles modelling group to let them know the data is being put into the Ensembles questionnaire -ongoing
  • Sophie and Marie-Pierre - look at re-structuring the CV paper - ongoing

Completed actions

  • Mark, email Gerry - expose re-published documents at a different url - done
  • Eric - send revised version of BAMS article and response to reviewers comments to the list - done
  • Gerry - arrange time for demo session of Ensembles questionnaire - dropped
  • Gerry and Bryan - have a talk off-line about user authentication in the Ensembles questionnaire - dropped
  • Mark M - establish a starting list of what's needed for portal to be production ready - done
  • Sylvia - summarise things that need to happen for portal to be production ready - done
  • Gerry - send Sylvia link to questionnaire stats collected in the questionnaire - done
  • Gerry - email list with example of use of RIP value to distinguish between decadal runs, in the form of a white paper - done
  • Charlotte, Eric, Nicola - find out whether PIMMS is footing the bills for the telcos - email discussion started - done
  • Bryan - take a first pass over the GMD papers, then set up a telco to discuss - done
  • Gerry and Charlotte - set up wiki page to list all question requests to add into the questionnaire - done
  • Eric and Bryan - discuss new questions issue with Karl and send him a draft email to questionnaire users asking what questions they'd like to ask - ongoing - no point bothering Karl at the moment, so draft email and send it to Karl to approve - done


  • Telcos being funded by NCAS contingency until IS-ENES2 takes over.
  • Have telcos every 2 weeks from now on - Charlotte and Sylvia to coordinate.
  • questionnaire news (Charlotte, Gerry)

Questionnaire - can only use an model name/institution name that conforms to the DRS.

Simulation documents coming out of the questionnaire have start year appended to allow 1 to 1 mapping between data and metadata.

For linked experiments (e.g. decadal and decadal-i) users were expected to manage RIP values themselves. New system ensures that users don't replicate RIP values when documenting similar experiments.

Documents in the feed already have been migrated. Model documents all migrated. Simulation documents re-published for the Met Office. Moving onto other simulation documents - slight differences between official names and names given by the modelling groups.

Questionnaire is a bit tempermental at the moment. Pages hanging etc. In touch with BADC to sort this out. Think it's the underlying server.

Additional questions page in the questionnaire - still having questions added. How best to put these into the questionnaire and ingest this new info into the CIM documents.

Need to store forcings table in database so that it doesn't get lost.

PIMMS We're beginning to scope out requirements for a tool to link data and metadata. We've got Stephen Pascoe on board for this but we expect that Metafor may well want to have some input. Here's the link to the relevant wiki

Up for decision as to whether we use one or many methods to link data and metadata. Could give people a suite of ways, and they could decide according to the requirements of their system.

Hannah has managed to load the documents that she created by harvesting metadata from papers into the CIM viewer (The metadata harvester uses ChemicalTagger? software and the Metafor CV to parse text in papers). This has brought up a few errors that need fixing (eg the viewer only lists the first variable in a list) but also the more interesting question of how to interpret harvested CIM documents vs those created by modellers. The thing is that the CIM was designed to be populated by modellers with the assumption that if something isn't in the CIM document then it isn't in the model. But CIM documents created by harvesting information from papers will naturally not cover everything about a model, so missing info doesn't mean that those things weren't included. So PIMMS will need to describe different protocols for interpreting CIM documents depending on how they were created, but we probably want to ensure that that CIM accounts for missing data more intelligently in future releases.

- WP3 CIM population and testing news (Sébastien)

  • implementation of ENSEMBLES questionnaire

Questionnaire is set up and ready to be used.

  • ES-doc news (Mark M and Sylvia)

Marks' deliverables:

  1. CIM Viewer

1.1 Development continues and is moving forward. 1.2 Integration with the web-service API to the backend database is complete. 1.3 The view rendering engine has been extensively refactored and looks good. 1.4 Target go live date is 1st June so as to permit esgf p2p system to go live if stakeholders concur.We expect to have a 1.0 Version of the Viewer ready by 1 June. For us, 1.0 means that it will render all of the CIM documents in a simple way. Whether we allow others to install and use the software at that point will need to be decided by the PIs. Follow on work will still be required to merge information across documents to present the information in a more user friendly way.

  1. CIM web service API

2.1 DRS api has been implemented - this supports resolving a CIM document set from a CMIP5 dataset id (essential functionality). 2.2 Rest api is fully implemented for document, ongoing for document set.

  1. es-doc portal (formerly CIM Portal)

3.1 CIM portal has been rebranded es-doc portal. 3.2 Look & feel is being updated to reflect re-branding. 3.3 Home page text needs to be specified. 3.4 Target go live date is 1st June - the portal also hosts the api hence this date is linked to point 1.4 above. 3.5 Discussions with Martina from DKRZ have been held in order to clarify DOI landing page requirements.

  1. es-doc process

4.1 Management process is being established leveraging COGS, PivotalTracker?, GitHub?. 4.2 Sylvia and I have reviewed user stories in Pivotal Tracker - comparison stories have been added recently. 4.2 Stephen Pascoe has agreed to be a formal code reviewer (python). He will deliver first report next week.

  1. Other

5.1 Peter Fox is this week working on a CMIP5 visualisation application - when CIM api is released he may integrate this into the visualtion application. 5.2 Coding sprint - doodle poll is being set up to elicit interest in an is-enes coding sprint with an es-doc track.

Other deliverable is the auto-generated questionnaire based on the CIM. Software can cope with model components, simulations, grids and a few other bits. Toggles for users to select sub-objects. Adding in nice features to get it to work.

Customising it for a dynamical core project being run in the US in the summer. Also customising it for CASCADE.

Baseline code is available on github.

Gerry helping test the CASCADE version.

URL plan submitted to group - need buy-in from the group.

Transition plan from gateway to peer-to-peer software and plugin.

Meeting every other week at telco timeslot on Wed. Es-doc mailing list also available. Next meeting next Wed 16th May, 16:00 London, 17:00 CET.

  • Dissemination news (Sarah)

GMD papers sent to list for comment and feedback received. Paper not submitted yet.

BAMS paper resubmitted, needs some format tweaking before it goes back for review.

  • Misc

IS-ENES2 funded, with a 20% cut in funding.