Metafor Issues Document

This document is intended to give an overview of the major issues experienced by the Metafor team in general terms. It is anticipated that this will be useful for other projects attempting similar things to Metafor, allowing them to see the issues that we spent a lot of time discussing, and what our decisions on them were.

1) Breaking the CIM up into packages.

This needs to be done, because to get the CIM completed in reasonable time, it requires multiple people to work on it. Packages are the sensible way of diving the CIM up into reasonable sized chunks, and can provide an over-arching structure for the CIM.

However, the downside is that different packages may wind up dealing with the same concepts in different ways, resulting in clashes and/or over-complication.

The working solution Metafor used for this issue was the appointment of a CIM architect, who was responsible for keeping an eye on all the development done in all the packages, making sure all the packages fit together properly. Also, the project team had weekly teleconferences to discuss CIM issues and active discussions on the Metafor email list.

2) Syntax and Semantics

This is still an on-going issue, and it's vital to be sure that everyone knows exactly what is meant by the CIM terms (e.g. ModelComponent?, TechnicalProperty?, Activity etc).

This will be especially important later on, when producing the training material to train other researchers in how to use the CIM.

3) What is a model?

At the simplest level, a model is something that takes data in to produce an output. By this definition (data_in+5) in the following is a model:

data_out = (data_in+5)

The general perception of models in the climate change community is of a large and complicated mathematical structure, expressed as computer code. Hence we reserve the term "model" for something more complex than (data_in+5).

Confusion arises when we try to define what level of complexity is required for something to be awarded the status of model.

4) Translating UML into xml

This translation process has thrown up a lot of issues. In general it's been helpful for identifying problems with the UML, but a lot of time has been spent trying to come up with almost-automatic ways of doing the conversion.

It's helped us to have the CIM architect be the sole person doing the translation, as they then can make sure everything ties together as it should.

Providing instances to the CIM architect has also proved to be useful during this process.

5) Definition of model properties

There was a lot of discussion on whether model properties should be further divided into technical/scientific/numerical properties.

There were many suggestions to make technical and numerical properties specialisations of (generic) model properties. Each property might be realised by a name/value pair. And the names of (specific) technical and numerical properties may be hard-coded. This has the advantage of making the model much simpler, and of allowing any type of property to be coupled.

6) When does a tweaked model become a new model?

Most climate modellers work by taking a model from another source (person or place) and modifying it slightly to fit their own experiments. This can lead to the problem of versions of models having the same name, but being slightly different, and these differences not being properly documented. The "Change property" was proposed to document these differences.

Over time, models can diverge from the original, while still being assumed to be the same as the original.

Tracking authorship and provenence of models can help reassure the user that they've really got the model they think they do.

7) We've worked out how to put data into our CIM but not how to get it out

We need to think about how people will be using the CIM to get to the data that the CIM is describing. So which classes need to be liked to elements in the data package. We're doing ok at thinking about how we put data in. Thinking about how to get data out shouldn't be an afterthought.

8) Working procedures

• For the CIM definition, we started with informal discussions on a forum. This followed a "bazaar" approach, which took a lot of time to follow the forum and no working procedure was established to summarize or draw conclusions out of the informal forum discussions. The conclusion is that the "bazaar" approach could have been more successful if someone was officially charged of summarizing the forum discussions, proposing conclusions, making sure that everybody agreed, and writing them down for futher reference.

• The relationship between the different project sites (wiki, trac joomla etc) needed to be clarified earlier in the project. The (joomla) website is now used for more formal documentation (minutes, meeting arrangements etc) while the wiki and tickets are used for on-going project notes and discussions. The wiki page summarising the rules for working on the trac and wiki is  http://metaforclimate.eu/trac/wiki/admin/metaforRules

9) Use cases (and how we use them)

Use cases were defined at the beginning of the project, but the level of detail that should put in the use cases and their relations to the "requirements" should have been better specified. The question raised is "Do the use cases feed the requirements or is it the other way around?"

Bryan gave the following in answer to this question:

...I think we all do something like this:

1) we're in this project because we know that there is some functionality we, or folk we know need. That is, we have a poorly defined use case ...

2) ... but we've thought about it a bit, and so we know that the use case will require a specific piece of functionality, and so

3) we need to specify it, so we specify the functionality/requirement, and then harden the formal use case to match the functionality/requirement we know/knew it needed.

and *often*

4) we find that our requirements themselves are modified and/or extended in the process, and we cycle through 3-4-3-4 a few times.

Clearly of course we never *formally* do 3-4-3-4 over and over again, but sometimes in a group activity it is worth exposing the thinking ... some of those times ... 

which is just to totally agree that use cases are not simple (but because they exist in  a continuum of activities and we tend to forget that in formal project management, and especially at the beginning of projects).

The conclusion for this document is that use cases are not necessarily simple to work with!

10) Examples of the CIM (CIM instances) and sequence diagrams

Instances and sequence diagrams explain how the CIM will get used (and highlight areas where the CIM gets broken by specific model examples). How these are useful needs to be well-communicated to the project team (especially if they're not used to generating these things)

Edits and dates

Sarah and Charlotte, 2nd December 2008

Charlotte Dec 8th 2008

Sarah, 24th March 2009 from email discussions with Sophie and Bryan