wiki:CMIP5/QuestionaireBetaTesting

Comments, queries and issues raised by our beta testers

Beta testing reports are being collected by our  online survey, each survey has space for 5 reports, to log further issues simply complete the survey again. Beta testers will be returned to this page on completing the survey. The cmip5q team will check the survey for new reports daily so please expect a small delay between entering your report and seeing it here. New reports will appear at the top of the table.

You can share day to day queries about the questionnaire on our mailing list at cmip5qhelp at badc.nerc.ac.uk . Alternatively you can raise queries directly to the support team at cmip5qteam at badc.nerc.ac.uk. The beta testing period is also a test of our support infrastructure!

 Return to the open letter to our beta testers

Id InstituteIssueDescriptionLocationResponses
36DKRZ (GER)BugWhen I copy a simulation with the copy button, save this new simulation with a different name and make another copy of this new simulation I can't open this new second simulation http://q.cmip5.ceda.ac.uk/cmip5/11/simulation/12/edit/
35MOHC (UK)BugI completed an input to the Atmosphere sub-model for HadGEM2-ES and resolved it - no errors shown on the model page and the summary page indicates that it is resolved using a file. The text on the model page indicates that resolving inputs in the model is a default setting and should be used where the same input linking is used for most simulations. However when I get to the simulations page the required input that I have just resolved at the model level is shown as incomplete - with no files linked. I now have to resolve the input all over again (and remember to enter the same information - it does allow you to put completely inconsistent information in at this point). If there is no point in defining default input resolutions - why not just remove it and make the process a little simpler. http://q.cmip5.ceda.ac.uk/cmip5/5/simulation/5/edit/
34MOHC (UK)BugValidation of simulation goes to input error http://q.cmip5.ceda.ac.uk/cmip5/5/simulation/5/edit/
33MOHC (UK)OtherHaving completed one input - input resolution - conformance - file - variable process for a single variable, single file, single simulation, and single input (and a simple input at that) - I am overwhelmed by the amount of work that it is going to take to complete this task for a single simulation. Assuming I can treble my work rate as I get more proficient I am looking at another 35+ hours work to complete the conformances for a single simulation - and I have all the information from the modellers already written down. Subsequent simulations should be quicker but we are still looking at 10+ hours per simulation - is this what we intended? I would like to discuss this before committing to complete the beta testing. If we continue I can't promise to complete this work before the end of the month.
32MOHC (UK)QuestionWhen I use the files drop down it present files that I have not entered - are these standard files (which may be duplicates with the one entered) or just an artefact of the test system. http://q.cmip5.ceda.ac.uk/cmip5/5/component/848/coupling/
31MOHC (UK)BugThe drop down on Variable also contains variables that I hadn't entered - I had assumed this would be constrained by the file I selected in the previous drop down http://q.cmip5.ceda.ac.uk/cmip5/5/component/848/coupling/
30MOHC (UK)UsabilityThe information on model modifications and inputs on the conformance page is unreadable after they are entered. http://q.cmip5.ceda.ac.uk/cmip5/5/simulation/5/conformance/
29MOHC (UK)UnclearThe whole input - resolve inputs - conformance - files - variables area of the questionnaire is going to be an area of much confusion. I thought I new what I was doing on this, but it still took me two hours work to resolve a single conformance based on a single input, single file and single variable. I was especially confused on where to put a specific description of inputs - we will need to be very explicit on which bits of the puzzle go into which boxes. Actually I need to be very clear about this UNLESS information is gathered in a very prescriptive way (which we will need to carefully document) it will be an awful job trying to complete this information with any useful degree of rigour. http://q.cmip5.ceda.ac.uk/cmip5/5/simulation/5/conformance/
28MOHC (UK)Usability"What is the information in the question ""mostly used for experiment"" use for - is it necessary. A good proportion of the variables/files I am defining are used for 80%+ of the experiments. This will require me to tick 85+ boxes for each file. A ""select ALL button"" would help - but not that much. Why are we asking the user to do this when they will be providing this information through the conformance process." http://q.cmip5.ceda.ac.uk/cmip5/5/list/file/
27MOHC (UK)UsabilityFiles - only accept one reference. Doesn't work well for us - especially if we want to combine variables into a 'file'. In effect restricts us to one variable per file (and even then is still a little restrictive) http://q.cmip5.ceda.ac.uk/cmip5/5/list/file/
26MOHC (UK)OtherAre the additional authorisation screens necessary on the open_id dialogue. For our purposes just the login screen is necessary - one fewer thing to explain to usersfirst page after selecting institute
25MOHC (UK)UsabilityIt is not clear to me (and others) how we use the model part of the questionnaire to deal with different model configurations. For example how do we differentiate between HadGEM2-ES used with its full configuration for one set of runs and HadGEM2-ES used in its atmosphere only config for the AMIP runs. One school of thought is we describe each major configuration as a separate model (copied from a master) - indicating which components are included for each configuration. This works but may be hard to maintain as we will have multiple copies of the same descriptions. I think this was how it was supposed to work. However Karl and Eric favour all model configurations having the same name. Logically this would mean that we need to be able to indicate which components are used/not used for a specific simulation. The concept of model mod could be used for this - though it doesn't really do it well at the moment. Obviously a change to the data model could overcome this difficulty, but probably can't be accomodated now. So we need an agreed work around that all users will adopt. Note: currently the names given to models are not constrained by the agreed names provided to PCMDI - is this an issue for consistency with the netCDF metadata.
24MOHC (UK)"3.2.stc.1850_156yr:None - What does this requirement mean? Is the question meant to be - ""have you run for 156 years?"". I will answer ""not compliant"" to this as we only run for 146 years on our historic." http://q.cmip5.ceda.ac.uk/cmip5/5/simulation/5/conformance/
23MOHC (UK)Initial conditions are from experiment 3.1.piControl The initial conditions are defined by a dump file from the control run - I don't want to have to list every single variable in the dump file (i.e. all atmosphere variables at all model levels + loads of diagnostic values. Our approach is pretty common. Can we not have an option for a dump file from a parent run. This would then be consistent with the internal netcdf metadata. http://q.cmip5.ceda.ac.uk/cmip5/5/simulation/5/conformance/
22MOHC (UK)Bug"ran ""Validate"" and got an internal error page" http://q.cmip5.ceda.ac.uk/cmip5/5/simulation/5/edit/
21MOHC (UK)Vocabulary"I am entering properties for a small number of components. Generally easy to do. On this page I found: - the questions in each group were almost in the reverse order of the previous version (any reason for the change - doesn't particularly matter in this case, just wondered why it changed?) - the multi option tick box solution works well but could do with a little more space between the tick box and the text - the single option drop down might be better just as a simple drop down rather than a radio button - but it works ok - in vegetation types had to resort to ""other"" - we need ""scrub"" as an option - we couldn't understand the ""describe the treatment of vegetation biogeography"" - could we have a better description of what you are looking for." http://q.cmip5.ceda.ac.uk/cmip5/5/component/885/edit/
20MOHC (UK)BugI tried to complete the properties on this page and it gave me an internal error - repeatedly. I tried then to enter properties one section at a time. - bizarrely the questions were asked in different order as I refreshed the page (not good) - it worked one section at a time http://q.cmip5.ceda.ac.uk/cmip5/5/component/852/edit/
19MOHC (UK)Unclear"The question AerosolSchemeScope? has options of ""yes"" or ""no"" - not sure how to answer that! Question also repeated below" http://q.cmip5.ceda.ac.uk/cmip5/5/component/845/edit/
18MOHC (UK)OtherSummary: basic approach to completing information is fine. There appears to be some flakiness in the display order of the questions and there is at least one duplicate question. This is really going to need a very thorough review on each page. I don't have the time to complete all of the properties - sorry - and I can only do a data ask from the scientists once, so it has to be stable. I think it will be OK to extend vocabulary options during operations as long as we publicise it.
17MOHC (UK)Usability"If you tick the box to indicate that ""my model does not include this component"" - it still requires that the long name for the component is completed. If the component is not included can the entry validation checks be skipped (or at least not reported)" http://q.cmip5.ceda.ac.uk/cmip5/5/component/861/edit/
16MOHC (UK)Vocabulary"Why do we have a subcomponent to a key properties component (see Atmosphere). This is confusing and has no physical reality. Would be better to combine the properties from the lower level component (Top of Atmosphere Insolation) into key properties component (or in this specific case into Atmosphere Radiation component). This comment should not be used to imply that I support the ""key properties"" component . . as I previously stated I would prefer the properties to be consolidated into the top level component."
15MOHC (UK)QuestionWhy is component called River Routing when the scheme used throughout would imply it should be called Land Surface River Routing
14MOHC (UK)UsabilityWhen you have a large number of references - I would expect we will end up with around 160-200. It can be hard to quickly distinguish between references from the same author, and scrolling to find a reference can become a pain. It would be nice if a tooltip could give you the full reference onmouseover and maybe three columns would help with the scrolling http://q.cmip5.ceda.ac.uk/cmip5/5/assign/reference/component/886/
13MOHC (UK)Other"The Land Surface Key Properties component asks in a short text field for information on ""BasicApproximations?"" - this either needs some clarifying text or a LOT more real estate. To describe the approximations in our land model would be several pages of text. More information on what you are looking for please" http://q.cmip5.ceda.ac.uk/cmip5/5/component/879/edit/
12MOHC (UK)UnclearWhy can't the key properties be stored under the master component rather than creating a new component. http://q.cmip5.ceda.ac.uk/cmip5/5/component/NNNNN/edit/
11MOHC (UK)UsabilityMore information on using the questionnaire. I spent 4 hours today adding about 20% of the descriptions (no input on parameters at the moment) - this was just cutting and pasting from our internal database. Generally worked well apart from the Save button - if you move away to set up a new party or add a reference - everything is lost and has to be reentered. In the applications I write I usually use sub_modal windows to capture the ancilliary stuff so that the main window remains populated - probably too much effort now, but we ought to think if there are any other approach we can use to avoid this problem
MOHC (UK)BugIf you add a new component it correctly gets populated with the top level contacts, but not with the top level grid (i.e. when boxes are ticked to auto-populate sub-components) http://q.cmip5.ceda.ac.uk/cmip5/5/component/1262/edit/
10MOHC (UK)Unclear"Purpose of long name - its a required field and complains if left blank. But we are using a longish text name for the short name (i.e. not a code) - and therefore in MOST cases it could be prepopulated to the short name but allow the user to over write if they want to have an extended name" http://q.cmip5.ceda.ac.uk/cmip5/5/component/1262/edit/
9MOHC (UK)UsabilityTo provide some reference on the time taken to complete the questionnaire I will add some metrics. In the last day or so I have entered the 114 references used to describe HadGEM2-ES. This took me around 5 hours. Many organisations will already have their references in a digital form - a CSV import option would be a very useful way to reduce the time spent working on the questionnaire. If this is not possible - would it be possible to change the entry process so that I can enter one after another rather than having to go back to the reference list screen, scroll to the bottom and then enter the next one. This gets to be painful once you have a more than a screenful of references in the list. http://q.cmip5.ceda.ac.uk/cmip5/5/list/reference/
8MOHC (UK)OtherSome of the references in the MOHC list are nothing to do with our model (and doesn't appear to be generic to CMIP5 project) - I assume its an artifact of setting up the beta test. I can't delete these references as they don't have a (delete) option. http://q.cmip5.ceda.ac.uk/cmip5/5/list/reference/
7MOHC (UK)OtherWhen using the validation option for a model the questionnaire presented a standard internal error page. Would be nice to have a back button to go back to where I was.
6MOHC (UK)UsabilityWhen entering a party could you default the abbreviation to be the same as the Name field - unless the user specifies something different. http://q.cmip5.ceda.ac.uk/cmip5/5/list/parties/
5MOHC (UK)QuestionI think I understand the use of the three contacts at the model level - I don't understand what they should be used for at the sub-model level. Should the funder be used for the government department that provides the funding for the model (tricky for us as it is spread across three departments) or the 'owner' of the model (e.g. MOHC). I think we need clear definitions of what the three contacts are to be used for (in tooltips if possible). http://q.cmip5.ceda.ac.uk/cmip5/5/component/843/edit/
4MOHC (UK)UnclearWhen specifying the number of maximum processors and the number of cores per processor - are you looking for the total size of our compute facility, the maximum size of the resources used to run our model, the typical size of resources to run our model for CMIP5, or the minimum size of resources necessary to run the model. We are running different simulations with different resource levels. http://q.cmip5.ceda.ac.uk/cmip5/5/platform/3/edit/
3MOHC (UK)UnclearIf we have different grids for the Ocean and Atmosphere sub-models are we supposed to leave the Grids question at the model level (top level component e.g. HadGEM2-ES level) blank? http://q.cmip5.ceda.ac.uk/cmip5/5/component/842/edit/
2MOHC (UK)UnclearHybridization - what kind of values were you expecting here? A one-line summary of the hybrid vertical coordinate scheme perhaps? http://q.cmip5.ceda.ac.uk/cmip5/5/grid/40/edit/
1MOHC (UK)UnclearTopModelLevel? - for our 38-level model I was tempted to enter 38 here! But this merely duplicates the NumberOfLevels? property. Therefore I suspect that this field is in fact asking for the vertical coordinate value at the top of the model, which is what I've entered. Is that what you intended? http://q.cmip5.ceda.ac.uk/cmip5/5/grid/40/edit/

bug

version is: DRAFT CMIP5 Metadata Questionnaire (rc 1)

short description: try to create a new simulation under experiments, choose button 'new' fill in the needed fields and press 'save', result: 'Internal Error'

reporter: hans