If you’re familiar with simulation models you will undoubtedly recognize this… Once the simulation model is made available, you dive into it with the usual enthusiasm. Selection options are traced and (dynamic) input fields and result areas are located. At first glance, everything looks flawless. The model seems to be working well  and the supplier of the simulation model is valued as a professional modeler. Fortunately, usually this is the case, especially when the model has been built recently, using a primary dataset.

However, additional indications can be approved over time and/or additional clinical insights are gained, generating adjustments of the initial model.

The model therefore goes through a life cycle in parallel with the regulatory processes. To this end, clinical data are updated and adjustments are needed. Additional comparators are defined and integrated. As a result of all these adjustments, flaws can quickly creep into the programming.

Selection options are shortened or expanded as the developer literally adds or removes lines or columns… And a beauty turns into a beast. Quite disturbing to discover there are still residual fractions of earlier models hidden in the most recent model, or that the titles suddenly appear several lines above tables, or have been cut out.  Tight numerical series are frequently repeated into series with no logic.

Some vendors like to give almost every cell a cell name. Beautiful and fancy but we doubt whether there’s always an added value, especially when long (sometimes cryptic) cell names are used. Our recommendation would be to limit the use of cell names, and, when needed, to use simple short names. It would for instance be possible to work with conventions and use certain letters (eg. ‘c’ for cost and ‘n’ for number). Additional frustration occurs when cell names are no longer valid or  have a different meaning throughout the life cycle.

Models that have already undergone multiple update cycles almost invariably include historical ‘scars’ due to suboptimal model management. Someone with modeling experience can confirm that some data is generated from numerous former steps spread over multiple worksheets, to  sometimes land at a final source cell nearby the adjacent cells. Pretty frustrating when it becomes clear that the whole loop could be covered in one step! The record we experienced as a consultant was a loop of 14 steps! Our recommendation is to use one single source cell and to refer to that single source cell while avoiding ‘clones’ as much as possible.

Evidence of historical manipulations can also be found in hidden cell properties. For example, when using a blank cell to make some extra notes or to validate a calculation, you expect the same font as previously, yet sometimes the cell value appears in a different color, font or property, or even completely outside the style of the general model. In such case, we are almost certain that an underlying template (or model for another customer) has been used. This is not a technical problem as such, but it does create a kind of mistrust.

Occasionally, complex VBA scripts are used. All well and good, but a heart attack is imminent when observing unexpected end results that only become visible after ‘running’ a macro. As soon as a model is received, one of the first actions is to check the ‘health’ of the model by running the macros and putting the logic of the outcomes into perspective. An additional frustration arises when the macro takes several hours to generate a result. One reason may be that other macros are called up within the macro. Avoidable in our opinion. It can save many (over)hours. A probability analysis should take up to 15 min. and certainly not 6 hours.

Everything that benefits the speed or interpretability of a model falls under the heading of ‘model hygiene’. In a first step, we always check the physical health of a model and test the flexibility to make (simple) adjustments.

We recommend, wherever possible, to simplify ‘models’ in order not to unnecessarily burden the recipient with model constructions that are difficult (or impossible) to interpret. As you  see, many frustrations for you and the team can be avoided by dealing appropriately with models.