What variables are used to check the reliability of an MMM model?

Eulerian Customer Success teams act as advisors to help you select the best MMM model among the different iterations of models offered.

  • For each Client, the Eulerian platform will create multiple iterations of models using different combinations of variables, smoothing, and transformations. Each iteration will produce slightly different results.

  • The Eulerian platform will evaluate the performance of model iterations using validation metrics:

  • R² (Coefficient of Determination): Measures the proportion of variance explained by the model. The higher the R², the better, but avoid overfitting the model.

  • NRMSE (Normalized Root Mean Squared Error): Indicator of the accuracy of the model by normalizing the prediction errors. A lower NRS indicates better accuracy.

  • Cross-validation : Eulerian uses cross-validation to assess the robustness of the model. The platform ensures that performance remains stable across different cross-validation cuts.

  • Decomp.rssd (Decomposition Root Sum of Squared Differences) : The decomp.rssd is a specific metric used in Robyn to assess the stability and quality of the decomposition of contributions of model variables (such as advertising expenditure) relative to the total sum of contributions.
  • The decomp.rssd measures the deviation between predicted contributions and expected (or average) contributions over a series of simulations or iterations. A lower decomp.rssd indicates better stability and consistency in the distribution of the effects of the explanatory variables.
  • This metric is particularly useful for checking the robustness of the marginal contributions of the model and for ensuring that the model does not exhibit large variability in the decomposition of effects during simulation iterations.