Welphi as tool to inform the shape of value functions– 2nd step in the process of an index creation; Combining the multi-criteria method MACBETH (Measuring Attractiveness by a Categorical Based Evaluation Technique) with Web-Delphi social processes.

Nowadays the process of creating an index, or the process of decision making, has drawn-out of its classical approach, having to consider aspects such as multiple criteria and stakeholders’ opinions in the same scope, which usually leads to a satisfactory outcome rather than an optimal one. 

Such setups have been recognized as requiring innovative ways of combining evidence with the views and values of multiple stakeholders like decision makers, experts and ultimately the population affected by future decisions. This evolution and consciousness regarding index creation, and decision making, features has led to the Multi-Criteria Decision Analysis (MCDA). 

MCDA has been defined as “an umbrella term, which describes a collection of formal approaches that seek to take explicit account of multiple criteria in helping individuals or groups explore decisions that matter”. MCDA is a structured framework with two components: a technical and a social component. The technical component entails the employment of a set of technics to support the different steps of the development of an index creation, or multi-criteria evaluation model, whereas the social component is meant to capture the points of view of the participants involved, in order to create a “shared understanding of the issue”. 


Welphi has proven to be a great game changer ensuring the social strand of these work frames, beening a rich and effective way to collect information from an enlarged and geographically dispersed number of participants. Having set the necessary structuring work leading to the structuring components, i.e indicators or criterion, to be considered in an index creation, or decision making process, follows the task of articulating and modeling preferences.  

Let us resort to our latest introduced Welphi case study – The Euro-Healthy project – to broadly explain how Welphi was used to articulate and model preferences, in a 1^st instance accessing panelists’ views about the added value of improvements in different levels of indicator, informing the shape of the value functions resorting to the MACBETH (Measuring Attractiveness by a Categorical Based Evaluation Technique) socio-technical methodological approach, a novel methodology specially developed for the Euro-Healthy project’s needs.

In the scope of the Euro-Healthy project, and at this stage, Welphi processes, web-based Delphi processes, technically sound in MACBETH were conducted to inform the shape of value functions. The MACBETH based intuitive protocols of questioning were key to promote transparency along the processes, avoiding the eventual difficulty and cognitive uneasiness experienced by evaluators when trying to express their preference judgments numerically. These processes were developed in three sequential rounds. Along the three rounds, an “identity card” for each indicator was always available for on-line search of information and scientific evidence, as was also always available, for each indicator, the range of performance across the European regions. This was done resorting to Welphi’s features which you can find more about at http://support.welphi.com/article/add-indicators/. 

|Aim: To determine the shape of the value function on each indicator.

|Objects of study: A set of previously selected indicators, through Welphi processes. (https://welphi.blogspot.com/2019/01/welphi-for-selection-or-identification.html).

|Panel: 58 experts and stakeholders from different countries, distributed by four panels as follows Socio-economic, Demographic change and Health behaviors, Physical and Built environment and Healthcare and Mortality, according to their area of knowledge and expertise. A common design for the Welphi's value function processes was implemented simultaneously with the four panels.

|1^st round: The range of performance on each indicator was divided in three equal pieces representing three changes in performance. For each indicator, panelists judged the contribution to population health of each one of those three changes in performance, answering the question:

“To improve population health in Europe, what is the contribution of this change in performance on the indicator?”

Panelists’ answers were provided with the MACBETH qualitative judgement scale (from very weak contribution, to extreme contribution). A “Don’t know/Don’t want to answer” option was available and comments could be given on each indicator. To the sequence of three judgements given by each panelist as attributed an implicit a value function shape on each indicator.

|2^nd and 3^rd rounds: In the 2^nd round panelists were presented with the implicit value function of their first round judgments for each indicator. Moreover, feedback with the results of the 1^st round was provided on each indicator: the number and percentage of respondents for each type of implicit value function shape and the comments made. In this 2^nd round, panelists were given the chance to either confirm or change their implicit value functions shapes, at the light of the group information provided. At the beginning of the 3^rd round, panelists received again information synthesizing the answers of the panel after the 2^nd round. Accordingly, these had the opportunity to maintain or revise their answers.

|Results: The results of the process, along with other project’s outcomes guided a strategic group to set the value functions for the indicators in a decision conference.

Check out our support page (http://support.welphi.com/video-tutorials/) and watch our video tutorials to help setup your Welphi process today for articulating and modeling preferences towards your project’s goals, whichever your framework might be!