Welphi as tool to understand weighting coefficients – 3rd step in the process of an index creation; Combining the multi-criteria method MACBETH (Measuring Attractiveness by a Categorical Based Evaluation Technique) with Web-Delphi social processes

Following up the introduction made in our last article (https://welphi.blogspot.com/2019/01/welphi-as-tool-to-inform-shape-of-value.html), this week we pick up where we left off, broadly explaining how Welphi was used in our latest introduced Welphi case study – The Euro-Healthy project – to articulate and model preferences. This time Welphi was used to explore panelists’ views about the importance of closing gaps in indicators to qualitatively understand weighting coefficients. 

In the scope of the Euro-Healthy project Welphi processes, web-based Delphi processes, technically sound in MACBETH were conducted to qualitatively understand weighting coefficients. Once more, the MACBETH based intuitive protocols of questioning were key to promote transparency along the processes, avoiding the eventual difficulty and cognitive uneasiness experienced by evaluators when trying to express their preference judgments numerically. These processes were developed in three sequential rounds. Along the three rounds, an “identity card” for each indicator was always available for on-line search of information and scientific evidence, as was also always available, for each indicator, the gap between the worst and best performance across the European regions. This was done resorting to Welphi’s features which you can find more about at http://support.welphi.com/article/add-indicators/.  

|Aim: To collect qualitative weighting judgments on each indicator.

|Objects of study: A set of previously selected indicators, through Welphi processes (https://welphi.blogspot.com/2019/01/welphi-for-selection-or-identification.html).

|Panel: 58 experts and stakeholders from different countries, distributed by four panels as follows Socio-economic, Demographic change and Health behaviors, Physical and Built environment and Healthcare and Mortality, according to their area of knowledge and expertise. A common design for the Welphi weighting process was implemented simultaneously with the four panels.

|1^st round: Panelists were presented with a list of indicators and their gaps between the worst and best performance across the European regions. Each panelist individually was asked to answer a question concerning the gap on each indicator. The question was:

“To reduce inequalities in Europe, how important is to close this gap?”

While answering panelists considered “how big is the gap and how important it is to develop policies to close the gap in view of reducing health inequalities in Europe”. Panelists’ answers were provided with the MACBETH qualitative judgement scale (from not important, to extremely important). A “Don’t know/Don’t want to answer” option was available and comments could be given on each indicator.

|2^nd and 3^rd rounds: The extracted results of the 1^st round were presented at the beginning of the 2^nd round, allowing panelists to maintain or revise their answers.  At the beginning of the 3^rd round, panelists received again information synthesizing the answers of the panel after the 2^nd round. Accordingly, these had the opportunity to maintain or revise their answers.

|Results: The results of the process, along with other project’s outcomes guided a strategic group to assign quantitative weights for the indicators in a decision conference.

In the scope of the Euro-Healthy project, the outputs of both our introduced and distinct Welphi processes, to inform the shape of value functions and to understand weighting coefficients, together with scientific evidence collected, latter informed a decision conferencing face-to-face process in which a strategic group composed of 13 members, representing the diversity of viewpoints related to PH and covering the different areas of expertise and interest, successfully participated in the process of building the PHI model. The result was a set of multi-level PH indices based on the hierarchical multicriteria model. The PHI model underwent a set of testing, adjustment and validation procedures by the strategic group, until the PHI model has shown to be able to be a proper tool to assess PH across European regions.

Check out our support page (http://support.welphi.com/video-tutorials/) and watch our video tutorials to help setup your Welphi process today for articulating and modeling preferences towards your project’s goals, whichever your framework might be!