Friday, May 24, 2019

The Versatility of the Delphi Technique and of Welphi (inherently)



As originally developed by Dalkey and Helmer, the Delphi technique was designed for the technological forecasting of future events. Until today, however, a review of the literature indicates that it is considered a reliable qualitative research method with potential for use in problem solving, decision making, and group consensus reaching in a wide variety of areas. [1]

Delphi has been applied in a large number of domains including academia, administration, agriculture, automotive, banking, criminal justice, economics, education, environmental studies, finance, health care, housing, insurance, management, real estate, sales, strategic planning, tourism, training, transportation, and utilities. Some problem areas addressed using Delphi include project evaluation, sales forecasting, energy generation, programmer productivity, science and technology planning, urban analysis, bank automation, policy and trend analysis, planning regional health services, mental health care, impact of legislation and tax reforms on business and insurance companies, risk management, investment analysis, information usefulness in merger acquisition, policy formulation, insurance trend analysis, estimating the quality of teacher education, market research, financial planning, housing, office transportation, assessing police corruption, and curriculum development. [2]
Over time Delphi has continued to be a popular subject for researchers and practitioners. The number of Delphi publications peaked and continue to show a steady and sustained interest and enthusiasm in the planning and forecasting community over the last decades. [2]

Practitioners are often willing, and sometimes even eager, to modify Delphi to meet their decision-making and forecasting needs. In some cases, the modifications to Delphi are meaningful and contribute to a better understanding of the technique. [2]

Welphi does not propose a modification to the Delphi technique. Rather it adapts its classical “paper” development, e.g. resorting to printed questionnaires, to a web-based environment allowing for the development of time and cost effective Delphi processes. Welphi keeps all of Delphi key characteristics (anonymity, iteration, controlled feedback and statistical aggregation) and versatility while automating the processes and offering user-friendly dashboards, for both the administrator and the shareholders in the process, adapting the Delphi technique to the technological Era we are developing our processes in. 

Visit www.welphi.com to find out more about how Welphi can positively impact and change the your way of developing Delphi processes.

Sources:
[1] Murry, Jr.J.W. and Hammons, J.O. Delphi: A Versatile Methodology for Conducting Qualitative Research (1995). In: The Review for Higher Education, Vol. 18, nÂș4, pp. 423-436.
[2] Gupta, U.G and Clarke, R.E. Theory and applications of the Delphi technique: A Bibliography (1975-1994) (1996). In: Technological Forecast and Social Change, Vol. 53, Issue 2, pp. 185-211.

Friday, March 8, 2019

2nd round – Providing Controlled Feedback to the active participants in a Welphi process

If you’ve been following our weekly Welphi updates, it is completely acknowledged by now that the Welphi app is a tool for implementing Delphi processes. With this new update we hope to get more specific about Welphi and Controlled Feedback.

Controlled Feedback is one of the four key-characteristics of the Delphi method as a forecasting technic, and it takes action on the 2nd round of a Delphi process and onwards, in all the subsequent rounds.

In a Delphi process context Controlled Feedback is associated with the provision of anonymous feedback to all the active participants in the process, as to the number or density of responses to the question(s) in the previous rounds plus any new information/suggestion, typically in the form of comments provided by the participants. This anonymous feedback is provided through rounds to remove differences and try to streamline the opinions towards a focus topic, reducing variance in responses to the question(s), in order to improve the process’ output efficiency.

Here's how Welphi presents Feedback throughout the rounds of a Welphi process!

Starting the 2nd round, participants are presented with a screen containing all the percentages of anonymous answers provided in the 1st round of the process in a table, and are reminded of their individual answers, given in the 1st round, which appear pre-selected in a dark-grey cell. Assess to the previous round’s comments is also made available, anonymously, to the participants. To view these, participants simply have to click the “VIEW PREVIOUS ROUND COMMENTS” button, on the upper right corner of the questionnaire. By doing this, a window pops up showing participants’ comments relating to each criteria/indicator.  


In this 2nd round, participants are invited to either keep or change their answers, provided in the previous round, at the light of the group information provided. After revising its previously given answers, for a participant:

|To keep previous round’s answers: Press the “SAVE AND NEXT” button, throughout the questionnaire’s pages, to conclude the round. 

|To change the previous round’s answers: This can be done by clicking the “EDIT” button. By doing this all the answer’s options get automatically unlocked, enabling the participant to select new answers to the question(s). 

Comments can also always be provided at any point. If a participant chooses to do so, by clicking the “BALLOON” button, the balloon icon appears “painted” instead of blank, giving indication that a comment was placed for that criterion/indicator. It should be noted that a participant’s answers provided in the previous round stays highlighted in dark-grey during all the duration of the subsequent round, acting as a reminder. 

This example of providing Feedback to the participants in a Delphi process with Welphi was based on a 2nd round where aggregated data from the 1st round is presented to participants. However, Welphi always behaves in the same way for every subsequent round i.e, always showing the data from the precedent round.

Start your Welphi processes today and see for yourself these and other features that Welphi has to offer. Visit our continuously improving website at www.welphi.com and get registered today to start your Welphi experience with a free trial at www.app.welphi.com.  

Friday, February 22, 2019

Welphi - Enabling synchronicity in any asynchronous way


Time is a funny thing, isn't it? This video, by Nathan Yau, is but ONE exemple of how our day-to-day lives differ from one and other, in terms of time availability. Imagine just adding the geographical disperness to the picture as another time constrain... This and other set-ups make it hard, sometimes, to gather fundamental players, to help you cross your finish lines... Welphi is here to give you a helping hand ;) Welphi allows you to develop your Delphi processes, i.e. Welphi processes, synchronously in any asynchronous situation. Like we said, funny who time works! Check out our website for more on Welphi and it's features, including the ability to develop processes breaking time and geographical constrains, at www.welphi.com! Get in touch with us, we'd love to be a part in the process of achieving your goals! As always, stay tunned!

Friday, February 15, 2019

Designing a Welphi Questionnaire


An indicator/criteria can have several custom fields which you can create to add information about your indicators/criteria. These indicators/criteria custom fields can also be used to help you achieve your desired Welphi questionnaire design as tools to implement the overall follow and layout of your questionnaire, and to make information available to your Welphi panel of respondents.

Well, you may be asking: But how? Following is introduced a simple step-by-step guide on how you can use custom fields to help implement your desired Welphi questionnaire design!

1|Creat your custom fields: You should create your custom fields before you create any indicator/criteria. To add a field, go to the Indicators’ page, click the “ADD/EDIT COSTUM INDACATOR FIELD” button and then the “create indicators field” button. Each field must have a name and can have a description.


2|Select and customize your custom fields to get your desired Welphi questionnaire design. A custom field can be used to:

     2.1| Group indicators, making each different group to appear in a different page in your Welphi questionnaire. To do this choose the “Group all the indicators with the same value in this field. Each group will appear in a separate page.” when adding or editing your indicators field. The design achieved by this action is as shown in the screen featured in Figure 1, below. 

Figure 1 - Welphi screen, displaying groups of indicators appearing in different pages of a Welphi questionnaire. 

2.2|Subgroup indicators, creating different sections in each group page of your Welphi questionnaire. To do this choose the “Subgroup all the indicators with the same value in this field. Each subgroup will appear in a separate section.” when adding or editing your indicators field. The design achieved by this action is as shown in the screen featured in Figure 2, below.

Figure 2 - Welphi screen, displaying subgrouped indicators appearing in different sections on the same page of a Welphi questionnaire. 

2.3|Make information available on your indicator/criterion. If you don’t want to use the field values to group indicators, select the option “Don’t use this field to group indicators”. By doing this you will not be grouping or subgrouping your indicators/criterion as explained above in 2.1 and 2.2.. You will however be adding information to your Welphi questionnaire that you deem as valuable or even essential, to help your panel of respondents to provide their answers in a more informed way. You can input whichever information you see fit. For instance, you can make available tp the participants in the process descriptions of your indicators/criterion or scientific information on them. Information added to this custom field while be displayed on the Welphi questionnaire screen provided the respondents click the “EYE” button highlighted in the screen bellow (Figure 3).  

Figure 3 - Welphi screen, emphasizing the "EYE" button in a page of a Welphi questionnaire.

Note that in order to belong to the same group or subgroup, a set of indicators must have the same value in the selected field and that you can only select one field to group indicators and one field to or sub-group them in each process.

Every field can be edited later in the process by clicking the field’s name in the costum indicators field page. You can also add new fields if you want to. After creating the desired field, you can add the indicators adding values to its fields.

For more on this topic watch our video on how to add indicators on our support page at http://support.welphi.com/article/add-indicators/!

Friday, February 8, 2019

Using Welphi to build a multicriteria model for the selction of software in the oil&gas industry




Wondering if Welphi is the tool to help your decision problems while meeting corporate environments? We can help answer that question for you: definitely yes! Read below to find out how Welphi was successfully used to solve a real decision problem of the leading oil & gas company in Portugal regarding the selection and integration of a much needed software: a data integration platform.

Welphi was used in this project as a tool to collect information to latter inform the building of a multicriteria model meant to assess vendor’s proposals regarding the evaluation and selection of an Enterprise Management System, i.e. a data integration platform.  

Adopting a framework combining concepts of multicriteria value measurement with participatory processes, two Welphi processes where developed and implemented, which where both technically sound in MACBETH, in order to make clear the value system of the actors engaging in the processes these Welphi processes where used to either,
  • Determine the added value from a set of performance levels in their respective categories, i.e. within each criterion (criteria value functions).
  • Determine the partial value of each criterion considered in the model (weighting criteria).

 |Aim
  •  Delphi for Criteria value functions: Collect qualitative pairwise comparison judgments between performance scale levels on each one of multiple evaluation criteria.
  •  Delphi for weighting criteria: Collect qualitative judgments of importance of swinging between least and most preferred performance levels on the criteria.

|Objects of study: A previously selected set of evaluation criteria, determined by the company, considering both the functional and technical stands of software solutions applicable in the context.

|Panel: A total of 73 experts and future users of the platform belonging to district working areas within the company, distributed by two panels: Functional and Technical, according to their area of knowledge and expertise. A common design for the Welphi value function process was implemented simultaneously with the two panels.

|1st round:
  • Delphi for criteria value functions: For each of the considered criteria the respective performance intervals were displayed, each one representing an increase in performance. The participants were asked to answer the following question:

“With regard to this criterion, which do you consider to be the increase in preference between each two levels of performance?”

Answers were provided according to the MACBETH qualitative judgment scale (from “no increase” to “extreme increase”). The sequence of the participants’ answers enabled to extract their implicit main concerns for each criterion.
  • Delphi for weighting criteria: The participants were asked to answer the following question:

“Regarding the selection of the proposal for a data integration platform that best meets the company’s needs, suppose there is a proposal with neutral performances in all criteria. What would be the importance of improving it from neutral to good on each of the criteria?”
Answers were provided with the MACBETH qualitative judgment scale (from “no importance” to “extreme importance”).
In both Welphi processes a “don’t know/don’t want to answer” option was also available for selection and comments could be provided.
|2nd and 3rd rounds: The extracted results of the 1st round, participant’s percentage of answers and comments, were presented at the beginning of the 2nd round, allowing panelists to maintain or revise their answers. Specifically, for the Welphi for criteria value functions, participants were presented with information about the percentage of participants corresponding with each criterion and each one of the extracted implicit main concern. At the beginning of the 3rd round, panelists received again information synthesizing the answers of the panel after the 2nd round. Accordingly, these had the opportunity to maintain or revise their answers.

|Results: Resulting outcomes of this processes served as feed in information for the company to construct the aimed multicriteria evaluation model, having a strategic group make final decisions based in the provided compilation of judgments form the participants in the Delphi processes.

Friday, February 1, 2019

Welphi as tool to understand weighting coefficients – 3rd step in the process of an index creation; Combining the multi-criteria method MACBETH (Measuring Attractiveness by a Categorical Based Evaluation Technique) with Web-Delphi social processes




Following up the introduction made in our last article (https://welphi.blogspot.com/2019/01/welphi-as-tool-to-inform-shape-of-value.html), this week we pick up where we left off, broadly explaining how Welphi was used in our latest introduced Welphi case study – The Euro-Healthy project – to articulate and model preferences. This time Welphi was used to explore panelists’ views about the importance of closing gaps in indicators to qualitatively understand weighting coefficients. 

In the scope of the Euro-Healthy project Welphi processes, web-based Delphi processes, technically sound in MACBETH were conducted to qualitatively understand weighting coefficients. Once more, the MACBETH based intuitive protocols of questioning were key to promote transparency along the processes, avoiding the eventual difficulty and cognitive uneasiness experienced by evaluators when trying to express their preference judgments numerically. These processes were developed in three sequential rounds. Along the three rounds, an “identity card” for each indicator was always available for on-line search of information and scientific evidence, as was also always available, for each indicator, the gap between the worst and best performance across the European regions. This was done resorting to Welphi’s features which you can find more about at http://support.welphi.com/article/add-indicators/.  

|Aim: To collect qualitative weighting judgments on each indicator.

|Objects of study: A set of previously selected indicators, through Welphi processes (https://welphi.blogspot.com/2019/01/welphi-for-selection-or-identification.html).

|Panel: 58 experts and stakeholders from different countries, distributed by four panels as follows Socio-economic, Demographic change and Health behaviors, Physical and Built environment and Healthcare and Mortality, according to their area of knowledge and expertise. A common design for the Welphi weighting process was implemented simultaneously with the four panels.

|1st round: Panelists were presented with a list of indicators and their gaps between the worst and best performance across the European regions. Each panelist individually was asked to answer a question concerning the gap on each indicator. The question was:

“To reduce inequalities in Europe, how important is to close this gap?”

While answering panelists considered “how big is the gap and how important it is to develop policies to close the gap in view of reducing health inequalities in Europe”. Panelists’ answers were provided with the MACBETH qualitative judgement scale (from not important, to extremely important). A “Don’t know/Don’t want to answer” option was available and comments could be given on each indicator.

|2nd and 3rd rounds: The extracted results of the 1st round were presented at the beginning of the 2nd round, allowing panelists to maintain or revise their answers.  At the beginning of the 3rd round, panelists received again information synthesizing the answers of the panel after the 2nd round. Accordingly, these had the opportunity to maintain or revise their answers.

|Results: The results of the process, along with other project’s outcomes guided a strategic group to assign quantitative weights for the indicators in a decision conference.

In the scope of the Euro-Healthy project, the outputs of both our introduced and distinct Welphi processes, to inform the shape of value functions and to understand weighting coefficients, together with scientific evidence collected, latter informed a decision conferencing face-to-face process in which a strategic group composed of 13 members, representing the diversity of viewpoints related to PH and covering the different areas of expertise and interest, successfully participated in the process of building the PHI model. The result was a set of multi-level PH indices based on the hierarchical multicriteria model. The PHI model underwent a set of testing, adjustment and validation procedures by the strategic group, until the PHI model has shown to be able to be a proper tool to assess PH across European regions.

Check out our support page (http://support.welphi.com/video-tutorials/) and watch our video tutorials to help setup your Welphi process today for articulating and modeling preferences towards your project’s goals, whichever your framework might be!

Friday, January 25, 2019

Welphi as tool to inform the shape of value functions– 2nd step in the process of an index creation; Combining the multi-criteria method MACBETH (Measuring Attractiveness by a Categorical Based Evaluation Technique) with Web-Delphi social processes.




Nowadays the process of creating an index, or the process of decision making, has drawn-out of its classical approach, having to consider aspects such as multiple criteria and stakeholders’ opinions in the same scope, which usually leads to a satisfactory outcome rather than an optimal one. 

Such setups have been recognized as requiring innovative ways of combining evidence with the views and values of multiple stakeholders like decision makers, experts and ultimately the population affected by future decisions. This evolution and consciousness regarding index creation, and decision making, features has led to the Multi-Criteria Decision Analysis (MCDA). 

MCDA has been defined as “an umbrella term, which describes a collection of formal approaches that seek to take explicit account of multiple criteria in helping individuals or groups explore decisions that matter”. MCDA is a structured framework with two components: a technical and a social component. The technical component entails the employment of a set of technics to support the different steps of the development of an index creation, or multi-criteria evaluation model, whereas the social component is meant to capture the points of view of the participants involved, in order to create a “shared understanding of the issue”. 


Welphi has proven to be a great game changer ensuring the social strand of these work frames, beening a rich and effective way to collect information from an enlarged and geographically dispersed number of participants. Having set the necessary structuring work leading to the structuring components, i.e indicators or criterion, to be considered in an index creation, or decision making process, follows the task of articulating and modeling preferences.  

Let us resort to our latest introduced Welphi case study – The Euro-Healthy project – to broadly explain how Welphi was used to articulate and model preferences, in a 1st instance accessing panelists’ views about the added value of improvements in different levels of indicator, informing the shape of the value functions resorting to the MACBETH (Measuring Attractiveness by a Categorical Based Evaluation Technique) socio-technical methodological approach, a novel methodology specially developed for the Euro-Healthy project’s needs.

In the scope of the Euro-Healthy project, and at this stage, Welphi processes, web-based Delphi processes, technically sound in MACBETH were conducted to inform the shape of value functions. The MACBETH based intuitive protocols of questioning were key to promote transparency along the processes, avoiding the eventual difficulty and cognitive uneasiness experienced by evaluators when trying to express their preference judgments numerically. These processes were developed in three sequential rounds. Along the three rounds, an “identity card” for each indicator was always available for on-line search of information and scientific evidence, as was also always available, for each indicator, the range of performance across the European regions. This was done resorting to Welphi’s features which you can find more about at http://support.welphi.com/article/add-indicators/. 

|Aim: To determine the shape of the value function on each indicator.

|Objects of study: A set of previously selected indicators, through Welphi processes. (https://welphi.blogspot.com/2019/01/welphi-for-selection-or-identification.html).

|Panel: 58 experts and stakeholders from different countries, distributed by four panels as follows Socio-economic, Demographic change and Health behaviors, Physical and Built environment and Healthcare and Mortality, according to their area of knowledge and expertise. A common design for the Welphi's value function processes was implemented simultaneously with the four panels.

|1st round: The range of performance on each indicator was divided in three equal pieces representing three changes in performance. For each indicator, panelists judged the contribution to population health of each one of those three changes in performance, answering the question:

“To improve population health in Europe, what is the contribution of this change in performance on the indicator?”

Panelists’ answers were provided with the MACBETH qualitative judgement scale (from very weak contribution, to extreme contribution). A “Don’t know/Don’t want to answer” option was available and comments could be given on each indicator. To the sequence of three judgements given by each panelist as attributed an implicit a value function shape on each indicator.

|2nd and 3rd rounds: In the 2nd round panelists were presented with the implicit value function of their first round judgments for each indicator. Moreover, feedback with the results of the 1st round was provided on each indicator: the number and percentage of respondents for each type of implicit value function shape and the comments made. In this 2nd round, panelists were given the chance to either confirm or change their implicit value functions shapes, at the light of the group information provided. At the beginning of the 3rd round, panelists received again information synthesizing the answers of the panel after the 2nd round. Accordingly, these had the opportunity to maintain or revise their answers.

|Results: The results of the process, along with other project’s outcomes guided a strategic group to set the value functions for the indicators in a decision conference.

Check out our support page (http://support.welphi.com/video-tutorials/) and watch our video tutorials to help setup your Welphi process today for articulating and modeling preferences towards your project’s goals, whichever your framework might be!