Thursday, 5 June 2014

Consulting with citizens in the design of wellbeing measures and policies

This post by Michael Hogan, provides an example of how ‘Interactive Management’ was used in consulting with Irish citizens and experts in the development of wellbeing measures and policies. This blog is part of the Wikiprogress discussion on “Engaging citizens in well-being and progress statistics”.

Internationally, there is increasing interest in, and analysis of, human wellbeing and the economic, social, environmental, and psychological factors that contribute to it. Current thinking suggests that to measure social progress and national wellbeing we need something more than GDP, especially as the link between economic growth and psychological and social wellbeing is not always positive (Easterlin, 1974; Layard, 2005; Wilkinson & Pickett, 2009). When the Commission on the Measurement of Economic Performance and Social Progress (CMEPSP), chaired by Joseph Stiglitz, reported in September 2009 they not only proposed a range of wellbeing domains that should be the focus of national accounting, they also proposed a strategy for the development of wellbeing measures.  Specifically, the Stiglitz Commission proposes that “at the national level, round-tables should be established, with the involvement of stakeholders, to identify and prioritise those indicators that carry the potential for a shared view of how social progress is happening and how it can be sustained over time” (Stiglitz et al., 2009, p. 18).

Since the Stiglitz report was published, different countries have taken a variety of different approaches to the design of wellbeing measures. While a number of countries have adopted the national round-table approach to deliberation and design, the range and scope of stakeholder inputs has varied from country to country.  For example, in the United States of America, President Barack Obama signed the Key National Indicators Act in March 2010, which provides for the creation of a Key National Indi­cator System (KNIS) that will provide US citizens with both national and local information on wellbeing across a range of dimensions. While this initiative is undertaken in the hope of creating a more informed and accountable democracy, the Key National Indicators Commission is composed of just eight academics.  While it is currently unclear what impact, if any, the absence of cross-party political input will have on the future success of KNIS, a recent case study analysis of the international wellbeing movement (Wallace & Schmueker, 2012) suggests that limited stakeholder engagement may prove problematic for the United States moving forward.  Notably, Wallace and Schmueker (2012) evaluated six case studies with a view to identifying what needs to happen to ensure that measuring wellbeing is made to matter in policymaking practice. The study found that wellbeing measures are at their most effective when they are supported by a combination of strong leadership, technocratic policy processes, and wide buy-in from civil society, citizens, and the media.  Ensuring buy-in from civil society and consulting with citizens in the design of measures was argued as critical for the sustained success of any national wellbeing agenda.  However, the report did not specify how best to approach citizen consultations nor how best to use consultation data in the design of wellbeing measures and policies.
One approach to this challenge is the use of a multi-level systems thinking strategy, whereby the systems thinking derived from citizen consultations is imported into the systems thinking of roundtable expert groups.  For example, in Well-being 2030, A new vision for ‘Social Europe’, Dhéret, Zuleeg, Chiorean-Sime, and Molino (2011) note that:

Having a political debate on the development model Europe wants to promote is a precondition to revitalising the European social model and reconciling citizens with the European project. Key questions that have been central to the Well-being 2030 project are: What kind of societies do Europeans want to live in? What kind of growth do Europeans want and what are their perceptions of ‘social progress’? What price are they willing to pay to achieve societal progress? And what is the role of the European Union in all this? In the final analysis, these are not questions that can be answered by research or by experts; they are political questions which should be debated in our political processes, at local, regional, national and European level. But there is very little public debate on such topics. Policy-makers will have to find the answers to these questions if they want to meet people’s expectations, and the EU should take a leading role in fostering such a debate. This will need to be a politicised debate, including elected representatives at all levels. Only through developing Europe’s political priorities in the social policy field can well-being be enhanced in the long term. (p. 26) 

Consistent with the recommendations of the Stiglitz report, I believe that the roundtable is an essential part in the design of wellbeing policies.  However, the roundtable is best considered as one part of a broader group design process, the objective of which is to facilitate collective intelligence and collective action. Broader group design processes can facilitate joint actions at every level (e.g., the micro individual level, meso community level and macro policy levels), by a variety of stakeholders. When working with groups to facilitate systems thinking it is important to have: (1) a facilitation team that helps to structure group deliberations using (2) group methodologies, including software support systems, that help with generating, categorizing, structuring, and sequencing ideas and developing action agendas in the context of (3) a productive workshop space

There are a variety of different methods that can be used to facilitate systems thinking in this regard.  In Ireland, we have used the systems science methodology developed by John Warfield, Interactive Management (IM).  IM is a computer facilitated thought and action mapping technique that helps groups to develop outcomes that integrate contributions from individuals with diverse views, backgrounds, and perspectives. Established as a formal system of facilitation in 1980 after a developmental phase that started in 1974, IM was designed to assist groups in dealing with complex issues (see Ackoff, 1981; Argyris, 1982; Cleveland, 1973; Deal & Kennedy, 1982; Kemeny, 1980; Rittel & Webber, 1974; Simon, 1960). The theoretical constructs that inform IM, developed over the course of more than two decades of practice, draw from both behavioural and cognitive sciences, with a strong basis in general systems thinking. The IM approach carefully delineates content and process roles, assigning to participants responsibility for contributing ideas and to the facilitator responsibility for choosing and implementing selected methodologies for generating, clarifying, structuring, interpreting, and amending ideas.

In a typical IM session, a group of participants who are knowledgeable about a particular situation engage in: (a) developing an understanding of the situation they face, (b) establishing a collective basis for thinking about their future, and (c) producing a framework for effective action. In the process of moving through these phases, group members can develop a greater sense of teamwork and gain new communication and information-processing skills. IM utilizes a carefully selected set of methodologies, matched to the phase of group interaction and the requirements of the situation. There were five steps involved in this process: (1) generate and clarify ideas, (2) vote, rank order, and select elements for structuring, (3) structure elements using IM software, (4) evaluate graphical representation of group logic and amend if necessary, (5) transcribe group discussion and evaluate discourse and reasoning to further understand the nature of group thinking (see Figure 1).
Figure 1. Steps in IM Process (click on image to enlarge)

Consulting with Irish Citizens and Wellbeing Experts using Interactive Management

To provide an example of an ongoing project employing the interactive management methodology, Figure 2 illustrates a problematique generated by participants at the Wellbeing in Ireland Conference (NUI, Galway, 2012) in response to the trigger question, What are Barriers to Wellbeing in Ireland? The problematique is to be read from left to right, with paths in the model interpreted as ‘significantly aggravates’. Boxes with two or more elements together indicate reciprocally inter-related elements.  Tracing one path of negative influence through the model, conference participants identified the absence of holistic approaches to healthcare, lack of a space for dialogue on holistic views of wellbeing, lack of understanding as regards the nature of wellbeing and how to measure it, and no national measurement of wellbeing as critical barriers to wellbeing in Ireland (Hogan & Broome, 2012). 

Figure 2. Influence Structure of Barriers to Wellbeing in Ireland 
(click on image to enlarge).

We hosted a second Wellbeing in Ireland conference focused on wellbeing measurement and the design of a national wellbeing index for Ireland (Hogan & Broome, 2013).  In advance of the conference, we asked participants to reflect on the broad issue of wellbeing in Ireland and generate a list of strategic objectives in response to the following question: In the context of developing a new national wellbeing index for Ireland, what are the strategic objectives that should guide our efforts to enhance the wellbeing of the people of Ireland over the coming decade? We analysed the survey responses and identified ten domains of strategic objectives.  Conference participants then engaged in idea-writing and discussion to further develop strategic objectives within each domain.  Strategic objectives were posted on display walls and participants were given time to study all of the objectives and provide clarification before voting to select their top objectives from the full list.  Selected objectives across ten wellbeing domains are listed in Table 1.

Table 1. Top Ranked Strategic Objectives across Ten Wellbeing domains

 Conference participants next used the ISM software to structure interdependencies among the highest ranked objectives (see Figure 3).  The figure is to be read from left to right and arrows indicate ‘significantly enhances’.  As can be seen from Figure 3, participants argued that promoting leadership and governance with an emphasis on community participation is a fundamental driver in the system and promoting this objective is thus likely to increase our chances of achieving all other objectives in the system of interdependent objectives. Members of the wellbeing in Ireland conference group are now engaging with political leaders in Ireland, specifically calling for a national consultation to extend the analysis of wellbeing objectives in advance of designing a national wellbeing index. 

Notably, the work of the Health and Wellbeing Cluster on Collective Intelligence and Wellbeing was published as part of a major Government Report on Citizen Engagement with Local Government. The report of the working group on citizen engagement with local government recommends that a People’s Participation Network (PPN) be established in every Local Authority area in Ireland to ensure extensive input by citizens into the decision-making process at local government level. The Working Group, which was set up in September 2013 under the chairmanship of Seán Healy, Director of Social Justice Ireland, sets out how the PPN should operate to enable the community and voluntary and environmental sectors to take an active formal role in the policy making and oversight activities of the Local Authority. It proposes a framework for public participation and formal engagement with the community and will be underpinned by regulations and guidelines from the Department. Engagement of this nature has been provided for under the Local Government Reform Act 2014 in Ireland. The regulations and guidelines, while allowing for the set-up of the framework for public participation across all Local Authorities, will also allow for more diverse engagement of citizens through other mechanisms, including the use of social media.
In an effort to pilot the type of large-scale national consultation that will be needed to fulfil the aims of the PPN and to foster a lifespan perspective in relation to wellbeing goals (NESC, 2009), we recently ran eight separate IM sessions with older adults (3 groups, total N = 26, Mean age = 69.84, 7 males, 19 females), younger adults (3 groups, total N = 21, Mean age = 31.6, 6 males, 15 females), and children (2 groups, total N = 14, Mean age = 14.7, 6 males, 8 females).  Similar to our wellbeing conference we asked participants to generate and clarify strategic objectives that should guide our efforts to enhance the wellbeing of the people of Ireland over the coming decade.  A total of 325 strategic objectives were identified by the eight groups and categorised into 39 categories by four expert coders. Analysis of wellbeing priorities according to hierarchies of category votes (Figures 3) indicated that health infrastructure, health and wellbeing promotion, and physical activity were the highest priority wellbeing goal categories amongst older adults; identity and perspective taking, equality of access to services, and health infrastructure were the top three goal categories for the working age groups; and changing the school curriculum to support greater wellbeing amongst schoolchildren, accepting and promoting diversity, and identity and perspective taking were top priorities for adolescents. 

A number of highly rated strategic goals from different categories were selected for structuring by each group (i.e., 10 – 12 elements were structured by each group).  Subsequently, it was possible to calculate average influence scores for each wellbeing goal category based on individual influence scores for each strategic goal structured by the groups (see McMoreland, Hogan, and Walsh, 2014 for more details).  While votes provide information as to the relative importance of goals and goal categories, IM structuring provides information as to the relative influence of goals and goal categories in a system of interdependent goals.  These influence scores are calculated based on both their position in the structure (i.e., with elements to the left receiving a higher score compared with elements on the right) and the number antecedents and succedents they have in the structure (i.e., elements with more succedents relative to antecedents have a higher overall influence on other goals in the system). Influence scores provide an indication of the relative influence of specific goals in a system of interdependent goals and this information can be used at an aggregate level to examine the average influence scores for different categories of goals. A meta-analysis of categories based on thematic overlaps provides insight into the level of influence of clusters of categories (see Figure 4). Meta-categories with higher average influence scores are those which have the greatest potential for supporting the accomplishment of the overall set of strategic goals. Thus, achieving the strategic goals for those categories in level one will significantly enhance the ability to accomplish strategic goals in the subsequent levels. For example, drawing on the logic of participants, improving public transport in rural areas (theme one) could help to ensure that elderly people are better able to maintain their independence because of easier access to local amenities (theme two). This would help to reduce loneliness and isolation amongst this cohort (theme four), and potentially facilitate greater inclusion and involvement of the elderly in their local community (theme five).

Figure 4. Whole group influence model of high-level category themes (click on image to enlarge)

Figure 4. Whole group influence model of high-level category themes (click on image to enlarge)
Overall, the findings from our pilot national consultation work further emphasise the need for new methods to be introduced in order to measure a broader range of indicators of wellbeing, both for the population as a whole and for different age cohorts, who may highlight contrasting goals and hierarchies that may inform subsequent national wellbeing policy and practice. 

Embedding Systems Science in the International Wellbeing Movement
As can be seen from the above example, consulting with citizens and wellbeing experts can provide valuable input that can be used to advance the wellbeing agenda.  Consultations using interactive management offer a number of benefits, including:  

1)      Providing insight into the values, goals, and preferences of stakeholders;
2)      Engaging participants in a democratic, consensus building process that facilitates buy-in and enhances the legitimacy of decision-making groups;
3)      Facilitating transparent understanding of the reasoning that informs the systems thinking of groups;
4)      Creating opportunities to merge data across multiple groups and analyse group differences and similarities using meta-analysis, given the standardized and structured approach used;
5)      Providing an option to link qualitative structural models of system interdependencies to quantitative modelling efforts by drawing upon data from national surveys and other forms of national accounting;
6)      Creating the ability to establish feedback loops between multiple working groups and multiple levels of analysis such that: (a) there is growing awareness amongst stakeholders of a diversity of perspective, and (b) quantitative models are evaluated in light of qualitative reasoning and qualitative models are considered in light of rigorous quantitative analysis;
7)      Developing a multi-level approach that incorporates systems thinking across geographical space and demographic groupings, which can be used to inform both a synthesis and perspective at a macro-economic and societal perspective (i.e., for central government policy), while also offering a synthesis, perspective and collective input at local level that is needed to translate national wellbeing policies into local government and community group projects and practices.  In other words, well-designed national consultations can facilitate better linkages between Micro- (local) and Macro- (national) level wellbeing goals grounded in a systems thinking approach to social progress.

Figure 5. Well-Being in Ireland Conference Strategic Objectives Structure (click on image to enlarge).
Importantly, in order to be truly effective in practice, citizen consultations in relation to wellbeing measurement and policy implies the need for a broader view in relation to collaboration, cooperation, and social and scientific problem solving.  Effective cooperative and collaborative dynamics are fundamental to successful problem solving in science and society. Research suggests that cooperative relationships are characterized by reciprocity, discussion, mutual respect, perspective taking, and a coordination of each individual’s views with those of others. When people have the opportunity to share their views they are more likely to develop a stake in the process and therefore become motivated to learn and work toward common goals (Wells & Arauz, 2006).  This collective approach encourages dialogical interaction (Ballantyne & Varey, 2006); mutual framing, collaborative learning and extended active participation between multiple groups within and across many levels, to scale up new shared values for national wellbeing.

However, we also recognise that, in the broader context of political engagement and policy change, there are many barriers to the implementation of wellbeing policies.  As one part of our wellbeing in Ireland conference, we worked with a small group of four wellbeing experts, specifically, directors of the UK and Scottish wellbeing projects, chair of the Irish National Statistics Board, and a research director from the Carnegie-UK trust.  The expert group worked to generate, clarify, and structure barriers to implementing national and international wellbeing policies.  The problematique generated by this expert group (Figure 6) indicated that conceptual confusion, shortage of case studies, and failure to recognise diversified needs across gender, class, disability, and ethnicity were three primary driver barriers that significantly aggravated a number of other barriers in the system.  Rather than attempting to define wellbeing directly, consistent with Sen’s capability approach and the approach taken in Scotland, we suggest that working with stakeholder groups to identify capabilities and goals needed to enhance wellbeing provides a more pragmatic approach to developing consensus-based wellbeing programme agenda that may overcome some of the challenges associated with wellbeing measurement and conceptual confusion in the area.  In addition, a focus on goals and goal pursuit parallels the approach adopted by governments in developing, adopting, and implementing policies. Thus, a focus on wellbeing goals allows for a shared language between citizens and politicians.  A focus on strategic goals also allows for a greater correspondence between the language of citizens, politicians, and wellbeing scientists, particularly if wellbeing scientist can evaluate evidence in relation to the impact of pursuing specific goals on wellbeing outcomes and communicate the results of their work clearly to citizens and politicians.
 Figure 6:  Problematique of barriers to implementing national and international wellbeing policies (click on image to enlarge)

To consult or not to consult, that is the question?

Since the Stiglitz report was published, different countries have taken different approaches to the design of wellbeing measures and policies. While some countries have worked diligently to consult with citizens in a meaningful way in the deliberation and design process, other countries have not engaged with citizens in any meaningful way, and no country has adopted a collaborative, systems thinking approach to national consultations or wellbeing policy design.  While this is somewhat surprising and disappointing given the broad goals of the international wellbeing movement, it is only recently that case study analyses have highlighted how limited citizen engagement may prove problematic for any national wellbeing agenda (Wallace & Schmueker, 2012).  Consistent with Wallace and Schmueker (2012), who concluded that wellbeing measures will be most effective when they are supported by a combination of strong leadership, technocratic policy processes, and wide buy-in from civil society, citizens, and the media, we proposed a multi-level systems thinking approach be adopted to citizen and expert roundtable consultations to inform national and local wellbeing policy and practice.  While some countries have worked to develop their wellbeing measurement strategy by drawing upon expert round tables, other countries including the Canada, United Kingdom, Italy, and Australia have also consulted with citizens in the wellbeing measurement design process.  It has been suggested by Kroll (2011) that these participatory approaches can enhance collective intelligence and increase the sense of procedural justice and perceived relevance of wellbeing indicators in the population, while also increasing democratic legitimacy and amplifying the potential political attention paid to progress indicators.

This may be the ideal scenario but without such a vision we may end up repeating the mistakes of the past.

Co-Leader of the Health and Wellbeing Theme at the Whitaker Institute for Innovation and Societal Change and School of Psychology, NUI, Ireland

Twitter: @michaelhogannui


Ackoff, R. L. (1981). Creating the corporate future: Plan or be planned for. New York: John Wiley and Sons.

Argyris, C. (1982). Reasoning, learning, and action: Individual and organizational. San Francisco: Jossey-Bass.

Ballantyne, D., & Varey, R. J. (2006). Creating value-in-use through marketing interaction: the exchange logic of relating, communicating and knowing. Marketing theory, 6(3), 335-348. doi: 10.1177/1470593106066795
Cleveland, H. (1973). The decision makers. Center Magazine, 6(5), 9-18.
Deal, T. E., & Kennedy, A. A. (1982). Corporate cultures: The rites and rituals of corporate life. Reading, MA: Addison-Wesley.
Dhéret, C., Zuleeg, F., Chiorean-Sime, S., & Molino, E. (2011). Well-being 2030: A new vision for 'Social Europe'. EPC Issue Paper, 65(18).
Easterlin, R. A. (1974). Does economic growth improve the human lot? Some empirical evidence. In R. David & M. Reder (Eds), Nations and households in economic growth: Essays in honor of Moses Abramovitz. New York: Academic Press.
Hogan, M.J. & Broome, B. (2012).  Well-Being in Ireland: Overcoming Barriers to Well-Being in Ireland, Collective Intelligence Conference Report, NUI, Galway.
Hogan, M.J. & Broome, B. (2013).  Wellbeing in Ireland – Designing Measures and Implementing Policies, Collective Intelligence Conference Report, NUI, Galway.
Kemeny, J. (1980). Saving American democracy: The lesson of Three Mile Island. Technology Review, 83(7), 64-75.
Kroll, C. (2011). Measuring progress and well-being: Achievements and challenges of a New global movement. Berlin: Friedrich Ebert Foundation.
Layard, R. (2005). Happiness: Lessons from a new science. London: Penguin.
NESC. (2009). Well-being matters: A social report for Ireland [NESC Report No. 119]. Dublin: NESC.
NUI Galway. (2012) Overcoming barriers to well-being in Ireland: Workshop Report. Retrieved from [12 April 2013]
Rittel, H., & Webber, M. (1974). Dilemmas in a general theory of planning. DMG-DRS Journal, 8, 31-39. Doi: 10.1007/BF01405730

Simon, H. A. (1960). The new science of management decisions. New York: Harper & Row.

Stiglitz, J. E., Sen, A., & Fitoussi, J. P. (2010). Report by the commission on the measurement of economic performance and social progress. Paris: Commission on the Measurement of Economic Performance and Social Progress. Retrieved from [10 April 2013].
Wallace, J., & Schmueker, K. (2012). Shifting the dial: From wellbeing measures to policy practice. Dunfermline: Carnegie UK Trust.
Wells, G., & Arauz, R. M. (2006). Dialogue in the classroom. The journal of the learning sciences, 15(3), 379-428.
Wilkinson, R., & Pickett, K. (2009). The spirit level: Why more equal societies almost always do better. London: Allen Lane.

No comments:

Post a Comment