3) Чувство связанности (постижимость, управляемость, осмысленность жизни).
4) Оптимизм.
5) Самооценка.
6) Самоэффективность (вера в свою способность справляться).
7) Толерантность к неопределенности.
8) Самоконтроль.
Как отмечает Анна Хегай, включение соответствующих опросников (например, тесты жизнестойкости, шкала удовлетворенности базовых потребностей, скрининг тревоги и депрессии, опросник восприятия будущего) в проведение оценки социальных проектов и программ позволяет получить новый важный пласт знаний об их воздействии.
ЗАКЛЮЧЕНИЕ
Как показала конференция АСОПП 2023 года, подходы к оценке благополучия и счастья многообразны и очень интересны. Как отмечают эксперты, при их использовании важно, чтобы субъективный и объективный подходы использовались в комплексе, гармонично дополняя друг друга. Оценка эффективности каждого из этих подходов требует разработки собственного инструментария. При этом, участники конференции неоднократно отметили, что разного рода аналитические инструменты, используемые в рамках обоих подходов, помогают принимать управленческие решения, основанные на данных.
Говоря о других сессиях конференции АСОПП, большое внимание участников привлекли сессии трека «Оценка социокультурных проектов» – по данным организаторов, в них приняли участие более 150 человек. Доклады, представленные в рамках трека, были посвящены опыту как больших фондов, поддерживающих культурные проекты, которым нужно оценивать результаты своих программ, так и небольших НКО, которые оценивают свои проекты.
В следующих выпусках журнала «Позитивные изменения» мы подробно раскроем некоторые темы докладов, звучавших на конференции. Посмотреть записи всех сессий конференции можно в группе АСОПП во ВКонтакте[15 - https://vk.com/asopp] и на странице ассоциации в YouTube[16 - https://www.youtube.com/@asopp].
What’s Your Happiness Index? A Glimpse into the Annual Conference of the Association of Specialists in Program and Policy Evaluation
One of the pivotal events of this autumn for Russian professionals engaged in the field of evaluation is the annual conference hosted by the Association of Specialists in Program and Policy Evaluation. The central theme of the 2023 conference was assessing the quality of life and well-being. In this piece, we explore the experiences and practices shared by conference participants regarding the implementation and application of monitoring and evaluation in the social sphere.
Yulia Vyatkina
Editor, Positive Changes Journal
SHIFTING THE SPOTLIGHT TO HAPPINESS
The IX conference of the Russian Association of Specialists in Program and Policy Evaluation (ASPPE) took place online on October 17–18, 2023. This year’s conference featured 42 presentations and hosted 14 thematic and discussion sessions. The event drew over 500 attendees from all corners of Russia, spanning from Yuzhno-Sakhalinsk in the Far East to Kaliningrad in the West and Makhachkala in the South, the organizers say.
The conference program encompassed two main tracks: “Wellbeing-Focused Evaluation” and “Evaluation of Sociocultural Projects,” in addition to thematic segments dedicated to evaluation in specific domains – such as social entrepreneurship, working with children with special developmental needs, individuals with disabilities, and various forms of chronic illnesses. As usual, several sessions were also devoted to evaluation methodologies.
“The program was exceptionally rich, packed with engaging and substantive presentations. The relevance of the issues on the table and the profound discussions surrounding them underscore that interest in evaluation remains unwavering. I believe that thanks to our speakers and partners, whom I’d like to extend my heartfelt appreciation to, the conference offered an opportunity for all participants to expand their knowledge, enhance their professional competencies, delve into existing practices, and introduce their real-world working experiences to colleagues. In my view, this event holds great significance and utility, nurturing a sense of unity within the evaluation community and inspiring fresh achievements,” remarked Anna Lygina, President ASPPE.
The Accounts Chamber of the Russian Federation, which is responsible for evaluating the implementation of state programs and policies, among other things, has been an active participant in the conference for several years. In recent times, this regulatory body has been actively championing an evidence-based approach.
“In 2021–2022, we organized an extensive competition for government and municipal employees to encourage the adoption of an evidencebased approach in decision-making.[17 - Accounts Chamber of the Russian Federation. (2022). Competition of the Accounts Chamber of the Russian Federation “Practices and initiatives of evidence-based approach to management decision-making” – 2022. Retrieved from: https://ach.gov.ru/page/contest-2022?ysclid=lq6lwqk97z789634494. (accessed: 11.12.2023).] In 2023, in collaboration with the Financial University, we are hosting a research project competition for undergraduate and graduate students.[18 - Accounts Chamber of the Russian Federation. (2023). Research Project Competition “Application of Evidence-Based Approach for Evaluation of State Programs, Projects, and Measures.” Retrieved from: https://ach.gov.ru/statements/konkurs?ysclid=lq6m h5we6i912394087. (accessed: 11.12.2023).] We aspire to cultivate a community of individuals passionate about evaluation among students, researchers, and decision-makers,” stated Anastasia Kim, Deputy Head of Inspection at the Research and Methodology Department of the Accounts Chamber of the Russian Federation, during the ASPPE conference’s opening session.
The central theme of this year’s conference revolved around the evaluation of quality of life and well-being. As early as the 1960s, researchers began to conclude that gross domestic product (GDP) and the standard of living were not the ideal indicators to describe society’s life and individual well-being. That is how the concept of “quality of life” emerged, oriented toward a comprehensive assessment of various aspects of societal life, including non-material ones.[19 - Almakaeva, A. (2018). How is Quality of Life Researched? 21.05.2018. PostNauka. Retrieved from: https://postnauka.org/faq/86535. (accessed: 11.12.2023).]
In 1972, the concept of Gross National Happiness was introduced by the King of Bhutan in place of the commonly used GDP. This immediately elevated a small South Asian nation to become a trailblazer among countries making happiness the cornerstone of national policy. In 2008, the kingdom developed the corresponding tool – the Gross National Happiness Index. Happiness indices have since been calculated for various countries. One example is the World Happiness Report, published by the UN’s Sustainable Development Solutions Network.[20 - Helliwell, J. F., Layard, R., Sachs, J. D., Aknin, L. B., De Neve, J.-E., & Wang, S. (Eds.). (2023). World Happiness Report 2023 (11th ed.). Sustainable Development Solutions Network. Retrieved from: https://worldhappiness.report/. (accessed: 11.12.2023).] According to the 2023 report, Russia ranks 70th out of 137 countries in terms of happiness, with Finland, Denmark, and Iceland leading the chart.
THROUGH THE PRISM OF HAPPINESS AND WELL-BEING: AN OVERVIEW OF INTERNATIONAL EXPERIENCE
Approaches to assessing the quality of life, well-being, and happiness are intricately interlinked, as noted by Natalia Kosheleva, a member of the ASPPE Board and a consultant specializing in monitoring and evaluating socially oriented programs and projects.
According to the expert, these approaches can be categorized into three major groups, distinguished by their impact scale:
1. National/Territorial
Programs and policies aiming to enhance the quality of life, well-being, and happiness of people within a specific country or territory.
2. Organization-Specific
Programs or projects whose objective is to promote the quality of life, well-being, and happiness of individuals within a given organization.
3. Individual
The goal of programs and policies is to ensure the quality of life, well-being, and happiness of specific individuals.
The national/territorial approach can be illustrated by Bhutan, where the Gross National Happiness Index encompasses 9 spheres of human life:
• Psychological well-being;
• Health;
• Time utilization (e.g., adequate sleep is considered an important part of well-being);
• Education;
• Cultural diversity and sustainability;
• Quality management;
• Quality of life in local communities;
• Ecological diversity;
• Standard of living.
The survey questionnaire for the Gross National Happiness Index comprises 249 questions. For instance, in the Psychological Well-Being section, residents of the kingdom are asked to identify the 6–7 most important factors and elements that contribute to their happiness, rate the quality of their life on a scale of 0 to 10, and express how much pleasure they derive from it.
Which units can we measure well-being in? The Happiness Research Institute suggests a unit of measurement known as WALY (Wellbeing Adjusted Life Years) – life years adjusted based on the subjective level of well-being. This approach originates from the field of medicine, where health is measured in QALYs (Quality Adjusted Life Years) and DALYs (Disability Adjusted Life Years).
In essence, WALY is akin to a kilowatt-hour: if we have an electrical device with a power of one kilowatt, it consumes one kilowatt-hour of electricity in an hour of continuous operation. When calculating WALY, subjective well-being ratings on a scale of 0 to 10 are used as “power,” where 10 signifies complete life satisfaction. If a person is completely satisfied with their life for a year, the well-being “consumed” amounts to one WALY, Natalia Kosheleva explains.
Researchers also strive to calculate well-being losses and gains resulting from various factors and policies. Moreover, WALY enables the measurement of the Happiness Return on Investment (HROI).
According to Natalia Kosheleva, there is a significant development in the assessment of how projects impact individuals’ subjective well-being. For example, substantial work in this direction has been carried out in the United Kingdom. There is a center dedicated to evidence-based approaches in projects aimed at enhancing well-being. Since 2011, the annual population survey features four questions related to personal well-being. Respondents are asked to rate their current life satisfaction on a scale of 1 to 10, assess the significance of their life pursuits, reflect on their happiness the previous day, and indicate their level of anxiety during the same period.[21 - Office for National Statistics. Surveys using our four personal well-being questions. Retrieved from: https://www.ons.gov.uk/peoplepopulationandcommunity/wellbeing/methodologies/surveysusingthe4officefornationalstatisticspersonalwellbeingquestions. (accessed: 11.12.2023).]
Approaches to assessing the quality of life, well-being, and happiness are interlinked. They can be categorized into three groups, based by their impact scale.
In the UK, creative approaches to measuring well-being have also emerged. Convinced that the purpose of exhibitions includes influencing people’s emotional and psychological states, museums have particularly excelled in this area. To assess this impact, all visitors are invited to select a leaf of a specific color at the entrance to the exhibition and attach it to a tree. Following a color-coded system, a red leaf signifies “I feel good and prosperous,” a yellow leaf indicates a neutral state, and a green leaf expresses “I feel sad.” At the exit, visitors encounter another tree. Subsequently, the number of leaves of each color at the entrance and exit is tallied to gauge whether the exhibition has affected people’s emotional states.
RUSSIAN EXPERIENCE
Over the past three years, various indices for measuring happiness and well-being have also made their debut in Russia. The pioneer was the Russian Cities Quality of Life Index, developed in 2021 by VEB.RF in collaboration with PwC (now known as Trust Technologies) and the Russian Academy of National Economy and Public Administration, in partnership with the Agency for Strategic Initiatives (ASI). This comprises an information and analytical system that, as of 2023, consists of several components:[22 - VEB.RF (2023). Quality of Life Index. Retrieved from: https://citylifeindex.ru/. (accessed: 11.12.2023).]
• A database encompassing data on over 300 indicators for 218 cities across Russia. It allows analyzing and comparing cities, including with their foreign counterparts (based on individual indicators).
• City profiles, offering a detailed portrait of each city and facilitating comparisons of specific indicators with the national average or a cluster of similar cities.
The Timchenko Foundation also developed the General Index of Child Well-being, introduced in 2021, followed by the RUSAL Cities Quality of Life Index in 2022. These indices share a similar structure and rely on specific indicators. In the following sections, we will delve into each of these indices in more detail and explore how they can be utilized to assess conditions in various regions.
RUSAL’S CITIES QUALITY OF LIFE AND SUSTAINABLE DEVELOPMENT INDEX
This index, introduced in 2022, forms the basis of the company’s new social investment strategy. Irina Bakhtina, RUSAL’s Chief Sustainability Officer, explained that it is founded on criteria used by the Organization for Economic Cooperation and Development (OECD) as well as the VEB.RF Quality of Life Index.[23 - VEB.RF (2023). Quality of Life Index. Retrieved from: https://citylifeindex.ru/. (accessed: 11.12.2023).] The key distinction between RUSAL’s index and VEB.RF’s database lies in the population size of the territories under examination. RUSAL is interested in evaluating the quality of life and sustainability in cities and communities with populations of below 50,000.
66 indicators were considered in the development of this index, including statistical data (32 indicators), survey data (19 indicators), and information obtained through requests to regional and local authorities (15 indicators). RUSAL’s comprehensive tool covers 21 territories, including cities and municipal districts, across 12 dimensions of quality of life, categorized into three aspects: Values, Environment, and Potential.
“Values” include the following characteristics: 1) Work-life balance, leisure opportunities, and consumption of goods and services.
2) The ability to maintain regular contact with family and acquaintances, trust in the community, and a willingness to participate in social activities.
3) Satisfaction with life and confidence in the future.
4) Public safety.
“Environment” includes:
1) A comfortable, appealing, and diverse urban environment, encompassing streets, parks, and public spaces.
2) Good health and access to advanced medical care.
3) Favorable environmental conditions and an advanced waste management system within the city.
4) Availability of high-quality, well-maintained, and conveniently located housing.
The Happiness Research Institute suggests a unit of measurement known as WALY – life years adjusted based on the subjective level of well-being.
“Potential” comprises the following characteristics:
1) Access to employment opportunities that enable income and professional growth.
2) Extensive opportunities for receiving quality education for both children and adults.
3) Convenient transportation within the city and the ability to travel beyond its borders.
4) Effective governance and protection of civil rights.
“We consistently emphasize: this is not a ranking; it’s a diagnostic tool. It is crucial for us to identify areas where we may be falling short and to understand why. It is essential to examine the results in a comparative context. Why is this important? If our employees and their families move from one plant to another, we want to assess how their quality of life might change,” explains Irina Bakhtina.
Based on the results of 2022, regional centers such as Volgograd, Krasnoyarsk, Sayanogorsk, Novokuznetsk, along with the Sharypovsky district of the Krasnoyarsk Krai and Kandalaksha in the Murmansk region, took the lead in the index rankings. Conversely, the territories of the Irkutsk region lagged behind. For instance, Taishet, a city in the Irkutsk region with a population of 34,000, occupied the 21st position out of 21 in terms of socio-economic well-being. Respondents provided a low subjective rating of urban improvement quality, highlighted inadequate street lighting, noted low participation of children in extracurricular activities, reported low scores in the Unified State Exam (USE), mentioned a scarcity of events and cultural institutions in the city, pointed out a high accident rate, and cited limited affordability of housing.
Understanding the landscape of these issues serves as a crucial reference point for making decisions regarding social investments. In this regard, the company hopes that by addressing significant problems in Taishet and other cities and regions, it will witness improvements reflected in the index in the years to come.
CHILD WELL-BEING INDEX
The Child Well-Being Index serves as a data source to gauge the quality of life for children. Such indices are employed in numerous countries across the world. For instance, there is a report from Ireland’s Department of Children’s Affairs on the state of the child sector in the country, the Child Poverty and Hardship Index in the Kyrgyz Republic, the National Cohort Study and School Health Surveys in Finland, and Kazakhstan’s Child Wellbeing Index.
In Russia, the Child Well-Being Index[24 - The Timchenko Foundation. (2022). About the Child Well-Being Index. Retrieved from: https://cwb-index.timchenkofoundation.org/about/. (accessed: 11.12.2023).] was developed with contributions by the Timchenko Foundation, the Russian Academy of National Economy and Public Administration, Lomonosov Moscow State University, Moscow State University of Psychology and Education, National Research University Higher School of Economics, the Public Chamber of the Russian Federation, and the Russian Ministry of Education.
This instrument consists of two indices: the Statistical Index of Child Well-Being, calculated based on 27 indicators published by Rosstat, and the Subjective Index of Child Well-Being, derived from 18 indicators obtained through surveys of children and their parents.
