key: cord-0005398-9g2w16d6 authors: Shek, Daniel T. L.; Wu, Florence K. Y. title: The Social Indicators Movement: Progress, Paradigms, Puzzles, Promise and Potential Research Directions date: 2017-01-11 journal: Soc Indic Res DOI: 10.1007/s11205-017-1552-1 sha: 2fa8d0a839aab34ac1118d26dab943298237a93d doc_id: 5398 cord_uid: 9g2w16d6 This paper is a response to the article entitled “Fifty years after the Social Indicators Movement: Has the promise been fulfilled?” by Ken Land and Alex Michalos (2015) which constitutes a careful review of the historical development of the Social Indicators Movement, utility of social indicators in shaping the concept of quality of life and subjective well-being, and issues deserving social indicators research in future. In this response paper, we join in the discussion by highlighting five issues—progress, paradigms, puzzles, promise, and potential research directions of social indicators research. In terms of progress, while we have accomplished many tasks proposed by Solomon et al. (The quality of life, Sage, London 1980), some of them are yet to be achieved. Regarding research paradigms surrounding social indicators, researchers have primarily used positivistic or post-positivistic orientation to conduct and interpret social indicators research, with relatively fewer studies using interpretive, constructionist or critical theory perspective. There are also several puzzles deserving consideration. These include (a) the use of “other types of evidence”, particularly qualitative data; (b) evaluation of social programs; (c) feasibility of assessing “social progress”; (d) choice of social indicators; (e) interpretation of findings; (f) methodological debates; and (g) explanations for social change. Finally, the promise of social indicators research to promote quality of life and potential future research directions of social indicators research are discussed. It is indeed my honor to write this paper responding to the groundbreaking paper entitled ''Fifty years after the Social Indicators Movement: Has the promise been fulfilled?'' by Ken Land and Alex Michalos (2015) . After reading the paper, my immediate reaction is that the paper is an outstanding contribution to the field of social indicators because it carefully charts the history of the social indicators movement and highlights some important future research directions. As it is a ''heavy weight'' paper written by two gurus in the field of social indicators, it is a ''must read'' paper by students and researchers in the field of social indicators research. The paper by Land and Michalos (2015) is a comprehensive paper that gives both a historical and critical perspective on the social indicators movement. The paper starts by describing the social indicators movement in the 1960s and its evolution in the past half of a century such as the development of different social indicators systems and academic journals. Then the authors describe the contemporary state of social indicators, quality of life and well-being studies and conclude that ''numerous composite indices exist at all levels of analysis-from international comparisons to national-level indices to subnational/ regional levels and for various sub-populations' ' (p. 28) and five examples are presented to illustrate such indicators. Finally, agenda for future with reference to possible research directions is included in the paper. In this response article, we will add my two cents on several issues, including progress, paradigms, puzzles, promise and potential research directions (i.e., 5 ''Ps''). I will discuss the progress of the Social Indicator Movement with reference to the tasks that social indicators researchers should accomplish as described by Solomon et al. (1980) . Concerning paradigms of research on social progress, we argue that quality of life research in different paradigms generates different pictures on ''social progress''. Regarding puzzles in social indicators research, we consider several issues, including the role of qualitative data, role of evaluation in social programs, feasibility of assessing social progress, choice of social indicators to be used, interpretations of findings, methodological debates and explanations for social change. We will then examine, how social indicators research findings can inform policy according to its original promise. Finally, future research directions will be highlighted. Solomon et al. (1980) suggested a list of tasks to be undertaken at the initial stage of the Social Indicators Movement. It would be interesting to ask the extent to which such tasks are accomplished after roughly 36 years. • Task 1-''Studying objective life conditions and subjective life quality experience and their inter-relationships'' As pointed out by Land and Michalos (2015) , many frameworks for measuring objective life conditions have been developed and numerous studies have been conducted. Similarly, many studies on subjective life quality experience have been conducted. However, studies on the inter-relationships between these two domains seem not to have been the primary focus in the field. Thus, it is important to find ways to align these two domains in future. • Task 2-''Beyond description of conditions and experiences and to explore relationships amongst interdependent factors'' While models of objective indicators and experiences of life quality have been constructed and tested, more work is needed to explore the ''interdependent'' factors. • Task 3-''To examine life quality not just on the societal level but also group and individual levels, and investigate their inter-relationships'' While most of the existing studies have examined quality of life at the societal level, there are comparatively fewer studies examining group and individual life quality. There are even fewer studies investigating the inter-relationships amongst quality of life on the different levels. • Task 4-''To understand how individuals and groups participate to create their life quality'' Most of the related studies focus on quality of life outcomes (social indicators of outcomes) with few studies examining how people create their life quality and the related process. There is a need to focus more on the human agency. • Task 5-''To treat quality of life as a dynamic process than a static state'' Although objective social indicators are usually collected over time, researchers usually regard the construct as static state. Explanations of the changes are commonly ad hoc in nature. Moreover, longitudinal research on subjective life studies is not common. • Task 6-''Foster cross-cultural comparisons and encourage institutions and international organizations to collaborate'' There are emerging studies examining crosscultural comparisons but the development is still in its infancy (Shek et al. 2005a; Shek 2010 Shek , 2011 . Collaboration amongst institutions and international organizations is growing but is yet to mature. • Task 7-''To stimulate quality of life research in places without related research tradition'' We have seen some progress in non-Western contexts, such as studies conducted in Asia (Chua et al. 2010; Ip and Shek 2014) . However, the five illustrative examples given by Land and Michalos are all projects in English speaking communities. • Task 8-''Reflection and recognition about cultural differences'' This is a neglected aspect in the movement. Social indicators researchers have commonly assumed that the indicators are universally valid, and can be used in different cultures. Even though some social indicators researchers recognize cultural differences, the related reflection is not substantial. • Task 9-''Promoting inter-disciplinary projects'' We witnessed inter-disciplinary collaboration amongst sociologists, political scientists, social workers, and social policy scholars who were more focused on the ''macro'' aspect of quality of life. However, the link between macro and micro disciplines (such as Psychology) remains to be achieved. For example, epidemiological studies are commonly conducted in psychology, psychiatry or mental health. However, such findings are seldom mentioned in social indicators research. • Task 10-''Regard biophysical and social environment and their perception as basic elements of quality of life'' Most studies have actually mostly done this. However, the spiritual dimension, despite its importance, is not commonly included in social indicators research. • Task 11-''Conduct projects that have relevance for decision-makers'' Some studies have attempted to do this, although some social indicators research (particularly those involving ranking of different sites) simply promote understanding without much relevance to decision-makers. Social science research is inevitably shaped by the research paradigm one adopts (Kuhn 1962) . The choice of research paradigm does not only dictate the research methods but also the interpretations of findings (Neuman and Kreuger 2003) . Hence, it is important to reflect about social indicators research conducted within different paradigms. To be simple, we can regard paradigm as a lens through which we can see and understand the reality, such as social progress. It addresses the issue of ontology (i.e., the nature of reality one assumes, including whether it exists or not), epistemology (i.e., the question of ''how do we know?'') and methodology (i.e., ways through which the reality can be known). Looking back in history, social indicators research is guided primarily by statistical data and analyses. In terms of paradigm, it is a product within positivism or post-positivism Lincoln 1998, 2011) . Positivistic or post-positivistic research on social progress demands the use of social science methods involving deductive logic and precise observations with the research objectives of discovering and testing generalizable nomothetic universal laws, which can predict human behavior. Researchers adopting a positivistic or post-positivistic paradigm typically assume that social reality is stable over time and preexisting regularities can be scientifically studied. Through precise instruments and objective research methods, social indicator researchers can know the reality just like pressing a button of the camera. To maintain neutrality in research, common sense and values do not enter into the picture of positivistic social indicators investigation. While social science has made much progress based on positivistic or post-positivistic paradigms, there is growing dissatisfaction regarding whether it can give a ''true snapshot'' of the reality Lincoln 1998, 2011) . First, while positivistic research assumes that we can understand social reality in a ''social vacuum'', there is growing criticism of this ''context stripping'' view and there is a call for contextual understanding of observed social phenomena. Second, the dualist assumption of the separation between theories and observations of facts is questionable and many researchers argue that ''facts'' are in fact theory-laden (i.e., thinking shaped by some existing preoccupation). Third, the thesis that it is possible to be totally objective may not be tenable because values and facts are value-laden and interdependent. Finally, there is the criticism that it is impossible to have impartial observers because inquirers might influence the observation process Lincoln 1998, 2011) . While positivists put up theories and test the related hypotheses, interpretive and constructionist social scientists define research studies as attempts to investigate meaningful social action and create as well as maintain understandings and interpretations of people. Instead of relying on hypothetical-deductive models and supposedly ''objective'' observations, the purpose of research is to understand social reality that cannot be fully understood in terms of causal laws. As it is assumed that social reality is fluid and there are no preexisting regularities, individual interpretations and social constructions are important. According to this paradigm, common sense is an important source of understanding and values are indispensable in the research process. In contrast to the positivistic practice of using precise measurement, interpretive and constructionist social scientists commonly use interviews, narrative research, natural observations, qualitative studies, and ethnography. Hence, profiles based on social indicators are regarded as ''naive realism'' because social reality is always fluid and thus beyond the description by static statistical indicators. Statistical models can generate good maps but such maps are without ''blood and flesh'' and ''lived experience'' that cannot help people understand the experience of individuals and social groups. While interpretive and constructionist social science provides an alternative paradigm, through which we can understand social progress, there are several problems associated with this position. Primarily, adopting a relativistic epistemology, no objective indicators of social progress are possible, because there can be different interpretations and constructions of the term. Second, while interpretive studies can generate unique pictures based on individual or group experiences, how to construct an aggregated and presumably consistent picture on the ''profile'' would be a problem. Thirdly, interpretive and constructionist research studies have been criticized as subjective and unscientific. Critical social science endorses a critical process of inquiry which attempts not just to look at the ''observables'' (which are superficial illusions) but to examine the underlying structures and processes, which shape the real world phenomena (Neuman and Kreuger 2003) . Besides looking at the underlying power relations that cannot be seen on the surface, critical social scientists help people build a better world for themselves. For example, according to neo-Marxian thoughts, growth in GDP does not necessarily reflect social progress because the working class is exploited by the capitalists. In fact, because of its dynamic nature, exploitation, inequalities and injustice cannot be fully reflected in statistical measures. As far as preferred research methods are concerned, although quantitative and qualitative research methods can be used, critical theorists argue against the use of ''dominant'' methods (which create illusory pictures) and favor action research and participative research. Obviously, different paradigms have different views on how to use social indicators to perform social auditing. Positivistic and post-positivistic researchers would use valid and reliable measures to collect data from large samples to generate information about social progress. Using housing and quality of life as an example, indices and profiles of housing affordability have been developed which help to assess the quality of life of the residents. For interpretive and constructionist researchers, instead of primarily collecting figures and statistical profiles, they tend to collect lived stories and listen to the voices of the unheard who are neglected in the mainstream discourses so that the fluid social reality can be better illustrated. Using housing quality of life as an example, instead of building up ''profiles'' of housing quality of life, interpretive and constructionist social scientists would attempt to illustrate the subjective experiences of the informants and socially constructed realities in different groups of people on housing experience. For example, they would understand the subjective experiences of those who have to pay high mortgage or live in places with very bad environment, such as sub-divided flats and ''coffin rooms'' (very small rooms in Hong Kong). Finally, critical theorists argue that the profiles related to housing quality of life shown by social indicators may be illusory and misleading. While positivistic researchers chart changes in housing affordability, critical social scientists look at the underlining power relations leading to such fluctuation, which is not only shaped by the home owners and buyers, but also by the interests and power relations amongst the real estate developers (who build the houses), bankers (who lend money to the buyers for mortgage) and the Government (who owns the land and makes related regulations). In the seminal work on social indicators, Bauer (1966) stated that ''this volume as a whole is devoted to the topic of social indicators-statistics, statistical series, and all other forms of evidence-that enable us to assess where we stand and are going with respect to our values and goals, and to evaluate specific programs and determine their impact'' (p. 1, emphasis added). Obviously, there has been an abundance of statistical studies and statistical series in this area in the past 50 years. Nevertheless, the use of ''other forms of evidence'' such as qualitative data and mixed-methods data to reflect our position and direction is comparatively less widespread. There are several fundamental features of quantitative research (Leung and Shek 2011; Lincoln and Guba 2000; Patton 2002a, b) . Firstly, its philosophical orientation is positivistic or post-positivistic and it assumes that the nature of reality is independent of human consciousness and that it is governed by causal laws. Secondly, it aims at discovering universal laws (i.e. nomothetic emphasis). Thirdly, the knowledge is achieved through ''fact accumulation'' where the facts are presumed to be derived from sense-impressions, which are value-neutral. Fourthly, the role of research is to explain and predict phenomena using measured variables with reliability and validity as quality criteria. Fifthly, quantitative research attempts to confirm theories through testing hypothetical deductive models. Lastly, to ensure the objectivity of a study, quantitative research favors artificial research arrangement where the variables can be controlled. Concerning quantitative researchers' characteristics, they are expected to be neutral and value free. To maintain objectivity, quantitative researchers should serve as outsiders controlling bias through error elimination and maintaining a distant, detached and neutral relationship with the subjects. On the other hand, subjects are expected to be passive in quantitative research. Quantitative researchers also have certain beliefs about the research process: literature review is regarded as fundamental and as the skeleton for theory and hypotheses; operationalization of concepts; structured, predetermined and strictly planned research design; preference for random and representative samples; use of standardized and validated instruments; use of complex statistical analyses; generalization of findings. On the other hand, the philosophical orientation of qualitative research is derived from constructionist and interpretive philosophies which assume that the nature of reality is individually or socially constructed. On the nature of science, qualitative research takes an idiographic stand focusing on individual or group uniqueness. It looks at the nature of knowledge as construction of reality and understanding of meanings and interpretations, which are fluid in nature. Qualitative research primarily attempts to induce theories through naturalistic and context-sensitive methodologies. Instead of asking researchers to be value free, qualitative research expects that researchers to be reflexive and serve as the research instrument. To ensure context-specific study, qualitative researchers favor naturalistic research arrangements in the real world and regard research as understanding the reality focusing on individual unique experiences and/or group characteristics. For the quality criteria, it highlights trustworthiness and authenticity, including credibility, transferability, dependability and confirmability. Concerning qualitative researchers' characteristics, they are expected to be empathetic and sensitive to feelings of people. To gain an in-depth understanding of human behavior, qualitative researchers should serve as insiders with close and empathetic neutrality. At the same time, participants are expected to be active in qualitative research. Researchers doing qualitative research believe that in the research process literature review is auxiliary and helps to gain a better understanding of human experiences, and that there is no need to operationalize concepts. The research design should be unstructured, flexible and evolving. There is no preference for random and representative samples, and analyses should be qualitative and thematic. Researchers are instruments in the research process. Lastly it is believed that in qualitative research the findings should be conceptually generalized. According to Flick (2014) , the dominant positivist approach to doing research requires the researchers to be emotionally distanced, unbiased and absolutely objective. However, qualitative researchers argue that it is virtually impossible to be totally value-free or objective. In mainstream social sciences research, quantitative research methods and statistical analyses are the conventional strategies while qualitative methods are regarded as a less preferred option which is less rigorous (Flick 2014; Griffin 2004; Marshall and Rossman 2011) . Patton (2002c) proposed five sets of criteria for judging the quality and credibility of qualitative research. These include: (a) traditional scientific research criteria (e.g., objectivity of inquirer, correspondence of findings to reality, systematic rigor of research procedures), (b) constructivist criteria (e.g., authenticity, connects and moves the audience, reflexivity), (c) artistic and evocative criteria (e.g., aesthetic quality, creativity, interpretive vitality), (d) critical change criteria (e.g., increase of consciousness about injustice, voices of less powerful group, historical and value contexts); and (e) pragmatic criteria (e.g., utility, feasibility and balanced approach). Patton (2002c) pointed out that ''many researchers mix and match approaches'' (p. 271) and he has ''worked with and mixed criteria from all five frameworks to match particular designs to the needs and interests of specific stakeholders and clients' ' (p. 271-272) . The advantages and limitations of using qualitative research have been discussed in the scientific literature (Flick 2014; Griffin 2004; Marshall and Rossman 2011; Willig 2001) . One of the advantages of using qualitative research is that it can enable researchers to focus on the uniqueness and individuality of the informants (i.e., meaning-making process of the informants). Qualitative research best addresses the complexity and multiplicity of individual's lived experiences. The meanings of these experiences are usually understood through in-depth qualitative investigation on informants' frames of reference. This allows the researchers to understand the inconsistencies and contradictions within themselves and across different people. In addition, the face-to-face interactions between the researchers and the informants help develop the relationship of trust and they help to understand sensitive issues. Another advantage of qualitative research is its flexible research design. The iterative process of the qualitative research practice helps the researcher situate himself/herself in the field of study and develop more and more knowledge about the field and people in it. In addition, qualitative researchers can explore in detail the experiences, motives and opinions of others and learn to see the world from perspective other than his/her own by using in-depth interviewing. With this new perspective, the long-held assumptions may be challenged and thus recasting reflections (Griffin 2004; Marshall and Rossman 2011) . The more the researchers find out, the wider the perspective and horizons would be formed in the specific field. The third advantage of administering qualitative research is providing an emic understanding of the personal experiences of the informants (Lincoln and Guba 1985; Marshall and Rossman 2011; Rubin and Rubin 2012 ). The qualitative researchers are able to investigate the complexity and multiplicity of the context offers. By situating the researchers in the field and going into the contexts, researchers can elicit multiple constructed realities and study holistically the phenomena and thus obtain the tacit emic knowledge and subjective understandings and interpretations (Marshall and Rossman 2011; Patton 2002a) . Whilst qualitative research methods can inquire the processes in particular contexts in considerable depth, sample size in qualitative research is usually small (Flick 2014) . The statements are often based on analyses of relations, conditions and process in certain contexts. This rootedness in contexts often allows qualitative researcher a specific expressiveness and thus the knowledge produced might not generalize to other contexts. As such, academics, practitioners or policy makers may not take the related findings seriously (Griffin 2004) . Recently, some eminent qualitative researchers (see Maxwell 2012; Flick 2014) set forth some possible ways of mapping out the path from the case to the theory that will allow the qualitative researchers to reach at least a certain level of generalization. Another limitation of utilizing qualitative research is that the analysis procedures are time-consuming and therefore expensive (Griffin 2004) . While crafting the analyses, qualitative researchers have to carefully examine the recordings, field notes, verbatim and other relevant documents in detail and the process takes quite much time in digging in the depth of the informants' inferences. Hennink et al. (2011) alerted the qualitative researchers the significance of this crafting process to gain Verstehen, which is to understand the life of the people from their own perspective in their own contexts, though the researchers might spend prolonged time in the field and the analysis process. Besides quantitative and qualitative methods, there is also a growing tendency to mix quantitative and qualitative data (Creswell 1994 (Creswell , 2003 Tashakkori and Teddlie 2003) . According to Datta (1994) , there are five forceful and pragmatic reasons for integrating these two methods: (a) researchers have used both methods for a long time; (b) there are arguments supporting the use of both methods; (c) both methods are supported by funding bodies; (d) both methods and related findings have been used in social policies; and (e) we have learned much from both paradigms. Greene et al. (1989) also noted five reasons to combine quantitative and qualitative methods in a single study. First, mixing methods (especially triangulation) can help to seek convergence of findings. For example, it is possible to show by using statistical data and subjective interviews that ordinary people cannot afford to buy a flat. Second, complementarity mixed methods study is able to bring out different opinions about a phenomenon. For example, while quantitative housing affordability index suggests that housing is not affordable, qualitative interview data can give some indication how paying the high mortgage affects negatively people's lives. Third, researchers can combine qualitative and quantitative methods in a developmental manner where the first method helps to inform the second method. For example, based on the housing affordability index information which suggests that housing is a problem, researchers can then carry out field observations about the lived experiences of people. Fourth, mixed methods research can help to deal with contradictions in findings with the emergence of fresh perspectives (i.e., initiation). For example, researchers can use qualitative methods to understand the contradiction observed in the quantitative data, such as increase in income but decrease in happiness. Finally, mixed methods can be used to add more understanding to a study (i.e., expansion). For example, qualitative research can be carried out to understand the information provided by the housing affordability index. There are an increasing number of mixed methods studies investigating quality of life. (e.g., Mitra et al. 2013) . Roy et al. (2015) used quantitative survey, qualitative interviews and participatory approach to construct social indicators of well-being ''by exploring the limits of quantification and considering an alternative action theory'' (p. 689). Tonon (2015) also advocated qualitative methodology in quality of life research and proposed mixed methods as the ''third methodological approach'' in quality of life studies, which are guided by some conceptual and methodological criteria (Shek et al. 2005b ). Besides describing and monitoring social progress, social indicators can also be used as outcome indicators to measure changes after implementation of social intervention. Interestingly, two observations can be highlighted on the use of social indicators in the evaluation of social programs. First, comparing with description and monitoring functions of social indicators, the evaluation function of social indicators has been given relatively weaker weight. One possible explanation is that macro social indicators such as indicators at the national level may not be sensitive measures for social programs, which may be more individualistic and community-based. It also takes time for social intervention programs to create outcomes, which can impact on changes in social indicators. Nevertheless, for specific areas such as welfare, security and health, social indicators have been frequently used to reflect the effect of social intervention-poverty rates have been used to reflect the effectiveness of poverty alleviation programs; substance abuse rates have been used to reflect community anti-drug programs; infection diseases rates have been used to reflect flu shot programs. Second, often social indicators research and evaluation research have been regarded as two separate research areas in the field and the literature has not been seriously addressing the possible linkages between these two domains. Program evaluation has been defined by Rossi et al. (2004) as ''the use of social research methods to systematically investigate the effectiveness of social intervention programs in ways that are adapted to their political and organizational environments and are designed to inform social action in ways that improve social conditions'' (p. 16, emphasis added). According to Patton (2008) , program evaluation is ''the systematic application of research to inform evaluative judgments. It involves the systematic collection of empirical information about the activities, characteristics, and outcomes of programs to make judgments about the program's merit or worth, improve program effectiveness, and/or inform decisions about future programming'' (p. 683, emphasis added). Obviously, it is possible to use social indicators on different levels as outcome indicators in evaluation research. The standards and principles in different evaluation models (e.g., American Evaluation Association 2004; European Commission 2003; Joint Committee on Standards for Educational Evaluation 2008; United Nations Evaluation Group 2005a, b) enlighten researchers in the field of social indicators research on at least two aspects. First, it is important to engage different stakeholders in the process. Instead of just involving the experts in the process, other relevant stakeholders should be engaged as well. Second, multiple types of evaluation and multiple data should preferably be used to evaluate social programs. Different paradigms define social progress in different ways. Positivists and post-positivists see social progress as something that can be assessed by social indicators. For interpretivists and constructionists, social progress is either subjectively experienced or socially constructed. Hence, static averaged social indicators cannot capture the fluid nature of social reality. For critical theorists, it is possible to chart social progress, but it must be understood in terms of the historical perspective that helps to reveal the underlying contradictions. Moreover, the concept of social progress is a rather philosophical one with an evaluative component attached to it. For example, while we enjoy many benefits of industrialization and urbanization, such as systematic provision of public utilities and affluence of material life, we also witness many problems, such as environmental pollution. Similarly technological advancement has brought us the Internet and smart phones. These technologies have freed many from the traditional working hours, but they have also often brought along increased stress and decreased face-to-face interaction between people. Thus, many skeptics would argue that the costs we have to pay for all these social progresses are much greater than the benefits. Unfortunately, there is a clear lack of discussion on the value bases of social indicators of the different schemes in the field. Assuming that social progress can be measured, the next question that should be asked is what indicators should be used to chart social change. For researchers with different orientations, they have different views on what indicators constitute good measures of social indicators. There are three other issues that should be taken into consideration. First, for many social indicators research frameworks, there is a lack of credible and critical theoretical model. Second, while some of the social indicators are common to all paradigms, there are also variations. While it is reasonable to gradually refine the indicators and find out whether it would be possible to use same indicators, the existing variations may also simply suggest that different value judgments are attached to different indicators. Third, experts are usually involved in deciding what indicators should be included in a system. According to social constructionists, this is a form of social construction as it would be likely that the selected social indicators would be biased to fit the world views of the experts or elite academics. The choice of indicators is particularly critical when selecting social indicators to measure non-Western cultures. For example, in the report titled ''Reconceptualizing social indicators in the Caribbean: A review and discussion'' (Economic Commission for Latin America and the Caribbean 2000), it is noted that a list of ''new indicators proposed by the ECLAC Latin American Meeting has been adapted to the Caribbean reality for the purpose of inclusion into the framework of existing indicators on the family'' (p. 16). To illustrate this problem further, an example from the European Union can be used. Atkinson et al. (2002) found in their analysis several properties that should be used in constructing social indicators that help to monitor national performance of the countries in the EU: 1. ''An indicator should identify the essence of the problem and have a clear and accepted normative interpretation. 2. An indicator should be robust and statistically validated. 3. An indicator should be responsive to effective policy interventions but not subject to manipulation. 4. An indicator should be measurable in a sufficiently comparable way across member states, and comparable as far as practicable with the standards applied internationally by the UN and the OECD. 5. An indicator should be timely and susceptible to revision. 6. The measurement of an indicator should not impose too large a burden on member states, on enterprises, or on the Union's citizens. Of these principles, three refer to the portfolio of indicators as a whole. 7. The portfolio of indicators should be balanced across different dimensions. 8. The indicators should be mutually consistent and the weight of single indicators in the portfolio should be proportionate. 9. The portfolio of indicators should be as transparent and accessible as possible to the citizens of the European Union' ' (p. 190) . There are at least four further implications that can be drawn from this. First, it is not clear why a ''clear and accepted normative interpretation'' is important and whether it is possible to achieve. Second, it is difficult, if not impossible for a social indicator not to be subjected to manipulation because of political consideration and the rise of social media. Third, the requirement that social indicators ''should not impose too large a burden on member states'' is problematic because the concept of large is very vague and leaves too much opportunity for interpretation. Fourth, the requirement of ''mutually consistent'' is a conceptual ideal only. For example, is the inclusion of indicators of economic development and poverty mutually consistent? The requirement of proportionate weight of single indicators is also unclear because weighting is commonly done in research. As statistical profiles are commonly used in social indicators research, many statistical issues are involved. Perhaps a critical evaluation of the Human Development Index can illustrate the existing methodological and conceptual issues highlighted in Puzzle 4. For example, Chakraborty (2002) argued against aggregation of scores across different domains for several reasons. First, aggregation would lead to a loss of important information because a composite index can contain only the information that the individual indicators are providing. ''It only presents that information in a form that is more convenient, more clearly understandable, and more amenable to some forms of analysis. But in the process, much useful information may be lost. This loss should be weighed against the gain from avoidance of the trouble of handling a large set of data'' (p. 1199). Chakraborty (2002) further pointed out that ranking in HDI is problematic because ''even for planning purposes, the usefulness of an ordinal ranking of districts is not very clear'' (p. 1200). As the assessment units in HDI are not measured at an individual level, it is impossible to tell how human development attributes are distributed amongst individuals. Finally, Chakraborty (2002) argued that HDI only gives a state rather than a process view on human development. According to Kovacevic (2011) , there are many criticisms of the Human Development Index, including inability to assess human development accurately, over-simplification based on low quality data, high correlation between different components of HDI and HDI, biases and measurement errors in international data, problems of using composite measures, focusing on averages with under-focus on the vulnerable groups. Several measures were proposed to improve the quality of the HDI. Klugman et al. (2011) further pointed out several limitations of the HDI. They noted that the HDI was criticized as ''too simplistic, while others who accepted its self-imposed limitations still questioned its choice of indicators and its computational methodology'' (p. 249). There was also a criticism on its lack of conceptual or theoretical basis (p. 259). Regarding the choice of indicators, HDI was criticized as excluding other important aspects of well-being such as equity, social justice, political openness, and happiness. Besides, the choice of indicators was questioned and there were queries on the choice of using equal weights. Although additional measures have been developed Paradigms, Puzzles… 985 Adjusted HDI, the Gender Inequality Index, and the Multidimensional Poverty Index), such measures have not received enough attention from the field. Klugman et al. (2011) further presented three reasons why it is not recommendable for societies to try to maximize the HDI. First, the indicators in HDI are not able to reflect various aspects of human development, and also their weighting is problematic. Second, focus on capability should not be regarded as the overriding concern. Finally, it is very likely that policymakers in any particular country possess much more superior information to help them in decision-making. In response to the criticisms on the Human Development Index several researchers have suggested alternative indicators. For example, Pereira and Mota (2016) proposed an alternative approach using the ELECTRE TRI-C multicriteria method. Neri (2016) argued for the use of perceived income, health and education to construct the Perceived Human Development Index (PHDI). One key issue in social science research is interpretation of findings. It is noteworthy that social indicators research (particularly those on social indicators profiles) is primarily descriptive in nature, leaving the question of explanation and interpretation a thorny one. Noll (2004) argued that ''using objective indicators starts from the assumption that living conditions can be judged to be favourable or unfavourable by comparing real conditions with normative criteria like values, goals or objectives. An important precondition, however, is that there is a societal or even political consensus about three key issues: first, about the dimensions that are relevant for welfare considerations; second, about good and bad conditions; third, about the direction in which society should move' ' (p. 158) . To illustrate the difficulties highlighted by Noll (2004) , we can refer to an example based on divorce rates. Can we interpret that an increase in divorce rates is an indication of ''negative'' social development? People with religious beliefs may think so. However, radical feminists and those who are committed to liberation of women might interpret it as a good sign for women to liberate themselves. Similarly, can we say that science and technology development brings forth positive social development? With rapid industrialization and urbanization, we witness the global warming effect, which might adversely affect the climate of the earth. In fact, the progress of humans also means worse development or even extinction for non-human species. These examples clearly highlight one important issue surrounding the interpretation of findings of social indicators researchthat value base affects how the findings are interpreted. In future, it would be helpful to further re-think the value choices and priorities in social indicators research. It is difficult to explain changes in social indicators and there are several issues involved. First, for cross-sectional studies utilizing social indicators, it is difficult to locate the causal relationships between changes in policies and social indicators outcomes. Because data are collected at a single time point, it would be impossible to ascertain the cause-effect relationship. Second, for longitudinal studies using macro social indicators, explanations may become post hoc in nature, if no a priori models are proposed. Third, as sample size based on social indicators at the national level is small (such as income per capita in different countries and places), multivariate statistical analyses may not necessarily meet the assumptions of multivariate statistics. Fourth, for subjective quality of life studies, it would be easier to propose models because data are usually collected at the individual level. Nevertheless, we also need good theoretical models on the determinants of subjective well-being, which are not commonly seen in the field. In short, we need more effort to formulate models that can account for changes in social indicators across time and place. The original idea of social indicators research is to utilize social reporting or auditing to identify gaps in social policies that will lead to social intervention. Thus, the problem of whether it is possible to use these findings to make policy and services initiatives arises. There are two observations regarding the potential use of social indicators research. First, application of findings to social intervention appears to be not sufficient. Brown and Corbett (1997) pointed out that social indicators have five basic policy uses where the ''hierarchical typology of uses which incur progressively exacting demands: description, monitoring, setting goals, outcomes-based accountability and evaluation'' (p. iii). According to Noll (2004) , ''while social indicators and social reports have successfully been used as descriptive monitoring tools, their application and use for purposes like setting goals and priorities, or the choice and evaluation of political programs still seems to be problematic and questionable'' (p. 176). Fortunately, they also pointed out that there was a rising trend of using social indicators for policy making particularly in European countries. Second, while some overarching social indicators such as the ones used in the Human Development Index may not be strongly linked to social policies, social indicators in specific areas have impact on social intervention. For example, in the area of substance abuse, rise in the substance abuse figures in the last quarter would alert the law enforcement agencies to see whether there are any changes in the supply of or demand for drugs and what additional initiatives should be stepped up. Similarly, rise in suicide rates among the elderly would prompt the Government to re-think about the elderly policies and whether adequate support is given for old people. In the area of health, statistics on infectious diseases would engender quick response from the Government to curb the problem as early as possible. In short, the relevance of social indicators related to social problems may capture greater attention of the Government than social indicators related to the general well-being of the place. Obviously, how those general social indicators can lead to social policies is an important issue to be addressed. In their paper, Land and Michalos pointed out several limitations in existing social indicators research. These included negligence of human agency, lack of multi-level studies, few longitudinal studies and few initiatives in building theories. They further pointed out that the drastic changes in social and economic structures (e.g., post-industrialization, globalization, digitalization, and rise of social media) have generated interesting topics for future research, such as social stratification, issues related to income disparity and migration issues. Obviously, these are important and interesting directions for future research. There are several other areas that researchers in the field of social indicators research may consider (Shek 2014) . The first issue is quality of life in the notion of ''uncertain world'' (Farber 1999; Inglehart 2000; Manski 2013) where uncertainty emerging from different domains constitutes excellent research opportunities. Regarding economic uncertainties, the economic cycle has become short and economic turbulence such as the financial tsunami in 2008 has become more frequent. Besides, vocational uncertainties caused by the use of automation and information technology have reduced the demand for labor and employment opportunities. Concerning political uncertainties, with the lost promise of capitalism and socialism, people in different parts of the world are also losing their confidence in the government. In particular, people lose confidence in those governments where welfare expenses and related debts are huge (e.g., Greece). For social uncertainties, the emergence of the ''M-shaped'' society (i.e., growing upper and lower class with shrinking middle class) and lack of upward mobility amongst young people are also important social conditions affecting quality of life. Besides, growing relationship uncertainties such as rise of marital disruption rates (e.g., divorce rates) and inter-group conflict also have adverse impact on quality of life. Culturally speaking, uncertainties surrounding changing values and world views are emerging. With growing materialism, consumption and moral diversity, it is theoretically and practically important to ask how such cultural uncertainties influence quality of life. Finally, growing global uncertainties such as health hazards (e.g., SARS and Ebola virus disease), global warming resulting in more extreme weather conditions, and mass extinction of non-human species also constitute quality of life issues for human beings throughout the world. Obviously, issues surrounding the uncertain world constitute excellent research opportunities for future studies on social indicators and quality of life. In fact, research in response to these issues clearly underscores the importance of social indicators research in monitoring the improving quality of life. The second research direction is to promote more inter-disciplinary collaboration. An examination of the current picture shows that social indicators research involves primarily scholars and researchers from the more ''macro'' disciplines, including economics, sociology and political science, although the concept of health-related quality of life is also embraced by psychologists, psychiatrists, nurses, occupational therapists, counselors and other allied health professionals on the more ''micro'' level. It is noteworthy that many studies conducted by such ''micro'' researchers have great relevance to social indicators, such as epidemiological studies on mental disorders and surveys on happiness in different patient populations. Hence, how to promote the collaboration between the ''macro'' and ''micro'' researchers would be a challenge. As a start, it would be conceptually stimulating to examine the concept of quality of life and the role of social indicators in different disciplines so that some conceptual integration can be achieved. For example, how we can shape the discourse that epidemiological studies on mental health issues address quality of life issues and that the related findings are regarded as social indicators would be exciting topics for future studies. As social scientists, critical appraisal is important. For positivists and post-positivists, it is important to use logical reasoning and critical debates in understanding human societies. For constructivists and critical theorists, adopting a reflective stand is important. Hence, it is indispensable to reflect on some of the fundamental methodological and interpretational issues in the social indicators movement. Essentially, we believe that human beings are capable of taking a step backward to ask questions about the reality. In this response article, the progress, paradigms, puzzles, promise and possible future directions surrounding the Social Indicators Movement are presented. Through this modest effort, we earnestly hope that the field can thrive further. Guiding principles for evaluators Social indicators: The EU and social inclusion Social indicators Social indicators and public policy in the age of devolution Issues in social indicators, composite indices and inequality Social development in Hong Kong: Development issues identified by social development index (SDI) Research design: Qualitative and quantitative approaches Research design: Qualitative, quantitative, and mixed methods approaches Paradigm wars: A basis for peaceful coexistence and beyond The landscape of qualitative research Reconceptualizing social indicators in the Caribbean: A review and discussion. Caribbean: ECLAC subregional headquarters for the Caribbean Evaluation standards and good practice Eco-pragmatism: Making sensible environmental decisions in an uncertain world An introduction to qualitative research Toward a conceptual framework for mixed-method evaluation designs The advantages and limitations of qualitative research in psychology and education Globalization and postmodern values Special issue: A tale of three Chinese societies: The quality of life and well-being of Chinese people in a changing world Program evaluation standards The HDI 2010: New controversies, old critiques United Nations Development Programme Human Development Reports Research Paper The structure of the scientific revolutions Fifty years after the social indicators movement: Has the promise been fulfilled? An Assessment and an Agenda for the Future, forthcoming in Social Indicators Research The Social Indicators Movement: Progress, Paradigms Quantitative and qualitative approaches in the study of poverty and adolescent development: Separation or integration? Naturalistic inquiry Paradigmatic controversies, contradictions, and emerging confluences Public policy in an uncertain world: Analysis and decisions Designing qualitative research Qualitative research design: An interactive approach: An interactive approach Implementing a multidimensional poverty measure using mixed methods and a participatory framework A perceived human development index Social work research methods: Qualitative and quantitative approaches Social indicators and quality of life research: Background, achievements and current trends Qualitative research & evaluation methods Designing qualitative studies. Qualitative research and evaluation methods Two decades of developments in qualitative inquiry Program evaluation Human development index based on electre tri-c multicriteria method: An application in the city of Recife Evaluation: A systematic approach The challenges of participatory construction of social indicators of well-being Qualitative interviewing: The art of hearing data Special issue: Quality of life of Chinese people in a changing world Special issue: Quality in life research: Responses to emerging issues in a changing world Applied research in quality of life (ARQOL): Where are we and issues for consideration Special Issue: Quality of life research in Chinese, Western and global contexts Evaluation of evaluation studies using qualitative research methods in the social work literature (1990-2003): Evidence that constitutes a wake-up call UNESCO's policy-relevant quality of life research program Handbook of mixed methods in social and behavioral research Qualitative studies in quality of life: Methodology and practice Standards for evaluation in the UN system Introducing qualitative research in psychology: Adventures in theory and method Acknowledgements The preparation for this paper is financially supported by The Hong Kong Jockey Club Charities Trust.