7.0       Criteria for Indicators: Selecting the right ones

As the potential list of indicators for any one environmental public health issue or relationship is exceptionally exhaustive, within a program some selection must be made to identify and retain a manageable number that still allows the program to meet its set goals or influence / achieve its desired outcomes.  The purpose of indicator selection and the fact that any such selection will appear, for other purposes incomplete, must be kept in mind.  Any list of chosen indicators will also be somewhat temporary, reflecting our current state of knowledge and ability to act at this time.  In order to guide indicator identification and selection, criteria must be chosen that ensure that the appropriate indicators are retained to meet the desired goals.  The literature is rich in potential criteria and indicators for a variety of purposes.  It is essential that each program develops its own set of criteria, however, some are common and should be included in most, if not all cases.  We propose a rationale for filtering the candidate indicators in which we apply two categories of criteria (science-based and use-based; Eyles et al. 1996) which include consideration of practical program needs criteria such as those discussed by Rump (1996) (e.g. relevant to the program goals, relevant to regional culture and context, clearly understood by target audiences, etc.).  The two categories of criteria proposed here are directed at ensuring that indicators of high scientific quality and those best suited to meet the specific needs and goals of the program are identified and retained. 

 

Scientific Criteria

Scientific criteria are generic to the issue of scientific quality and according to Eyles et al. (1996) include:

1.         Data availability and suitability: Because of cost and time limitations in many programs consideration must be given to the current availability of data and in considering data already available, the original intent or purpose of its collection must be thought of so as not to compromise the data in meeting other scientific criteria;

2.         Indicator validity (assessed in a variety of forms):

-        Face validity: the indicator is a reasonable measure as assessed by the users;

-        Construct validity: variables that claim to describe the same dimensions do so;

-        Predictive validity: the measure correctly predicts a situation which would be caused by the phenomenon being measured;

-        Convergent validity: several measures collected or structured in different ways all move similarly;

-        Content validity: the fit between the indicator and the object being observed;

-        Theoretical and empirical validity, as discussed by Hancock et al. (1999), adds to this list, assessing whether the indicator measures an important health determinant or dimension;

3.         Indicator representativeness: a measure of the indicators appropriateness to represent a specific dimension of concern within the phenomenon of interest;

4.         Reliability: measured by consistency over a number of repetitions, to ensure the measurement is the same, or very close to (minimal error variance), over a variety of measurements and under a variety of conditions.  Reliability is a prerequisite to validity;

5.         Ability to disaggregate: disaggregating indicators are those that are able to be broken down into other variables telling us much more than the single measure it represents.  The OECD (1976) identify disaggregation by ascribed groups (e.g. age, sex, race, region), well-being (e.g. years of education, employment status) and contextual (e.g. size of community, type of occupation) characteristics.