Primary Source: Patton, C. V, and Sawicki, D. S. (1993) Basic Methods of Policy Analysis, Second Edition. Prentice Hall: Englwood Cliffs, NJ.

Quasi-experimental designs are useful for real-world evaluations when a true experiment cannot be conducted – when we cannot randomly assign persons to treatment or control groups, when we cannot control the administration of the program or policy or restrict the policy to a treatment group or when programs are not directed at individuals.   The term quasi-experimental unfortunately implies to some persons that there is something wrong or second-rate about the design. Quite to the contrary, quasi-experimental approaches seek to maintain the logic of full experimentation but without the procedures, hardware, techniques, or control of the laboratory. Cook and Campbell’s Quasi-Experimentation provides extensive coverage of this approach and the design options available, including a description of appropriate statistical tests for the designs. There are two basic designs that planners and analysts should find very useful: the non-equivalent control group and the interrupted time-series designs. The non-equivalent control-group design involves the comparison of a treatment group and a similar (but not randomly selected) group before and after the policy or program is implemented. The interrupted time-series design involves the comparison of a treatment group several times both before and after the policy or program is implemented.

Nonequivalent Control-Group Design

The non-equivalent control-group design (Table 1) is read with the Ts and Cs indicating observations or measurements for the treatment and control groups, respectively, and the subscripts 1 and 2 indicating, respectively, preprogram and postprogram measurements. The dashed line indicates that the control group is logically selected so as to be similar, but not necessarily equivalent, to the treatment group. In short, a group, locale, or other entity [...]

Primary Source: Patton, C. V, and Sawicki, D. S. (1993) Basic Methods of Policy Analysis, Second Edition. Prentice Hall: Englwood Cliffs, NJ.

This approach compares actual postprogram data to targets set in prior periods, usually before implementation of the program. The analyst sets specific goals and targets for pre-established evaluation criteria for known time periods, and obtains data on the performance that actually occurs. Finally, the analyst compares actual performance to target performance, and seeks plausible explanations for differences that might have been brought about by program and nonprogram factors. In practice, this approach has been modified to involve comparison of actual program performance with implied rather than explicit targets. Targets can be set each year for one or more years in advance, and annual evaluations can be made of programs that have existed for a number of years (where preprogram data may not have much relevance). Although this method may be helpful in revealing year-to-year and other short-term changes, it does not allow us to determine the extent to which changes can be attributed to the policy or program.

These types of designs, which do not have equivalent experimental and control groups or pretests and posttests to help measure and determine the causes of change in key criteria, present a variety of interpretation problems, referred to as problems of internal and external validity. Internal validity refers to the ability todetermine whether unequivocal conclusions can be drawn about the experiment itself, and external validity refers to the ability to generalize from the experiment to other settings. In general, internal-validity problems for simple evaluation designs described above include the inability to determine whether observed changes occurred because of the program or because of non-experimental events. Such events [...]

Primary Source: Patton, C. V, and Sawicki, D. S. (1993) Basic Methods of Policy Analysis, Second Edition. Prentice Hall: Englwood Cliffs, NJ.

Perhaps the most widely used evaluation method, the before-and-after approach, involves comparing conditions (of peoples or locales) before a policy or program is implemented and after it has had a chance to make an impact. Although steps might be taken to identify program objectives and relevant evaluation criteria and to collect data prior to program implementation (rather than to reconstruct the data or rely on data collected for other purposes), this method requires that we assume that any differences between before-and-after data are a result of the policy or program.

This approach can be modified somewhat to compare actual postprogram data with no-action alternative as it was projected before implementing the program. This variation still fails to identify unanticipated consequences of no action and requires us to assume that the trend extrapolation reflects what would have indeed occurred.

Primary Source: Jeswani, H., and Azapagic, A. (2006) Sustainable Process Design SWOT Analysis in Report on the SWOT analysis of concepts, methods, and models potentially supporting LCA. Eds. Schepelmann, Ritthoff & Santman (Wuppertal Institute for Climate and Energy) & Jeswani and Azapagic (University of Manchester), pp 194-197

Level of analysis: Micro (processes and products)

Assessed aspects of sustainability: Environmental, Economic and Social

Main purpose of the assessment: To aid sustainable process design on a life cycle basis, integrating technical, economic, environmental and social criteria.

Description of the methodology: This approach enables identification of relevant sustainability criteria and indicators, comparison of alternatives, sustainability assessment of the overall process design and identification of ‘hot spots’ in the life cycle of the system.

Detailed description8

Conventional process design is based on technical and economic criteria, with the system boundary drawn around the process itself. This often leads to designs which are optimised economically around the plant but under-optimised upstream and downstream of the process. Recognising this deficiency in process design, Azapagic et al. (2004 & 2006) have proposed a general methodology for Sustainable Process Design (SPD) to enable consideration on a life cycle basis of economic, environmental and social criteria, alongside the usual technical criteria.

The methodology consists of the three following steps:

Identification of sustainability design criteria and stakeholders

The starting point in SPD is identification of sustainability criteria on which the alternatives, and later the whole design, will be assessed. At the initial stage of design when little information on the whole system may be available, it is proposed to use generic sustainability criteria that apply to any process; some examples are listed in Table 1. Further examples are provided by IChemE (2003), who have developed a set of sustainability indicators specifically for the [...]

Primary Source: Jeswani, H., and Azapagic, A. (2006) Strategic Assessment SWOT Analysis in Report on the SWOT analysis of concepts, methods, and models potentially supporting LCA. Eds. Schepelmann, Ritthoff & Santman (Wuppertal Institute for Climate and Energy) & Jeswani and Azapagic (University of Manchester), pp 182-188

Level of analysis: Micro, meso and macro.

Assessed aspects of sustainability: Environmental, Economic and Sociall possible, with a focus on environmental aspects

Main purpose of the assessment: Integrated assessment of environmental, economic and social sustainability.

Description of the methodology: SA framework allows using combination of various tools to assess all aspects of sustainability (economic, environmental and social) of policies, projects and plans.
Detailed Description
Sustainability Assessment is an umbrella term that can include a range of approaches or methodologies such as Sustainability Appraisal, Sustainability Impact Assessment, Integrated Sustainability Assessment5, or Integrated Assessment6, amongst others. Sustainability Assessment may be conducted by regulators for approval purposes, in what might be termed “external” Sustainability Assessment; while “internal” Sustainability Assessment is increasingly conducted by proponents themselves as a tool to improve internal decision-making and the overall sustainability of the final proposal (Pope, 2006). It is being applied to an ever-increasing range of decisions across the world, from policies, to strategic plans, to projects to trade agreements (Dalal-Clayton and Sadler, 2005), at different levels (local, regional, national, international, sectoral) and with different timing (ex ante, during, ex post).

There are many variations of the forms of sustainability-based assessment existing throughout the world. Many international organisations and associations have adopted sustainability and have been developing indicators for sustainability assessments. Some of these organisations and associations include Organisation for Economic Cooperation and Development (OECD), World Trade Organisation (WTO), World Economic Forum’s (Pilot Environmental Sustainability Index), the Dow Jones Sustainability Indexes, [...]