Starting your policy design: alternatives, theory of change and evaluation dimensions

1.

SCHOOL OF MANAGEMENT
POLICY Analysis 2022
– Policy Design practically.
LECTURE 3 – STARTING YOUR POLICY DESIGN:
ALTERNATIVES, THEORY OF CHANGE AND
EVALUATION DIMENSIONS
Giancarlo Vecchi

2.

1. CONSTRUCTING ALTERNATIVES
2
At this stage you have a definition of your policy
problem and evidences regarding the causes
of the problem, the categories affected, the
amount and localisation of the needs, etc.
By “alternatives” we mean something like “policy
options” or “alternative courses of action” or
“alternative strategies of intervention to solve
or mitigate the problem”.
Giancarlo Vecchi

3.

How many alternatives?
3
In general, it is better to elaborate not only one but
some alternatives (at least two!) to address the
policy problem.
In the last stages of the analysis three or four
alternatives are enough,
But in the beginning, we can make a list of all the
alternatives that are useful to consider.
Later on we will discard some obvious losers, combine
others, and reorganize still others into a single
“basic” alternative.
Giancarlo Vecchi

4.

4
Collect the alternatives that the scientific and
professional literature suggest regarding the
problem or similar cases
• Note if key political/social actors or institutions are
actively proposing or seem to have on their mind
and
• Try to invent alternatives that might prove to the
alternatives currently being discussed by the key
political actors. It’s good to brainstorm, to try to be
creative.
Giancarlo Vecchi

5.

Rules of a Brainstorming
5
Brainstorming or: everyone says something he/she thinks could
be important or in any case connected with the goal
• Set a Time Limit (e.g. 1 hour)
• A word is enough (do not talk to much)
• All the involved individuals must contribute and propose
something (we invite people that we think are useful)
• Nobody criticizes and nobody talks when another is talking
• Every proposal, even if it seems to be unlikely or very
strange, should be presented (we want develop creativity!)
• It is very useful to develop ideas proposed by others
• The list of proposals should be as long as possible (to help
connections and further developments)
• Assign a monitor to keep the brainstorm focused
Giancarlo Vecchi

6.

6
3.
A game on creativity
Connect the dots by drawing four straight, continuous lines
that pass through each of the nine dots, and without lifting the
pencil from the paper
Giancarlo Vecchi

7.

7
3.
Solution: Thinking outside the box
Giancarlo Vecchi

8.

(Brainstorming)
8
Exercise in group, Brainstorming
How to develop the teamwork for this Policy Analysis
course, in a condition of ‘distance working’ (remote
learning), with colleagues in many countries, with the
aim to involve all the members
15 minutes
Giancarlo Vecchi

9.

9
• Always we should include, in our first approach
to the problem solution, the alternative “Let
present trends continue undisturbed”.
• In fact we have to consider that the world is full of
naturally occurring changes, and some of these
ongoing changes might mitigate the problem on
which we are working without any intervention
(note: we are not characterizing this alternative
as “Do Nothing”; it is not possible to “do
nothing”); or the option “Let present trends
continue undisturbed” will reach accepted
outcomes at a minor costs.
Giancarlo Vecchi

10.

10
In most cases, however, the “let-present-trendscontinue” option will drop out of our final analysis.
If our problem definition will work well, we will end up
with an hypothesis of intervention that in our sights
could mitigate/solve the problem to some degree, on
the basis of a specific theory of change and a
specific implementation program, considering the
context conditions.
Giancarlo Vecchi

11.

Survey “Best/Good Practices”
11
11
• The rationale here is that the problem you are
studying is not unique, and that policymakers, public
managers in other jurisdictions and experts, have
dealt with a similar critical problem in some fashion,
perhaps not very different from the one you are
studying.
• See if you can track down some of these
past/current solutions and see if you can extrapolate
something to adapt to the situation you are studying.
Giancarlo Vecchi

12.

Use analogies
12
12
• Sometimes it pays to gather information about things
that are, on the surface, quite unlike the problem you
are studying but that, under the surface, show
instructive similarities.
• EG the solutions realized in other sectors
Giancarlo Vecchi

13.

2. Theory of Change
13
Theory of change = Program Theory => connecting
the contents of the policy (independent variable) with
the expected results (dependent variable =
outcomes), on the basis of a cause-effect model
Theory of Change = the process by witch change
comes about (for a category of individuals or
companies, an organization, a community, etc.)
The theory of change explain how an intervention will
be able to reach some expected results.
But how to define the ‘expected results’?
Giancarlo Vecchi

14.

2. Main dimensions of the THEORY OF CHANGE 14
Giancarlo Vecchi

15.

2. (Forward Mapping)
15
A strategy of Forward Mapping is needed:
We should use an hypothesis, based on a theory, to
connect, with a logical chain, an intervention with the
expected esteemed results:
• outcomes = the level of change in the problem that
we are expecting to reach with the intervention; we
can define:
• final outcomes and (some) intermediate outcomes
Giancarlo Vecchi

16.

2.
16
Outcomes
We should articulate the outcomes in:
a) Outcomes related to the direct beneficiaries of the
intervention (the target population)
b) Outcomes that regards other categories of
citizens/companies. Often, interventions will affect,
in a positive or negative way, not only the direct
recipients, but even other categories (side effects of
the policy)
Giancarlo Vecchi

17.

2.
17
The second relevant dimension
• Outputs = the final result of the production
processes; the realizations that we have to produce
to achieve the expected outcomes. We can have
intermediate and final outputs
• Implementation activities = the activities
(production and delivery processes, coordinated
tasks, etc.) to realize the ‘coherent’ outputs
• Inputs = all the needed resources to realize the
outputs and to reach the expected outcomes
Giancarlo Vecchi

18.

Chart showing the main steps of a change chain 18
Giancarlo Vecchi

19.

19
Example 1.
Problem: People with health problems (eg. Young
students with vitamins shortage)
Policy:
But how apples contribute to improved health?
Giancarlo Vecchi

20.

Theory of change: Cause-effect model
20
FIRST STEP: define the theory of change that can sustain the
positive modification of the problem situation
Additional
process:
Quercitin
Giancarlo Vecchi

21.

The solution-outcome chain
21
2d step: final outcome definition based on the theory of change
3d step: complete the chain
Giancarlo Vecchi

22.

Theory of change e the input-output-outcome design
Giancarlo Vecchi
22

23.

The outcomes chain
23
The chain of outcomes can be long
Giancarlo Vecchi

24.

24
Example/2
Problem: unemployment or occasional jobs and low or
no incomes, of specific categories of people: eg.:
single mothers with dependent children/s, exoffenders, dropouts of the school, etc.
Theory of change: IF X (independent variable =
causes): a period (of 18 months approx.) in a
protected work environment – providing work
attitudes, skills, habits etc. with supervisors, to
produce goods and services –
THEN Y (dependent variables/s = expected outcomes:
X should form the basis for the successful entry in the
regular labour market, improving wages Improving
income through a regular job in the labour market
Giancarlo Vecchi

25.

25
Example 2
Giancarlo Vecchi

26.

INDICATORS
26
• Broadly speaking, an indicator is a measure,
generally quantitative, that can be used to illustrate
and communicate the state/condition and the trend
of phenomena simply
• E.g. the performance of an organization, the
outcomes of a policy, the progress over time of
interventions.
Giancarlo Vecchi

27.

INDICATORS/2
27
• Objective indicators: the occurrences of given
phenomena which are measurable on an interval or
ratio scale, and amenable to the usual methods f
data analysis (e.g. the indicators and trends of air
pollution)
• Subjective indicators: those based on reports from
individuals on the ‘meaning’ aspects of their reality
and as such they represent psychological or
experience variables (see questionnaires,
interviews, opinion polls, etc.)
Giancarlo Vecchi