View this notebook on a map
![]() |
|
Choice under uncertainty
This area represents the heartland of decision theory. The procedure now
referred to as expected
value was known from the 17th century. Blaise Pascal invoked it in his famous wager
(see below), which is contained in his Pensées, published in 1670. The idea of expected value is that, when faced with
a number of actions, each of which could give rise to more than one possible
outcome with different probabilities, the rational procedure is to identify all
possible outcomes, determine their values (positive or negative) and the
probabilities that will result from each course of action, and multiply the two
to give an expected value. The action to be chosen should be the one that
gives rise to the highest total expected value. In 1738, Daniel Bernoulli published an influential
paper entitled Exposition of a New Theory on the Measurement of Risk, in
which he uses the St. Petersburg paradox to show that
expected value theory must be normatively wrong. He also gives an example in which
a Dutch merchant is trying to decide whether to insure a cargo being sent from
Amsterdam to St Petersburg in winter,
when it is known that there is a 5% chance that the ship and cargo will be lost.
In his solution, he defines a utility function and computes expected utility
rather than expected financial value.
In the 20th century, interest was reignited by Abraham Wald's 1939 paper pointing out that the two central concerns of
orthodox
statistical theory at that time, namely statistical hypothesis testing
and statistical estimation theory, could both be regarded as
particular special cases of the more general decision problem. This paper
introduced much of the mental landscape of modern decision theory, including loss functions, risk functions, admissible decision rules, a priori
distributions, Bayes decision
rules, and minimax decision rules.
The phrase "decision theory" itself was first used in 1950 by E. L. Lehmann.
The rise of subjective probability theory, from the
work of Frank
Ramsey, Bruno de
Finetti, Leonard
Savage and others, extended the scope of expected utility theory to
situations where only subjective probabilities are available. At this time it
was generally assumed in economics
that people behave as rational agents and thus expected utility theory also provided a theory
of actual human decision-making behaviour under risk. The work of Maurice Allais and Daniel Ellsberg showed
that this was clearly not so. The prospect theory of Daniel Kahneman and Amos Tversky placed behavioural
economics on a more evidence-based footing. It emphasised
that in actual human (as opposed to normatively correct) decision-making "losses
loom larger than gains", people are more focused on changes in their
utility states than the states themselves and estimation of subjective
probabilities is severely biased by anchoring.
Pascal's Wager of choice under uncertainty
Pascal's Wager is
a classic example of a choice under uncertainty. The uncertainty, according to
Pascal, is whether or
not God exists. The personal belief or
non-belief in God is the choice to be made. However, the reward for belief in
God if God actually does exist is infinite. Therefore, however small the
probability of God's existence, the expected value of belief exceeds that of
non-belief, so it is better to believe in God.
Alternatives to probability theory
A highly controversial issue is whether one can replace the use of
probability in decision theory by other alternatives. The proponents of fuzzy logic, possibility
theory, Dempster-Shafer theory and info-gap
decision theory maintain that probability is only one of many alternatives
and point to many examples where non-standard alternatives have been implemented
with apparent success. Advocates of probability theory point to
- the Dutch book paradoxes
of Bruno de
Finetti as illustrative of the theoretical difficulties that can arise from
departures from the probability axioms and to
- the complete class
theorems which show that all admissible decision rules are
equivalent to a Bayesian decision rule with some prior distribution (possibly improper) and
some utility function. Thus, for any decision rule generated by
non-probabilistic methods, either there is an equivalent rule derivable
by Bayesian means, or there is a
rule derivable by Bayesian means which is never worse and (at least) sometimes
better.
Economics and business
Economists have used game theory to analyze a wide array of economic phenomena, including auctions, bargaining, duopolies, oligopolies, social network formation, and voting systems. This research usually focuses on particular sets of strategies known as equilibria in games. These "solution concepts" are usually based on what is required by norms of rationality. The most famous of these is the Nash equilibrium. A set of strategies is a Nash equilibrium if each represents a best response to the other strategies. So, if all the players are playing the strategies in a Nash equilibrium, they have no incentive to deviate, since their strategy is the best they can do given what others are doing.
The payoffs of the game are generally taken to represent the utility of individual players. Often in modeling situations the payoffs represent money, which presumably corresponds to an individual's utility. This assumption, however, can be faulty.
A prototypical paper on game theory in economics begins by presenting a game that is an abstraction of some particular economic situation. One or more solution concepts are chosen, and the author demonstrates which strategy sets in the presented game are equilibria of the appropriate type. Naturally one might wonder to what use should this information be put. Economists and business professors suggest two primary uses.
Descriptive
A three stage Centipede Game
The first use is to inform us about how actual human populations behave. Some scholars believe that by finding the equilibria of games they can predict how actual human populations will behave when confronted with situations analogous to the game being studied. This particular view of game theory has come under recent criticism. First, it is criticized because the assumptions made by game theorists are often violated. Game theorists may assume players always act rationally to maximize their wins (the Homo economicus model), but real humans often act either irrationally, or act rationally to maximize the wins of some larger group of people (altruism). Game theorists respond by comparing their assumptions to those used in physics. Thus while their assumptions do not always hold, they can treat game theory as a reasonable scientific ideal akin to the models used by physicists. However, additional criticism of this use of game theory has been levied because some experiments have demonstrated that individuals do not play equilibrium strategies. For instance, in the Centipede game, Guess 2/3 of the average game, and the Dictator game, people regularly do not play Nash equilibria. There is an ongoing debate regarding the importance of these experiments. [2]
Alternatively, some authors claim that Nash equilibria do not provide predictions for human populations, but rather provide an explanation for why populations that play Nash equilibria remain in that state. However, the question of how populations reach those points remains open.
Some game theorists have turned to evolutionary game theory in order to resolve these worries. These models presume either no rationality or bounded rationality on the part of players. Despite the name, evolutionary game theory does not necessarily presume natural selection in the biological sense. Evolutionary game theory includes both biological as well as cultural evolution and also models of individual learning (for example, fictitious play dynamics).
Normative
| The Prisoner's Dilemma |
|
Cooperate |
Defect |
| Cooperate |
2, 2 |
0, 3 |
| Defect |
3, 0 |
1, 1 |
On the other hand, some scholars see game theory not as a predictive tool for the behavior of human beings, but as a suggestion for how people ought to behave. Since a Nash equilibrium of a game constitutes one's best response to the actions of the other players, playing a strategy that is part of a Nash equilibrium seems appropriate. However, this use for game theory has also come under criticism. First, in some cases it is appropriate to play a non-equilibrium strategy if one expects others to play non-equilibrium strategies as well. For an example, see Guess 2/3 of the average.
Second, the Prisoner's Dilemma presents another potential counterexample. In the Prisoner's Dilemma, each player pursuing his own self-interest leads both players to be worse off than had they not pursued their own self-interests. Some scholars believe that this demonstrates the failure of game theory as a recommendation for behavior.
A somewhat more abstract view of probability
Mathematicians usually take probability theory to be the study of probability
spaces and random variables — an approach introduced by Kolmogorov in the
1930s. A probability space is a triple  ,
where
- Ω is a non-empty set, sometimes called the
"sample space", each of whose members is thought of as a potential outcome of a
random experiment. For example, if 100 voters are to be drawn randomly from
among all voters in California and asked whom they will vote for governor, then
the set of all sequences of 100 Californian voters would be the sample space Ω.
is a σ-algebra of subsets
of Ω - its members are called "events". For example
the set of all sequences of 100 Californian voters in which at least 60 will
vote for Schwarzenegger is identified with the "event" that at least 60 of the
100 chosen voters will so vote. To say that
is a σ-algebra implies per definition that it contains Ω, that the complement of any event is an event, and that
the union of any (finite or countably infinite) sequence of events is an event.
It is important to note that P is a
function defined on
and not on Ω, and often not on the complete powerset
either. Not every set of outcomes is an event.
If Ω is denumerable we almost always define
as the power set of Ω, i.e
which is trivially a σ-algebra and the biggest one we can create using Ω. In a discrete space we can therefore omit 
and just write (Ω,P) to define it. If on the
other hand Ω is non-denumerable and we use
we get into trouble defining our probability measure P because
is too 'huge', i.e. there will often be sets to which it will be impossible to
assign a unique measure, giving rise to problems like the Banach–Tarski paradox. So we have
to use a smaller σ-algebra
(e.g. the Borel algebra
of Ω, which is the smallest σ-algebra that makes all
open sets measurable).
A random variable
X is a measurable function on Ω. For example, the number of voters who will vote for
Schwarzenegger in the aforementioned sample of 100 is a random variable.
If X is any random variable, the notation
 ,
is shorthand for  ,
assuming that " "
is an "event".
For an algebraic alternative to Kolmogorov's approach, see algebra of random variables.
Anti-patterns, also referred to as pitfalls, are classes of commonly-reinvented bad solutions to problems. They are studied, as a category, in order that they may be avoided in the future, and that instances of them may be recognized when investigating non-working systems.
- Accidental complexity: Introducing unnecessary complexity into a solution
- Action at a distance: Unexpected interaction between widely separated parts of a system
- Accumulate and fire: Setting parameters for subroutines in a collection of global variables
- Blind faith: Lack of checking of (a) the correctness of a bug fix or (b) the result of a subroutine
- Boat anchor: Retaining a part of a system that no longer has any use
- Busy spin: Consuming CPU while waiting for something to happen, usually by repeated checking instead of proper messaging
- Caching failure: Forgetting to reset an error flag when an error has been corrected
- Checking type instead of interface: Checking that an object has a specific type when only a certain contract is required
- Code momentum: Over-constraining part of a system by repeatedly assuming things about it in other parts
- Coding by exception: Adding new code to handle each special case as it is recognised
- Cryptic code: Using abbreviatures in variables or functions instead of complete (self-descriptive) names
- Double-checked locking: Checking, before locking, if this is necessary in a way which may fail with e.g. modern hardware or compilers.
- Hard code: Embedding assumptions about the environment of a system at many points in its implementation
- Lava flow: Retaining undesirable (redundant or low-quality) code because removing it is too expensive or has unpredictable consequences
- Magic numbers: Including unexplained numbers in algorithms
- Procedural code (when another paradigm is more appropriate)
- Spaghetti code: Systems whose structure is barely comprehensible, especially because of misuse of code structures
Statistics and the Theory of Measurement
D. J. Hand
Journal of the Royal Statistical Society. Series A (Statistics in Society), Vol. 159, No. 3 (1996) , pp. 445-492
Abstract
Just as there are different interpretations of probability, leading to different kinds of inferential statements and different conclusions about statistical models and questions, so there are different theories of measurement, which in turn may lead to different kinds of statistical model and possibly different conclusions. This has led to much confusion and a long running debate about when different classes of statistical methods may legitimately be applied. This paper outlines the major theories of measurement and their relationships and describes the different kinds of models and hypotheses which may be formulated within each theory. One general conclusion is that the domains of applicability of the two major theories are typically different, and it is this which helps apparent contradictions to be avoided in most practical applications.
A Microeconomic View of Data Mining
Jon Kleinberg1  , Christos Papadimitriou2 and Prabhakar Raghavan3
| (1) |
Department of Computer Science, Cornell University,
Ithaca, NY, 14853 |
| (2) |
Computer Science Division, Soda Hall, UC Berkeley,
CA, 94720 |
| (3) |
IBM Almaden Research Center, 650 Harry Road, San
Jose, CA, 95120 |
Abstract We present a rigorous
framework, based on optimization, for evaluating data mining operations such as
associations and clustering, in terms of their utility in decision-making. This
framework leads quickly to some interesting computational problems related to
sensitivity analysis, segmentation and the theory of games.
market
segmentation - optimization - clustering
| Adversarial
classification |
|
| Full text |
 Pdf (215 KB)
|
| Source |
Conference on Knowledge
Discovery in Data archive
Proceedings of the tenth
ACM SIGKDD international conference on Knowledge discovery and data mining
table of contents
Seattle, WA, USA
Pages: 99 - 108
Year of Publication: 2004
ISBN:1-58113-888-1 |
| Authors |
| Nilesh Dalvi |
University of Washington - Seattle, Seattle,
WA |
| Pedro Domingos |
University of Washington - Seattle, Seattle,
WA |
| Mausam |
University of Washington - Seattle, Seattle,
WA |
| Sumit Sanghai |
University of Washington - Seattle, Seattle,
WA |
| Deepak Verma |
University of Washington - Seattle, Seattle,
WA | |
| Sponsors |
SIGMOD: ACM Special Interest Group on Management of Data
SIGKDD: ACM Special Interest Group on Knowledge Discovery in
Data ACM:
Association for Computing Machinery |
| Publisher |
ACM Press New York, NY, USA
| |
 ABSTRACT
Essentially all data mining algorithms assume that the
data-generating process is independent of the data miner's activities. However,
in many domains, including spam detection, intrusion detection, fraud detection,
surveillance and counter-terrorism, this is far from the case: the data is
actively manipulated by an adversary seeking to make the classifier produce
false negatives. In these domains, the performance of a classifier can degrade
rapidly after it is deployed, as the adversary learns to defeat it. Currently
the only solution to this is repeated, manual, ad hoc reconstruction of
the classifier. In this paper we develop a formal framework and algorithms for
this problem. We view classification as a game between the classifier and the
adversary, and produce a classifier that is optimal given the adversary's
optimal strategy. Experiments in a spam detection domain show that this approach
can greatly outperform a classifier learned in the standard way, and (within the
parameters of the problem) automatically adapt the classifier to the adversary's
evolving manipulations. |
| Adversarial learning |
|
| Full text |
Pdf (774 KB)
|
| Source |
Conference on Knowledge
Discovery in Data archive
Proceeding of the eleventh
ACM SIGKDD international conference on Knowledge discovery in data mining
table of contents
Chicago, Illinois, USA
Pages: 641 - 647
Year of Publication: 2005
ISBN:1-59593-135-X |
| Authors |
|
| Sponsors |
SIGKDD: ACM Special Interest Group on Knowledge Discovery in
Data ACM:
Association for Computing Machinery |
| Publisher |
ACM Press New York, NY, USA
| |
ABSTRACT
Many classification tasks, such as spam filtering, intrusion
detection, and terrorism detection, are complicated by an adversary who wishes
to avoid detection. Previous work on adversarial classification has made the
unrealistic assumption that the attacker has perfect knowledge of the classifier
[2]. In this paper, we introduce the adversarial classifier reverse engineering
(ACRE) learning problem, the task of learning sufficient information about a
classifier to construct adversarial attacks. We present efficient algorithms for
reverse engineering linear classifiers with either continuous or Boolean
features and demonstrate their effectiveness using real data from the domain of
spam filtering. |
Writing a good grant proposal Simon Peyton Jones and Alan Bundy
Writing a good research grant proposal is not easy. This document is an
attempt to collect together a number of suggestions about what makes a good
proposal. It is inevitably a personal view on the part of the authors; we would
welcome feedback and suggestions from others.
APPROACHING A PROPOSAL The first and most obvious thing to do is to
read the advice offered by your funding agency. In the case of EPSRC, the
primary funding body for computing science research, there is a "Guide to EPSRC
Research Grants". We make no attempt to duplicate the material in the EPSRC
guide or any other; you must get yourself a copy and follow the guidance
closely.
The most substantial part of any grant application is some form of "Case for
Support". It is this case which will persuade, or fail to persuade, your funding
body of the value of your proposal. Proposals range very widely indeed in their
quality. You can improve your chances enormously simply by ruthlessly writing
and rewriting. This document is entirely about improving your case for support.
There are two vital facts to bear in mind:
- Your case for support will, with luck, be read by one or two experts in your
field. But the programme manager, and most members of the panel that judges your
proposal against others, won't be expert. You must, must, must write your
proposal for their benefit too.
- Remember that programme managers and panel members see tens or hundreds of
cases for support, so you have one minute or less to grab your reader's
attention.
Based on these facts, here are two Golden Rules:
CRITERIA FOR A GOOD GRANT PROPOSAL Most funding agencies apply similar
criteria to the evaluation of proposals. We discuss these below. It is important
to address these criteria directly in your case for support. A proposal which
fails to meet them will be rejected regardless of the quality of its source.
Otherwise, there is a danger of discriminating unfairly in favour of well-known
applicants.
Major criteria Here are the major criteria against which your proposal
will be judged. Read through your case for support repeatedly, and ask whether
the answers to the questions below are clear, even to a non-expert.
- Does the proposal address a well-formulated problem?
- Is it a research problem, or is it just a routine application of
known techniques?
- Is it an important problem, whose solution will have useful
effects?
- Is special funding necessary to solve the problem, or to solve it
quickly enough, or could it be solved using the normal resources of a
well-found laboratory?
- Do the proposers have a good idea on which to base their work? The
proposal must explain the idea in sufficient detail to convince the reader that
the idea has some substance, and should explain why there is reason to believe
that it is indeed a good idea. It is absolutely not enough merely to identify a
wish-list of desirable goals (a very common fault). There must be significant
technical substance to the proposal.
- Does the proposal explain clearly what work will be done? Does it
explain what results are expected and how they will be evaluated? How would it
be possible to judge whether the work was successful?
- Is there evidence that the proposers know about the work that others
have done on the problem? This evidence may take the form of a short review
as well as representative references.
- Do the proposers have a good track record, both of doing good research
and of publishing it? A representative selection of relevant publications
by the proposers should be cited. Absence of a track record is clearly not a
disqualifying characteristic, especially in the case of young researchers, but a
consistent failure to publish raises question marks.
Secondary criteria Some secondary criteria may be applied to separate
closely-matched proposals. It is often essentially impossible to distinguish in
a truly objective manner among such proposals and it is sad that it is necessary
to do so. The criteria are ambiguous and conflict with each other, so the
committee simply has to use its best judgement in making its recommendations.
- An applicant with little existing funding may deserve to be placed ahead of
a well- funded one. On the other hand, existing funding provides evidence of a
good track record.
- There is merit in funding a proposal to keep a strong research team
together; but it is also important to give priority to new researchers in the
field.
- An attempt is made to maintain a reasonable balance between different
research areas, where this is possible.
- Evidence of industrial interest in a proposal, and of its potential for
future exploitation will usually count in its favour. The closer the research is
to producing a product the more industrial involvement is required and this
should usually include some industrial contribution to the project. The case for
support should include some `route to market' plan, ie you should have thought
about how the research will eventually become a product --- identifying an
industrial partner is usually part of such a plan.
- A proposal will benefit if it is seen to address recommendations of
Technology Foresight. It is worth looking at the relevant Foresight Panel
reports and including quotes in your case for support that relate to your
proposal.
Cost-effectiveness Finally, the programme manager tries to ensure that
his or her budget is to be used in a cost-effective manner. Each proposal which
has some chance of being funded is examined, and the programme manager may lop
costs off an apparently over-expensive project.Such cost reduction is likely to
happen if the major costs of staff and equipment are not given clear, individual
justification.
COMMON SHORTCOMINGS Here are some of the ways in which proposals often
fail to meet these criteria.
A new idea is claimed but insufficient technical details of the idea are
given for the committee to be able to judge whether it looks promising.
Since the committee cannot be expert in all areas there is a danger of
overwhelming them with technical details, but it is better to err by
overwhelming them than by underwhelming them. They will usually get an expert
referee to evaluate your idea.
The proposers seem unaware of related research. Related work must
be mentioned, if only to be dismissed. Otherwise, the committee will think that
the proposers are ignorant and, therefore, not the best group to fund. The case
for support should have a list of references like any paper, and you should look
at it to check it has a balanced feel - your referee will do so. Do not make the
mistake of giving references only to your own work!
The proposed research has already been done - or appears to have been
done. Rival solutions must be discussed and their inadequacies revealed.
The proposal is badly presented, or incomprehensible to all but an
expert in the field. Remember that your proposal will be read by
non-experts as well as (hopefully) experts. A good proposal is simultaneously
comprehensible to non-experts, while also convincing experts that you know your
subject. Keep highly-technical material in well-signposted section(s); avoid it
in the introduction.
The proposers seem to be attempting too much for the funding requested
and time-scale envisaged. Such lack of realism may reflect a poor
understanding of the problem or poor research methodology.
The proposal is too expensive for the probable gain. If it is easy
to see how to cut the request for people/equipment/travel, etc. to something
more reasonable then it might be awarded in reduced form. More likely, it will
be rejected.
The proposers institution should be funding it. Research agencies
will usually only fund research that requires resources beyond that which might
be expected in a "well-found laboratory" --- indeed, this is part of the charter
of the research councils. If it looks like your proposal might be done by a PhD
student on the departmental computer then that is what should happen. If the
proposer's laboratory is not "well-found" then this is taken to be a vote of
no-confidence in the proposer by his/her institution. Doubtless there
are other common grounds for failure that have been omitted. If you know of any
please let us know!.
Often, one can tell from independent knowledge of the proposers or by reading
between the lines of the proposal, that the criteria could have been met if a
little bit more thought had gone into the proposal. There is a clear question
being addressed by the research, but the proposers failed to clarify what it
was. The proposers are aware of related research, but they failed to discuss it
in the proposal. The proposers do have some clear technical ideas, but they
thought it inappropriate to go into such detail in the proposal. Unfortunately,
there is a limit to which a funding agencies can give such cases the benefit of
the doubt. It is not fair for referees to overlook shortcomings in proposals of
which they have personal knowledge if similar shortcomings are not overlooked in
proposals which they have not encountered before. In any case, proposals which
do meet the criteria deserve precedence.
CONCLUSION We hope that this document will help you to write better
grant proposals, and hence to be more successful in obtaining funds for your
research. This article is not just about writing better grant proposals to
obtain more money. The basic set-up of peer-reviewed grants of limited duration
is a sensible one. It compels researchers regularly to review and re-justify the
direction of their work. Behind poorly presented grant proposals often lie
poorly-reasoned research plans. Perhaps if we can improve the quality of
Computer Science proposals we will also improve the quality of Computer Science
research.
Simon Peyton Jones, simonpj@microsoft.com
Alan Bundy, bundy@aisb.ed.ac.uk
Writing a good grant proposal Simon Peyton Jones and Alan Bundy
Writing a good research grant proposal is not easy. This document is an
attempt to collect together a number of suggestions about what makes a good
proposal. It is inevitably a personal view on the part of the authors; we would
welcome feedback and suggestions from others.
APPROACHING A PROPOSAL The first and most obvious thing to do is to
read the advice offered by your funding agency. In the case of EPSRC, the
primary funding body for computing science research, there is a "Guide to EPSRC
Research Grants". We make no attempt to duplicate the material in the EPSRC
guide or any other; you must get yourself a copy and follow the guidance
closely.
The most substantial part of any grant application is some form of "Case for
Support". It is this case which will persuade, or fail to persuade, your funding
body of the value of your proposal. Proposals range very widely indeed in their
quality. You can improve your chances enormously simply by ruthlessly writing
and rewriting. This document is entirely about improving your case for support.
There are two vital facts to bear in mind:
- Your case for support will, with luck, be read by one or two experts in your
field. But the programme manager, and most members of the panel that judges your
proposal against others, won't be expert. You must, must, must write your
proposal for their benefit too.
- Remember that programme managers and panel members see tens or hundreds of
cases for support, so you have one minute or less to grab your reader's
attention.
Based on these facts, here are two Golden Rules:
CRITERIA FOR A GOOD GRANT PROPOSAL Most funding agencies apply similar
criteria to the evaluation of proposals. We discuss these below. It is important
to address these criteria directly in your case for support. A proposal which
fails to meet them will be rejected regardless of the quality of its source.
Otherwise, there is a danger of discriminating unfairly in favour of well-known
applicants.
Major criteria Here are the major criteria against which your proposal
will be judged. Read through your case for support repeatedly, and ask whether
the answers to the questions below are clear, even to a non-expert.
- Does the proposal address a well-formulated problem?
- Is it a research problem, or is it just a routine application of
known techniques?
- Is it an important problem, whose solution will have useful
effects?
- Is special funding necessary to solve the problem, or to solve it
quickly enough, or could it be solved using the normal resources of a
well-found laboratory?
- Do the proposers have a good idea on which to base their work? The
proposal must explain the idea in sufficient detail to convince the reader that
the idea has some substance, and should explain why there is reason to believe
that it is indeed a good idea. It is absolutely not enough merely to identify a
wish-list of desirable goals (a very common fault). There must be significant
technical substance to the proposal.
- Does the proposal explain clearly what work will be done? Does it
explain what results are expected and how they will be evaluated? How would it
be possible to judge whether the work was successful?
- Is there evidence that the proposers know about the work that others
have done on the problem? This evidence may take the form of a short review
as well as representative references.
- Do the proposers have a good track record, both of doing good research
and of publishing it? A representative selection of relevant publications
by the proposers should be cited. Absence of a track record is clearly not a
disqualifying characteristic, especially in the case of young researchers, but a
consistent failure to publish raises question marks.
Secondary criteria Some secondary criteria may be applied to separate
closely-matched proposals. It is often essentially impossible to distinguish in
a truly objective manner among such proposals and it is sad that it is necessary
to do so. The criteria are ambiguous and conflict with each other, so the
committee simply has to use its best judgement in making its recommendations.
- An applicant with little existing funding may deserve to be placed ahead of
a well- funded one. On the other hand, existing funding provides evidence of a
good track record.
- There is merit in funding a proposal to keep a strong research team
together; but it is also important to give priority to new researchers in the
field.
- An attempt is made to maintain a reasonable balance between different
research areas, where this is possible.
- Evidence of industrial interest in a proposal, and of its potential for
future exploitation will usually count in its favour. The closer the research is
to producing a product the more industrial involvement is required and this
should usually include some industrial contribution to the project. The case for
support should include some `route to market' plan, ie you should have thought
about how the research will eventually become a product --- identifying an
industrial partner is usually part of such a plan.
- A proposal will benefit if it is seen to address recommendations of
Technology Foresight. It is worth looking at the relevant Foresight Panel
reports and including quotes in your case for support that relate to your
proposal.
Cost-effectiveness Finally, the programme manager tries to ensure that
his or her budget is to be used in a cost-effective manner. Each proposal which
has some chance of being funded is examined, and the programme manager may lop
costs off an apparently over-expensive project.Such cost reduction is likely to
happen if the major costs of staff and equipment are not given clear, individual
justification.
COMMON SHORTCOMINGS Here are some of the ways in which proposals often
fail to meet these criteria.
A new idea is claimed but insufficient technical details of the idea are
given for the committee to be able to judge whether it looks promising.
Since the committee cannot be expert in all areas there is a danger of
overwhelming them with technical details, but it is better to err by
overwhelming them than by underwhelming them. They will usually get an expert
referee to evaluate your idea.
The proposers seem unaware of related research. Related work must
be mentioned, if only to be dismissed. Otherwise, the committee will think that
the proposers are ignorant and, therefore, not the best group to fund. The case
for support should have a list of references like any paper, and you should look
at it to check it has a balanced feel - your referee will do so. Do not make the
mistake of giving references only to your own work!
The proposed research has already been done - or appears to have been
done. Rival solutions must be discussed and their inadequacies revealed.
The proposal is badly presented, or incomprehensible to all but an
expert in the field. Remember that your proposal will be read by
non-experts as well as (hopefully) experts. A good proposal is simultaneously
comprehensible to non-experts, while also convincing experts that you know your
subject. Keep highly-technical material in well-signposted section(s); avoid it
in the introduction.
The proposers seem to be attempting too much for the funding requested
and time-scale envisaged. Such lack of realism may reflect a poor
understanding of the problem or poor research methodology.
The proposal is too expensive for the probable gain. If it is easy
to see how to cut the request for people/equipment/travel, etc. to something
more reasonable then it might be awarded in reduced form. More likely, it will
be rejected.
The proposers institution should be funding it. Research agencies
will usually only fund research that requires resources beyond that which might
be expected in a "well-found laboratory" --- indeed, this is part of the charter
of the research councils. If it looks like your proposal might be done by a PhD
student on the departmental computer then that is what should happen. If the
proposer's laboratory is not "well-found" then this is taken to be a vote of
no-confidence in the proposer by his/her institution. Doubtless there
are other common grounds for failure that have been omitted. If you know of any
please let us know!.
Often, one can tell from independent knowledge of the proposers or by reading
between the lines of the proposal, that the criteria could have been met if a
little bit more thought had gone into the proposal. There is a clear question
being addressed by the research, but the proposers failed to clarify what it
was. The proposers are aware of related research, but they failed to discuss it
in the proposal. The proposers do have some clear technical ideas, but they
thought it inappropriate to go into such detail in the proposal. Unfortunately,
there is a limit to which a funding agencies can give such cases the benefit of
the doubt. It is not fair for referees to overlook shortcomings in proposals of
which they have personal knowledge if similar shortcomings are not overlooked in
proposals which they have not encountered before. In any case, proposals which
do meet the criteria deserve precedence.
CONCLUSION We hope that this document will help you to write better
grant proposals, and hence to be more successful in obtaining funds for your
research. This article is not just about writing better grant proposals to
obtain more money. The basic set-up of peer-reviewed grants of limited duration
is a sensible one. It compels researchers regularly to review and re-justify the
direction of their work. Behind poorly presented grant proposals often lie
poorly-reasoned research plans. Perhaps if we can improve the quality of
Computer Science proposals we will also improve the quality of Computer Science
research.
Simon Peyton Jones, simonpj@microsoft.com
Alan Bundy, bundy@aisb.ed.ac.uk |
|
