Studies – Scientific or Empirical

Scientific research methods include: case studies, surveys, controlled experiments, and correlational studies – of these case studies are the least desirable in terms of evidence which may be able to rule out alternative hypotheses.

Empirical investigations begin with a research question or a related series of questions. If the question for which an answer is being sought is not clear to the researcher, the effectiveness of the study is likely to be blunted. Such research questions can often be stated in terms of a problem and concern matters of causation.

Does x exist or what are its characteristics or limits? Of what components is x comprised?

Is there an association between x and y? What are the causes of y? Does x cause changes in y or prevent such changes? Does x cause more changes in y than does z? Or are any such changes more (or less) pronounced under certain conditions but not others?

The criteria for establishing causation are:

1.        Co-variation: two events must change together;

2.        Time ordering: one event must occur before the other;

3.        Alternative explanations: other plausible explanations for the co-variation must be eliminated.

When practical or other difficulties count against conducting controlled experiments Prospective and Retrospective Causal Studies can be used, however, any conclusions reached must be tempered on the basis of the ‘lesser’ quality of evidence generated.

A Prospective Causal Study involves segments of the population and, for example, focuses on people who may either have or lack the suspected causal agent, tracking the two segments over time to see if different levels of effect emerge.

A Retrospective Causal Study examines the histories of subjects in segments of the population to see if there are significantly different levels of the suspected causal agent between those different groups.

Case studies – the study of one subject – provide little basis for generalization to the larger group or all people and are plagued by subjectivity, bias, and observer error, while surveys can be a valuable source of information if conducted with rigour and viewed with a sensible caution about any conclusions drawn, but suffer from difficulties with subject reporting: neither allows the inference of causation.

Field studies and naturalistic observation have the advantage of collecting information in a natural setting without the artificiality of laboratory research, but have the marked disadvantage of little or more likely no control over eliminating extraneous variables and isolating the behaviours of interest, while posing very great obstacles to a uniform, consistent level of observation.

Correlational research allows for a quantitative relation between two or more variables to be ascertained as well as one variable to be predicted from another, but cannot control some extraneous variables, determine time order, or allow the inference of causation.

Quasi-experiments are similar to experiments, but don’t allow the manipulation of an independent variable. Subjects are selected on the basis of some pre-existing characteristic, rather than randomly-assigned. Comparisons can be made between subject groups with some control of extraneous variables, but does not allow the inference of causation.

Controlled experiments do allow causal inferences to be drawn. The goal is to show that the effect on the dependent variable was due to the independent variable: the extent to which the experimental outcome truly depends upon the independent variable as opposed to confounding variables and error, this is called the experiment’s internal validity.

The central elements in scientific research methods are, firstly, that information is pursued in the form of publicly verifiable data (see: peer review). Any individual or single scientific study must be seen in the context of the larger picture of research findings. Consequently to what degree strong conclusions may be drawn about the phenomenon under study depends to an important extent upon replication of research results. 

Secondly, control through special procedures is used to reduce error in observations and in the interpretation of findings. Control is especially important with investigations in the human social world as it is difficult to successfully apply scientific research to questions of complex human behaviour.

Precision in language is the third major component. Concepts can be confusing, obscure, and ambiguous. Scientific thinking attempts to be precise and consistent with its use of language.

It should be emphasized that scientific research, when well-conducted, is one of the best sources of evidence due to these characteristics of verifiability, control, and precision.

(see also: ethics, selection bias, statistical significance, experimental research, peer review, scientific approach, science, scientific methodology, pseudo-science, systematic evidence, anecdotal evidence, (conceptual) models, operational definition, critical thinking, theory (scientific), falsifiability, testability) . 

» Return to belief, judgement, and clear thinking »