20101127 - Evaluation Research (Wikipedia)

Comments: Below is my reading on Evaluation Research, taken from Wikipedia on Sat, 27 Nov 2010. Evaluation research is used to determine the impact of a social intervention. A social intervention is an action taken within a social context designed to produce an intended result. Evaluation research thus analyzes the impact of a particular program on a certain social problem the program is trying to solve. Appropriate Topics Appropriate topics for evaluation research are as diverse and extensive as any other social research. Many topics exist that are in need of evaluation; however, the ones selected for evaluation have a practical significance. Practical significance refers to a topic's application in the real-world. Evaluation research has to be conducted during real life situations, meaning researchers need real life participants who are willing to be cooperative. Types of Evaluation Research Studies In the field, there are three main types of studies: 1 Needs assessment studies Particular studies directed to determine the existence and extent of problems, usually pertaining to a specific population. 2 Cost-benefit studies Studies that decide whether the results of a program justify the expense. The cost could have been financial or non-financial. 3 Monitoring studies Studies that provide a steady flow of information about a topic of interest. These studies are usually conducted over an extended period of time. In some cases, monitoring studies require incremental interventions, meaning the results may change slightly as monitoring methods alter and changes within the topic being studied are made. Common monitoring studies focus on crime rates or epidemic outbreaks. Issues of Measurement While executing evaluation research, a researcher may encounter issues that limit the overall quality and validity of their findings. With all evaluation research , researchers must be able to operationalize, observe, and recognize the presence or absence of what is under study. A common problem is the attempt to measure a variable that is simply unmeasurable. Also, researchers must be fully aware of outside factors that may influence the variables they are studying. Researchers must also note not just the presence or absence of a variable, but they must analyze the extent to which the variable is present and the amount of its involvement. New vs. existing measures When designing an evaluation research study, a researcher must decide whether to create his or her own measures or to make use of existing ones. One consideration is that existing measures may not necessarily meet a researcher's goals; however, they do allow for comparison. Types of Research Design 1 Experimental design One evaluation research design type is experimental design. An experimental design for a study whose aim is to evaluate a new medical treatment would likely adhere to the following steps: Step 1: identify the population; identify used definitions (operationalizations) and pre-test information on patients used in the treatment Step 2: divide patients into experimental and control groups; consider ethical issues: in particular, is it ethically sound to deny treatment of the control group? Step 3: determine the length of the measurement Step 4: determine the details of measurement, context, interventions, etc. Step 5: wait for the treatment to run its course, measure what's needed, and determine how the response variable was changed (if at all) 2 Quasi-experimental design Another evaluation research design type is quasi-experimental design. Quasi-experimental designs are non-rigorous inquiries that somewhat resemble controlled experiments; however, they lack key elements that are indicative of experiments, including pre- and post-testing and/or control groups. 3 Time-series design Time-series design is an additional evaluation research design type. This design type entails measurements made over a fixed time period, such as the study of traffic accident rates before and after the lowering of the speed limit in an area. 4 Multiple time-series design Multiple time-series designs utilize more than one set of data collected over a period of time, such as accident rates over time in several states or cities. This design type allows comparisons to be made. 5 Social indicators research Social indicators research is a popular avenue that combines evaluation research with the analysis of existing data. Social indicators are measurements that are reflective of the quality or nature of social life. Social indicators are frequently monitored in order to determine the nature of social change within a society. Some commonly used social indicators include crime rates, infant mortality rates, and the number of physicians per 100,000 population. 6 Nonequivalent control groups Nonequivalent control groups share similarities with the experimental group, but they are not selected based on random assignment. For example, two comparable schools might be used; one as a nonequivalent control group, one as the experimental group. Common Problems Researchers who utilize evaluation research must contend with numerous problems. They may experience difficulty ensuring subjects' full cooperation. Additionally, they may have trouble finding willing participants, as research and participation in it is a priority for few. Ethics Like other forms of social research, evaluation research must take into account ethical considerations. As mentioned earlier, sometimes evaluation research studies may require the split of subjects into experimental and control groups. When considering controversial topics, such as sex education programs, it can be the evaluation research itself that raises ethical problems. In this case, the following question might be raised: is it ethically sound to expose only the experimental group to sex education programs? Results and Interpretation Results from evaluation research are not always put into practice. One reason for this is that implications may not be presented in such a manner that is understandable to non-researchers. Additionally, results may contrast with researchers' beliefs. Researchers may have a vested interest in the results. They may also be pressured to come up with certain results, such as positive findings for a pharmaceutical company. In this case, ethical concerns regarding the researchers' objectivity must be raised. When laypersons interpret research findings, they must always be aware of the highly political aspect of research. Political agendas can be difficult to identify, and they often cause research to be biased. This involves not just party politics, but also politics that are at play within the scientific and academic communities. Comments: I concur that the research community may tend to be biased, e.g. toward their norms, beliefs and practices. I have been advised by 2 R&D key persons to redirect (is that bias?) my research in certain manner. Do I want my PhD in easier manner? If yes, it is best to follow the advice of the University Research Community. I am in dilemma. Source: http://en.wikibooks.org/wiki/Social_Research_Methods/Evaluation_Research