Nature Of Research Design 3p2x12

NATURE OF RESEARCH DESIGN I. GENERAL RESEARCH DESIGN A research design is a plan for comprehensive data collection to answer research questions and/or test research hypotheses. It consists of detailed prescriptions for solving problems from either a scientific or humanistic perspective. A. Can classify research designs by 8 pairs of schema (Smith): 1. Quantitative (counting and measuring data; analyze statistically) and qualitiative (examining narrative data, categorizing into themes, etc.) 2. Interpretive (study of meaning) and functional (meaning is baseline for inferring conclusions) 3. Experimental (manipulation of research environment and observing reactions; quantitative & functional) and naturalistic (observing and recording ongoing behavior in natural settings–no manipulation of the environment. More interpretive but may be quantitative or qualitative) 4. Laboratory (bringing subjects to a controlled environment–usually experimental, although that isn’t essential; artificial) and field (occurs in a more natural environment, but can still be experimental–not identical to naturalistic). 5. Participant (investigator actively participates with subjects who may or may not be aware of researcher’s identity) and non-participant (researcher is outside; subjects may not even be aware part of a study, although that raises ethical questions) 6. Overt research (subjects know they are part of a study and there may be various researcher & subject effects) and unobtrusive (researcher removed from what is being studied so won’t influence results–either by using confederates to gather data, or by using textual data) 7. Cross-sectional (examines behavior at a single point in time with present emphasized) and longitudinal (time series analysis–examines behavior over time as it changes) 8. Basic (theoretical with little or no concern for practical application; more experimental and laboratory) and applied (concerned with praxis as well as theory–aims to solve problems in everyday life; more field or naturalistic) B. General Quantitative Methods (a review)–Note, not all researchers would classify quantitative methods this way--may have slightly different terminology or categorizing (e.g. survey and content analysis usually treated separately).

1. Experimental/Quasi-Experimental–investigate possible cause-effect relationships by exposing one or more experimental groups to one or more treatment conditions and comparing results to a control group (for quasi, the conditions and setting is less controlled). 2. Causal-comparative (“ex post facto”)–investigate possible cause-effect relationships by observing an existing consequence and searching through data (i.e. accident rates, census data, etc.) for plausible causal factors. 3. Correlational–to investigate the extent to which variations in one factor correspond with variations in on or more other factors based on correlation coefficients. 5. Descriptive–to describe systematically a situation or topic as accurately as possible, to describe characteristics of a population and/or situation, etc. Includes most survey research, some historical research, some textual research (i.e. content analysis), & some case or field studies (can be combined with qualitative research). Might also be developmental–to investigate patterns or sequences of change or growth over time. C. General Qualitative Methods Confusing, because no one set definition of what constitutes qualitative research. However, it is generally agreed that this research involves the use of symbols, classification, etc. to describe, explain, interpret and/or evaluate mostly textual data (without seeking generalizations). 1. Ethnomethodology/Ethnography–studying and observing a group over time to understand their own interpretations of things, their “common sense” knowledge, etc.; the examination of “everyday social life.” Can be macro or micro, using a variety of techniques (to create what Geertz calls “thick description”); can include autoethnography. 2. Observations–can be participant or non-participant, “lab” or field; for example, Bales IPA (may be part of ethnomethodology or not; can also generate a “text” as in Conversational Analysis). 3. In-depth interviews–using open-ended questions to understand attitudes, behaviors, etc. (can be part of ethnomethodology or stand-alone) 4. Focus Groups–group interview to understand attitudes and behaviors 5. Case Studies–uses multiple data sources to study a particular case (organization, event, person, etc.) 6. Historical analysis–descriptive and interpretive studies of historical persons, events, ideas, places, documents, etc. (can be combined with quantitative analysis) 7. Critical Analysis:

a. Descriptive Textual Analysis–a study by a researcher of the content of a text, sometimes mistakenly referred to as “content analysis.” There may be counting, and/or a particular critical methodology employed. Can be on a written text or from transcripts based on conversation (see above) b. Rhetorical Criticism–describing and/or evaluating a text’s persuasiveness (or other factors) using critical rhetorical methods (including neo-Aristotlean, Burkean, metaphoric, narrative, feminist, etc.) c. Media Criticism/Cultural Studies–describing and/or evaluating a media text, using one or more critical theories and methods (including semiotic, psychoanalytic, neo-Marxist or ideological, feminist, etc.). II. QUANTITATIVE RESEARCH DESIGN A. Assumptions and characteristics 1. Based in logical positivism–all knowledge is derived from direct observation (empiricism) combined with logical inferences drawn from that observation (rationalism) 2. Seeks explanation, prediction, and control through observation, description, and verification 3. Assumes that phenomenon is observable (directly or indirectly), is ordered (not random), and is explainable. 4. Relies on statistical treatment of data–examines patterns, relationships, etc. Two basic types: a. Descriptive–describe patterns, construct simple descriptions about the characteristics of a set of quantitative data. [see non-parametric statistics] b. Inferential–describe and predict (generalize probabilities)–permit the researcher to extend past description to make statements and descriptions of a much larger group [see parametric statistics] 5. Approaches can be primarily inductive (data to theory, aka “Grounded Theory”) where explanations emerge from observations, or primarily deductive (theory to data, aka “the Hypothetico-Deductive Approach”), which begins with a general theory to be tested against data, and then modified or affirmed (through hypotheses-testing). 6. Depends on reductionism–focusing on the smallest units of observation. 7. Creates hypotheses about variables in specific relationships (since we’ve already discussed this, won’t elaborate at this point).

8. Examines nominal, ordinal, interval and/or ratio data using a variety of measurement instruments, such as measurement scales. 9. Depends on accurate sampling from an overall population. B. Measurement instruments--scales A measurement scale is a composite measure of a variable, usually based on more than one item. Scales usually are used with complex variables that don’t lend themselves to single item measurements. Data such as age, height, weight, but also number of TVs in a house, can be measured without scaling techniques. The most common types are associated with the level of data analysis (which is hierarchical). 1. Measuring dimensions of concepts a. Unidimensional concepts are measured by a set of indices added together equally to derive a single overall score. A scale of such items is called a summated scale. For example, the Humor Orientation Scale (see p. 95, Frey et. al.). B. Multidimensional concepts refer to concepts that have more than one factor (or subconcepts) which must be measured by more than one set of scale items (e.g. credibility is made up of at least three independent factors, see p. 94, Frey et. al.) 2. Nominal scales–measure discrete variables which are differentiated on the basis of type or category (instead of degree or amount) a. Must be classifiable into at least two different categories that are mutually exclusive, equivalent, and exhaustive (e.g. male-female). b. Often used for background variables, or for dependent variables (esp. in a pilot type study using open-ended questions that generate categories or taxonomies, as in the original compliance-gaining research). c. Can also be used for checklist responses d. May yield interesting data, but limited (must use descriptive statistics only) 3. Ordinal scales–classify into nominal categories, but also rank order along some type of dimension (such as a “most liked to least liked” ranking or “greater than” and “less than,” etc.). Usually used on discrete variables, but in some cases there is an underlying continuous variable dimension. a. Two basic types

1) Ipsative scale–each rank can be used only once (often used in judging, by the way) 2) Normative scale–permits ranked ties b. Provide more info than nominal scales, but also are limited–cannot tell how much more or less of the variable is present (so also merely descriptive) 4. Interval scales–categorize (as in nominal), and rank order along a dimension (as in ordinal), but also establish equal distances between each of the adjacent points along the measurement scale. Can measure both discrete and continuous variables. a. Likert & Likert-type scales (also known as the summated rating approach) 1) developed by Rensis Likert to identify the extent of a person’s attitudes, beliefs, or feelings toward a subject (and was originally designed to be ordinal). 2) a “true” or traditional Likert scale asks people the extent they agree or disagree with a statement on a 5 point scale 3) can be used with other types of positive to negative opinion responses (such as “truth to falseness”) on a topic with a 5 to 7 point scale (then called Likert-type or Likert-like) 4) Each person’s response is summed, with usually the strongest or most favorable beliefs are given the highest scores with the weakest or least favorable given the lowest scores 5) A forced-choice unidimensional scale with four important features: a) all scales in one instrument should relate to a single factor or idea b) should contain approximately the same number of positive and negative statements c) is flexible enough so that it can be altered to suit the needs of the researcher d) can be treated as interval data (assume equal distance between response items) 6) 4 steps to construct: a) determine the constructs to be measured b) compile a battery of possible scales c) through pre-testing, etc., determine the discriminative power of each scale (ability to measure only one construct under investigation) d) select final set of scales

7) Potential problems a) possibility of response bias–a tendency to mark the same response for each item [less likely if have an equal number of negative items; can also do reversed items] b) tendency of respondents to choose the middle or neutral score [can use a 4 or 6 point scale to force a choice–however, will frustrate those who truly have no opinion or feelings on a topic]

b. Semantic Differential scale 1) Developed by Osgood, et. al. to measure the meanings people ascribe to a specific stimulus (a word, phrase, sentence, etc.) 2) Because measure multifaceted meanings of construct, are more multidimensional. a) for example, can be used to measure evaluation (good-bad) b) another dimension might relate to potency (strong-weak) c) can do many other dimensions (active-ive, trust-distrust, like-dislike, etc.) 3) Consists of a series of response scales with a 7 point continuum bounded by bipolar adjective pairs at extreme points. 4) Respondents are asked for a first impression to immediately mark on of the seven positions, assumed to be equidistance with an arbitrary zero point (a type of forced choice); the scale is scored by summing or averaging responses on all scale items. 5) Several plus features a) can measure constructs in which different dimensions are known (e.g. emotions, which have a cognitive and affective component) b) are flexible, and can measure both unidimensionally and multidimensionally (as well as assign high number to negative construct instead of just to positive) c) can identify unknown dimensions of constructs 6) Basic steps to construct a) identify the concept and its dimensions b) select appropriate adjective pairs–need to be representative of each dimension of a concept (with at least 2 but not more than 10 scales per dimension, with an optimal number of 4). This choosing is not arbitrary or random; it must also be geared to the understanding of the respondents.

c) format the scales with each concept presented separately, but the different dimensions mixed and polarity alternated (to prevent response bias). d) ister to a pilot group and analyze to figure out dimension scores [can determine unknown dimensions with a factor analysis] e) revise instrument for regular istration 7) As with Likert-type scales, a person might respond in the middle position for each item, or might mark the same response for each item. c. Thurstone scales (aka Equal-appearing interval scales)–used to measure the attitude toward a given concept. 1) Named for researcher L.L. Thurstone, the scale is intended to solve the problem of equi-distance in a Likert-type scale or semantic differential scale, thus making it more truly interval level measurement. 2) Procedures a) collect a large number of statements (at least 100) related to the concept b) have judges rate these statements along an 11-category scale in which each category expresses a different degree of favorableness toward the concept. c) Items are ranked according to mean or median ratings assigned by the judges, then used to construct a questionnaire of 20-30 items chosen evenly across the range of ratings. d) Statements are worded so that a person can agree or disagree with them. e) scale is istered to a sample of respondents whose scores are determined by computing the mean or median value of agreedupon items (someone who disagrees with all the items has a score of zero) 3) Problems a) it is difficult and time-consuming to construct b) can never be absolutely sure that the items actually represent equal intervals. c) due to this, this scale is rarely used in communication research. 5. Ratio scales–not only categorize and rank order a variable with equal intervals, but also establish a true or absolute zero point. 1. Although common in the physical sciences, rare in social sciences, because difficult to establish an absolute zero point.

2. Sometimes used in communication research when determining frequency of behaviors (see pp. 93-94 in Frey, et. al.) C. Measurement Techniques 1. Questionnaires & Interviews–probably the technique used most frequently in communication research. A. Can be part of the scales noted above, or as part of experimental research, or as part of survey research; can also be used to elicit messages in textual research or naturalistic research–esp. more open-ended questions used in interviews). B. In general, questionnaires tend to use written responses, whereas interviews require interaction between the researcher and respondent. C. Two general types of questions used in both: 1. Closed/Closed ended/Structured–preselected answers from which respondent chooses (see the interval scales, above, but also can be used in a “yes-no” format). a. Limited answers permits quantitative treatment. b. Since easier to compile and compare, often used with large samples. c. Tend to be researcher biased (what researchers deem important, etc.) 2. Open/Open-ended/Unstructured–respondents used their own words in answering questions. a. Time-consuming to prepare and ister b. More difficult to categorize and analyze responses (and more time-consuming as well), esp. quantitatively c. Provide more information on individuals, and don’t lead the respondent to answer in preconceived ways; used with small samples and in pre-research. d. Often used in interviews; researchers can use both in the same study. D. Question Strategies and Formats. 1. Can be directive (fully structured in a predetermined sequence) or nondirective (initial response determines the next question)–which chosen depends on the researcher’s intent (for generalizable data, must use more structured approaches).

2. A list of questions may be called the interview schedule or protocol. 3. Interviews may be semistructured, using a mix of directive and nondirective questions (a basic list with follow up to be determined by the interviewer). 4. Various ways to sequence the questions (known as the question format) a. Tunnel–a straight series of similarly organized questions, with a consistent set of responses to code b. Funnel format–broad open questions start the questionnaire or interview, with narrow ones following. c. Inverted funnel–the opposite (start with narrow, closed questions, then move to more open-ended ones)–often used with taboo or controversial subjects (can move from low risk questions into more high-risk) 5. Need to structure in such a way as to avoid question order effects a. Consistency effects–respondents believe that later answers must be consistent with those to earlier questions (when may not be) b. Fatigue effects–respondents grow tired, and less accurate c. Redundancy effects–respondents breeze over items that seem to repeat previous questions. d. Response set effects–tendency for respondent to answer the same way (e.g. using only one side of the scale) e. The “good subject” or acquiescent response style–the person answers positively to each question (or “yes” in a yes-no format, or “always agrees”). f. The “bad subject” or quarrelsome response style–the person answers negatively to each question (or “no” or “never agrees”) g. Can deal with these in a number of ways, including balancedscale approaches (“flipping” the wording and responses), requiring both positive and negative answers, etc. E. Relative advantages of Questionnaires and Interviews are noted in Figure 4.7 on p. 13 of the Frey et al. text. 2. Observations–the systematic inspection and interpretation of behavioral phenomena. Two primary types A. Direct observation

1. May occur in a lab setting or in the field 2. Respondents may or may not know part of an observational study (covert or overt–if covert, it is a type of unobtrusive measure) B. Indirect observation (unobtrusive measures) 1. Examination of artifacts or texts produced by people (e.g. transcripts, video and audio tapes, etc.) 2. Can also examine trace measures (physical evidence left behind–like a detective’s clues; some non-verbal research does this, as does historical); two basic types: a. Measures of erosion–how objects are worn down by use over time (can be natural or controlled) b. Measures of accretion–how physical traces build up over time (e.g. in latrinalia studies); one type is garbology–going through people’s garbage. As with erosion, can be natural or controlled. C. Use a variety of coding schemes to code or classify observations, a process that ranges from closed to open. a. Closed schemes used checklists and predetermined categories, used in experimental and some textual analysis (content and interaction analysis, esp.); developing effective closed observational schemes is a complex task (see p. 107, Frey et. al.) b. Open schemes are more likely to be used in ethnography and critical analysis (although critical work doesn’t use this language) Next time–we’ll cover sampling and then move into statistics. Continue with chapter 5, but also chapter 11.