Course: CMM 128 Interviewing Practices

Lecture Notes

Go to page (1/2)
Survey Interviews

As the most meticulously planned and executed interviews, putting together one follows a serious of steps to ensure reliable and valid results.

What is reliability and validity?

- Reliability involves getting the same or similar answer each time the same or similar question is asked.
- Validity involves getting truthful and meaningful answers.

Some examples:

- If you ask me for my name and I reply, "Rick" the first time, "Richard" the second time, and "Ricky" the third time, my answers are truthful, they have something to do with the question you ask (i.e., they are meaningful), but they are not as reliable as if I said "Rick" for all three requests.
- On an interviewing test with the purpose of assessing your knowledge of interviewing concepts, if I ask you what 1+1 equals and you reply "2," and you give this answer each time I ask you, your answer is reliable (i.e., you gave a consistent answer each time) but the answer is not valid. Even though you gave the answer that you thought was true, this answer was not meaningful since it did not have anything to do with how much you know about interviewing concepts.

The process to get reliability and validity

1. Determine your purpose. This will be the benchmark for whether questions asked in the survey are meaningful.

2. Randomly select people for your study. Randomness means that everyone has an equal chance to be in a study. When randomness does not exist, a bias occurs in the study. This may make the study invalid. When a bias occurs, the surveyor has to ask two questions to decide if the study is invalid:

- Is the bias significant?
- Is the bias meaningful?

For example, if you want to call people on the telephone and survey them about how they protect their privacy, using phone numbers from the phone book would produce an invalid survey because there are a significant number of people who have an unlisted/unpublished phone number and people have an unlisted/unpublished phone number because they are trying to protect their privacy.

Even if you randomly called a sampling of all possible phone numbers (i.e., phone numbers between 999-999-9999 and 000-000-0000), not everyone will be home when you call (they're working, sleeping, at school, etc.), not everyone will answer the phone if they are home (caller ID, answering machines), not everyone will be equally likely to be the one who answers (teenagers are more likely to answer than adults) and not everyone will be equally willing to answer survey questions.

3. Design your study. There are two basic kinds of designs:

- Longitudinal studies. These look at one group over two or more time periods. For example, you might give students a test of their interviewing knowledge at the start of the semester and again at the end of the semester to see what effect the course had on their knowledge.
- Cross-sectional studies. These look at two or more groups in one time period. For example, students who have taken an interviewing course are compared to students who have not taken an interviewing course to see what differences exist in their knowledge of interviewing.
- Combination designs. These combine elements of both. For example, the improvement by men in an interviewing course is compared to the improvement by women in an interviewing course.

A complex version of this design might look like this:

Group 1 (Men) | Test them in time period 1 | Have them take an interviewing course | Test them in time period 2
Group 2 (Women) | Test them in time period 1 | Have them take an interviewing course | Test them in time period 2
Group 3 (Men) | Test them in time period 1 | (No interviewing course) | Test them in time period 2
Group 4 (Women) | Test them in time period 1 | (No interviewing course) | Test them in time period 2
Group 5 (Men) | (No test) | (No interviewing course) | Test them in time period 2
Group 6 (Women) | (No test) | (No interviewing course) | Test them in time period 2

- Results: Interpretation:

Group | Score in time period 1 | Score in time period 2
1 (Men) | 100 | 200
2 (Women) | 100 | 200
3 (Men ) | 100 | 150
4 (Women) | 100 | 150
5 (Men) | Unknown but assume 100 | 125
6 (Women) | Unknown but assume 100 | 125

1. Gender does not affect a person's score since men and women in identical survey settings had identical scores.
2. Initially it looks like taking an interview course increases a person's score by 100 points. However, part of this increase must be due to some other variable such as getting older since group 3 and 4 had a 50 point increase in their score even though they did not take an interviewing course. Having taken the exam once before seems to have affected the interviewing score in time period 2. Groups 3 and 4 scored higher than groups 5 and 6 and the only difference between the groups was that groups 3 and 4 had taken the exam once before. Thus, experience with the exam increases a person's score by 25 points. Getting older increases a person's score by 25 points, and taking an interview course increases a person's score by 50 points.