Validity and reliability
A Report on the Validity and Reliability of Personal Spacing Behavior Verbal Frequency QuestionnaireMore than just a physical construct, personal space is defined as “the distance component of interpersonal relations” (as cited in Gifford & Sacilotto, 1993). One of the earliest studies on this matter defines the concept as the space around an individual which is felt to be personal, belonging to self (Dosey & Meisels, 1969). Accordingly, this personal space can be manifested through various behaviors that may be consciously or unconsciously exhibited by individuals. Likewise, personal space is apparent in a broad range of situations – even in crowded conditions where in the physical space is very limited. An example of such situation happens in trains which are always packed with passengers. In this case, people engage in certain behaviors in order to maintain their personal space given the lack of physical space. Moreover, though it is not a physical concept, it is possible to quantitatively measure personal space through the use of a dimensional scale. In an attempt to do so, a questionnaire, in the form of a verbal frequency scale, was developed that contain ten items, describing different spacing behaviors that people engage in.
In order to be more specific, these behaviors were limited to those commonly exhibited in the train – being a common setting for personal space studies. For each item, five response options were provided which are coded from 1 to 5 – the highest score corresponding to “Always” and the lowest to “Never”. However, there are two different types of items made – negatively and positively stated statements, respectively – such that coding of responses was reversed for those items which are negatively stated. The total scores for all the items would be added and the average would be computed. Given this scoring, the variable, personal space, can be operationally defined as the degree to which individuals engage in different behaviors in order to prevent interpersonal relations. The assumption is that individuals who keep a large personal space are likely to engage to different spacing behaviors; hence, a high score in the test entails a large personal space.
In order to guarantee that that the questionnaire would measure what it is supposed to measure, two types of validity were established prior to the administration of the questionnaires. First, the content validity of the questionnaire was established by creating items that were all based on observations made by previous researchers in their studies of personal spacing behaviors in the train. As such, previous observations found in published journals were translated into personal statements that constitute the items covered.
Second, while it is not a good indication of usefulness, the face validity of the questionnaire was maintained in order to appear acceptable and relevant to the respondents. The questions were worded in a simple manner to facilitate better understanding. Also, the questionnaire was formatted to the convenience of the respondents such that they would only have to encircle their response for each item.
After establishing content and face validities, the questionnaires were then administered to ten respondents who can easily be contacted by the researcher any time. In order to ensure that the respondents can relate to the questions covered, only regular train passengers were chosen to participate in the study. Consequently, the same questionnaires were again administered to the same respondents after a period of one week in order to confirm the stability of results. All responses were coded and inputted in statistical software for analysis.
Based on the results, the respondents scored high in the variable that was measured. This means that people do engage in certain activities and behaviors that would protect their personal space. These findings confirm the established theories and expectations on the said topic. As such, it can be deduced that the questionnaire possesses construct validity.
The questionnaire was tested for both types of reliability measures: internal consistency and repeatability. For the first measure, three methods of reliability analysis were employed to determine the internal consistency of the questionnaire: split-half method, coefficient alpha and item-total correlation. On the other hand, test-retest method was used to test the repeatability that refers to the consistency of measures on repeated applications.
The items in the questionnaire were divided into halves and the scores on each half were correlated with one another to estimate the degree to which the items are equivalent to each other. The coefficient of equivalence by means of Spearman-Brown correlation formula was computed to be at 0.6 which indicates moderate correlation. Therefore, it is safe to presume that there is no significant difference in the scores derived from both forms of the scale. This entails that the respondents’ responses are consistent regardless of the particular sampling of items. To confirm, Cronbach’s alpha was computed and based on the obtained value of 0.6, it can be deduced that the scale is reliable. This means that the items in the scale measure the same thing – which is, personal space.
On the other hand, in order to know if the test will yield consistent scores when administered successively after a particular time span, a test-retest method of reliability was employed. The post-test was given to the same sample a week after the administration of the pre-test. The results of the two sets of tests were compared through Pearson Product Moment Coefficient of Correlation. A correlation coefficient of 0.5 was obtained indicating a moderate positive association between the two sets of tests. This association was found to be significant at p<0.0.5. This shows that the questionnaire is moderately stable such that it provides the same results for the same subjects on two or more occasions.
Summary of Results
The questionnaire aimed to measure the personal space of individuals. The significance of measuring this variable lies in the fact that individuals have different perceptions of personal space depending on the factors that account for it such as culture, gender, etc. In terms of content, the questionnaire is valid since it was able to probe various aspects of the variable it is supposed to measure. It said to be complete and comprehensive such that it was able to satisfy the research objectives. This was determined by a good review of literature to encompass several facets of the topic as studied in previous researches.
Moreover, in terms of construct, the questionnaire remains valid as it was able to confirm the current theory which holds that defense of space takes the form of a variety of behaviors like assuming a particular posture, feigning a sleep, staring in an empty space, etc. (Fried & De Fazio, 1974). Since the questionnaire supports this view, it demonstrates a degree of construct validity. In addition, while statistical analysis was not employed to determine validity, it seems that there is no significant difference between the results of the present test and those of the established tests measuring the same variable.
Nevertheless, while the questionnaire seems to be valid in terms of content and construct, it seems that it lags behind reliability given the moderate correlation obtained from statistical analysis. But then again, given the obtained values, there is a possibility that the items in the questionnaire are consistent such that they measure the same construct. Also, as far as the analysis of the test-retest results is concerned, it appears that the questionnaire is stable and can be used to repeated occasions.
Given that the questionnaire is administered to a small number of people, its predictive validity was not established such that it cannot be assumed that the present questionnaire can predict behavior in the future. In addition, the concurrent validity of the test was also not confirmed since it was not tested with an independent external criterion. The limited number of items covered by the questionnaire also made it impossible to statistically confirm construct validity through factor analysis.
Furthermore, while the statistics confirmed, to some extent, the internal consistency of the questionnaire, it should not be assumed that it is completely reliable as far as homogeneity is concerned. This is because the number of items covered by the questionnaire is too small for statistical analysis. On the other hand, despite the moderate association between the results in the test-retest analysis, it is difficult to say that the questionnaire is indeed stable since the period of time between the two tests is too short and recall bias may have affected the reliability analysis. Perhaps, addition of more items in the questionnaire administered to a larger sample size will strongly establish reliability and validity.
Dosey, M.A. & Meisels, M. (1969). Personal spae and self-protection. Journal of Personality and Social Psychology. 11(2), 93-97.
Fried, M. L. & De Fazio, V. J. (1974). Territoriality and boundary conflicts in the subway. Psychiatry Journal for the Study of Interpersonal Processes. 37(1), 47-54.
Gifford, R. & Sacilotto, P. (1993). Social isolation and personal space: A field study. Canadian Journal of Behavioural Science, 25(2).
*Scores will be reversed for negatively stated statements.
1 – Never 2 – Seldom 3 – Neutral 4 – Often 5 – Always
- I look out through the window during the trip.
- I feign a sleep.
- *I maintain eye contact with other passengers.
- I engage in different activities (e.g. listening to music, reading, texting, etc.) during the trip.
- *I talk with other passengers.
- When a passenger asks me to move, I move out immediately to avoid further conversations.
- I sleep throughout the trip.
- When other passengers get close to me, I move farther from them.
- When a space near me is vacated, I adjust my position against other passengers.
- If possible, I place my things around my body to separate myself from other passengers.