My goal in this post is to draw attention to an important study that used advanced statistical techniques to analyze the CoI questionnaire (Abbitt & Boone, 2021). While exploring statistical anomalies may not be front of mind for most practitioners, or even those doing research with the CoI framework, this provides significant conceptual and analytical impact regarding the CoI questionnaire. By providing a brief overview of the study I hope to attract interest in pursuing research into the refinement and further validation of the CoI quantitative questionnaire.

However, before addressing suggestions for improvement, it is important to highlight and emphasize the findings of this research regarding the existing strengths of the CoI questionnaire. In this regard, Abbitt and Boone (2021) state that the CoI framework has demonstrated its value in providing insight into online and blended learning environments and the CoI “instrument exhibits strong measurement properties as evaluation of item reliability and person reliability suggested strong reliability” (p. 389).

Recognizing the strengths of the CoI questionnaire is not in contradiction to suggested area for possible improvement. First, Abbitt and Boone (2021) provide important insights concerning a problem with a specific cognitive presence (exploration) item; “Online discussions were valuable in helping me appreciate different perspectives.” Using Rasch measurement techniques, they found there was a possible misfit with this item and the cognitive presence scale. That is, the item may not be consistent with the other items on the cognitive presence subscale. This is notable as another study found this particular cognitive presence item loaded on the social presence factor (Kovanović et al., 2018). Similarly, there was also a possible misfit for a teaching presence (direct instruction) item: “The instructor provided feedback in a timely fashion.” As a result, the authors suggested that consideration may be given to revising or removing these items from the CoI questionnaire. My preference would be to test rewording of these particular items. This would contribute to future research of the CoI framework.

Another issue revealed by this study relates to possible item difficulty where it is easier or harder to agree with an item. For example, by examining the ordering and spacing of the items on the teaching presence scale, the items easiest for respondents to agree with were related to communication while the most difficult related to instructor feedback. Similarly, the easiest items on the social presence scale were related to open communication while the most difficult were related to affective expression. Finally, regarding cognitive presence, respondents found it easier to agree with the application and relevance of course material were the hardest. The implications are:

When viewed through the lens of the CoI framework, it is reasonable to examine and comment upon the item difficulty and spacing of items … for SP, we see that items relating to the dimensions (Affective Expression, Group Cohesion, and Open Communication) fall along distinct and different portions of the scale. For the CP and TP scales, however, this pattern is less distinct. These unique characteristics of item difficulty as well as spacing … provide insights into aspects of construct validity and also the online courses on which future continuous improvement efforts can focus. (Abbitt & Boone, 2021, p. 390)

Without getting into the details, this study identifies areas for productive research with the potential to improve understanding of communities of inquiry and provide opportunities to explore the development of a second generation of the CoI questionnaire. At the same time it should be kept in mind that this research also makes it clear that the CoI questionnaire has proven to be an essential tool to study communities of inquiry and that it offers “strong measurement properties.” Further refining the conceptual integrity of the questionnaire does not discount its current strengths.

To reiterate, the purpose of this post is to draw attention to this research and encourage work on developing a second generation of the CoI survey questionnaire. It would also seem to me that given the complexity of this challenge, it may be effectively addressed through a collaborative approach much like the one that led to the creation of the original CoI questionnaire (Arbaugh, et al., 2008).