Since this study includes student-generated artifacts, we have shared our project with the chair of the IRB and secured permission to collect the materials for analysis.
We also verified with the students that we had their permission to use copies of their maps in any potential publications.
See the book, A Practical Guide to Information Literacy Assessment for Academic Librarians (Radcliff et al), to understand more about concept mapping and to evaluate the advantages & drawbacks of this approach.
Google Books offers limited access to Chapter 10, Concept Maps.
Concept maps facilitate quick and holistic assessments of student learning, as we discovered while teaching information literacy courses for two academic departments. This method has been used as a teaching tool and for curriculum mapping, but very little library literature exists about its use for assessment.
Both information literacy courses require an annotated bibliography for the final assignment. Although this project makes use of resources and skills taught during the semester, it did not directly address how well students had incorporated course content into their existing research practices. We initially used the maps at semester’s end to review how much content students retained and to see how they integrated what they learned into the research process. Would students be able to articulate specific resources and search skills that we taught during the semester? Visual inspections of the concept maps indicated that students could identify and remember specific databases and search skills.
The end-of-semester maps were informative, but lacked effective assessment data. We began the next semester by asking students to draw concept maps and concluded the semester by repeating the exercise. The pre- and post-concept maps created a baseline to gauge student knowledge and data to show what changes had taken place.
The process for analyzing the pre- and post-maps began with transcribing the words into a spreadsheet. We then standardized the ideas, creating our own thesaurus of terms. After the ideas were standardized, the thesaurus terms were then mapped to the ACRL Information Literacy Competencies and the data was analyzed for more rigorous results.
Our information literacy courses emphasize discipline-specific resources, understanding database construction, source evaluation, and citation skills. This approach is evident in the assessment results, as many concepts mapped to Standards 1.2 and 2.2. It was also clear with the post-maps that more students could identify specific resources and search techniques. As such, the post-map content showed that students had expanded their notions about the research process. We were thus able to measure changes in how students conceive of the research process and how they look at the process as a whole. Based on the data, then, the courses clearly demonstrated value-added aspects for both departmental programs. Concept map differences have already been observed: graduate students describe the research process in greater detail than undergraduates, while students in the social sciences use different terminology than students in the humanities.
Using concept maps as reviewing and assessment tools allowed us to conduct outcomes assessment, show that we were adding value to undergraduate education, and facilitate continuous improvement. The maps helped us to consider how we might change course content when we perceive gaps in the research process or to drop course content when we observe class mastery on certain parts of the research process, based on pre-test results.
