Abstract
An awareness of how and where to search the education literature, and how to appraise it is essential to be a teacher scholar (an academic who takes a scholarly approach to teaching), to develop high quality education research, and to perform the scholarship of teaching and learning. Most pharmacy faculty scholars do not receive training in searching the education literature. Thus, a framework for searching the education literature is needed. The framework presented here on where and how to search for evidence in the education literature, referred to as the WHEEL for teaching, is meant to serve as a guide for faculty members in conducting comprehensive and exhaustive literature searches for the publication of scholarship of teaching and learning projects, educational research, or approaching one's teaching in a scholarly manner. Key resources to search and methods for searching the education literature are listed and described.
INTRODUCTION
Pharmacy faculty members are faced with many teaching challenges today including an emphasis on incorporating active-learning strategies using instructional technology tools; measuring learning and outcomes through such things as progress examinations, objective structured clinical examinations, rubrics, and portfolios; and incorporating various instructional strategies such as team-based learning, high-fidelity simulations, and interprofessional education. In the recommendations from the American Association of Colleges of Pharmacy Task Force on Best Evidence Pharmacy Education, Hammer and colleagues recommended that a series of papers in The American Journal of Pharmaceutical Education (AJPE) on “how to’s” for the scholarship of teaching and learning (SoTL) and educational research is needed.1 During the last few years, researchers have provided the academy with numerous key articles in the area of SoTL that guide the development of projects.2-5 What is lacking is a guide to searching the educational literature. Even if faculty members do not perform SoTL or educational research, faculty members should have a scholarly approach to teaching, which requires knowing how and where to search for education literature beyond the pharmacy discipline that supports teaching. Having an awareness of the levels of evaluation used to appraise the educational literature is also important. The intent of this article is to guide faculty members in conducting a comprehensive and exhaustive literature search and in appraising the literature in preparation for publication of scholarship of teaching and learning, for educational research projects, and in being a teacher scholar. These guidelines are summarized by the acronym WHEEL, which stands for where and how to search for evidence in the education literature. The analogy of a wheel is appropriate because as new information is learned and the search process is repeated, the wheel moves forward.
WHY TO SEARCH THE EDUCATION LITERATURE
Health professionals are familiar with the practice of evidence-based medicine for the best treatment of patients. For this purpose, they identify a question, search the literature, appraise the evidence, apply the evidence, and evaluate the outcomes.6,7 This process is also applicable to teaching methods used to educate pharmacy students. In order for educators to take a scholarly approach to teaching, they must first review and be familiar with what is already known about different teaching methods. Evidence, including published literature as well as data collected internally, should be used whenever educators are doing anything from trying a new active-learning technique in the classroom to revising an entire curriculum. Searching the literature is a key component in this process. Reviewing the evidence prior to implementing a change can help to inform educators about potential pitfalls and best practices. A comprehensive literature search should be conducted to review all potentially valuable information. Assistance with how and where to search for education-related literature is useful as most pharmacy faculty scholars do not receive this training.
Published literature provides background support to indicate what is known about a specific area. It also assists in identifying the needs or unknowns in the particular area of study. Reviewing the background support and data then allows educators and researchers to formulate their specific ideas or research in an evidence-based manner and provides focus to support a unique project that would have a great impact in the classroom and could add to the literature. Literature from other disciplines or from higher education in general may have already evaluated a similar teaching method and what they found may be applicable to pharmacy education. Analyzing the existing evidence in light of the current proposed project is an important step for conducting research that matters.8 Incomplete searches may lead to a biased publication and an inappropriate interpretation of the impact of a project.9 It is important to know what has been done in the past in order to build on established knowledge. Without first consulting published literature and data, researchers will be repeating work already accomplished by other scholars.
WHERE TO SEARCH FOR EDUCATION LITERATURE
Because there is no one database that houses all education literature, multiple searches within multiple sources are necessary in order to find all pertinent literature. Common databases (listed in Table 1) include thousands of citations relevant to health education. Unfortunately, the databases may not always have appropriate subject headings for searching health education literature and not all health education resources are indexed. Multiple searches using various terminology and use of related citations may be helpful to ensure complete searches in databases. Foundations and organizations focusing on education are good sources of information that may be useful to supplement literature searches in databases. Many of these are listed in Table 2. Search functions within these resources vary greatly. Ancestry searching, which involves reviewing the bibliographies from any useful citations found in these searches, should be used to identify additional citations.
Databases Useful in Researching the Education Literature
Education Resources Available Within Organizations and Foundations
It is important to also manually search pharmacy-related education journals such as the American Journal of Pharmaceutical Education and Currents in Pharmacy Teaching and Learning. Additionally, scanning other education-related journals and information from other health professions is essential as well. Databases may not include all education-related journals, and indexing of these journals, if they are included, may not be comprehensive. Table 3 includes a list of key journals related to teaching and notes if they are currently indexed within the most commonly used databases. This may include only partial indexing of the journals (eg, indexing only 1 year of a journal’s articles). These journals may be added or deleted from the databases over time. The resources listed in the tables are not all-inclusive lists, but may be ideas for beginning searches.10
Key Teaching and Learning Journals for Educators in the Health Professions and Their Inclusion in Popular Databasesa
Finally, experts in the field may be contacted and Web searches for grey literature (any literature that is not controlled by commercial publishers) may be conducted. Grey literature for health education research may include items such as dissertations, theses, committee reports, government reports, technical reports, etc.11
HOW TO SEARCH THE EDUCATION LITERATURE
The systematic approach to searching for any type of literature also applies to searching for education literature. The steps to searching for education literature are summarized in Figure 1. The figure includes numbers to order the steps; however, it is a circle (wheel) because as new information is learned, the process may be repeated to incorporate the new information. The first step in the process is to clearly define the question or the hypothesis and the scope of the question. If a team of scholars are looking to create a tool to assess an interprofessional experience including nursing and pharmacy students, they would need to think about specifically which aspect(s) of the interaction they are looking to assess. Are they looking to assess students’ learning, contributions, attitude, etc? Are they looking only at experiential learning vs classroom lectures, or a one-time event vs multiple interactions? They should think about the participants, intervention, comparison, outcome, and inclusion and exclusion criteria when determining the scope of the query. This is followed by determining essential concepts for which more information is needed. In the interprofessional experience information, this may include understanding what is meant by interprofessional vs interdisciplinary.
WHEEL for Searching the Education Literature. (*Involve experts as the question is defined and again as methodology is developed.)
Search terms should be developed, taking time to think about all possible related terms, different spellings, and synonyms to ensure a complete search. Related citation searching may be used to identify additional appropriate citations. Searching the term “interprofessional” in Medline links to the Medical Subject Heading (MeSH) terms interprofessional relations or interdisciplinary communication. Searching the term “interdisciplinary team” links to the MeSH term “patient care teams.” Definitions for the MeSH terms should be reviewed to make sure that the chosen term is the most appropriate for the search. Subject trees may also be reviewed to identify sub-themes. Multiple combinations of MeSH terms and keywords should be used.
Once the key concepts are identified, resources should be identified in which to complete searches. Searches should begin in tertiary resources (resources that provide an overview and summarize information from other resources) followed by searches in databases.11,12 All resources likely to cover the essential concepts should be included in the search. Databases differ in the types of journals and publication dates for the journals they index. Brief descriptions of the content of several databases are included in Table 1. When searching for information about evaluating interprofessional interactions between nursing and pharmacy students, any resources that would have information about nursing, pharmacy, and interprofessional assessment should be accessed. Medline, The Cumulative Index to Nursing and Allied Health Literature (CINAHL), and education databases would be appropriate resources in which to begin. On the other hand, it would likely be a waste of time to search a database such as PsychINFO for this type of literature.
A Web search for information especially from appropriate government agencies, organizations, and foundations would help to ensure a complete search. For assessment of interprofessional interactions, there are multiple organizations listed in Table 2 that may be appropriate to search and may also include searching accreditation standards for each discipline included for requirements for interprofessional education. If the interaction involved managing patients with diabetes, a search for organizations related to diabetes management and education should also be conducted. Searches should not be limited only to pharmacy literature, but should include interprofessional interactions between any health fields.
These searches should be supplemented by manual searching of appropriate journals that may not have been indexed in the databases searched or may not have been completely indexed, and ancestry searching of useful citations. Once all of the information found is organized and appraised, areas where information is lacking should be identified; the process should be repeated to identify and fill in any gaps.
APPRAISING THE EVIDENCE
The final step in the process is to manage the information found by appraising the evidence and determining what is relevant to the project. The concept of using the “best evidence” is recommended.1,8 This concept has parallels to the Cochrane Collaboration, which has a mission to gather the best evidence for medical practice. The Best Evidence Medical Education Collaboration and the Campbell Collaboration are 2 groups that strive for the systematic review of studies in teaching and education domains.1 The BEME Collaboration suggests appraising the evidence in medical education using the QUESTS criteria.8 Details of the QUESTS dimensions are provided in Table 4. The use of the educational information retrieved from the research literature should be selected based on the highest quality of evidence in design, utility, extent, strength, target, and setting. Guidelines for evaluating educational manuscripts have been published.8,13 Following appraisal of the current literature, areas of additional research should be identified for researchers to target their education projects. Learning to search for high-quality research in education and being able to appraise the quality is critical for the enhancement of SoTL projects as well as in using a scholarly approach to teaching.
The analysis of the design and methodology of high-quality education literature is not dissimilar from that of basic science or clinical science literature evaluation. Kirkpatrick described the different levels of evaluation that can be used in SoTL or in evaluating effectiveness of teaching in terms of a hierarchy.14 These 4 levels from lowest to highest include: evaluation of reaction such as satisfaction, evaluation of learning such as knowledge or skills acquired, evaluation of behavior such as changes brought to the real world, and evaluation of results such as impact of behavior changes on society, outcomes, or costs. As the scholar progresses in this hierarchy, the level of evidence for the educational research is strengthened. Ideally, researchers in education should be striving to identify strategies that have impacts on the higher levels of evaluation so that these strategies may be reproduced. Higher levels of evidence also would be more useful for enhancing teaching. In searching the education literature, more of the evidence found is often at the lower levels of evaluations, ie, levels 1 and 2, in contrast to evidence published in the basic science or clinical science literature.
SUMMARY
In the same way that pharmacy educators take an evidence-based approach to research and to caring for patients, it is important for them to take a scholarly approach to teaching. It is essential to effectively search for and use evidence to improve teaching, to identify where there are needs for further research in teaching and learning, and to enhance the application and integration of SoTL. Ideas should be defined and search terms should be identified. A comprehensive search should be conducted in all appropriate resources. Additionally, published literature outside of pharmacy education may be useful in guiding applications to pharmacy. All information found should be appraised to determine its value in improving pharmacy education.
- Received November 21, 2013.
- Accepted January 16, 2014.
- © 2014 American Association of Colleges of Pharmacy
