The academic success of faculty members in higher education has been based primarily on the scholarship of research and its resultant publication. This is a rather narrow approach to faculty evaluation because other activities (practice, service, and teaching) are also required of faculty members but are not given the same emphasis in terms of faculty evaluation for promotion. In 1990, Ernest Boyer of the Carnegie Foundation published a book entitled Scholarship Reconsidered, which stimulated discussions among academicians regarding different categories of scholarship.1 He proposed that 4 areas be applied to the evaluation of faculty members and these included the scholarship of discovery (original research), integration (linking discovery among disciplines or analytical approaches), application (translation from basic science to the clinical practice arena), and teaching. 1 Difficulties in implementing this concept have included defining the scholarship of teaching and measuring excellence in teaching. 2 Glassick proposed that measuring excellence in teaching should be similar to that for research or discovery. These standards are to have (1) clear goals, (2) adequate preparation, (3) appropriate methods, (4) significant results, (5) effective presentation, and (6) reflective critique. 3 These features should be applied to manuscripts that present curricular development or innovation. The writings of Boyer and Glassick have the underpinnings that for scholarship to be truly revered as such, the outcomes of such efforts must be peer-reviewed and publically disseminated with the intention that the scholarly efforts could be reproduced by others.

In 2004, a Task Force of the American Association of Colleges of Pharmacy was charged with developing guidelines for manuscripts submitted to the Journal in the category of Instructional Design and Assessment. The work of the Task Force was published in the Journal in 2004 and introduced a format for these papers (Introduction, Design, Evaluation and Assessment, and Summary) and the resulting acronym: IDEAS.4 The creation of the IDEAS format was mainly influenced by a reported outline for curriculum development articles for the medical literature 5 and Boyer's descriptions of the scholarship of teaching. 1,3 The main premise of the IDEAS format was that articles describing the scholarship of teaching and learning (SOTL) should adhere to the same rigorous standards required of those dealing with other areas of scholarship, such as discovery, integration, and application. Thus, the goal of the IDEAS format was to identify and standardize the crucial elements required for the publication of quality SOTL studies in the Journal.

Since the publication of the IDEAS format, many SOTL articles have appeared in the Journal using the Task Force's recommendations and authors' adherence to these guidelines have increased the quality of the manuscripts/articles published in this category. However, the concept of SOTL has now become widespread enough in pharmacy education that Journal editors felt the IDEAS format should be updated to reflect a higher level of rigor, especially in the area of assessment. While the initial IDEAS format simply included a requirement for an assessment of the effectiveness of an educational strategy, the revised IDEAS format goes further, requiring authors to use an evidence-based approach to document that learners' knowledge or performance improved via appropriate assessments.6 Therefore, the main purpose of this article is to update the original IDEAS format by providing clear guidelines and specific examples of an enhanced level of rigor that will now be expected in SOTL articles appearing in the Journal. Additionally, the suggested outline for such manuscripts has been modified to reflect the new recommendations (Appendix 1).

These guidelines apply to manuscripts that describe and evaluate instructional design and include new courses, parts of courses, integration of selected competencies across the curriculum (eg, service learning, critical thinking, communication), assessment of instructional outcomes, and the use of technologies and new delivery methods. Authors wishing to publish their work should consider that new curricular design/innovations should have been offered and evaluated at least once before submitting the manuscript to the Journal. Preferably, the manuscript should be offered more than once, but the degree of innovation will determine whether it is deemed publishable after one offering.

The sections below describe the various components of the IDEAS format that should be incorporated into the manuscript. This format should be adhered to for curriculum development and SOTL manuscripts and can serve as a guide for planning new curriculum development or innovations that may in fact be replicated at other colleges and schools of pharmacy.

INTRODUCTION SECTION OF THE IDEAS MANUSCRIPT

The introduction section of the article should provide a brief overview of the manuscript and include a statement explaining the need to address a perceived curricular deficit or other reasons for the design of the innovation. The significance of the curricular effort from the perspective of the Accreditation Council on Pharmaceutical Education (ACPE)7 competencies and Center for Advancement of Pharmacy Education (CAPE) educational outcomes should also be noted. 8 This section should also provide important background information, including relevant literature from pharmacy, medicine, and other health professions, as well as literature from the field of education that addresses similar curricular efforts. Specific curricular goals or hypotheses to be evaluated or tested should be stated. The instructional component's context within the entire curriculum should be addressed. Additionally, any specific educational terminology used in the manuscript should be defined in the Introduction.

DESIGN SECTION OF THE IDEAS MANUSCRIPT

The Design section of the manuscript should provide a detailed description of and rationale for the methods used to design and evaluate the curricular innovation/instructional component. Further, the author should explain how the curricular goals or stated hypothesis were achieved. Irrespective of the evaluation strategy, the methods should be stated clearly and be of sufficient rigor to qualify as the “scholarship of teaching,” as defined by Boyer,1 and where appropriate, should be supported by the literature. Enough detail should be provided in this section so that the interested reader could reproduce the instructional design and evaluation. The following points should be addressed in this section: expected outcomes and learning objectives, educational environment, pedagogy/andragogy, content, and assessment/evaluation methods. Each of these points is described in more detail below.

Expected Outcomes and Learning Objectives

If applicable, the general outcomes expected from the curricular design should be stated in specific ability terms (ie, what students will be able to do after completion of the instructional module). Additionally, specific objectives and competencies should be stated in the form of knowledge, skills, and/or attitudes. The authors are encouraged to consult both Bloom's9 and Fink's 10 taxonomy of learning for identifying the cognitive domain of each specific objective or competency in their curricular design (Table 1 and Table 2).

Educational Environment

A section addressing the educational environment that provides a description of and rationale for the student audience, characteristics of the school, class or “sample” population, required faculty manpower and credentials, and placement of the innovation within the curriculum or the course should be provided. Logistical issues such as required facilities, single- and/or multiple-campus audiences and/or faculty (eg, Web-based or satellite programs), study settings, and other logistical challenges or concerns should also be addressed including the time required for development, implementation, delivery, and assessment.

Pedagogy/Andragogy

The primary pedagogy or andragogy of the curricular design or innovation should be included. The educational literature defines pedagogy as “teacher focused” and andragogy as more “learner focused.11,12 The discussion should include a description of and rationale for the use of specific teaching and learning methods (eg, active learning, didactic teaching), use of technology (eg, synchronous or asynchronous teaching, or a hybrid), and the role of students and faculty members. Additionally, assessment methods used to evaluate student learning (eg, traditional examinations, skill-based examinations, and formative or summative assessments) should be described and examples of student assessment tools provided. Estimates of time needed for students to prepare outside of class time when appropriate, should also be included.

Content

Author(s) should provide information regarding the content of their curricular design/innovation including a broad overview of the curricular design or innovation and how/why the curricular content was chosen for redesign. Additionally, one unit, case, or problem used within the curricular design or innovation should be described in detail. This detailed description may include examples of innovative or interesting products developed by the instructor(s) (eg, cases, problem-based learning (PBL) problems, descriptions of active-learning sessions, instruction sheets given to students for activities or assignments, and examples of student work in the course). A list of required textbooks and/or Web resources that students are asked to use should also be provided.

Materials such as assessment tools and learning assignments should be submitted for peer review, recognizing that Journal space limitations may not permit publication of all of these. If the Journal is unable to publish the materials, the author is encouraged to add a note stating the materials are available upon request. Authors can also provide electronic examples of tools or material online by posting them on a personal or course Web site and including the Internet link (URL) as a reference.

Where necessary, investigational review board approval by the author's institution should be documented or a statement that the work has been determined to be exempt from review should be included.

EVALUATION AND ASSESSMENT SECTION OF THE IDEAS MANUSCRIPT

The methods used to assess or evaluate the achievement or the curricular design or innovation, or to test the stated hypothesis, including a rationale for their selection, should be described here. The author(s) are encouraged to provide examples of their assessment tools. Relevant health sciences or educational literature should be cited where appropriate as support for the methods used. A description of all statistical and analytical tests used should be provided, along with a rationale for their use.

The evaluation and assessment section has 2 major goals: (1) evaluation of the teaching strategy or curricular innovation, including perspectives of the faculty member(s) involved or other stakeholders involved; and (2) determination of the performance of the learner (eg, gained knowledge and skills). Hence, this section has 2 major components: curriculum evaluation and learner evaluation. In order to meet the 2 main goals of this section, authors must generate and analyze outcomes data that directly link to the “expected outcomes” described in the manuscript's design section.

Evaluation and Assessment Data

Tables and figures should succinctly summarize the key assessment findings and provide documentation of the outcomes. Tables should emphasize the most relevant findings and be of sufficient detail to stand alone, and redundancy of table content in the text should be avoided. Figures may consist of an assessment tool (eg, online surveys) or highlight key aspects of the instructional strategy.

The scholarship of teaching must have comparable scientific rigor as other research areas defined by Boyer.1 Thus, providing data that support evidence for learning is critical. In addition to curricular evaluation, the assessment should consider students' perceptions of the teaching innovation and objectively measure student learning (eg, traditional examinations, pretest/posttest examinations, use of formative vs summative assessments, computer-assisted instruction, etc). Summary data regarding qualitative student responses can be included (eg, reflective comments on what was learned and how it occurred as well as behavioral changes). Instructor self-assessment and peer review are also useful to evaluate the teaching approach. Where appropriate, other stakeholder opinions within and outside the academic pharmacy environment should be included, especially if the innovation involves a long-term perspective related to attitudinal or behavioral changes.

Both quantitative and qualitative data and collection methods may be used to measure expected learning outcomes; however, the types of data and collection methods selected must be appropriate to measure the stated objectives.13 For example, while qualitative data (eg, student comments) might be collected via an informal “minute paper” exercise to support an objective such as “students appropriately compare and contrast the benefits versus the limitations of using course content in the practice setting,” most SOTL studies require a more formal methodology that generates quantitative data. Other important factors to consider as part of evaluation and assessment are the time for student and instructor preparation and institutional costs. If the teaching strategy or curricular innovation requires extensive preparation time, this should be quantified. If technology is part of the teaching approach, appraisal of the technology should occur to determine whether it is more or less time efficient. Lastly, if additional resources (eg, financial costs and use of personnel) were required to implement the teaching or curricular innovation/instructional design, this should be quantified.

Table 3 contains a partial listing of types of data that could be used in an IDEAS format manuscript and related strengths and limitations. Although objective data are preferred, authors should not be dissuaded from using qualitative data, where appropriate. Additionally, when the data are based on attitudinal scales, such as in questionnaires, pre- and post-course evaluations, or instructor evaluations, the authors should adhere to the best practices outlined in the Journal for this type of data.14, 15 For example, the issues of representativeness and response rates should be clearly addressed. Authors should consult with experts in survey research to assure that appropriate methods are used. Individuals considering publishing using the IDEAS format are encouraged to consider recent publications in the Journal that clearly document learning related to new teaching strategy or curricular innovation.

Several recent papers have appeared in the Journal since the original publication of the IDEAS report in 2004 that include rigorous types of evidence-based research data described in Table 3. One investigator used performance and attitudinal data in their study using progress assessment to evaluate students' knowledge prior to beginning advanced pharmacy practice experiences (APPEs).16 These data were subsequently used by the college's assessment committee to supplement their ongoing curricular mapping processes. A second study used student attitudinal surveys that were longitudinal and spanned into the postgraduate years, including qualitative interviews to evaluate simulated patients versus standardized patients in objective structured clinical examinations (OSCEs). 17 Observational data also have been published that evaluated the impact of a laboratory course on the manual blood pressure and heart rate measurement skills of students. 18 These studies highlight that diverse types of data are acceptable and encouraged for the assessment section of papers to be submitted in the category of instructional design and assessment.

DISCUSSION SECTION OF THE IDEAS MANUSCRIPT

The Evaluation and Assessment section must be followed by a Discussion section, where the authors reflect on and critically evaluate the key findings. Examples of issues appropriate for inclusion in the Discussion section are listed in the Appendix.19 Restating the hypothesis at the beginning of the Discussion focuses the reader on the purpose of the curricular design/innovation and allows the author(s) the opportunity to convince the reader that it is valuable and that sufficient information is provided to develop and implement the described program at another institution. The outcome of the curricular design/innovation should be interpreted and placed into context for the reader. For example, if the purpose of the curricular design/innovation is to eliminate a perceived gap in the existing curriculum, then describe how the void was discovered and whether the curricular design or innovation was successful in bridging the gap. Moreover the authors should highlight the relevance of the curricular design or innovation, the process of development and implementation, feedback from students or learners, an assessment and evaluation of student performance, and discussions of whether the desired outcomes were achieved. If statistical analyses were performed, place the results into perspective, whether results are statistically significant and whether the results are relevant. Critique the suitability or usefulness of supporting materials, including any required texts or reading assignments that were used. In addition, relate the findings to pertinent work that others have published and point out similarities and differences. 4 Additionally, this section should be concluded with an overall discussion of the effectiveness of design strategy including its limitations. This is a critical section of manuscripts in this category as it documents, in measurable terms, the results of the teaching strategy or innovation.

Keep in mind that the curricular design or innovation, content, and teaching strategies described may be used as a template for others to use in developing similar instructional strategies in another program. Therefore, it is important to review the obstacles and other barriers that were encountered during this process and how they were overcome. Likewise, the manner in which anticipated barriers were avoided or overcome should be discussed. If there were unresolved problems with the instructional design, offer a plan to remedy the problem. The salient take-home points should be highlighted. It is important not to generalize the findings of the report beyond the parameters of the instructional design.

Because pharmacy, medicine, nursing, and other health professions education are rapidly changing, curricular design and innovation are expected to evolve with these changes. Thus, plans for how the institution or instructors will keep the design/innovation current should be discussed in this section as well.

SUMMARY SECTION OF THE IDEAS MANUSCRIPT

The major goals of the instructional design or the original hypothesis should be restated in the Summary section of the paper. Additionally, the main findings in relation to the stated goals and/or hypothesis should be summarized.

THE IDEAS FORMAT AND SOTL

These guidelines for preparing manuscripts for the Instructional Design and Assessment (IDEAS) category of the Journal are written to (1) encourage the use of a scholarly approach in curricular revision or innovation; (2) provide educators with guidelines for the organization and content of an article that will undergo peer review; and (3) facilitate the timely distribution and dissemination of curricular innovation to the education community.

The scholarship of teaching is gaining credibility as a valued criteria for academic education and being given full consideration with respect to promotion and tenure.20,21 While quality instruction and teaching are the most important goals for higher education, the scholarship of teaching can also result in higher quality instruction and recognition for the academic community. Sharing the successes as well as the disappointments of curricular development and innovation within the academy in a timely manner is critical for the entire academic community. The goal for both the original and this updated IDEAS format is to provide a template for those in academic pharmacy to successfully share what they learn about curricular development and assessment in a manner that promotes the scholarship of teaching.

The revised outline in Appendix 1 reflects the changes made in this update. The assessment portion of the manuscript should be scientifically rigorous using evidence-based approaches. The discussion of the implications of the scholarly work should be included as subheadings in the evaluation and assessment section. The summary should reflect more traditional conclusions where major findings of the work are highlighted.

As the expectations for the scientific rigor for articles on instructional design and assessment increases, the quality of manuscripts submitted to the Journal will increase and reflect more scholarly outputs. It is not the authors' intention to dissuade the sharing of innovative instructional design ideas that lack rigorous assessment data. It is appropriate that innovative ideas lacking in assessment rigor should be presented at professional meetings and the corresponding abstracts published in the Journal.