Tests remain the cornerstone of summative development in most formal instructional programs. They are accused of not being authentic or comprehensive. The dangers of incorrect interpretation have made some of us learning practitioners cautious if not skeptical.
Certainly, no instructional designer would make the claim that tests should stand on their own as the whole solution; however, most know that a valid and reliable exam can tell so much about student performance and instructional effectiveness to serve as firm starting ground for making planning and evaluation decisions. When interpreted correctly, learners, teachers, instructional designers, developers and managers can carry out meaningful conversations with quantifiable evidence from which to plan and improve learning experiences: the curriculum.
Even more so, learning practitioners realize the practical nature of exam development. Firstly, multitudes of students and teachers use them extensively gaining strong familiarity with their application. The learning curve is small. Although test-taking skills can confuscate subject-matter skills, effective tests can minimize this effect. Secondly, tools, applied research and best practices exist and shape the field. Practitioners can develop within an existing and evolving community of practice. Given the constraints of formal education and its expanding and changing mandate, practical means it can be done.
In the end, we can do tests. They can be a part of summative assessment. They can work with alternative assessments as mirrors to each other. The lesson noted is that we must do tests correctly or not at all. To do them correctly, we need to ensure that the exams we create are valid, reliable and pretty. They measure what is intended without bias. They promote confidence in their results over time and applications. They are professional creating ownership. With these goals in mind, we must select the right tools, recruit and develop capable people and we must establish a responsive process. In short, we must apply the principles and practices of collaborative, rapid development.
In the world of development, the investment up front expands the room within which we can build success. Analysis and design gives power to the development work to be done. With exam development, creating a blueprint, the crux of analysis and design, forms the first strong step and the primary deliverable. This exam blueprint establishes the objectives to be measured and the standards of performance as evaluation criteria.
An exam blueprint simplifies development clearly communicating what items should be included. It clarifies what the items should look like. It makes item development more the immediate focus than exam development. It sets up item classification necessary for item analysis after exams have been administered and graded. In this matter, the blueprint sets up evergreening with revision and renewal. More importantly, the blueprint makes additional forms and versions possible increasing exam security and eliminating bias. This blueprint is essential for rapid development allowing practitioners to focus on exams and items and not be stuck figuring out development processes. Templates can be made, deadlines met, and budgets preserved. It can be started and repeated without the typical turmoil and anxiety. Deliverables can be produced, evaluated and maintained. Iterative design and development processes can be executed to ensure quality. A kanban project model can be incorporated to keep workflow moving by tracking progress and determining needs and priorities.
Moreover, the exam blueprint invites all parties to the table. Item writers and reviewers have a shared map. Instructional designers bring evaluation best practices to the table. Managers and development leads gain assurance in results, being able to see the consistency and well-thought out plans for exams. Subject-matter experts and teachers can have confidence in what is being assessed matching what has been taught. Students, although they may not directly access blueprints, will experience clearer communications about what they will be assessed on and to what level. These blueprints invite collaboration. Working together will create teams and take full advantage of leadership and social energy.
"What does this mythical creature called the exam blueprint look like and how do I make one?", you may ask. The exam blueprint in its basic format is a table that lists the learning objectives to be measured in rows and the categories of difficulty in columns. The cells contain item numbers, tallies and percentages for each objective and difficulty. Moreover, the blueprint states the distribution of items based on objective weighting and required difficulty spread.
In a collaborative and rapid development context, these tables can be easily created and augmented in Google Sheets. This tool, Google Sheets, as a cloud-computing application, allows for multiple users to work in real-time with multiple devices in a variety of contexts. As a spreadsheet, Sheets offers the conditional formatting, filtering, sorting, calculation and analysis tools to make data entry, analysis and interpretation easier.
Typically, the exam blueprint contains several sheets to organize the master and forms. The first sheet, the master, outlines the objectives and standards to be used in developing the exam. This distribution is the ideal against all the forms will be measured. The following sheets outline the actual distribution of the items based on objectives and standards. On these sheets, conditional formatting provides a check against the Master sheet specifications. Using this check, instructional designers can edit or request new items to make the blueprint work.
Certainly, no instructional designer would make the claim that tests should stand on their own as the whole solution; however, most know that a valid and reliable exam can tell so much about student performance and instructional effectiveness to serve as firm starting ground for making planning and evaluation decisions. When interpreted correctly, learners, teachers, instructional designers, developers and managers can carry out meaningful conversations with quantifiable evidence from which to plan and improve learning experiences: the curriculum.
Even more so, learning practitioners realize the practical nature of exam development. Firstly, multitudes of students and teachers use them extensively gaining strong familiarity with their application. The learning curve is small. Although test-taking skills can confuscate subject-matter skills, effective tests can minimize this effect. Secondly, tools, applied research and best practices exist and shape the field. Practitioners can develop within an existing and evolving community of practice. Given the constraints of formal education and its expanding and changing mandate, practical means it can be done.
In the end, we can do tests. They can be a part of summative assessment. They can work with alternative assessments as mirrors to each other. The lesson noted is that we must do tests correctly or not at all. To do them correctly, we need to ensure that the exams we create are valid, reliable and pretty. They measure what is intended without bias. They promote confidence in their results over time and applications. They are professional creating ownership. With these goals in mind, we must select the right tools, recruit and develop capable people and we must establish a responsive process. In short, we must apply the principles and practices of collaborative, rapid development.
In the world of development, the investment up front expands the room within which we can build success. Analysis and design gives power to the development work to be done. With exam development, creating a blueprint, the crux of analysis and design, forms the first strong step and the primary deliverable. This exam blueprint establishes the objectives to be measured and the standards of performance as evaluation criteria.
An exam blueprint simplifies development clearly communicating what items should be included. It clarifies what the items should look like. It makes item development more the immediate focus than exam development. It sets up item classification necessary for item analysis after exams have been administered and graded. In this matter, the blueprint sets up evergreening with revision and renewal. More importantly, the blueprint makes additional forms and versions possible increasing exam security and eliminating bias. This blueprint is essential for rapid development allowing practitioners to focus on exams and items and not be stuck figuring out development processes. Templates can be made, deadlines met, and budgets preserved. It can be started and repeated without the typical turmoil and anxiety. Deliverables can be produced, evaluated and maintained. Iterative design and development processes can be executed to ensure quality. A kanban project model can be incorporated to keep workflow moving by tracking progress and determining needs and priorities.
Moreover, the exam blueprint invites all parties to the table. Item writers and reviewers have a shared map. Instructional designers bring evaluation best practices to the table. Managers and development leads gain assurance in results, being able to see the consistency and well-thought out plans for exams. Subject-matter experts and teachers can have confidence in what is being assessed matching what has been taught. Students, although they may not directly access blueprints, will experience clearer communications about what they will be assessed on and to what level. These blueprints invite collaboration. Working together will create teams and take full advantage of leadership and social energy.
"What does this mythical creature called the exam blueprint look like and how do I make one?", you may ask. The exam blueprint in its basic format is a table that lists the learning objectives to be measured in rows and the categories of difficulty in columns. The cells contain item numbers, tallies and percentages for each objective and difficulty. Moreover, the blueprint states the distribution of items based on objective weighting and required difficulty spread.
In a collaborative and rapid development context, these tables can be easily created and augmented in Google Sheets. This tool, Google Sheets, as a cloud-computing application, allows for multiple users to work in real-time with multiple devices in a variety of contexts. As a spreadsheet, Sheets offers the conditional formatting, filtering, sorting, calculation and analysis tools to make data entry, analysis and interpretation easier.
Typically, the exam blueprint contains several sheets to organize the master and forms. The first sheet, the master, outlines the objectives and standards to be used in developing the exam. This distribution is the ideal against all the forms will be measured. The following sheets outline the actual distribution of the items based on objectives and standards. On these sheets, conditional formatting provides a check against the Master sheet specifications. Using this check, instructional designers can edit or request new items to make the blueprint work.
 |
| Sample Unit Exam Template - Master sheet |
Comments
Post a Comment