Curriculum Leadership Institute

Pathways to School Improvement

E-Hints Related to Assessment

Below is an archive of E-Hints related to assessment. We welcome you to use them to improve assessment practices in your classroom or share them with a colleague who might benefit.

Thank you for making an effort to make sure kids are learning, even when it means revising a beloved lesson!

Assessment Tip - Align your assessments to the content AND skill (verb) to make sure your assessment item is valid.

3 Ways to Avoid the Dreaded “Teacher Lottery”

by

We’ve all heard it. Kids talking about their schedules like this, “Yes! I’m going to get an A in math this year because Ms. So-and-so is so easy!” Or, “Dangit, science is going to be so hard. I got Mr. Tough-stuff, and he doesn’t let anything slide.” As teachers, you never want to be considered the easy teacher, but you also don’t want to be the hard teacher that the students dread. But, what if there was no easy teacher or hard teacher, and all learning environments were equally fair? Here are three ways to even the playing field for your students so that the “teacher lottery” becomes less varied.

Set Clear Curricular Expectations

First things first, standards are not curriculum, and neither is a textbook! Depending on the state and content area, some standards are written to be very broad and general. Some are written for a grade band rather than a specific grade level. In many cases, a standard can be interpreted differently by different teachers. Additionally, standards are not organized into teachable units and may not include a level of emphasis or rigor. Textbooks, on the other hand, may include levels of emphasis and rigor, but may not reflect local priorities and may have more lessons than can be taught in one school year. In my Intro to Business class for example, my textbook included 28 chapters. But in any given year, I could only get through 13 of them with my students. The textbook that my school had adopted for that class included more than twice the amount information that I could teach my students.

The solution to both of these issues, is to develop a local curriculum that is organized into teachable units made up of learning targets that are written clearly (so all teachers interpret them the same way), are measurable (avoid verbs like understand, learn, etc.), and is appropriate for the amount of time that you have with your students. This way, the expectations are clear for both teachers and students.

Make End-of-Unit Assessments Common

When all teachers for the same course are using the same assessment to determine success, students are held to the same expectations. To further ensure that teachers are administering the common assessment fairly, include a set of “administration guidelines” as a cover sheet. Information that should be clarified in the Administration Guidelines includes:

  • the amount of time a student is allowed to complete the assessment (must the assessment be completed within a single class period so students cannot discuss answers when they gather after class?),
  • any materials they are allowed (are math students permitted to use a calculator, or language arts students a dictionary?),
  • the amount of assistance a teacher is allowed to give when a student has a question (may a teacher explain the definition of a word, or clarify directions?),
  • and the criteria for successful completion.

Please note that the Administration Guidelines as explained above are meant for general education students that are not on IEPs. Any student with an IEP must be given the modifications that are outlined within their IEP. Similarly, when administering common assessments to ELL students, consult their ELL teacher to determine appropriate accommodations.

Collaborate as a grade-level team

The big idea for all curriculum and assessment work is to improve classroom practices. By collaborating with your grade-level team, you’re able to share what worked and what didn’t. So, if something didn’t work in your lesson, ask a colleague who saw success to share what strategies they used. It’s OK to be vulnerable, and in fact, improving your teaching depends on it. Look for areas that you can improve and help those that can use your help to find more success in their classroom.

Students have enough to deal with without having to worry about which teacher they’re going to have. It will take a bit of work up front to complete the steps outlined above, but your school will be better for it. Your school doesn’t need easy teachers or hard teachers, they need good teachers.

Need some help with these things? Let us know!

Proficiency Scales and the CLI Model

by

Many of our school districts have begun work with proficiency scales, a tool introduced by Robert Marzano and Marzano Research to determine levels of student learning. Teachers have seen success in their classrooms when sharing proficiency scales with their students and allowing students to track their progression from level-to-level; therefore, allowing students to take ownership of their learning. One benefit of including proficiency scales in your curriculum, instruction, and assessment work is that both the teacher and student can use them.  Let’s take a closer look at what proficiency scales are, their purposes, and how they work within the CLI Model.

Proficiency scales, typically a four-point scale, include related curricular targets and scores which are intended to clarify a progression of learning. They allow teachers and students to identify student performance. Please review this sample template. The use of proficiency scales is a decision to be made by your Curriculum Coordinating Council (CCC).  Keep in mind, that proficiency scales can be altered to meet the needs of your school or district.

Additionally, a proficiency scale can aid Subject Area Committee (SAC) members in identifying which curricular targets are a priority and must be on a common assessment. Therefore, creating proficiency scales fits perfectly as the first step of assessment work in year three of the CLI Model. This graphic organizer, revised from the Common Assessment Development Cycle graphic from Marzano Research, illustrates the assessment process that we use in our model.

By creating proficiency scales first, SAC members identify priority curricular targets (components) and use this information to better plan their assessments. Jan Hoegh, a consultant for Marzano Research, recommends the following questions as a way to identify what should be included on a proficiency scale.

What is the primary topic of the outcome? Outcomes are summary statements for a unit of instruction. They are meant to encompass all of the components listed as steps to achieving the outcome. Determine the primary topic of the outcome and use that information to develop your level 3.0 score. Sometimes, one of your components may fully communicate the primary topic of the outcome and can, therefore, be copied and pasted into the level 3.0 field of the proficiency scale.

Are there any components that don’t directly relate to the primary target? Some states include standards that are meant to be repeated from grade-to-grade or are more supplemental in nature. Although these standards are important and work well within an outcome, they may not directly relate to the primary target. If this is the case, they may be excluded from the proficiency scale.

Please remember, however, that even if a component is not included on the proficiency scale, it must still be taught and assessed. Everything that is included in your district’s guaranteed and viable curriculum is essential, and students are still expected to learn it. But, components of this nature may be evident in the final product without having specific items that measure them; they tend to be critical understandings or skills that contribute to the end result.

Are there any components that are pre-requisite knowledge or skills? Evaluate the remaining components to determine if they meet this criterion. If so, these may be included in your level 2.0 score.

Similar to other steps of your curriculum, instruction, and assessment work, please think of these questions as guiding questions, and understand they will not always provide the answers that you need to complete your proficiency scales fully. That does not mean that your curriculum was written poorly, but it does mean that you need to think about the end result and include information on the proficiency scale that will prove beneficial to teachers and students. Remember, the purpose of a proficiency scale is to clarify the learning progression for teachers and students and further allow them to identify where a student lies on that progression.

Using Data to Improve Instruction in Five Steps or Less

by

In a recent conversation, a principal at a Curriculum Leadership Institute (CLI) client district expressed concern about the sixth-grade math team. The district received state test results and it was clear that the sixth-grade students, as a whole, underperformed on one specific state standard. Unsure of the correct course of action, his initial plan was to re-evaluate their current resource. It is possible that the issue may be a misalignment between the current resource and the curriculum, but there are a few things to consider prior to spending the money on adopting a new resource.

Before being able to analyze and address the solution to the problem, the school district must have some essential pieces in place. The first piece is a locally-written curriculum. As stated in a previous E-Hint, state standards or an adopted resource are not synonymous with curriculum. But rather, curriculum is what your individual district defines as essential skills and knowledge for all students. The second task for the district is to have common assessments in place that are carefully aligned to the curriculum.  Each skill and piece of knowledge included in the curriculum must also be assessed in a way that allows students to show they can demonstrate the required learning.  Additionally, instructional plans must be created throughout the district. We, at CLI, recommend using a standard planning format for all teachers. We use the Instructional Planning Resource that includes formative assessment (component assessment) with predetermined criteria, teacher methods, student activities, resources, and differentiation. Instructional plans should be housed in an easily accessible manner so they may be evaluated for effectiveness and, if effective, used from year-to-year.

When all of these pieces are in place educators may begin to look at assessment data. Remember, there are three different types of assessments: formative, interim, and summative. Formative assessments are created and used by individual teachers and PLCs to immediately drive instruction and predict student success on major assessments or high stakes tests.  Interim assessments are larger assessments that incorporate several targets (components) and assess

at the unit level (outcome). These are formative in nature as re-teaching and re-assessing are still possible. Finally, districts are state and federally required to implement summative assessments, often called state or other standardized tests. These tests are typically used to evaluate a district as a whole. They are considered summative assessments as they are not used to drive instruction for the same population of students since scores are not known during the same school year and re-teaching and re-assessing are not possible.

When teachers receive data from formative and interim assessments, it may be clear what the next steps are in terms of re-teaching and re-assessing.  But, when teachers receive data from the previous school year’s summative assessments, oftentimes the next steps may be somewhat confusing or daunting.  The first thing to keep in mind when analyzing this data, is to look for big things that stand out. In the example in the introduction, the big “Aha!” was the sixth-grade students under-performing on one entire standard. So now what?

Now, review the three essential pieces already mentioned above: curriculum, assessment, and instruction. Begin with the locally-written curriculum. Check the alignment of the curriculum to the state standards; particularly the standard that students performed poorly on. Ask yourself or your PLC these questions:

  • Does the local curriculum require the same skills and knowledge as those in the state standards?
  • Is the “essence” of the standard the same when it was transferred to curriculum language?

If the curriculum checks out, and you’ve determined that it is appropriately aligned to the standards, take a look at the corresponding local common assessment (interim) that is in place. Determine if the common assessment is aligned to the curriculum. You may cross-reference the common assessment with the curriculum rather than the standards (since you’ve already critiqued the curriculum for alignment). Ask yourself or your PLC these questions:

  • Does the content (knowledge) in the common assessment meet that which is in the curriculum? Does it go beyond?
  • Do the verbs (skills) in the common assessment align with those in the curriculum? Do they go beyond?

Then determine if the expected performance on the common assessment is appropriate. Ask yourself or your PLC these questions:

  • Is it developmentally or task appropriate?
  • Is the rigor aligned to descriptions of student performance from the curriculum?
  • Did the student performances on the common assessment match the performances on the summative assessment? Are there commonalities in the performances of students that match the data from the summative assessment performance?

If you’ve determined that your common assessment is properly aligned with the local curriculum, you should move on to evaluating instruction. Ask yourself or your PLC these questions:

  • Do in-class activities allow for practicing the same types of tasks appearing on the assessment?
  • Does the content being taught (amount and type) align with the curriculum/assessment?
  • Is the rigor of learning activities high enough to prepare students for the assessment?
  • Is there sufficient instruction, practice, and remediation?

Typically, after evaluating the local curriculum, common assessments, and instruction, you’ll be able to determine the issue and correct it. But sometimes, the issue may not fall within the documented curriculum, assessment, or instruction. If that is the case, ask yourself or your PLC these questions:

  • Do the common assessment and the state assessment value the same skills?
  • Does curriculum pacing allow for instruction and assessment of appropriate learning prior to state tests?
  • Have interventions been effective?

In the example being used, it turned out that the sixth-grade math team was teaching the standard in question after the summative assessment was given. So in their case, they were able to re-organize their pacing to solve the problem, rather than purchase or create a whole new resource. Had they not taken the time to disaggregate the summative data, they could have arbitrarily changed the curriculum, instruction, and/or assessments, or purchased a new resource without seeing any improvement in scores. This type of thorough data analysis undoubtedly leads to overall evaluation of curriculum, instruction, and assessment which benefits both teachers and students.

Initial CLI District Response to 6 Key Elements of the Every Student Succeeds Act

by

download_pdf_smAnalysis of relevant elements of the Every Student Succeeds Act (ESSA) suggests topics of discussion for the CLI district’s curriculum coordinating council.  Although those who prepared ESSA say the act places most responsibility for establishing provisions to ensure school accountability on states, many stipulations included at the federal level do not allow much leeway for creativity or alternative interpretations.  It is therefore suggested that curriculum coordinating councils place the selected relevant topics found in the table below on discussion agendas, with the expectation that whatever the state ultimately decides will need to fall within the stipulated parameters is incorporated into the district’s response.  Familiarity with the entire ESSA document, which speaks to state level options and opportunities, is also advisable.

RELEVANT ESSA STIPULATION INTERPRETATION IN THE CONTEXT OF THE CLI MODEL SUGGESTED CLI DISTRICT RESPONSE
Each state’s plan must provide an assurance that the state has adopted challenging academic content standards and aligned academic achievement standards (challenging state academic standards) that include not less than three levels of achievement. This was written with the knowledge that some states will use national standards such as the Common Core State Standards or Next Generation Science Standards, and others will either write new standards or retain those used during the NCLB era. The ESSA stipulation neither requires nor eliminates the use of state tests in specific subject areas, but obviously suggests the development of local assessments that can classify and record student achievement according to proficiency levels. A CLI district that has completed all elements of its K-12 curriculum should find it easy to align with any new state standards, since the wording and formatting are already in place. CCC directives to SACs when modified state standards are prepared are to cross-check for wording, especially in terms of measurable verbs and academic content. New or additional high achievement unit outcomes may need to be written. Also, SACs may need to examine the summative assessments and formative assessment categories in instructional planning resource (IPR) documents to ensure the inclusion of point totals or rubrics that meet the “three levels of achievement” indicated in the ESSA stipulation.
Each state is required to adopt ELP (English language proficiency) standards aligned to the state’s reading or language arts content standards. Standards must be aligned to ensure students who meet the standards are on track for postsecondary education or employment. At this point we cannot be certain how a given state will “align” ELP to its reading or language arts content standards. However, there will doubtless be some kind of proficiency indicator with regard to speaking, reading and writing the English language that includes vocabulary tests measuring how well students understand commonly applied technical terms, or phrases associated with other forms of higher education. Once a clearly articulated English Language Arts curriculum is established, English language learner specialists can help to identify the additional language needs of English language learners. Those learner outcomes or accommodations would be added to the English curriculum. A CLI district also often develops and uses a vocabulary/spelling proficiency list for each of the grade levels and subject areas, especially at the elementary school level. Teachers are required to include those words in their instructional programs. Building principals would also play a role in working with teachers in improving instructional programs.
Assessments must measure multiple measures of student achievement, including higher order thinking skills and student growth. States are permitted to meet these requirements by administering a single summative assessment or multiple assessments during a school year. Each method must result in a single summative score. Portfolios, extended performance tasks and computer-adaptive assessments are permitted testing practices. Your state may have already decided on the kind of assessments it will use, or will just continue to use what it already has. The problem is that many older tests do not test for high order thinking or student growth. Mass testing requires mass scoring, which should include the creation of very sophisticated assessment items as well as extensive training for those who evaluate them. This is especially true for portfolios and extended performance tasks. CLI districts are already acquainted with high achievement unit outcomes that require multi-faceted student responses. They are often used in the summative assessment category of instructional planning resource documents. In fact, the assessments written by CLI districts might serve as examples of how state assessments could be created to measure higher order thinking skills.
Accountability is measured through academic achievement measured by proficiency on annual assessments, which may include student growth for high schools. Allowed in elementary and middle grades is a measure of student growth or other valid and reliable indicator that allows for differentiation in student performance. Progress (in all grades) must be achieved in English language proficiency. At least one indicator of school quality or student success must be deemed valid, reliable, comparable and allow for meaningful differentiation. Clearly, a one size fits all state assessment of the kind used predominately in the NCLB era would be difficult to use to measure student growth in terms of “differentiation in student performance.” Mixed with the assessment of English language proficiency, it is even more difficult. An indicator of school quality in the context of validity and reliability, and a measure of comparability and differentiation, is difficult to define. Let us assume that state officials will need to depend on model district and school programs to define and apply the stipulation found here. As with the above response, student growth in the academics and English language can be measured using CLI processes, primarily because they are built on specific and carefully worded intentions for student learning (outcomes/components). CLI’s IPR and teacher use of it will certainly facilitate validity (meeting intended purposes) and reliability (meeting such purposes consistently). Again, the existence of a well-crafted and written curriculum provides a much better chance that comparisons among proficiency scores can be made and differentiations noted in the quality of student learning.
For each school identified as low-performing, districts must develop a comprehensive support and improvement plan that includes evidence-based interventions. The plan should be developed after a needs assessment and the identification of resource inequities in each low performing school. The assumption in this stipulation is that a district would use an intervention only when a school is low-performing. The more efficacious method would be to have a continuously applied plan to maintain quality among ALL schools, making modifications when evidence indicates implementation problems in those schools which are not meeting expectations. The CLI Model is already established as an ongoing plan to continuously improve ALL schools in a district, and the data generated by the implementation of that plan makes any deficiencies clear. The academic governance structure of the CCC/SAC system, working in concert with the administrative team and board of education, can quickly make necessary adjustments in low performing schools.
States must establish and implement, with consultation with school districts, standardized statewide entrance and exit procedures for English learners. Educator professional development and preparation activities must be established to improve teaching skills in meeting the diverse needs of English learners. Districts in each state must obviously work in concert with the state in making this kind of system work. States might be advised to study progressive districts who have designed ELL entrance and exit programs that serve the described needs of English learners. Professional development at the district and building levels should incorporate training in working with English learners along with all other students. The CLI Model is both a program development and maintenance construct, as well as an ongoing and integrated form of teacher training. Again, in the CLI Model the curriculum is actually developed and implemented by teachers, as led by the CCC, SACs, and administrative teams. Such constant improvement efforts and monitoring are ongoing and effective.

The above areas of interest are primarily focused on matters pertaining to a district’s curriculum, instruction, and assessment.  They are very relevant to the work of the district’s curriculum coordinating council and subject area committees.  While the excitement around ESSA focuses on the return of powers to state and local boards, it is still important for district officials to become familiar with all the ESSA regulations with regard to requirements of states, including funding distributions, data collection of student progress, and other matters that will impact the allocation of federal funds for school improvement activities.

Curriculum is a Roadmap

by

download_pdf_sm

“Curriculum” is discussed on a daily basis in conversations within schools among administrators, teachers, support staff, and outside stakeholders.  Strangely enough, it is a term that carries fundamental misconceptions that make those conversations difficult.  Unless everyone involved in the conversation has the same definition for curriculum, what is said and what is heard are often very different.

Perhaps the most common misconception about curriculum is that a textbook or series is the curriculum. Textbooks or any other instructional materials are actually resources. Resources are used to teach curriculum, but are not the curriculum itself. Many textbook series have a curriculum embedded into the resource; however, it is critical that a district determine its curricula locally before resources are adopted. This is to ensure that adopted resources properly support the learning that the district feels is essential – not the hidden curriculum that the publisher values.

Another misconception about curriculum has to do with the federal government’s requirement that each state determine a set of standard skills and content that must be taught at each grade level for each content area.  Although these sets of skills and content are clearly called state standards, many educators are under the false assumption that they are curriculum. It is clearly stated in most standards documents – including the Common Core State Standards for Math and English Language Arts – that they are not curriculum and that districts need to determine their curricula locally to assure that students meet the identified standards for content and skills.

True, local curriculum must fully employ state standards, but the district may value additional skills and content. Furthermore, standards documents may not be organized into a teachable sequence, but rather they are typically categorized according to similar attributes (strands or domains) such as reading skills, number sense, or presentational skills that would not necessarily be taught in isolation of the other strands or domains within the subject area standards. Note the following graphic representation of curriculum as a roadmap and note the role that resources and standards play in the student’s learning journey. 
curriculum-roadmap-diagram

In this illustration, the journey begins with a teacher and her students. This teacher is tasked with leading her students to their final destination (the end of the “road”) which includes their ability to demonstrate skills and knowledge of the state standards and the district-defined values. This could be a daunting task, particularly if the “road” is not broken down into teachable chunks or units. The breakdown of the journey, the roadmap, is the curriculum.

Relationship between Curriculum, Resources, Standards, and Differentiation

Curriculum should be organized to include curriculum targets that can be accurately assessed after a unit is completed. The desired “outcomes” for each of these teachable units are depicted as blue flags along the road in the illustration. Essentially, these outcomes are significant checkpoints of student learning along the way.

To ensure that students can be successful at the unit outcome level, learning must be broken down further into smaller steps that will be formatively assessed on a regular basis (daily or every few days, at least). These narrow curriculum targets are depicted as footprints along the road. Each unit outcome (flag) includes its own set of footprints.

So where do resources, best practices, and differentiation fit in to this analogy? Resources are what you pack in your suitcase. They are the supplemental materials that make the trip fun, engaging, and successful. There is rarely one resource that will align perfectly to your curriculum, which is why we use the suitcase to represent resources; it is filled with many great tools and materials – some are essential, others are nice to have along.

Incorporating best practices (e.g., 21st century skills, math practices, ELA text complexity, and ISTE standards) into instruction is like the sun that sheds light on the trip. Best practices help to produce a well-balanced experience for students.

Lastly, differentiation activities (i.e., remediation and enrichment) are the rest areas along the way. It is essential to pause to ensure all students are on board and ready to progress, using formative assessments as indicators of who might need remediation or intervention activities.  It is equally important to provide engaging, relevant activities for those students who are already with you and are able to keep up pace.

There are so many elements that go into effective teaching; but, the foundation of everything is to determine what students need to know and be able to do – the curriculum ­­– and how it will fit into the time that you’re given. If you haven’t already made your roadmap, make it a priority to identify curriculum targets. You’ll feel more confident for having a plan and the chances of students arriving at the destination are greatly improved!

Reassessment Done Right

by

The words Time to Evaluate on an ornate white clock, counting down to the moment a manager will perform an evaluation, review, assessment or reevaluation of a worker, property or process
Reteaching is not about punishment, it is about the end goal—learning.

download_pdf_smWhen thinking about mastery and student learning over time, it is almost impossible not to think about reassessment.  Philosophically, if you believe in allowing students to retake assessments, you may struggle with the application of this belief in actual classrooms.  The question is not whether students shall be allowed to reassess, but instead, how to reassess effectively.  So, when a student takes an initial assessment and performs below required proficiency, what happens next?  The following steps will guide the creation of your own reassessment procedures.

1.  Conference with the student.  Part of building relationships with students is communicating openly with them about expectations and learning.  As soon as you realize a student is not meeting proficiency, have a conversation and begin to make a plan.

2.  Set a timeline for reteaching and reassessment. During the conference, set up a schedule for reteaching to provide time and support for the student to demonstrate successful learning.  This must always occur before students are allowed to reassess.  If reteaching does not occur, how can a student be expected to perform at a higher level?  Also, schedule the reassessment.  Due to obvious time constraints, there must be a sense of urgency to complete the reassessment.  Some districts use a reassessment contract requiring student and/or parent signatures in order for students to retake assessments.

3.  Provide reteaching opportunities. Just like teachers, students are busy! They are taking many classes and may also be involved in activities outside of the normal school day.  It is important to provide varied opportunities for them to receive help within the reteaching timeframe.  Maybe the student needs to come in before or after school or during study hall, or maybe they can watch a video at home and complete alternate assignments to demonstrate their learning, or perhaps another teacher could provide additional instruction.  Remember, reteaching is not about punishment, it is about the end goal—learning.  It doesn’t have to be epic; in actuality, reteaching may only take a few minutes.

4.  Contact help when needed. Oftentimes, if a student is struggling on multiple assessments in one subject area, he or she will also be having trouble in other subjects.  A teacher won’t always know how a student is performing in other courses; however, counselors and parents should.  If a student continually struggles and is caught up in a reassessment cycle, it is imperative that the cycle is broken before the student gives up.  There could be a placement issue or a need for a special education referral.  Maybe the student is going through some emotional struggles due to something happening at home.  A sure sign of trouble is when a student is not meeting the timelines agreed upon within the reassessment conference.  If this is the case, reach out to the people who can help as soon as you can.

5.  Provide a reassessment that is different from the original assessment. It is always wise to have more than one version of an assessment.  If students know the reassessment is exactly the same as the original assessment, it is very easy for them not to prepare and to use the original assessment as a “practice” assessment.  Remember, it is important that you know what they know—not what they memorized after taking the first assessment.

6.  Decide whether the student needs to retake the entire assessment, or just a part. In many cases, summative assessments contain individual questions that require students to apply multiple skills or varied knowledge in order to answer them.  In these cases, students would more than likely need to retake the entire assessment.  However, an assessment may also contain very specialized questions related to one specific skill or piece of knowledge.  If students perform poorly on these types of questions, they may not need to take the entire assessment over again.  They could just retake the parts that relate to that skill or knowledge.  Particularly on project-based assessments, the student may need to improve only portions of their project—not redo the entire project.

7.  Decide how retakes will be scored.  Grading typically falls under district guidelines and policies.  It is important for all teachers to follow the same reassessment rules.  Will the most recent reassessment score replace the original?  Will the highest score be taken?  Follow district approved procedures for recording the appropriate score.