You are on page 1of 452

Arizona State Board of Education

NOTICE OF PUBLIC MEETING


Pursuant to Arizona Revised Statutes (A.R.S.) 38-431.02, notice is hereby given to the
members of the Arizona State Board of Education and to the general public that the
Boards will hold a special meeting, open to the public, on Friday, August 14, 2015, at
9:00 AM at the Arizona Department of Education, Room 122, 1535 W. Jefferson,
Phoenix, AZ 85007. A copy of the agenda for the meeting is attached. The Board
reserves the right to change the order of items on the agenda, with the exception of
public hearings. One or more members of the Board may participate telephonically.
Agenda materials can be reviewed online at http://azsbe.az.gov.
Pursuant to A.R.S. 38-431.02 (H), the Board may discuss and take action concerning
any matter listed on the agenda.
Pursuant to A.R.S. 38-431.03(A)(3) and (4), the Board may vote to convene in
executive session for discussion or consultation for legal advice from the Boards
attorneys concerning any items on this agenda and/or for discussion or consultation
with the Boards attorneys in order to consider its position and instruct its attorneys in
pending or contemplated litigation or in settlement discussions conducted in order to
avoid or resolve litigation.
Persons with a disability may request a reasonable accommodation such as a sign
language interpreter, by contacting the State Board Office at (602) 542-5057. Requests
should be made as early as possible to allow time to arrange the accommodation.
DATED AND POSTED this 31st day of July, 2015.
Arizona State Board of Education

By: _______________________________________________________
Christine Thompson
Executive Director
(602) 542-5057
Friday, August 14, 2015
9:00 AM
Arizona Department of Education, Room 122
1535 W. Jefferson, Phoenix, AZ 85007

SPECIAL MEETING AGENDA


ARIZONA STATE BOARD OF EDUCATION
August 14, 2015
Page 2
9:00 a.m.

CALL TO ORDER, PLEDGE OF ALLEGIANCE, MOMENT OF SILENCE,


AND ROLL CALL
1. Presentation, discussion and possible action to adopt proposed
AzMERIT performance levels (cut scores).
2. Presentation, discussion and possible action to adopt a Move on
When Reading cut score for AzMERIT Grade 3, English Language
Arts (ELA)
3. Presentation and discussion regarding proposed performance levels
(cut scores) for the National Center and State Collaborative Alternate
Assessment (NCSC).
4. Presentation and discussion regarding the administration of the Move
On When Reading Program
5. Presentation, discussion and possible action regarding filling Board
staff vacancies in the positions of Deputy Director and Administrative
Assistant for the Investigative Unit, including consideration of the
Superintendents recommendation and those of other Board members.
Pursuant to A.R.S. 38-431.03(A)(3), the Board may vote to convene
in executive session, which will not be open to the public, for
discussion or consultation for legal advice with the Boards attorneys.
6. Presentation, discussion and possible action regarding the execution
of the May 18, 2015 Board policy requiring the Superintendent to grant
the employees of the State Board Investigation Unit access to
necessary documents, records and electronic information. Pursuant to
A.R.S. 38-431.03(A)(3) and (4), the Board may vote to convene in
executive session for discussion or consultation for legal advice with
the Board's attorneys.
7. Presentation, discussion and possible consideration regarding Douglas
v. State Board of Education (CV2015-006171). Pursuant to A.R.S.
38-431.03(A)(3) and (4), the Board may vote to convene in executive
session, which will not be open to the public, for discussion or
consultation for legal advice with the Boards attorneys and/or for
discussion or consultation with the Boards attorneys in order to
consider its position and instruct its attorneys in pending or
contemplated litigation or in settlement discussions conducted in order
to avoid or resolve litigation.

SPECIAL MEETING AGENDA


ARIZONA STATE BOARD OF EDUCATION
August 14, 2015
Page 3
8. CALL TO THE PUBLIC. This is the time for the public to comment.
Members of the Board may not discuss items that are not specifically
identified on the agenda. Therefore, pursuant to A.R.S. 38-431.01(H),
action taken as a result of public comment will be limited to directing
staff to study the matter, responding to any criticism or scheduling the
matter for further consideration and decision at a later date
ADJOURN

EXECUTIVE SUMMARY
Issue:

Arizona State Board of Education Special Meeting


August 14, 2015
Item 1
Page 1 of 3

Presentation, discussion and possible action to adopt AzMERIT


performance standards (cut scores)

Action/Discussion Item

Information Item

Background and Discussion


On November 3, 2014, the Board adopted AzMERIT as the statewide assessment to
measure the Arizona English Language Arts and Mathematics standards.
The March 2014 Board adopted values for the states new assessment guided the
AzMERIT Standard Setting Workshop held July 13-17, 2015 in Phoenix, Arizona. The
adopted values included expectations related to the transparency, validity, and inclusion
of Arizona stakeholders in the processes associated with AzMERIT. Specifically, the
values included test results that
measure a students mastery of the Arizona standards and progress towards
college and career readiness,
provide valid, reliable and timely data to educators and policy makers to advance
the academic success of Arizona students and inform the States accountability
measures,
communicate results to students, parents, and educators, in a clear and timely
manner to guide instruction,
provide an accurate perspective of the quality of learning occurring within
classrooms and schools, and
allow meaningful national or multistate comparisons of school and student
achievement.
In February 2015, Arizonas Technical Advisory Council (TAC), comprised of nationally
recognized assessment experts, reviewed all of the planned standard setting processes
and planned studies necessary for the establishment of the AzMERIT vertical scale and
for the determination of mode (paper-based, computer-based) comparability. All
recommendations from the TAC were incorporated in the final standard setting
procedures and plans for supporting studies. On July 22, 2015, the TAC met again to
review the results of the completed standard setting process and related studies. The
TAC endorsed the standard setting process and the findings and conclusions of the
studies.
Eighty-one Arizona educators participated in the AzMERIT Standard Setting Workshop.
These educators were divided into eight panels representing four grade bands (3-4, 5-6,
7-8, and 9-11) for each subject (ELA and math). Using the Bookmark method, these
panelists recommended performance standards (cut scores) for AzMERIT that measure
student progress toward college and career readiness and allow for meaningful national
Contact Information:
Irene Hunting, Deputy Associate Superintendent
Leila Williams, Associate Superintendent

EXECUTIVE SUMMARY

Arizona State Board of Education Special Meeting


August 14, 2015
Item 1
Page 2 of 3

and multistate comparisons of school and student achievement. The recommended


AzMERIT performance standards are generally comparable to performance standards
for NAEP and Smarter Balanced. For AzMERIT ELA 11 and AzMERIT Algebra II, the
recommended performance standards indicate a college readiness at least as rigorous
as ACTs college readiness.
A complete description of the standard setting process is included in the attached
report, Recommending AzMERIT Performance Standards ELA Grades 3-11, Math
Grades 3-8, Algebra I, Geometry, and Algebra II.
ADE recommends that the Board adopt these scale score ranges which reflect the
performance standards recommended by the AzMERIT Standard Setting panelists.
AzMERIT ELA
Scale Score
Ranges
Grade 3
Grade 4
Grade 5
Grade 6
Grade 7
Grade 8
Grade 9
Grade 10
Grade 11

Minimally
Proficient
2395-2496
2400-2509
2419-2519
2431-2531
2438-2542
2448-2550
2454-2554
2458-2566
2465-2568

Partially
Proficient
2497-2508
2510-2522
2520-2542
2532-2552
2543-2560
2551-2571
2555-2576
2567-2580
2569-2584

Proficient
2509-2540
2523-2558
2543-2577
2553-2596
2561-2599
2572-2603
2577-2605
2581-2605
2585-2607

Highly
Proficient
2541-2605
2559-2610
2578-2629
2597-2641
2600-2648
2604-2658
2606-2664
2606-2668
2608-2675

AzMERIT Math
Scale Score
Ranges
Grade 3
Grade 4
Grade 5
Grade 6
Grade 7
Grade 8
Algebra I
Geometry
Algebra II

Minimally
Proficient
3395-3494
3435-3529
3478-3562
3512-3601
3529-3628
3566-3649
3577-3660
3609-3672
3629-3689

Partially
Proficient
3495-3530
3530-3561
3563-3594
3602-3628
3629-3651
3650-3672
3661-3680
3673-3696
3690-3710

Proficient
3531-3572
3562-3605
3595-3634
3629-3662
3652-3679
3673-3704
3681-3719
3697-3742
3711-3750

Highly
Proficient
3573-3605
3606-3645
3635-3688
3663-3722
3680-3739
3705-3776
3720-3787
3743-3819
3751-3839

The adoption of these scale score ranges will result in the following estimated
performance on the Spring 2015 AzMERIT assessments.

EXECUTIVE SUMMARY

Arizona State Board of Education Special Meeting


August 14, 2015
Item 1
Page 3 of 3

AzMERIT ELA
est. % of students for
Spring 2015
Grade 3
Grade 4
Grade 5
Grade 6
Grade 7
Grade 8
Grade 9
Grade 10
Grade 11

Minimally
Proficient
44%
43%
37%
39%
41%
40%
47%
49%
54%

Partially
Proficient
15%
19%
33%
27%
26%
27%
26%
21%
20%

AzMERIT Math
est. % of students for
Spring 2015
Grade 3
Grade 4
Grade 5
Grade 6
Grade 7
Grade 8
Algebra I
Geometry
Algebra II

Minimally
Proficient
27%
29%
29%
38%
48%
43%
45%
47%
47%

Partially
Proficient
31%
29%
31%
30%
22%
24%
23%
24%
24%

Proficient
31%
33%
27%
30%
29%
26%
21%
22%
17%

Highly
Proficient
10%
5%
3%
4%
4%
6%
6%
8%
8%

Proficient
27%
32%
27%
21%
18%
20%
23%
24%
23%

Highly
Proficient
15%
10%
13%
11%
13%
13%
9%
6%
6%

Recommendation to the Board


It is recommended that the Board adopt the performance standards for AzMERIT as
proposed by the ADE in these materials.

Recommending AzMERIT
Performance Standards
English Language Arts Grades 3-11
Math Grades 3-8, Algebra I, Geometry,
and Algebra II
Technical Report
June 27, 2015

Arizona Assessments
Summer 2015 Standard Setting

Recommending Performance Standards for Arizonas


Measurement of Educational Readiness to Inform
Teaching (AzMERIT)
ELA Grades 3-11
Math Grades 3-8, Algebra I, Geometry, and Algebra II

Technical Report

June 27, 2015

Prepared by American Institutes for Research (AIR) in collaboration with the


Arizona Department of Education

AzMERIT

StandardSettingTechnicalReport

TABLE OF CONTENTS
Introduction....................................................................................................................................1
PerformanceStandardsandValidityofTestScoreInterpretations...........................................1
OverviewofStandardSettingApproach........................................................................................5
WorkshopDesign........................................................................................................................5
WorkshopLocation.....................................................................................................................7
WorkshopStaffing......................................................................................................................7
WorkshopPanelists....................................................................................................................7
HigherEducationPanel...............................................................................................................8
WorkshopTraining....................................................................................................................10
StandardSettingMaterialsandProcedures.................................................................................12
PerformanceLevelDescriptors.................................................................................................12
OrderedItemBooklet...............................................................................................................13
AzMERITBookmarkPlacement.................................................................................................15
BenchmarkInformation............................................................................................................17
PanelistFeedbackandImpactData..........................................................................................18
VerticalArticulation..................................................................................................................19
WorkshopEvaluation................................................................................................................20
RecommendedPerformanceStandardsandImpactData...........................................................21
EvaluationoftheStandardSettingWorkshop.............................................................................28
PanelistEvaluationofStandardSettingWorkshop..................................................................28
IndependentObserverReviewofStandardSettingWorkshop...............................................29
References....................................................................................................................................30
AppendixAWorkshopAgendas................................................................................................A1
AppendixBCompositionofPanels...........................................................................................B1
AppendixCTrainingPresentations...........................................................................................C2
AppendixDPerformanceLevelDescriptors.............................................................................D1
AppendixETestBlueprints.......................................................................................................E1
ii

AzMERIT

StandardSettingTechnicalReport

AppendixFSummaryofOrderedItemBooklets......................................................................F1
AppendixGOIBItemDataPlots...............................................................................................G1
AppendixHBookmarkPlacementReadinessForms.................................................................H1
AppendixIInvestigationofEquatingStudentScoresAcrossAzMERITTest
AdministrationModes..................................................................................................................I1
AppendixJAzMERITVerticalLinkingStudy...............................................................................J1
AppendixKPanelistEvaluationForms......................................................................................K1
AppendixLRecommendPerformanceStandardsbyRound.....................................................L1
AppendixMConvergenceofBookmarksacrossRounds........................................................M1
AppendixNEstimatedPercentageofStudentsatEachPerformanceLevelforPanelist
RecommendedPerformanceStandards,OverallandbyGenderandEthnicity.........................N1
AppendixOSummaryofPanelistEvaluations..........................................................................O1
AppendixPIndependentObserverReporttoStateBoardofEducation.................................P1

iii

AzMERIT

StandardSettingTechnicalReport

Introduction
In2010,ArizonaadoptednewacademiccontentstandardsinEnglishlanguagearts(ELA)and
math.TheArizonaCollegeandCareerReadyStandardsaredesignedtoensurethatstudents
acrossgradesarereceivingtheinstructiontheyneedtobeontrackforcollegeandcareerby
thetimetheygraduate.Inspring2015,theArizonaDepartmentofEducation(ADE)
administeredforthefirsttimeArizonasMeasurementofEducationalReadinesstoInform
Teaching(AzMERIT)toassessproficiencyonthenewArizonaCollegeandCareerReady
Standards.TheAzMERITmeasuresEnglishlanguageartsingrades311,andmathingrades38
andfollowingcompletionofhighschoolcourseworkinAlgebraI,Geometry,andAlgebraII.
TheAzMERITisaseriesoffixedformassessmentsthatareintendedtobeadministeredonline,
althoughtheassessmentisofferedasadualmode,onlineandpaper,assessmentto
accommodateschoolsthatarenotreadytotransitiontotheonlinetestingenvironment.A
commonoperationalbaseformwasadministeredtoallstudentswithinagiventestgradeand
subject.Eachassessmentiscomprisedoftwotothreediscretetestsessions.
ThefirstoperationaladministrationoftheAzMERITassessmenttookplaceinspring2015.
OnlineadministrationoftheAzMERIToccurredfromMarch30throughMay8,2015.Thepaper
versionoftheAzMERITwasadministeredbetweenApril13andApril24,2015.Followingthe
closeofthetestadministrationwindows,theAmericanInstitutesforResearch(AIR),under
contracttoADE,convenedeightpanelsofArizonaeducatorstorecommendperformance
standardsontheassessments.Thisdocumentdescribestheproceduresusedtoconductthe
standardsettingworkshopsaswellastherecommendedperformancestandardsandresulting
impacts.

Performance Standards and Validity of Test Score Interpretations


Validityreferstothedegreetowhichtestscoreinterpretationsaresupportedbyevidence,and
speaksdirectlytothelegitimateusesoftestscores.Establishingthevalidityoftestscore
interpretationsisthusthemostfundamentalcomponentoftestdesignandevaluation.The
StandardsforEducationalandPsychologicalTesting(AmericanEducationalResearch
Association,AmericanPsychologicalAssociation,andNationalCouncilonMeasurementin
Education,2014)provideaframeworkforevaluatingwhetherclaimsbasedontestscore
interpretationsaresupportedbyevidence.Withinthisframework,theStandardsdescribethe
rangeofevidencethatmaybebroughttobeartosupportthevalidityoftestscore
interpretations.1
Thekindsofevidencerequiredtosupportthevalidityoftestscoreinterpretationsdepend
centrallyontheclaimsmadeforhowtestscoresmaybeinterpreted.Moreover,thestandards
makeexplicitthatvalidityisnotanattributeoftests,butrathertestscoreinterpretations.
Sometestscoreinterpretationsmaybesupportedbyvalidityevidence,whileothersarenot.

ResponsivetoStandardsforEducationandPsychologicalTesting:Standard9.13

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Thus,thetestitselfisnotconsideredvalid,butratherthevalidityoftheintendedinterpretation
anduseoftestscoresisevaluated.
Centraltoevaluatingthevalidityoftestscoreinterpretationsisdeterminingwhetherthetest
measurestheintendedconstruct.Suchanevaluationinturnrequiresacleardefinitionofthe
measurementconstruct.ForArizonasnewAzMERITassessments,thedefinitionofthe
measurementconstructisprovidedbytheArizonaCollegeandCareerReadyStandards.
TheArizonaCollegeandCareerReadyStandards(ACCRS)specifywhatstudentsshouldknow
andbeabletodobytheendofeachgradelevelinorderforstudentstograduatereadyfor
postsecondaryeducationorentryintotheworkforce.Becausedirectlymeasuringstudent
achievementagainstofeachbenchmarkintheACCRSwouldresultinanimpracticallylongtest,
eachtestadministrationisdesignedtomeasurearepresentativesampleofthecontentdomain
definedbytheStandards.Toensurethateachstudentisassessedontheintendedbreadthand
depthoftheStandards,testformconstructionisguidedbyasetoftestspecifications,or
blueprints,whichindicatethenumberofitemsthatshouldbesampledfromeachcontent
strand,standard,andbenchmark.Thus,thetestblueprintsrepresentapolicystatementabout
therelativeimportanceofcontentstrandsandstandardsinadditiontomeetingimportant
measurementgoals(e.g.,sufficientitemstoreportstrandperformancelevelsreliably).Because
thetestblueprintdetermineshowstudentachievementoftheArizonaCollegeandCareer
ReadyStandardsisevaluated,alignmentoftestblueprintswiththecontentstandardsiscritical.
ADEhaspublishedtheAzMERITtestblueprintsthatspecifythedistributionofitemsacross
reportingstrandsanddepthofknowledgelevels.
AlignmentoftestcontenttotheArizonaCollegeandCareerReadyStandards(ACCRS)2ensures
thattestscorescanserveasvalidindicatorsofthedegreetowhichstudentshaveachievedthe
learningexpectationsdetailedintheACCRS.However,theinterpretationoftheAzMERITtest
scoresrestsfundamentallyonhowtestscoresrelatetoperformancestandardswhichdefine
theextenttowhichstudentshaveachievedtheexpectationsdefinedintheACCRS.AzMERIT
testscoresarereportedwithrespecttofourproficiencylevels,demarcatingthedegreeto
whichArizonastudentshaveachievedthelearningexpectationsdefinedbytheArizonaCollege
andCareerReadyStandards.ThecutscoreestablishingtheProficientlevelofperformanceis
themostcritical,sinceitindicatesthatstudentsaremeetinggradelevelexpectationsfor
achievementoftheArizonaCollegeandCareerReadyStandards,thattheyarepreparedto
benefitfrominstructionatthenextgradelevel,andthattheyareontracktopursuepost
secondaryeducationorentertheworkforce.Proceduresusedtoadoptperformancestandard
fortheAzMERITassessmentsarethereforecentraltothevalidityoftestscoreinterpretations.
FollowingthefirstoperationaladministrationoftheAzMERITassessmentsinspring2015,a
standardsettingworkshopwasconductedtorecommendtotheArizonaStateBoardof
EducationasetofperformancestandardsforreportingstudentachievementoftheArizona
CollegeandCareerReadyStandards.Thisdocumentdescribesthestandardizedandrigorous
proceduresthatArizonaeducators,servingasstandardsettingpanelists,followedto

ResponsivetoStandardsforEducationandPsychologicalTesting:Standard12.8and12.10

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

recommendperformancestandards.TheworkshopsemployedtheBookmarkprocedure,a
widelyusedmethodinwhichstandardsettingpanelistsusetheirexpertknowledgeofthe
ArizonaCollegeandCareerReadyStandardsandstudentachievementtomaptheperformance
leveldescriptorsadoptedbytheArizonaStateBoardofEducationontoanordereditembook
basedonthefirstoperationaltestformadministeredtostudentsinspring2015.
Panelistswerealsoprovidedwithcontextualinformationtohelpinformtheirprimarilycontent
drivencutscorerecommendations.Panelistsrecommendingperformancestandardsforthe
highschoolassessmentswereprovidedwithinformationabouttheapproximatelocationofthe
relevantACTcollegereadyperformancestandardforthegrade11ELAandAlgebraII
assessments,andProgrammeforInternationalStudentAssessment(PISA)performance
standardsforthegrade10ELAandGeometryassessments.Panelistsrecommending
performancestandardforthegrade38summativeassessmentswereprovidedwiththe
approximatelocationofrelevantNAEPperformancestandardsatgrades4and8,aswellas
interpolatedvaluesforgrade6.Panelistswereprovidedwiththeapproximatelocationsofthe
SmarterBalancedperformancestandardsforthegrade38and11assessmentsinELAand
mathtoprovideadditionalcontextaboutthelocationofperformancestandardsforstatewide
assessments.Additionally,panelistswereprovidedthecorrespondinglocationsfortheprevious
AIMSperformancestandards.Panelistswereaskedtoconsiderthelocationofthese
benchmarklocationswhenmakingtheircontentbasedcutscorerecommendations.When
panelistsareabletousebenchmarkinformationtolocateperformancestandardsthat
convergeacrossassessmentsystems,validityoftestscoreinterpretationsisbolstered.
Inaddition,panelistswereprovidedwithfeedbackabouttheverticalarticulationoftheir
recommendedperformancestandardssothattheycouldviewhowthelocationsoftheir
recommendedcutscoresforeachgradelevelassessmentsatinrelationtothecutscore
recommendationsattheothergradelevels.Thisapproachallowedpaneliststoviewtheircut
scorerecommendationsasacoherentsystemofperformancestandards,andfurtherreinforces
theinterpretationoftestscoresasindicatingnotonlyachievementofcurrentgradelevel
standards,butalsopreparednesstobenefitfrominstructioninthesubsequentgradelevel.
Basedontherecommendedcutscores,Table1showstheestimatedpercentageofstudents
meetingtheAzMERITproficientstandardforeachassessmentinspring2015.Table1also
showstheapproximatepercentageofArizonastudentsthatwouldbeexpectedtomeetthe
ACTcollegereadystandard,andthepercentageofArizonastudentsmeetingtheNAEP
proficientstandardsatgrades4and8.Table1alsopresentstheexpectedproficientrateforthe
SmarterBalancedAssessments,systemwide,basedonthespring2014fieldtest
administration.AsTable1indicates,theperformancestandardsrecommendedAzMERIT
assessmentsarequiteconsistentwithrelevantACTcollegeready,andtheNAEPandSmarter
Balancedproficient,benchmarks.Moreover,becausetheperformancestandardswere
verticallyarticulated,theproficiencyratesacrossgradelevelsaregenerallyconsistent.

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Table1.EstimatedPercentageofStudentsMeetingAzMERITandBenchmarkProficient
Standards.

Assessment

AzMERIT
Proficient

Grade3
Grade4
Grade5
Grade6
Grade7
Grade8
Grade9
Grade10
Grade11

41%
38%
30%
34%
33%
32%
27%
30%
25%

Grade3
Grade4
Grade5
Grade6
Grade7
Grade8
AlgebraI
Geometry
AlgebraII

42%
42%
40%
32%
31%
33%
32%
30%
29%

PercentofStudentsMeetingStandard
ArizonaACT
ArizonaNAEP
ProjectedSBAC
CollegeReady
Proficient
ELA
38%

28%
41%

44%

41%

38%

28%
41%

34%
41%
Mathematics
39%
42%
38%
33%

33%
33%
32%
32%

36%
33%

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Overview of Standard Setting Approach


TheBookmarkmethod(Mitzel,Lewis,Patz,&Green,2001)wasusedtorecommend
performancestandardsfortheAzMERIT.ADEpreviouslyusedtheBookmarkmethodto
recommendperformancestandardsfortheAIMSassessment.TheBookmarkmethodwas
implementedintworounds,providingpanelistswithbenchmarkinformationpriortoRound1
andpanelistfeedbackandimpactdatapriortoRound2.Tofacilitateverticalarticulationof
performancestandardsacrossgrades,workshoppanelistsbeganbyrecommending
performancestandardsforgrades4,6,8,10,and11(GeometryandAlgebraIIformath),
followingstandardBookmarkprocedures.Fortheremainingintermediategrades,followinga
verticalmoderationsessiontoarticulateperformancestandardsacrossgrades,panelistswere
providedwithinterpolatedperformancestandardsbasedontherecommendedstandardsfrom
theanchorgrades.Fortheintermediategrades,thejudgmenttaskusedbypanelistswas
modifiedsomewhat.Foreachperformancestandard,panelistswereaskedtoexaminetheitem
ontheinterpolatedpageandjudgewhetherstudentswhojustbarelyaredescribedbythe
performanceleveldescriptorcouldrespondsuccessfullytotheitem,andifso,toendorsethe
interpolatedOIBpageastheperformancestandard.Iftheycouldnotendorsetheinterpolated
OIBpageastheperformancestandard,panelistswereaskediftheycouldlocateanitemvery
nearthelocationoftheinterpolatedOIBpagethatstudentsjustbarelymeetingthestandard
couldrespondtosuccessfully.
Panelistsweretaskedwithrecommendingthreeperformancestandards(PartiallyProficient,
Proficient,andHighlyProficient)thatresultedinfourperformancelevels(MinimallyProficient,
PartiallyProficient,Proficient,andHighlyProficient).

Workshop Design
TorecommendperformancestandardsforeachoftheAzMERITassessments,ADEconvened
eightpanelsrepresentingfourgradebands(34,56,78,and911)foreachsubject.Thepanels
consistedofeducatorsfromtherespectivegradebandsandcontentareas.Thepanelists
recommendedperformancestandardsbasedprimarilyoncontentconsiderationswith
additionalcontextprovidedbyrelevantbenchmarkinformationfromstatewide(SBAC),
national(NAEP),international(PISA),andcollegeentrance(ACT)exams,aswellasestimated
studentperformanceontherecommendedstandardspriortoRound2.PanelistsusedOrdered
ItemBooklets(OIBs)andPerformanceLevelDescriptors(PLDs)toplaceperformancestandards
forallthreeperformancelevels,PartiallyProficient,Proficient,andHighlyProficient,intwo
rounds.Firstpanelistsrecommendedperformancestandardsfortheanchorgrades,4,6,8,
11/AlgebraII(grade10/Geometrywerealsoconsideredanchorgrades).Afterrecommending
performancestandardsfortheanchorgrades,amoderationsessionwasconductedwiththe
tableleadersfromeachofthepanelstoreviewtheverticalarticulationoftheperformance
standards,andtoimplementanyadjustmentstotheanchorgraderecommendationsto
facilitateverticalarticulation.Followingtheverticalarticulationsession,panelistscontinuedon
torecommendperformancestandardsfortheremaininggradelevelassessments,usingthe

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

interpolatedstandardstoprovidefurthercontextualinformationaboutthelikelylocationof
performancestandards.
TheAzMERITStandardSettingworkshopswereconductedoverfourdays,withthehighschool
panels,whichhadtorecommendperformancestandardsforthreeassessments,beginningon
Monday,andtheremaininggradelevelpanelsconveningonTuesday.Abroadoverviewofthe
workshopcalendarispresentedinTable2.Detailedagendasforthestandardsetting
workshopsareincludedasAppendixA.
Table2.CalendarDatesfor2015GradeLevelandHighSchoolELAandMathStandardsetting
Workshops
Workshop

Monday,July13

Tuesday,July14

Wednesday,July15

Thursday,July16

GradeLevel

N/A

StandardSetting
Day1

StandardSetting
Day2

StandardSetting
Day3

HighSchool

StandardSetting StandardSetting
Day1
Day2

StandardSetting
Day3

StandardSetting
Day4

Theworkshopsbeganwithabrieftableleaderorientationtoreviewwithtableleaderstheir
roleandresponsibilities.Theworkshopproperbeganwithalargegrouptrainingtoprovide
panelistswithanoverviewoftheworkshopactivitiesandinitialtraininginthebookmarking
procedures.Followingthelargegroupsession,theworkshoppanelsconvenedintheirmeeting
rooms,andbegantheirworkbyparticipatinginthesameAzMERITonlineassessmentthatwas
administeredtotheirstudentsinthespring.Paneliststhenspentseveralhoursworkingthrough
theperformanceleveldescriptors(PLDs)developedbyADE,anddevelopingmodified
descriptorstocharacterizethespecialsubsetofstudentswhojustbarelyqualifyforentryinto
eachoftheperformancelevels.Afterdevelopingdescriptorsforthejustbarelystudents,
panelistsspenttheremainderofdayonereviewingtheirordereditembooks(OIBs).
Panelistsdidnotbeginrecommendingperformancestandardsuntildaytwo,whichbeganwith
trainingonthebookmarkplacementtask.PaneliststhenworkedthroughtheirOIBsandplaced
theirbookmarksforRound1.AfterRound1,panelistswereprovidedfeedbackaboutthe
bookmarkplacementsoftheotherpanelistsanddiscussedthosebookmarkplacementsattheir
tablesandacrosstheroommoregenerally.Panelistswerethenalsoprovidedwithimpactdata
showingtheestimatedpercentageofstudentswhowouldmeeteachoftheperformance
standardsandengagedinpaneldiscussionsaboutanyimplicationsofthoseproficiencyrates.
Uponcompletionofpaneldiscussions,panelistsmadeasecondroundofbookmark
placements,andthenbegantheprocessoveragainforthesubsequentassessment.

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Workshop Location
TheworkshopswereheldattheHyattRegency,locatedat122North2ndStreetinPhoenix,
Arizona.ThelocationprovidedmeetingspacestoholdtheAzMERITworkshoppanels,aswellas
apsychometricworkroomforcompletionofanalysisactivitiesandstoragespaceforsecure
materialsthroughouttheworkshop.

Workshop Staffing
Aseniorworkshopcoordinatorwastaskedwithleadingthecrossworkshopintroductory
trainingandverticalmoderationmeetings,andwasresponsibleforworkingwitheach
facilitatorandmonitoringtheflowofactivitiesacrossworkshops.AIRtestdevelopmentstaff
servedasworkshopfacilitators,leadingeachpanelthroughtrainingactivitiesandexecutionof
thestandardsettingprocess.Additionally,anAIRresearchassistantwasassignedtoeachpanel
tosupporttheworkshopfacilitator.Becausetestdevelopmentstaffservedasworkshop
facilitators,theywerehighlyqualifiedtofacilitatethedevelopmentofjustbarelyperformance
leveldescriptors,andtoserveasasubjectmatterresourceforpanelistsastheynavigatedthe
OIB.AteamofthreeAIRpsychometriciansmanagedpsychometricactivitiesinsupportofthe
workshop,includingensuringaccuratedatacaptureofbookmarkplacements,presentationof
verticalarticulationresultsformoderationmeetings,andproductionoffinalresultsforthe
standardsettingtechnicalreport.Inaddition,AIRprojectstafffacilitatedorganizationof
meetingspaceandmealsandprovidedsupporttopanelistsasnecessary.
ADEstaffmonitoredallstandardsettingactivities,andalsoaddressedanypolicyortest
developmentquestionsforpanelists.WhileADEstaffansweredspecific,directquestions,they
werenotactivelyinvolvedinthefacilitationofthemeeting.

Workshop Panelists
ADEworkedtoobtainbroadlyrepresentativepanelsforthestandardsettingworkshopsthat
reflectedtheteacherpopulationinthestateofArizonaintermsofgender,race,ethnicity,and
geographicalrepresentation.Diversegroupsofpanelistsbringawiderangeofperspectivesand
experiencetothestandardsettingeffort,ensuringthattherecommendationsthatare
forwardedtotheStateBoardofEducationarethoughtfulandrepresentativeofbroad
educationalconstituencies,andrepresenttherangeofexpertiseandexperiencesfoundinthe
educatorpopulationacrossthestate.
WithineachoftheELAandmathpanels,atotalof12panelistspergradebandsubpanelwere
recruitedtorecommendstandards.ADEtargetedthenumberofmaleandfemalepaneliststo
mirrorthepopulationofeducators.Inthesameway,ADEworkedtoincludeproportional
representationofAmericanIndian/NativeAmerican,Asian/PacificIslanders,Black(Non
Hispanic),HispanicandWhite(NonHispanic)panelists,andaproportionalnumberofpanelists
fromrural,urban,andsuburbandistricts.Forcoursebasedassessmentsinmaththatrequire
specificcontentexpertise,ADEsoughttoincludeteacherswhohaveexpertiseinthecontent
standardsandcourseworkforallthreeareastheyrecommendedperformancestandardsfor:

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

AlgebraI,Geometry,andAlgebraII.Inaddition,ADEworkedtoincludespecialeducationand
EnglishLanguageLearners(ELL)teachers.
Withineachsubpanel,tableswerebalancedtoincludepanelistswithvaryingcontentexpertise
anddemographicrepresentationineachgroup.
ADEdesignatedthreetableleadersforeachpanel.Tableleadersattendedanadditional
orientationmeetingandweretaskedwithassistingstandardsettingstaffby

facilitatingdiscussionswithintheirtable;

distributingandcollectingreadinessandrecordingsheetsandsecurematerials;

alertingworkshopstaffofconfusionorconcernswithintheirtables;and

representingtheirtableandpanelduringverticalarticulationmeetings.

Letterscontaininglogisticalinformationandremindersaboutthepurpose3oftheworkshop
wereemailedtoconfirmedpaneliststwoweekspriortothestandardsettingworkshop.Inthe
weekprior,testingcontractorstaffcontactedallpanelistsviaphonetoconfirmreceiptof
information.Throughouttheprocess,ADEcontinuedtorecruitreplacementsforpanelistswho
withdrewtheirparticipation.
AppendixB4presentsthecompositionofthestandardsettingpanels.Foreachpanel,thetable
includesarecordforeachpanelistandindicatesthegeographicregionheorsherepresents
andhisorhergender,ethnicity,andmainexpertise.Whileitiscriticallyimportanttoincludea
rangeofstakeholdersinthestandardsettingprocess,experiencehasshownthatitisessential
forpaneliststohavedirectknowledgeofacademicstandardsandstudentgradelevel
performancetoparticipatemeaningfullyintheBookmarkingprocedure.Forthisreason,panel
participationwasrestrictedtoclassroomteachersandcurriculumspecialistswithexpertisein
ELAandmathcurriculumandinstruction.

Higher Education Panel


Priortothestandardsettingworkshops,ADEengagedahighereducationpanelintwoactivities
intendedtosupporttheassertionthatstudentswhoachievetheProficientlevelonAzMERIT
inELA11andAlgebraIIareontracktobecollegereadyupongraduationfromhighschool.This
highereducationpanelincluded10participantsrepresentingallthreeofArizonaspublic
universitiesandthreeofthestatescommunitycollegesystems.Eachwasfamiliarwiththe
requirementsforstudentstobesuccessfulineithercreditbearingentrylevelcollege
mathematicscoursesorcreditbearingentrylevelcollegeEnglishcourses.

3
4

ResponsivetoStandardsforEducationandPsychologicalTesting:Standard5.0,5.21,5.22,and7.0
ResponsivetoStandardsforEducationandPsychologicalTesting:Standard7.5

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

ThefirstactivityforthispanelwasareviewofthedetailedPLDsforELA11andAlgebraIIheldat
ADEsofficesonMay13,2015.Tosetthestageforthisactivity,anoverviewofAzMERIT,the
detailedPLDs,andhowthedetailedPLDswouldbeusedintheupcomingAzMERITStandard
Settingwasprovidedtoall10participants.ThepanelthenbrokeoutintoseparateELAand
mathgroupstofirstdeterminethecollegecoursethatbestfitsthedescriptorcreditbearing
entrylevelcollegecoursefortheircontentarea.ForELA,thatentrylevelcoursewas
determinedtobeFreshmanComposition,whiletheentrylevelcourseformathematicswas
determinedtobeCollegeMath.Whilestillintheirsubjectareagroups,thepaneliststhen
reviewedanddiscussedtheskillsandabilitiesdescribedintheELA11orAlgebraIIdetailedPLDs
forstudentsintheProficientlevelandwhetherthatlevelofskillorabilitywassufficienttobe
preparedforentrylevelcoursework.TheconsensusdecisionofboththeELAgroupandthe
mathgroupwasthatstudentswhohadtheskillsandabilitiesdescribedintheProficientlevel
wouldbeadequatelypreparedforthetargetentrylevelcourseupongraduation.
Additionally,boththeELAgroupandthemathgroupfeltitwasimportanttoindicatethattheir
endorsementofcollegereadinessincludedtheexpectationthatstudentswouldtakeonemore
yearofhighschoolEnglishaftertheELA11testandonemoreyearofhighschoolmathafter
theAlgebraIItest.Thisisnotanunreasonableexpectationsincemoststudentswouldbetaking
theELA11testattheendoftheirthirdoffourrequiredhighschoolEnglishcoursesandwould
betakingtheAlgebraIItestattheendoftheirthirdoffourrequiredhighschoolmathematics
courses.
ThesecondactivityforthispanelwasareviewoftheitemsincludedintheELA11andAlgebraII
testtodeterminewhichitemsdemonstratedtheskillsandabilitiesneededforstudentstobe
adequatelypreparedforentrylevelcoursework.Toaccommodatevacationschedules,panelists
participatedinthisonlineactivityindividuallyatthetimeandlocationoftheirchoosinginearly
July.Thisonlineactivityincludedatrainingmodulefollowedbyanitemreviewbasedona
variationontheItemDescriptorMatchingprocedure(Ferrara,Perie,&Johnson,2008).
LikethebookmarkingprocedureusedtorecommendperformancestandardsforAzMERIT,the
IDMatchingprocedurereliesonanordereditembook(OIB).Thisbookcontainstestitemsthat
appearinorderfromeasiesttomostdifficult,basedonstudentperformanceinthespring2015
testadministration.ThevariationoftheIDMatchingprocedureusedforthisactivityaskedthe
paneliststodeterminewhethertheknowledgeandskillsnecessarytoanswereachitem
correctlywereprerequisiteskillsforsuccessinentrylevelcoursework,thatis,CollegeMathor
FreshmanComposition.
HighereducationpanelistsbeganbyreviewingtheOIBfollowingthesameproceduresusedby
thestandardsettingworkshoppanelists.BeginningwiththefirstpageintheOIB,participants
answeredtwoquestionsastheyreviewedeachitem:
Whatdoesastudentneedtoknowandbeabletodotosuccessfullyrespondtothisitem?
Whyisthisitemmoredifficultthantheprecedingitems?

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Thisactivitywasdesignedtofocusparticipantsontheknowledgeandskillsmeasuredbyeach
item,aswellastocommunicatetoparticipantsthefullrangeofknowledgeandskillsmeasured
intheassessments.UponcompletionoftheOIBreview,participantswerepreparedtoperform
theIDMatchingtask.
ToperformtheIDMatchingtask,participantswereaskedtoconsiderwhatknowledgeand
skillsareprerequisiteforsuccessinentrylevelcourseworkattheirrespectiveinstitutions.
ParticipantsrepresentingCollegeMathperformedtheIDMatchingtaskfortheAlgebraIIOIB,
whileparticipantsrepresentingFreshmanCompositionperformedtheIDMatchingtaskusing
theGrade11ELAOIB.
ToperformtheIDMatchingtask,participantsjudgedwhethertheknowledgeandskills
necessarytoanswertheitemsuccessfullywereprerequisitetosuccessintherelevantentry
levelcourse.ForeachitemintheOIB,participantsansweredyesornothattocorrectly
answertheitemrequiredknowledgeandskillsthatareprerequisiteforsuccessintheentry
levelcourse.
Becauseitemswereorderedbydifficulty,theexpectationwasthatparticipantswouldgenerally
identifytwodistinctregionsofachievement,aloweronewhereitemsclearlywereprerequisite
forsuccessinentrylevelcollegecoursework,andahigheronethatreflectedachievement
beyondwhatwouldbeconsideredprerequisiteknowledgeforsuccessincollege.Itwasalso
expectedthattherewouldbearegionofuncertaintybetweenthetwo,withthenotionthata
likelycollegereadyperformancestandardwouldliewithintheregionofuncertainty.Whenthe
responsesoftheparticipantsweretabulatedtogetherhowever,therewasnodetectableregion
intheOIBwheretheknowledgeandskillsassessedbytheitemswerereliablynotprerequisite
forsuccessinentrylevelcoursework.Inotherwords,itemsconsideredprerequisiteforcollege
successwerereliablyidentifiedacrosstheentirerangeoftheOIB.Therefore,itwasnot
possibletoprovidethestandardsettingworkshoppanelistswithaconstrainedregioninwhich
acollegereadystandardmightbeidentifiedbaseduponthereviewbythispanelofArizona
highereducationrepresentatives.

Workshop Training
Thoroughtrainingisanessentialelementofastandardsettingworkshop.Trainingatthe
meetingshelpedpanelistsbecomefamiliarwiththeassessmentsystemandthestandard
settingprocess.Italsoinvolvedareviewanddiscussionoftheassessments,thestudent
populationsthatparticipatedineach,andtheperformanceleveldescriptors(PLDs).Inaddition,
trainingincludedindepthdiscussionofconceptskeytobookmarkplacement,suchasthe
notionofwhatwouldconstituteastudentjustbarelyinaperformancelevel.Allpanelists
wereadministeredanoperationaltestinordertounderstandthetestcontent,thetesting
interface,andvariousitemtypesthroughwhichstudentknowledgeandskillswereassessed.A

10

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

sampleofthepresentationslidesusedtoconducttheintroductorytraining,andthoseusedto
facilitateeachworkshopareprovidedinAppendixC5.
Tobegintheworkshop,thepanelistswereconvenedforabriefintroductorytrainingthat
focusedonthepurposeofthestandardsettingworkshopandareviewofthemainworkshop
activities.Followingthislargegroupintroduction,panelistsjoinedtheirassignedworkshop
panelswheretheworkshopleaderforeachassessmentguidedpaneliststhroughthestandard
settingactivitiesandprovidedindepthtrainingthroughoutthecourseoftheworkshop.
Tableleadershadtheadditionalresponsibilitiesofensuringthattableactivitiesremained
focusedonthetaskathand,helpingtoverifythatpanelistsunderstoodtheirtasks,andalerting
workshopleaderstoanyissuesencounteredbypanelistsastheyengagedintheirworkshop
tasks.Tableleaderswerenotexpectedtoprovidetrainingtopanelistsbutratherserveas
liaisonsbetweenthepanelistsandworkshopleaderstoensurethatworkshopactivitieswere
implementedcorrectly,alertingworkshopleaderstoanyissuesthataroseduringthecourseof
conductingworkshopactivities,andrepresentingtheirtablesinthecrosspanelmoderation
deliberations.Atableleaderorientationmeetingwasconvenedpriortothestandardsetting
workshoptofamiliarizetableleaderswiththeirrolesandresponsibilities,includingsuggestions
onhowtoprovideleadershipatthetablesduringthestandardsettingprocessandhowto
managethesecurematerials.

ResponsivetoStandardsforEducationandPsychologicalTesting:Standard7.5

11

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Standard Setting Materials and Procedures


Performance Level Descriptors
Performanceleveldescriptors(PLDs)definethecontentareaknowledgeandskillsthatstudents
ateachperformancelevelareexpectedtodemonstrate.Thestandardsettingpanelistsbased
theirjudgmentsaboutthelocationoftheperformancestandardsonthePLDsaswellasthe
ArizonaCollegeandCareerReadinessStandards.
Priortoconveningthestandardsettingworkshops,AIR,inconsultationwithADE,draftedPLDs
foreachtestthatdescribedtherangeofachievementencompassedbyeachperformancelevel
onthetest.ThePLDsweredesignedtobeclear,concrete,andreflectArizonasexpectationsfor
proficiencybasedontheArizonaCollegeandCareerReadyStandards.Followingacycleof
revisionstothedraftPLDs,ADEinvitedArizonaeducatorstoreviewPLDsforeachofthe
assessments.Basedonfeedbackfrom166educators,PLDswerefurtherrevised,andthe
resultingdraftswereusedbystandardsettingpanelists.ADEconsideredanyneedfor
clarificationorrevisionthatarosethroughoutthestandardsettingprocesspriortopublishing
thefinalversionsofthePLDsfollowingthestandardsettingworkshop.Performancelevel
descriptorsthatwereusedbypanelistsinthestandardsettingworkshoparepresentedin
AppendixD.
Centraltotheirtraininginthebookmarkmethod,panelistsusedthePLDstodevelopa
representationofstudentswhoarejustbarelydescribedbyeachoftheperformancelevel
descriptors.Duringthistrainingtask,panelistslearnedthatwhilePLDsarewrittento
characterizetypicalmembersofeachperformancelevel,theirbookmarkplacementswouldbe
directedtowardcharacterizingandidentifyingthemostminimallyqualifiedmembersofeach
performancelevel.Characterizingjustbarelymeetsstudentsisnotanintuitivejudgmentand
panelistsworkedtoidentifytheminimumcharacteristicsofstudentachievementforentryinto
eachperformancelevel.EachpanelproducedajustbarelyPLDtohelpguidetheirdiscussions
andbookmarkplacements.Todevelopacommonunderstandingamongpanelists,eachpanel
wasaskedto
1. reviewandparseperformanceleveldescriptors;
2. discusscharacteristicsofstudentsclassifiednearthresholdsofperformancestandards;
3. identifythecharacteristicsthatdistinguishstudentsjustabovetheperformance
standardfromthosejustbelow;
4. determinewhatevidencewasnecessarytoconcludethatastudentpossessedthe
minimumknowledgeandskillsneededtomeettheperformancestandard;and
5. summarizeknowledgeandskillsofstudentswhojustbarelymeeteachperformance
standard,orarejustbarelydescribedbyeachperformanceleveldescriptor

12

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Thesediscussionsyieldedcommondescriptionsofstudentsjustbarelycharacterizedbyeach
performanceleveldescriptorwithineachroom.

Ordered Item Booklet


Followingreviewofperformanceleveldescriptorsanddevelopmentofjustbarely
performanceleveldescriptors,panelistsreviewedordereditembooklets(OIBs).AnOIBisa
collectionoftestitemsorderedfromeasiesttomostdifficult.EachpageintheOIBcorresponds
toalevelofachievementontheAzMERIT,andpanelistsusetheOIBtorecommendthe
minimumlevelofachievementrequiredtoenterintoeachperformancelevel.
Composition of OIB
WithineachELAandmathtest,allonlinetesttakerswereadministeredatestformwitha
commonsetofitemsusedforoperationalscoring,aswellasasetofembeddeditemsusedfor
linkingorfieldtesting.Theoperationaltestformwasalsoadministeredonpaperwithitem
substitutionsforafewtechnologyenhanceditemsthatcouldnotberepresentedonpaper.The
operationalitemsadministeredonlineservedasthebasisfortheordereditembook.
Tominimizegapsintheordereditembooklets,theOIBswereaugmentedbyadditionalfield
testitemstomorefullyrepresenttherangeofacademicachievementencompassedwithin
thoseitembanks.EachmathOIBwasaugmentedwith1021fieldtestitems,andeachELAOIB
wasaugmentedwith712fieldtestitems.AllfieldtestitemsselectedforinclusionintheOIB
werereviewedforstatisticalintegrity;itemsflaggedforfurtherreviewduetolow
discriminationwereexcludedfromtheOIB.ItisimportanttonotethateachOIBwas
augmentedwithrespecttotheassessmentblueprint,whichspecifiesthecompositionofeach
testwithrespecttotherangeofcontentassessedbyeachoperationalform.Theaugmented
ELAandmathOIBswereproportionaltotheoperationaltestblueprints;theblueprintsare
presentedinAppendixE6.
Increasingthenumberofitemsacrosstherangeofitemdifficultiesprovidespanelistswith
greatercontexttoidentifyimportantshiftsintheknowledgeandskillrequirementsoftest
items.Oftenpanelistsbecomefocusedonthecognitivedemandsofasingleitemwhen
deliberatingonthelocationofaperformancestandard.Thispropensityisexacerbatedwhen
therearerelativelyfewitemsinagivenlocation,whichcancausejudgmentaboutoneitemto
takeontoomuchimportance.Evenwhentherearesufficientitemstoestablishreliable
performancestandardsforacentralproficientperformancestandard,therearetypicallyfewer
itemsavailableinlocationsassociatedwithperformancestandardscategorizingachievement
belowandaboveproficient;thus,movementofthebookmarkbyevenapageortwomay
resultinverylargeincreasesordecreasesinthepercentageofstudentsmeetingthestandard.
AugmentingtheOIBmoderatestheimpactassociatedwitheachOIBpage,especiallyfor
performancestandardsinthetailsoftheabilitydistribution.

ResponsivetoStandardsforEducationandPsychologicalTesting:Standard7.1and12.4

13

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Itemswereorderedaccordingtotheirresponseprobability(RP)levelbasedontheirItem
ResponseTheory(IRT)parameters.InIRT,theitemcharacteristiccurveforeachitemindicates
thelikelihoodofrespondingcorrectlyforeachpointalongthestudentachievementdimension.
Theresponseprobabilitycriterionreferstothelocationontheachievementscalethat
correspondstoagivenprobabilityofsuccess.Incontextofthestandardsettingworkshop,this
criterionisusedtodevelopacommonunderstandingofwhatconstitutesmasterywhen
evaluatingwhetherastudentcanrespondsuccessfullytoanitem.AnRPvalueof0.67wasused
asthemasterycriterionforallofstandardsettingworkshopsexceptthehighschoolendof
courseassessmentsinmath.Panelistswereaskedtoconsiderwhether,forexample,ajust
barelyproficientstudenthada0.67likelihoodofansweringtheitemcorrectly.Theywerealso
encouragedtoaskthisquestioninotherrelatedways,includingwhetherofjustbarely
proficientstudentswouldanswertheitemcorrectly,orwhetherajustbarelyproficientstudent
wouldrespondcorrectlytoitemtwoofthreetimes.Theendofcoursemathtestswerevery
difficultandthenumberofitemsonwhichstudentscoulddemonstratethatlevelofmastery
wasquitelow,resultinginaveryshortfunctionalOIB.Thus,anRPvalueof0.50wasadopted
fortheEOCmathtests,meaningthatajustbarelyproficientstudent,forexample,hada0.50
likelihoodofrespondingcorrectly,orthatofjustbarelystudentscouldrespondsuccessfully
toanitem,orthatajustbarelystudentcouldrespondsuccessfullytotheitematleastoneof
twotimes.
Dichotomouslyscored(e.g.,incorrectvs.correct)AzMERITitemswerecalibratedusingthe
Raschmodel.Multipoint,partialcredititemswerecalibratedusingMasterspartialcredit
modelwithorderingofscorepointpagesintheOIBbasedonstepleveldifficulties.
Theordereditembookletswerepresentedonline,allowingpaneliststoviewitemsinthesame
contextasstudenttesttakers.ThecompositionoftheELAandmathordereditembookletsby
assessmentandgradearesummarizedinTable3below.AtechnicalsummaryoftheOIBsare
presentedinAppendixF,includingforeachpageintheOIB,theitemscorepointassociated
withthepresenteditem,thedifficultyrepresentedbythepage,andthestandarderrorofthe
difficulty.Inaddition,theappendixindicatestheoverallpercentofstudentswhowouldscore
atorabovethestandardassociatedwitheachOIBpage,andthelocationofexternal
benchmarkswithinthebooklet.
Table3.CompositionofOrderedItemBooklets

Test
ELA3
ELA4
ELA5
ELA6
ELA7
ELA8
ELA9

42
42
42
42
42
42
44

NumberofItemsInOIB
Operational
FieldTest
Total
9
51
8
50
10
52
7
49
10
52
12
54
11
55

14

PagesinOIB
(TotalPoints)
67
67
68
68
68
71
69

American Institutes for Research

AzMERIT

Test
ELA10
ELA11
Math3
Math4
Math5
Math6
Math7
Math8
AlgebraI
Geometry
AlgebraII

44
44
45
45
45
47
47
47
47
47
47

StandardSettingTechnicalReport

NumberofItemsInOIB
Operational
FieldTest
Total
9
53
9
53
13
58
19
65
20
65
10
57
14
61
18
65
21
68
15
62
15
62

PagesinOIB
(TotalPoints)
68
67
58
66
66
57
61
70
70
66
66

Review of Ordered Item Booklets


ForeachitemintheOIB,panelistswereinstructedtoaskwhatastudentmustknowandbe
abletodotoanswereachquestionandwhatmakeseachitemintheOIBmoredifficultthan
theprecedingitem.ThisreviewoftheOIBallowedpaneliststogainnewperspectivesonthe
knowledgeandskillrequirementsofitemsandtoshareinformationregardingtheirthoughts
onthelocationofthethresholdregion.Duringthisdiscussion,theworkshopleadercirculated
throughtheroomtomonitorprogress,toassistpanelistswhomighthavehadtroublewiththe
task,andtoansweranyquestions.
OneachpageintheOIB,panelistsviewedthecontentoftheitem,theassociatedpassage,
contentalignment,andthescoringkeyorrubric.Inaddition,foreachpagethatpresenteda
writingitem,ELApanelistswereprovidedasamplestudentessayresponsethatscoredatthe
particularscorepoint.
PanelistswereinitiallyprovidedanitemmaptousewhilenavigatingtheOIB,whichincluded
passageandcontentalignmentinformationforeachpageintheOIB.Inaddition,panelistswere
presentedwithanitemplotthatdisplayedagraphicalrepresentationofthedifficultyofeach
pageintheOIB;thistoolshowedwherepageitemdifficultieswereclusteredtogetherversus
spreadout.OIBitemplotsarepresentedinAppendixG.

AzMERIT Bookmark Placement


PriortomakingtheirRound1bookmarkplacements,panelistswereprovidedtraininginthe
identificationofperformancestandardsintheordereditembooklets.Aspartofthistraining,
panelistslearnedtoidentifyalocationintheOIBthatbestdelineatestwoperformancelevels
(e.g.,betweenpagesonwhichstudentsmustdemonstratemasterytomeettheminimum
requirementsformembershipinthePartiallyProficientlevelfromthoseitemsonwhich
demonstrationofmasteryisnotnecessary).

15

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

UsingtheirjustbarelyPLDsasaguide,thepanelistsweretheninstructedtosetabookmarkon
theitemthatbestdelineatedeachoftheperformancelevels.Panelistswereremindedhowto
setbookmarks,andpriortomakinginitialplacements,facilitatorsledagroupactivitythat
reviewedthekeyconceptsofthebookmarkprocedure,allowingfacilitatorstoprovide
additionaltrainingifnecessary.Priortoplacingrecommendedperformancestandardsineach
round,panelistswereaskedtocompleteareadinessformtoindicatetheirpreparednessto
recommendperformancestandards.Thisformaskedpaneliststoasserttheirunderstandingof
thetoolsusedtorecommendperformancestandardsineachround.Ifapanelistindicatesthat
theydonotfeelpreparedtorecommendperformancestandards,theworkshopleader
providesadditionaltrainingandopportunitiesfordiscussion.Allpanelistshadtoindicatethat
theyfeltpreparedtomoveforwardbeforetheyrecommendedacut.AllAzMERITstandard
settingpanelistsindicatedtheyunderstoodthetaskathandandfeltreadytorecommend
performancestandards.Samplesofreadinessformsusedforcompletingthebookmarktaskare
presentedinAppendixH.
Bookmarkplacementwasconductedintworounds,allowingpaneliststomakeindependent
judgmentswhilestillbenefitingfromdiscussionwiththeirfellowpanelists.Panelistswere
instructedtoidentifytheirrecommendedcutsforProficient,PartiallyProficient,andHighly
Proficientineachround.TheplacementofthebookmarkisillustratedinFigure1.Eachpanelist
usedtheirjustbarelyPLDstoidentifywhichitemrepresentedthelowerboundofeach
performancelevel.Intheexample,apanelistconcludedthatstudentswhowerejustbarelyat
theProficientlevelwoulddemonstratemasteryontheitemonthepageindicatedbythe
arrow,whilestudentsbelowtheProficientlevelwouldnot.Therefore,thepanelistdecided
thattheProficientperformancelevelwouldbeginonthepageindicatedbyanarrow.The
panelistbelievedthatstudentsbelowtheProficientperformancelevelwouldnotbeableto
demonstratemasteryofitemsbeyondtheindicatedpageintheordereditembooklet.

16

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Figure 1. Example of Bookmark Placement

Benchmark Information
Panelistswerechargedtorecommendperformancestandardscomparabletootherimportant
assessmentsystems,includingnationalandinternationalbenchmarkssuchasNAEP,other
statewideassessments,andcollegeentryexams.TofacilitatecomparisonsofArizona
performancestandardswithothernationalandinternationalbenchmarks,panelistswere
providedwiththelocationsofperformancestandardsfromtheseotherassessmentssystemsin
theirOIBs.Inparticular,performancestandardlocationsforthefollowingassessmentswere
providedaspartofpanelistsOIBreview:

SmarterBalancedELAandmathperformancestandardsingrades38and11/AlgebraII,

PISAperformancestandardsingrade10ELAandGeometry,

NAEPperformancestandardsinreadingandmathingrades4and8(andinterpolated
forgrade6),

ACTcollegereadyperformancestandardingrade11ELAandAlgebraII,and

ArizonaspreviousAIMSassessment.

17

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Panelist Feedback and Impact Data


PriortoRound2,panelistswereprovidedfeedbackaboutthebookmarkplacementsmadeby
fellowpanelists.AftermakingtheirRound1bookmarkplacements,panelistsreconvenedand
beganwithadiscussionofpanelistfeedbackaboutthebookmarklocationsrecommendedby
eachpanelist,beginningwithtablelevelfeedbackanddiscussion,andprogressingtoroomlevel
discussion.Eachtablespenttimereviewinganddiscussingcutscoreplacements,focusingon
thelowestandhighestrecommendedperformancestandardsbothatthetableandacrossthe
panel.Panelistswereaskedtoreviewtheitemsbetweenthelowestandhighestperformance
standardsattheirtable,discussingthestandardsandthejustbarelyPLDs.Discussionwasthen
expandedtotheroomlevel,witheachtablereviewingthebasisfortheirown
recommendationsforthegroupatlarge.
Followingdiscussionofpanelistfeedback,panelistswerepresentedwithimpactdata,the
percentageofstudentsexpectedtoscoreatorabovetherecommendedRound1performance
standards.Panelistsdiscussedanyimplicationsoftheimpactdata,bothattheirtablesand
acrossthepanelmoregenerally,focusingonwhethertheimpactwasinlinewiththeir
expectations.Followingpresentationofimpactdata,panelistswereprovided,foreachitemin
theOIB,thepercentageofstudentsexpectedtoachievetheabilitylevelindexedbythatpage.
Aftercompletingtheirdiscussions,panelistsagainworkedthroughtheOIB,placingtheirRound
2bookmarksforallthreeperformancelevels,beginningwithProficientandfollowedby
PartiallyProficientandHighlyProficient.
Estimating Student Performance Data
WhiletheAzMERITOIBswereconstructedbasedoncalibrationoftheonlinetesting
population,thepercentageofstudentswithinthestatewhomeetorexceedeachpotential
performancestandard(i.e.,eachpageintheordereditembooklet)wasestimatedbasedonall
studentsparticipatinginthefirstoperationaladministrationoftheassessment,including
studentswhotestedonlineandstudentswhotestedonpaper.
Amatchedsamplesapproachwasusedtoestimatetheeffectsofmodeonstudent
performance.Previousyearstudentachievementresults,aswellasdemographicinformation,
includinggender,ethnicity,incomelevelstatus,Englishlanguagelearner(ELL)status,
IndividualizedEducationProgram(IEP),wereusedtoidentifymatchedsamplesforthemode
comparabilityanalyses.
Withmatchedsamplesinhand,itemparameterswerecalibratedseparatelyforthematched
samplesofpaperandonlinetestadministrations,andthelinkingconstantsnecessarytobring
thepaperitemparametersontotheonlinereferencescalewereidentified.Themodelinking
constantswereuniformlyquitesmall,indicatingvirtuallynoeffectoftestadministrationmode
onstudentperformance.Nevertheless,forthepurposeofestimatingstudentimpactforthe
standardsettingworkshop,themodelinkingconstantswereappliedtothepaperitem
parameterstoestimatestudentabilityforpapertestadministrations.Thus,thepercentageof

18

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

studentsestimatedtomeetorexceedeachpotentialperformancestandardontheAzMERIT
wasbasedonallstudentswhoparticipatedintheoperationalassessment.Asummaryofthe
modecomparabilitystudyispresentedinAppendixI7.
PriortoRound2oftheBookmarkprocedure,thepercentageofstudentsmeetingthe
standards,basedontheRound1mediancutscore,waspresentedtopanelists.

Vertical Articulation
Performancestandardsshouldideallybewellarticulatedacrossgrades.Unlessthereare
systemicdifferencesinthequalityofinstructionacrossgrades,theexpectationisthatstudents
whomeetthestandardsandarepreparedforinstructioninthesubsequentgradewilllikely
continuetomeetstandardsastheyprogressthroughtheirschoolyears,andthatthereforewe
wouldnotexpecttoseelargechangesintheproficiencyratesfromgradetograde.Whilethis
verticalarticulationisincorporatedintothedevelopmentoftheArizonaCollegeandCareer
ReadyStandardsaswellasthetestspecificationsforeachoftheAzMERITassessments,
maintainingandreinforcingthecrossgradearticulationinthesettingofmeaningful
performancestandardsisimportant,especiallyforELAandmath,wherestudentsareassessed
annually.Lackofarticulationinthesesubjectscanresultinconfusion,especiallywhenthereare
unreasonablylargeshiftsinstudentperformancelevelclassificationsfromgradetograde.
Articulationwasconsideredfromtwoperspectives:thepercentofstudentsmeetingstandards
acrossgradesandcourses,andthelocationoftheperformancestandardsonthevertically
linkedAzMERITscale,whichallowedpaneliststoevaluatetheirrecommendedperformance
standardswithrespecttoexpectedstudentgrowthfromgradetograde.Adescriptionofthe
proceduresusedtoyieldtheAzMERITverticalscaleispresentedinAppendixJ.
Tohelpfosterconsistencyintheidentificationofperformancestandardsacrossgrades,after
performancestandardswererecommendedfortheinitialgradelevelineachgradeband,table
leaderswereconvenedtoparticipateinaverticalmoderationsession.Tableleaderswere
shownthepercentageofstudentsscoringatoraboveeachoftheperformancestandards,and
thepercentofstudentsclassifiedateachperformancelevelacrosstests.Wherethepercentage
ofstudentsexpectedtomeetstandardsvariedgreatlybetweengradeorcoursebased
assessments,tableleaderswereaskedtoconsidermodificationstotherecommended
standardsthatwouldachieveamorearticulatedsystemofstandards.Intheseinstances,table
leadersreviewedtheordereditembookletsandconsideredwhetherthecontentoftheOIB
supportedtheadjustment.Thus,whiletableleadersworkedtoarticulatestandardsacross
grades,theyalsoensuredthatanychangesresultingfromthemoderationmeetingbe
consistentwiththeknowledgeandskillsdescribedinthePLDs.
Withanchorgradeperformancestandardsinhand,AIRevaluatedbothimpactdatafromeach
gradelevelassessment,aswellasstudentabilityestimatesfromtheverticallylinkedAzMERIT

ResponsivetoStandardsforEducationandPsychologicalTesting:Standard3.0,3.5,3.6,3.8,3.15,5.7,5.12,5.13,
5.14,5.15,12.3,12.17,and13.6

19

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

scale,tointerpolatethelikelylocationofeachperformancestandardforeachoftheremaining
gradelevelandEOCassessments.
Torecommendperformancestandardsinthesenonanchorgradeassessments,thestandard
bookmarkproceduresweremodifiedsothatpanelistswereinstructedtodeterminewhether
thejustbarelyPLDssupportedtheplacementofaspecificbookmarkontheinterpolatedpage.
IfthePLDsdidnotsupporttheplacementofthebookmarkontheinterpolatedpage,then
panelistswereaskedwhethertheycouldidentifyabookmarkplacementneartheinterpolated
pagethatwouldbesupportedbythePLDs.Panelistswereinstructedthattheirbookmark
placementsmustbeguidedbycontentconsiderations,whethertheyrecommendedthe
interpolatedpageintheOIBoradifferentbookmarkplacement.Otherwise,bookmark
placementsproceededaswiththeanchorgraderounds.FollowingRound1bookmark
placements,panelistsreceivedfeedbackaboutthebookmarkplacementsofpanelistsattheir
table,andfortheroomasawholeandimpactdata.
Afinalmoderationsessionwasconductedfollowingthecompletionofworkshopactivitiesfor
theinterpolatedgrades.Thisfinalmoderationactivityensuredthattableleadershadan
opportunitytoreviewtheentiresystemofrecommendedstandardsandtomakeanydesired
adjustmentspriortocompletionoftheworkshop.Aswiththeinitialmoderationsession,in
thoseinstanceswheretableleaderschosetoadjustaperformancestandardduringthefinal
moderationsession,theyreviewedtheirordereditembookletstoensurethattheadjustments
hadabasisintestcontent.
Theadvantageofthisapproachisthatitresultsinasystemofperformancestandardsthatare
moreconsistentacrossgradelevels.Atthemostbasiclevel,itensuresthattherearenotwide
fluctuationsintheproportionofstudentsmeetingeachperformancestandardacrossgrades.
Crossgradearticulationinformedbytheverticalscalealsoensuresthattherearenoreversals
inrecommendedperformancestandardsacrossgrades.

Workshop Evaluation
Panelistswereencouragedtoprovidefeedbackconcerningtheproceduresandoutcomesof
thestandardsettingworkshopthroughouttheprocess,viagroupdiscussions,practice
activities,andcompletionofreadinessformspriortoplacingtheirbookmarks.
Atthecompletionoftheworkshop,panelistswereaskedtocompleteaworkshopevaluation
formdesignedtoelicitfeedbackonallaspectsoftheworkshop,includingclarityoftrainingand
tasks,appropriatenessofthetimespentonactivities,andsatisfactionwiththeoutcomeofthe
workshop.SamplesoftheevaluationformsarepresentedinAppendixK.

20

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Recommended Performance Standards and Impact


Data
FortheAzMERITinELAandmath,AppendixLpresentstheminimum,maximum,andmedian
bookmarkplacementforeachroundofbookmarkplacements,aswellasanybookmarksplaced
duringModerationsessions,andresultingfinalrecommendationsfollowingthestandard
settingworkshops.Aspanelistsdiscussedthereasonsfortheirbookmarkplacementsinthe
contextoffeedbackfromotherpanelistsandimpactdata,variabilityacrosstablesoften
decreasedacrossrounds.ThefiguresinAppendixM,ConvergenceofBookmarksacross
Rounds,illustratevariabilityinmediantablebookmarkplacementsforthethreeperformance
standardsoverthetworounds.Thesefiguresillustratehowvariabilityinbookmarkdecisions
changedfromthefirsttothesecondround.Ingeneral,therewasconsiderableconsistencyin
theplacementofperformancestandardsacrossrounds.
Foreachtest,finalrecommendedperformancestandardistheoutcomefromthefinal
moderation,orintheabsenceofmoderation,themedianbookmarkpagefollowingRound2.
Thefinalrecommendedperformancestandardsforeachassessment,grade,andperformance
standardarepresentedinTable4,alongwiththeprojectedimpacteachperformancestandard
wouldhaveonArizonapublicschoolstudentstestedin2015.ThefinalrecommendedOIBpage
numbersarethemedianbookmarksofeachpanelfollowingRound2bookmarkplacement,and
subsequentmoderation.

21

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Table4.FinalRecommendedPerformanceStandardsforAzMERIT

Test
Grade3
ELA
Grade4
ELA

Grade5
ELA

Grade6
ELA

Grade7
ELA
Grade8
ELA

Grade9
ELA
Grade10
ELA
Grade11
ELA
Grade3
Math

Grade4
Math
Grade5
Math

PerformanceLevel
PartiallyProficient
Proficient
HighlyProficient
PartiallyProficient
Proficient
HighlyProficient
PartiallyProficient
Proficient
HighlyProficient
PartiallyProficient
Proficient
HighlyProficient
PartiallyProficient
Proficient
HighlyProficient
PartiallyProficient
Proficient
HighlyProficient
PartiallyProficient
Proficient
HighlyProficient
PartiallyProficient
Proficient
HighlyProficient
PartiallyProficient
Proficient
HighlyProficient
PartiallyProficient
Proficient
HighlyProficient
PartiallyProficient
Proficient
HighlyProficient
PartiallyProficient
Proficient
HighlyProficient

OrderedItem
BookletPage
18
25
49
19
32
57
15
32
53
16
30
58
18
36
61
19
38
62
17
32
56
13
32
59
13
29
52
10
33
52
10
35
58
4
27
52

22

Theta
0.09
0.29
1.36
0.14
0.60
1.80
0.13
0.63
1.80
0.12
0.58
2.03
0.02
0.61
1.90
0.06
0.64
1.72
0.12
0.59
1.57
0.11
0.58
1.42
0.02
0.52
1.27
0.16
1.04
2.43
0.31
0.76
2.20
0.65
0.41
1.74

EstimatedPercentage
ofStudentsAtor
AbovePerformance
Standard
56
41
10
57
39
5
63
30
3
61
34
4
59
33
4
60
33
6
53
27
6
51
30
8
46
26
8
73
42
15
71
42
10
71
40
13

American Institutes for Research

AzMERIT

Test
Grade6
Math
Grade7
Math
Grade8
Math
AlgebraI

Geometry

AlgebraII

PerformanceLevel
PartiallyProficient
Proficient
HighlyProficient
PartiallyProficient
Proficient
HighlyProficient
PartiallyProficient
Proficient
HighlyProficient
PartiallyProficient
Proficient
HighlyProficient
PartiallyProficient
Proficient
HighlyProficient
PartiallyProficient
Proficient
HighlyProficient

OrderedItem
BookletPage
9
26
46
11
30
46
15
29
47
17
33
56
16
30
52
15
29
49

StandardSettingTechnicalReport

Theta
0.48
0.41
1.55
0.19
0.59
1.51
0.69
0.09
1.15
0.69
0.03
1.27
1.37
0.58
0.96
1.49
0.78
0.57

EstimatedPercentage
ofStudentsAtor
AbovePerformance
Standard
62
32
11
52
30
13
57
32
13
55
32
9
53
30
6
53
29
6

Note:Followingthestandardsettingworkshop,recommendationsaresubmittedtoArizonasStateBoardof
Education.PerformancestandardsarenotfinalpriortoapprovalandadoptionbytheBoard.

23

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Table5showstheestimatedpercentageofstudentclassifiedateachperformancelevelbased
onfinalpanelistrecommendedstandardsforthestudentpopulationoverallacrossgradelevels
andcoursesfortheELAandmathassessments.TheresultsofTable5arerepresented
graphicallyinFigure2,forELA,andFigure3formath.AppendixNpresentstheestimated
percentageofstudentsclassifiedateachperformanceleveldisaggregatedbygenderand
ethnicity.
Table5.PercentageofStudentsatEachPerformanceLevelbasedonFinalRecommended
PerformanceStandards
Test
Grade3ELA
Grade4ELA
Grade5ELA
Grade6ELA
Grade7ELA
Grade8ELA
Grade9ELA
Grade10ELA
Grade11ELA
Grade3Math
Grade4Math
Grade5Math
Grade6Math
Grade7Math
Grade8Math
AlgebraI
Geometry
AlgebraII

MinimallyProficient PartiallyProficient
ELA
44%
15%
43%
19%
37%
33%
39%
27%
41%
26%
40%
27%
47%
26%
49%
21%
54%
20%
Math
27%
31%
29%
29%
29%
31%
38%
30%
48%
22%
43%
24%
45%
23%
47%
24%
47%
24%

Proficient

HighlyProficient

31%
33%
27%
30%
29%
26%
21%
22%
17%

10%
5%
3%
4%
4%
6%
6%
8%
8%

27%
32%
27%
21%
18%
20%
23%
24%
23%

15%
10%
13%
11%
13%
13%
9%
6%
6%

24

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Figure2.PercentageofStudentsatEachPerformanceLevelbasedonFinalRecommended
PerformanceStandardsAzMERITELA
5

10

27
33

30

29

26

21

22

17

33
20
19

13

44

43

G3ELA

G4ELA

33

26

39

41

40

G5ELA

G6ELA

G7ELA

G8ELA

PartiallyProficient

25

21

47

49

27

37

MinimallyProficient

27

26

G9ELA

Proficient

54

G10ELA G11ELA
HighlyProficient

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Figure3.PercentageofStudentsatEachPerformanceLevelbasedonFinalRecommended
PerformanceStandardsAzMERITMath

15

27

10

32

13

11

13

13

21

18

20

23

24

23

43

45

30
29

48
29

24

23

24

24

47

47

31

38
27

27

22

31

29

G3Math G4Math G5Math G6Math G7Math G8Math AlgebraI Geometry AlgebraII


MinimallyProficient

PartiallyProficient

26

Proficient

HighlyProficient

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

ADEintendstoreportstudentperformanceontheontheverticallylinkedAzMERITscale.
Becauseabilityestimatesofextremelylowandhighscoringstudentsarelessprecise,test
scoresforverylowandhighperformingstudentswillbemorepronetofluctuateovertime.To
minimizescalescoreinstabilityforverylowandhighscoringstudents,abilityestimateswillbe
truncatedat+3.5onthewithingradescalebeforebeingtransformedtotheverticallylinked
scale.
StudentabilityestimateswillthenbetransformedfromtheverticallylinkedRaschthetascale
tothesubjectspecificAzMERITreportingscale:

2500

30

3500

30

ApplyingtheAzMERITscalescoretransformationstotheperformancestandardsrecommended
bytheworkshoppanelsresultsinthesystemofscalescorerangesforeachoftheAzMERIT
performancelevelclassificationsidentifiedinTable6.
Table6.AzMERITScaleScoreRangesBasedonFinalRecommendedPerformanceStandards
Test

MinimallyProficient

PartiallyProficient

Proficient

HighlyProficient

ELA
Grade3ELA

23952496

24972508

25092540

25412605

Grade4ELA

24002509

25102522

25232558

25592610

Grade5ELA

24192519

25202542

25432577

25782629

Grade6ELA

24312531

25322552

25532596

25972641

Grade7ELA

24382542

25432560

25612599

26002648

Grade8ELA

24482550

25512571

25722603

26042658

Grade9ELA

24542554

25552576

25772605

26062664

Grade10ELA

24582566

25672580

25812605

26062668

Grade11ELA

24652568

25692584

25852607

26082675

Math
Grade3Math

33953494

34953530

35313572

35733605

Grade4Math

34353529

35303561

35623605

36063645

Grade5Math

34783562

35633594

35953634

36353688

Grade6Math

35123601

36023628

36293662

36633722

Grade7Math

35293628

36293651

36523679

36803739

Grade8Math

35663649

36503672

36733704

37053776

AlgebraI

35773660

36613680

36813719

37203787

Geometry

36093672

36733696

36973742

37433819

AlgebraII

36293689

36903710

37113750

37513839

27

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Evaluation of the Standard Setting Workshop


Panelist Evaluation of Standard Setting Workshop
Followingthecompletionofstandardsettingtasks,panelistswereaskedtoevaluatedifferent
aspectsoftheworkshop,andtheresultingrecommendations.Attheendoftheworkshop,all
butonepanelistindicatedthattrainingonthemaincomponentsandtoolsofthebookmark
procedurewasadequate,andthattheyunderstoodhowtouseeachcomponent.
Generally,panelistsindicatedthattheamountoftimeallottedfordifferentactivitieswithinthe
standardsettingworkshopwasaboutright.Overall,panelistsexpressedgeneralsatisfaction
withtheworkshopandofferedsuggestionsforimprovingtheexperienceinfuturemeetings.
Acrossallpanels,allbutoneparticipantindicatedtheyagreedthatstudentsclassifiedateach
performancelevelarefairlyclassifiedintoeachoftheperformancelevelclassificationsbased
ontheknowledgeandskillsdescribedintheArizonaCollegeandCareerReadyStandards,as
summarizedinTable7.AppendixOshowspanelistsresponsestotheevaluationforms.
Table7.SummaryofPanelistEvaluationofRecommendedPerformanceStandards
Strongly
Disagree

Disagree

Agree

Strongly
Agree

IamconfidentthatstudentsclassifiedasProficient
demonstrateafundamentalunderstandingofand
abilitytoapplythecontentknowledgeandskills
neededtobeontracktowardsArizonasCollegeand
CareerReadinessStandards(ACCRS)inMathematics
andEnglishLanguageArtsStandards.(Level3)

35

44

IamconfidentthatstudentsclassifiedasPartially
Proficientdemonstrateapartialunderstandingof
andabilitytoapplythecontentknowledgeandskills
neededtobeontracktowardsArizonasCollegeand
CareerReadinessStandards(ACCRS)inMathematics
andEnglishLanguageArtsStandards.(Level2)

34

45

IamconfidentthatstudentsclassifiedasHighly
Proficientdemonstrateanadvancedunderstanding
ofandabilitytoapplythecontentknowledgeand
skillsneededtobeontracktowardsArizonasCollege
andCareerReadinessStandards(ACCRS)in
MathematicsandEnglishLanguageArtsStandards.
(Level4)

30

49

WorkshopEvaluationQuestion

28

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Independent Observer Review of Standard Setting Workshop


ADEinvitedmembersoftheStateBoardofEducationtoattendandobservethestandard
settingworkshop.ThreedistrictobserversattendedandsubmittedareporttotheStateBoard
ofEducationdescribingtheirexperienceattheworkshop;thereportwasproduced
independentlywithoutinputorreviewfromADE.ThereportispresentedinAppendixP.

29

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

References
Ferrara,S.,Perie,M.,&Johnson,E.(2008).Matchingthejudgmentaltaskwithstandardsetting
panelistexpertise:TheItemDescriptor(ID)Matchingprocedure.JournalofApplied
TestingTechnology,9(1).
McLaughlin,D.,Scarloss,B.A.,Stancavage,F.B.,&Blankenship,C.D.(2005).UsingState
AssessmentstoImputeAchievementofStudentsAbsentfromNAEP:AnEmpiricalStudyin
FourStates.Washington,DC:AmericanInstitutesforResearch.Retrievedfrom
www.air.org/files/McLaughlin_AbsentStudents.pdf
Mitzel,H.C.,Lewis,D.M.,Patz,R.J.,&Green,D.R.(2001).TheBookmarkprocedure:
Psychologicalperspectives.InG.Cizek(Ed.),Settingperformancestandards:Concepts,
methods,andperspectives.Mahwah,NJ:Erlba

30

American Institutes for Research

AzMERIT

Appendix A Workshop Agendas

Appendix

StandardSettingTechnicalReport

AzMERIT

Standard Setting Technical Report

Document A1. AzMERIT Mathematics & ELA Grades 3-8 Standard Setting Agenda
July 14 - 16, 2015
(Grade 3-8 Panels)
Tuesday, July 14, 2015
7:30 8:00
7:30 8:00

Orientation for Table Leaders


Registration and morning refreshments

8:00 8:15
8:15 9:30

Panelists receive folders, sign security affidavit

Welcome and introductions from Arizona Department of Education


Large group introductory training

Welcome and introductions


Purpose of standard setting workshop
Description of the AzMERIT test design
General overview of standard setting procedures and key concepts
o Proficiency Level Descriptors
o Just Barely
o Ordered Item Book
o Mastery
o Bookmark task
o Benchmark Information
o Panelist feedback and impact data

9:30 9:45
9:45 10:00

Break, and separate into small group rooms


Introductions within panel

10:00 11:00

Participate in AzMERIT assessment

11:00 12:00

Review Performance Level Descriptors and develop Just Barely PLDs Grades
4, 6, and 8

12:00 1:00

Lunch

1:00 2:30

Review PLDs and develop Just Barely PLDs Grades 4, 6, and 8 (continued)

2:30 2:45

Break

2:45 4:45

Review of Ordered Item Book Grades 4, 6, and 8

4:45

Training on review of the OIB


o What do students need to know and be able to do to respond
correctly to each question?
o Why is each item more difficult than the preceding item?
Individual review of the OIB
Discuss areas of transition and skills with tables

Adjourn
Wednesday, July 15, 2015

7:30 8:00

Registration and morning refreshments

8:00 9:00

Training on Bookmark Placement task


A-1

American Institutes for Research

AzMERIT

Standard Setting Technical Report

(Grade 3-8 Panels)

9:00 10:15

Review of Bookmark Placement key concepts


o Proficiency Level Descriptors
o Ordered Item Book
Training on mastery and 2/3 likelihood
Training on bookmark placement judgment task, and procedure for
recording bookmarks

Round 1 bookmark placement for Proficient, Partially Proficient, and Highly


Proficient Grades 4, 6, and 8

Review of bookmark procedures and key concepts


Completion of Bookmark Placement Readiness Form
Review OIB and place each bookmark
o Proficient
o Partially Proficient
o Highly Proficient

10:15 10:30

Panelist Break, and concurrent production of feedback data

10:30 11:30

Review results of Round 1 Grades 4, 6, and 8

Presentation and discussion of Round 1 panelist agreement feedback


data
Review agreement feedback data
Discussion of percent of students achieving the Round 1 recommended
standards

11:30 12:30

Lunch

12:30 1:30

Round 2 bookmark placement for Proficient, Partially Proficient, and Highly


Proficient Grades 4, 6, and 8

Completion of Bookmark Placement Readiness Form


Review OIB and place each bookmark
o Proficient
o Partially Proficient
o Highly Proficient

1:30 3:30

Review Performance Level Descriptors and develop Just Barely PLDs Grades
3, 5, and 7

3:30 3:45

Break

3:45 4:45

Anchor Grade Moderation


*Table leaders required to participate, all panelists invited to attend
Adjourn

4:45

Thursday, July 16, 2015


7:30 8:00
8:00 10:00

Registration and morning refreshments

10:00 10:15

Panelist Break

10:15 10:45

Review Results of anchor grade vertical moderation

Review of Ordered Item Booklet Grades 3, 5, and 7

A-2

American Institutes for Research

AzMERIT

Standard Setting Technical Report

(Grade 3-8 Panels)


10:45 12:00

Round 1 bookmark placement for Proficient, Partially Proficient, and Highly


Proficient Grades 3, 5, and 7

Completion of Bookmark Placement Readiness Form


Review OIB and place each bookmark
o Proficient
o Partially Proficient
o Highly Proficient

12:00 1:00

Lunch

1:00 2:00

Review results of Round 1 Grades 3, 5, and 7

2:00 3:00

Round 2 bookmark placement for Proficient, Partially Proficient, and Highly


Proficient Grades 3, 5, and 7

Completion of Bookmark Placement Readiness Form


Review OIB and place each bookmark
o Proficient
o Partially Proficient
o Highly Proficient

3:00 3:30

Complete workshop evaluation forms

3:30 4:30

Final vertical moderation (if needed)


*Table leaders required to participate, all panelists invited to attend

A-3

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Document A2. AzMERIT Mathematics & ELA EOC Standard Setting Agenda July
13 - 16, 2015
High School Panels
Monday, July 13, 2015
7:30 8:00

Orientation for Table Leaders

7:30 8:00

Registration and morning refreshments

Panelists receive folders, sign security affidavit

8:00 8:15

Welcome and introductions from Arizona Department of Education

8:15 9:30

Large group introductory training

Welcome and introductions


Purpose of standard setting workshop
Description of the AzMERIT test design
General overview of standard setting procedures and key concepts
o Proficiency Level Descriptors
o Just Barely
o Ordered Item Book
o Mastery
o Bookmark task
o Benchmark Information
o Panelist feedback and impact data

9:30 9:45

Break, and separate into small group rooms

9:45 10:00

Introductions within panel

10:00 11:00

Participate in AzMERIT assessment ELA 11/ Algebra II

11:00 12:00

Review Performance Level Descriptors and develop Just Barely PLDs ELA 11/
Algebra II

12:00 1:00

Lunch

1:00 2:30

Review PLDs and develop Just Barely PLDs Algebra ELA/11 II (continued)

2:30 2:45

Break

2:45 4:45

Review of Ordered Item Book ELA 11/ Algebra II

4:45

Training on review of the OIB


o What do students need to know and be able to do to respond
correctly to each question?
o Why is each item more difficult than the preceding item?
Individual review of the OIB
Discuss areas of transition and skills with tables

Adjourn
Tuesday, July 14, 2015

7:30 8:00

Registration and morning refreshments

A-4

American Institutes for Research

AzMERIT

Standard Setting Technical Report

High School Panels


8:00 9:00

Training on Bookmark Placement task

9:00 10:15

Review of Bookmark Placement key concepts


o Proficiency Level Descriptors
o Ordered Item Book
Training on mastery and 2/3 likelihood
Training on bookmark placement judgment task, and procedure for
recording bookmarks

Round 1 bookmark placement for Proficient, Partially Proficient, and Highly


Proficient ELA 11/ Algebra II

Review of bookmark procedures and key concepts


Completion of Bookmark Placement Readiness Form
Review OIB and place each bookmark
o Proficient
o Partially Proficient
o Highly Proficient

10:15 10:30

Panelist Break, and concurrent production of feedback data

10:30 11:30

Review results of Round 1 ELA 11/ Algebra II

Presentation and discussion of Round 1 panelist agreement feedback data


Review agreement feedback data
Discussion of percent of students achieving the Round 1 recommended
standards

11:30 12:30

Lunch

12:30 1:30

Round 2 bookmark placement for Proficient, Partially Proficient, and Highly


Proficient ELA 11/ Algebra II

Completion of Bookmark Placement Readiness Form


Review OIB and place each bookmark
o Proficient
o Partially Proficient
o Highly Proficient

1:30 3:30

Review Performance Level Descriptors and develop Just Barely PLDs ELA 10/
Geometry

3:30 3:45

Break

3:45 4:45

Begin Review of Ordered Item Booklet ELA 10/ Geometry

4:45

Adjourn
Wednesday, July 15, 2015

7:30 8:00

Registration and morning refreshments

8:00 9:00
9:00 10:15

Complete OIB Review ELA 10/ Geometry


Round 1 bookmark placement for Proficient, Partially Proficient, and Highly
Proficient ELA 10/ Geometry

A-5

American Institutes for Research

AzMERIT

Standard Setting Technical Report

High School Panels

Completion of Bookmark Placement Readiness Form


Review OIB and place each bookmark
o Proficient
o Partially Proficient
o Highly Proficient

10:15 10:30

Panelist Break

10:30 11:30

Review results of Round 1 ELA 10/ Geometry

Presentation and discussion of Round 1 panelist feedback data


Review agreement feedback data
Discussion of percent of students achieving the Round 1 recommended
standards

11:30 12:30

Lunch

12:30 1:30

Round 2 bookmark placement for Proficient, Partially Proficient, and Highly


Proficient ELA 10/ Geometry
Completion of Bookmark Placement Readiness Form
Review OIB and place each bookmark
o Proficient
o Partially Proficient
o Highly Proficient

1:30 3:30

Review Performance Level Descriptors and develop Just Barely PLDs ELA 9/
Algebra I

3:30 3:45

Break

3:45 4:45

Anchor grade vertical moderation


*Table leaders required to participate, all panelists invited to attend

4:45

Adjourn
Thursday, July 16, 2015

7:30 8:00
8:00 10:00

Registration and morning refreshments

10:00 10:15

Panelist Break

10:15 10:45

Review results of anchor grade vertical moderation

10:45 12:00

Round 1 bookmark placement for Proficient, Partially Proficient, and Highly


Proficient ELA 9/ Algebra I

Review of Ordered Item Booklet ELA 9/ Algebra I

12:00 1:00

Completion of Bookmark Placement Readiness Form


Review OIB and place each bookmark
o Proficient
o Partially Proficient
o Highly Proficient

Lunch

A-6

American Institutes for Research

AzMERIT

Standard Setting Technical Report

High School Panels


1:00 2:00

Review results of Round 1 ELA 9/ Algebra I

2:00 3:00

Round 2 bookmark placement for Proficient, Partially Proficient, and Highly


Proficient ELA 9/ Algebra I

Completion of Bookmark Placement Readiness Form


Review OIB and place each bookmark
o Proficient
o Partially Proficient
o Highly Proficient

3:00 3:30

Complete workshop evaluation forms

3:30 4:30

Final vertical moderation (if needed)


*Table leaders required to participate, all panelists invited to attend

A-7

American Institutes for Research

AzMERIT

Appendix B Composition of Panels

Appendix

StandardSettingTechnicalReport

AzMERIT

StandardSettingTechnicalReport

TableB1.CompositionofELAGrades34Panels

Table

Table
Leader

Yes

Rural

ArizonaNavajoCentral

Male

n/a

Urbanized

PeoriaUnifiedSchoolDistrict

n/a

Urbanized

Yes

Rural

n/a

Urbanized

n/a

UrbanClusters

n/a

Urbanized

Yes

Urbanized

n/a

Urbanized

n/a

Suburban

Urban/Rural

District

CAFAINC.LearningFoundation
andPerformingArts
LegacyTraditionalSchoolsand
AthlosTraditionalAcademy
MaricopaCountyEducation
ServiceAgency
J.O.CombsUnifiedSchool
District
MadisonElementaryDistrict
WashingtonElementarySchool
District
CAFA,Inc.dbaLearning
FoundationPerformingArts
School
DeerValleyUnifiedDistrict

Gender

Ethnicity

GradeMost
Frequently
Taught

ElementaryEducation

Female Asian

ElementaryEducation

Female White,nonHispanic

SecondaryEducation

Female HispanicorLatino

Elementary,Administrative
Principal,Superintendent

Female HispanicorLatino

Elementary,Secondary

Female White,nonHispanic

ElementaryEducation

Female White,nonHispanic

ElementaryEducation

Female White,nonHispanic

Elementary,Administrative
Principal,Superintendent

Female HispanicorLatino

ElementaryEducation

Female White,nonHispanic

Elementary,Administrative

B-1

NativeAmerican

CurrentPosition

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

TableB2.CompositionofELAGrades56Panels
GradeMost
Frequently
Taught

Table

Table
Leader

Yes

UrbanClusters

MesaUnifiedDistrict

Female Black

ElementaryEducation

n/a

Rural

SuperiorUnifiedSchoolDistrict

Female White,nonHispanic

Elementary,Secondary

n/a

Suburban

MesaUnifiedDistrict

Female White,nonHispanic

ElementaryEducation

Yes

UrbanClusters

MayerUnifiedSchoolDistrict

Female White,nonHispanic

ElementaryEducation

n/a

Urbanized

LibertyElementaryDistrict

Female White,nonHispanic

ElementaryEducation

n/a

Urbanized

ChandlerUnifiedDistrict

Female White,nonHispanic

ElementaryandSpecialEducation

Yes

Urbanized

ParadiseValleyUnifiedDistrict

Female White,nonHispanic

ElementaryandSpecialEducation

n/a

Urbanized

AvondaleElementaryDistrict

Female White,nonHispanic

ElementaryEducation

n/a

Urbanized

PeoriaUnifiedSchoolDistrict

Female White,nonHispanic

Elementary,Administrative
Principal,Superintendent

Urban/Rural

District

Gender

Ethnicity

CurrentPosition

B-2

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

TableB3.CompositionofELAGrades78Panels
GradeMost
Frequently
Taught

Table

Table
Leader

Yes

Suburban

PeoriaUnifiedSchoolDistrict

Female

White,nonHispanic ElementaryEducation

n/a

Urbanized

AlhambraElementaryDistrict

Female

White,nonHispanic

n/a

UrbanClusters

J.O.CombsUnifiedSchool
District

Male

Yes

Urban

GeorgeGervinPrepAcademy

Female

n/a

UrbanClusters

DysartUnifiedDistrict

Female

White,nonHispanic SecondaryEducation

n/a

Urbanized

MesaUnifiedDistrict

Female

White,nonHispanic ElementaryandSpecialEducation

Yes

Suburban

PeoriaUnifiedSchoolDistrict

Female

White,nonHispanic Elementary,JuniorHigh

n/a

UrbanClusters

KyreneElementaryDistrict

Female

White,nonHispanic SpecialEducation

n/a

Urbanized

LaveenElementaryDistrict

Female

White,nonHispanic SecondaryEducation

Urban/Rural

District

Gender

Ethnicity

CurrentPosition

AdministrativePrincipal,
Superintendent
Secondary,Administrative
White,nonHispanic
Principal,Superintendent
ElementaryK8,SpecialEdK8,
HispanicorLatino
NBCTEnglishLanguageArts

7
8
8
7

B-3

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

TableB4.CompositionofELAGrades911Panels
Table

Table
Leader

Urban/Rural

Yes

Urbanized

n/a

Urbanized

n/a

Rural

n/a

District
AmericanCharterSchools
Foundationd.b.a.WestPhoenix
HighSchool
AmericanCharterSchools
Foundationd.b.a.SunValleyHigh
School

Gender

Ethnicity

CurrentPosition

GradeMost
Frequently
Taught

Male

White,nonHispanic Secondary,AdministrativeDuties

Female

White,nonHispanic

RedMesaUnifiedSchoolDistrict

Female

White,nonHispanic SecondaryEnglish&Art

11

UrbanClusters

PrescottUnifiedDistrict

Female

MultiRacial

SecondaryEducation

Yes

Rural

VailUnifiedDistrict

Male

White,nonHispanic SecondaryEducation

10

n/a

UrbanClusters

TucsonUnifiedDistrict

Female

White,nonHispanic SecondaryEducation

11

n/a

Urbanized

GlendaleUnionHighSchool
District

Female

White,nonHispanic SecondaryEducation

10

11

Secondary,Administrative
Principal,Superintendent

10

Yes

Rural

J.O.CombsUnifiedSchoolDistrict

Female

SecondaryEducation,Adult
White,nonHispanic Education,APLanguage&
Composition

n/a

Urbanized

TollesonUnionHighSchool
District

Female

White,nonHispanic

n/a

Urbanized

PimaPreventionPartnership

Female

Asian

11

SecondaryEducation

B-4

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

TableB5.CompositionofMathematicsGrades34Panels
Table

Table
Leader

Yes

Urbanized

n/a

Urban/Rural

District

CurrentPosition

GradeMost
Frequently
Taught

Gender

Ethnicity

RodelFoundationofArizona

Female

White,nonHispanic

Urbanized

ScottsdaleUnifiedDistrict

Female

White,nonHispanic ElementaryEducation

n/a

Urbanized

RodelFoundationofArizona

Female

White,nonHispanic

Yes

Rural

PimaUnifiedDistrict

Female

White,nonHispanic ElementaryEducation

n/a

Urbanized

GilbertUnifiedDistrict

Female

White,nonHispanic ElementaryEducation

n/a

Urbanized

DeerValleyUnifiedDistrict

Female

White,nonHispanic ElementaryEducation

Yes

Urbanized

MesaUnifiedDistrict

Female

White,nonHispanic ElementaryEducation

n/a

Urbanized

CartwrightElementaryDistrict

Female

White,nonHispanic (notprovided)

n/a

Urbanized

MadisonElementaryDistrict

Female

White,nonHispanic ElementaryEducation

n/a

Rural

LakeHavasuUnifiedDistrict

Female

White,nonHispanic ElementaryEducation

Elementary,Administrative
Principal,Superintendent

Elementary,Administrative
Principal,Superintendent

4
3
3

B-5

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

TableB6.CompositionofMathematicsGrades56Panels
GradeMost
Frequently
Taught

Table

Table
Leader

Yes

Urbanized

MesaUnifiedDistrict

Female

n/a

Urbanized

AvondaleElementarySchool
District

Female

n/a

Urbanized

ScottsdaleUnifiedDistrict

Female

White,nonHispanic ElementaryEducation

n/a

UrbanClusters

J.O.CombsUnifiedSchoolDistrict

Female

White,nonHispanic ElementaryEducation

Yes

Rural

WashingtonElementarySchool
District

Male

White,nonHispanic ElementaryEducation

n/a

Urbanized

DeerValleyUnifiedDistrict

Female

White,nonHispanic ElementaryEducation

n/a

Urbanized

FowlerElementaryDistrict

Female

White,nonHispanic ElementaryEducation

n/a

Urbanized

KyreneElementaryDistrict

Female

Black

ElementaryEducation

Yes

Urbanized

TanqueVerdeUnifiedDistrict

Female

White,nonHispanic

Elementary,Administrative
Principal,Superintendent

n/a

UrbanClusters

GlendaleElementaryDistrict

Female

White,nonHispanic ElementaryEducation

n/a

Suburban

ChandlerUnifiedDistrict

Female

White,nonHispanic ElementaryEducation

Urban/Rural

District

Gender

Ethnicity

CurrentPosition

Elementary,Administrative
Principal,Superintendent
Elementary,Administrative
White,nonHispanic
Principal,Superintendent
HispanicorLatino

5
6

B-6

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

TableB7.CompositionofMathematicsGrades78Panels
GradeMost
Frequently
Taught

Table

Table
Leader

Yes

Urban

DeerValleyUnifiedDistrict

Male

White,nonHispanic SecondaryEducation

n/a

Urbanized

OsbornElementaryDistrict

Female

White,nonHispanic Elementary,Secondary

n/a

Urbanized

DeerValleyUnifiedDistrict

Female

White,nonHispanic SecondaryEducation

n/a

UrbanClusters

LitchfieldElementarySchool
District

Female

White,nonHispanic

Secondary,Biology,MiddleGrades
Mathematics

Yes

Suburban

MesaUnifiedDistrict

Female

HispanicorLatino

Elementary,SecondaryMathematics

n/a

Urbanized

ArizonaSchoolfortheArts

Female

White,nonHispanic ElementaryEducation

n/a

Suburban

ChandlerUnifiedDistrict

Female

White,nonHispanic SecondaryMathematics

n/a

Urbanized

CartwrightElementaryDistrict

Male

HispanicorLatino

ElementaryEducation

Yes

Urbanized

TucsonUnifiedDistrict

Female

White,nonHispanic ElementaryEducation

n/a

UrbanClusters

BuckeyeElementaryDistrict

Male

White,nonHispanic SecondaryEducation

n/a

Urbanized

MesaUnifiedDistrict

Female

White,nonHispanic SpecialEducation

Urban/Rural

District

Gender

Ethnicity

CurrentPosition

B-7

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

TableB8.CompositionofMathematicsEOCPanels
Table

Table
Leader

Yes

UrbanClusters PhoenixUnionHighSchoolDistrict Female

n/a

Urbanized

n/a

Urbanized

Yes

Urbanized

n/a

Rural

n/a

Urban/Rural

District

Gender

Ethnicity

CurrentPosition

GradeMost
Frequently
Taught

White,nonHispanic Elementary,Secondary

10

Female

White,nonHispanic SecondaryEducation

Female

White,nonHispanic SecondaryEducation

Female

White,nonHispanic

Female

White,nonHispanic SecondaryMathematics

10

UrbanClusters LeadingEdgeAcademy

Female

White,nonHispanic SecondaryEducation

n/a

UrbanClusters DysartUnifiedDistrict

Female

Asian

10

Yes

Rural

LakeHavasuUnifiedDistrict

Female

White,nonHispanic SecondaryMathematics

n/a

Urbanized

ChandlerUnifiedDistrict

Female

White,nonHispanic SecondaryEducation

11

n/a

Urbanized

PrimaveraTechnicalLearning
Center

Female

White,nonHispanic SecondaryEducation

10

n/a

Urbanized

DeerValleyUnifiedDistrict

Male

White,nonHispanic SecondaryMathematics

11

AguaFriaUnionHighSchool
District
AmericanCharterSchools
Foundationd.b.a.EstrellaHigh
School
GlendaleUnionHighSchool
District
FlorenceUnifiedDistrict

Secondary,GiftedEndorsement,
NBPT

SecondaryEducation

11

B-8

American Institutes for Research

AzMERIT

Appendix C Training Presentations

Appendix

StandardSettingTechnicalReport

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

Thepurposeofthisworkshopistorecommendasystemofperformancestandardstothe
stateboard ofeducation.ForeachofthenewAzMERIT assessments,theworkshoppanels
willrecommendthreeperformancestandardsorcutscores:PartiallyProficient,Proficient
andHighlyProficient.Thesewillbeused toclassifystudentsintooneoffourperformance
levels: MinimallyProficient,PartiallyProficient,Proficient,andHighlyProficient
Tosetvalid,meaningfulcutscores,thatarepubliclyverifiable,standardsettingworkshops
areconducted.Thestandardsettingworkshopsemployresearchbasedproceduresthatare
usedbycommitteesofeducatorstoestablishcutscoresonastate'sassessments.
Performancestandardsimpactstudentsandtheeducationsystemstatewide.The
proceduresyouwillengageinaredesignedtogiveyouthetoolstomakeinformed
judgmentsthatyielddefensiblerecommendationsthatcanbesubmittedtostateboardof
educationforadoption.

C-1

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

Inthisworkshop,wewillberecommendingperformancestandardsforELAgrades3 11
and Mathgrade38,AlgebraI,GeometryandAlgebraII.Thereare8 separatepanelswhich
willworkindependently.Thepanelswillbesplitupbygradebands,ELAgrade34,56,78
and911andMathgrades34,56,78andAlgebraI,GeometryandAlgebraII.

ArizonasMathematics andEnglishLanguageArtsandLiteracystandardswereadoptedin
2010andaddressthemathematics,reading,writing,language,andspeakingandlistening
skillsthateachstudentwillworktomasterashe/sheprogressesthroughschooland
towardscollegeandacareer.

Eachpaneliscomprisedofabout 12panelists,splitintothreetables.
Eachtablehasadesignatedtableleaderwhowillhelptodistributeandcollectmaterials,
andwhocanserveasaliaisonbetweenyourtableandtheworkshopstaff.Pleasemake
suretoleaveallsecurematerialsintheroomsorturn intoyourtableleaderif directedby
theworkshopleader.

C-2

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

Following finalrecommendations,wewillconductadebriefingandwewillaskyouto
completeaworkshopevaluationform beforeyouleave.Theevaluationformsyouwill
receivethroughouttheprocessareanimportantpartofthestandardsettingandyouare
askedtocompletethemthoroughlyandthoughtfully.

Wewillbegintheworkshopwithanoverviewofthestandardsettingprocess.Thepanels
willemploytheBookmarkproceduretorecommendstandards.Youwillbetrainedonthe
specificsofthesemethodsasyougothrougheachstepoftheprocesswithinyourown
groups.
Followingthisintroduction,eachofyouwillhaveanopportunitytoparticipateinthesame
assessmentsadministeredtostudentsthisspring.Youwilltakethegradeandsubjecttest
foryourassignedcommitteeintheonlinetestingenvironmentthatstudentsexperienced.
Followingthat,youwillworkwithothermembersatyourtabletoreviewthePerformance
LevelDescriptorswhichdefinetheknowledgeandskillrequirementsofstudentsateach
level.
Next,youwillreviewabookoftestitemsorderedfromeasiesttomostdifficultbasedon
actualstudentperformancetorecommendperformancestandards,thinkingaboutwhat
studentshavetoknowandbeabletodoinordertorespondsuccessfullytoeachitem.This
isreferredtoasyourOrderedItemBookletorOIB.Youllthenrecommendperformance
standardsbyidentifyingpagesintheOIBthatserveascutsfordifferentlevelsof
achievement.Youllreceiveanddiscussfeedbackonyourinitialrecommendationswith
yourfellowpanelists,andthenmakeanotherindividualrecommendation.Then,youll
receivefurtherfeedbackandotherperformanceinformationtoprovideadditionalcontext
toyourrecommendations.

C-3

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

Standardsettingrefersgenerallytotheprocessofidentifyingapassingscoreonatest.The
centralquestionofthestandardsettingprocessistoidentifythelevelofperformanceona
testthatindicatesapassing,orgoodenough,performance.

FacilitatorNote:Pleasestresstheimportanceofsecurity.Theseareoperationalitemsthat
willbeusedonfutureadministrations.Absolutelynopicturetaking,cellphonesaretobe
putawayandnotoutatthetable,notextingwhileworking.Weunderstandemergencies
mayhappen,pleasetakeallcallsoutsidethemeetingroom.Donotsurfotherwebsites
whileusingthelaptops.

Apassingorgoodenoughperformanceisdeterminedbythepurposeoftheassessment.
Testsmay,forexample,certifyminimumcompetenceorselectoutonlythehighest
performers.

Wecantstresstheimportanceofsecurityenough.Wellbeworkingwithlivetestitems
thatwillbeadministeredtostudentsagaininthefuture,anditsimportantfromatest
scorevalidityperspectivethatitemsremainsecure.Thatmeansthatweaskthatyoukeep
cellphones,tablets,laptopsaway,andstepoutsideifyouneedtotakeacall.

TheAzMERIT arecriterionreferencedtests,meaningthattheydirectlymeasurea
representativesampleoftheknowledgeandskillsthatstudentsareexpectedtoachieveby
theendofeachschoolyear. Therefore,wewillemployatestcenteredapproachtosetting
performancestandards.Intestcenteredapproaches,cutscoresareestablishedbasedon
thedegreetowhichstudentsdemonstrateachievementofknowledgeandskillsmeasured
directlyintheassessment.Forthisreason,testcenteredapproachesdependcriticallyon
havingparticipantsinthestandardsettingwhoareveryknowledgeableaboutthestate's
contentstandardsandwillingtohelpthestatedefinethelevelofknowledgeandskill
expectedofastudentateachperformanceleveldemonstratedbythecutscores.

C-4

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

Performance levelsareregionsontheachievementscaledemarcatedbytheperformance
standards.Theyclassifystudentsbyhowmuchofthecontentstandardstheyknowandare
abletodo.Thethreeperformancestandardswillresultinfourperformance levels.

Thestandardsettingprocessthatwewillbeengagedinduringthenextseveraldaysis
designedtotranslatetheArizonaacademiccontentstandardsinEnglish andlanguagearts
and mathematic intoasetofperformancestandards,orcutscores.Twoimportant
documents,theOrderedItemBookletandthePerformanceLevelDescriptors,willbeyour
primarytoolsfortranslatingtheacademiccontentstandardsintoperformancestandards.
Throughoutthisworkshop,wewillrefertodifferenttypesofstandards.
Academiccontentstandardsspecifywhatstudentsshouldknowandbeabletodobyend
ofeachacademicyear.
Performancestandardsspecifyhowmuchofthecontentstandardsstudentsmustknow
andbeabletodoinordertomeeteachperformancelevel.Youwillrecommendthree
performancestandards,orcutscores,for eachanchorgrade(i.e.Grade4)andinterpolated
(adjacent)grade(i.e.Grade3).

C-5

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

YouwilldiscussthemechanicsoftheBookmark proceduresinmuchgreaterdetailwithin
yourowngroups.

Therearemanymethodsforsettingperformancestandards,includingexamineecentered
andtestcentered.Insomeemploymentapplications,forexample,testsmaybe
administeredtogroupsofpeoplewhoaremoreorlesssuccessful,andacutscoreis
identifiedthatbestdifferentiatesthetwogroups. Thisisanexampleofanexaminee
centeredapproach.

PerformanceLevelDescriptors,orPLDs,aredetaileddescriptionsoftheknowledgeand
skillsstudentsareabletodemonstratewithrespecttotheacademiccontentstandardsat
eachlevel.Inparticular,wellbeconcernedwithaspecialgroup ofstudents,thosewhojust
barelymeettheperformanceleveldescriptors.

However,becausetheAzMERIT arecriterionreferencedassessments,meaningthatthey
measurearepresentativesampleoftheacademiccontentthatstudentsareexpectedto
knowandbeabletodobytheendofeachschoolyear,weareemployingatestcentered
approachtorecommendedperformancestandards.Inotherwords,successful
performanceofitemsonthetestspeaksdirectlytostudentsperformanceofthe
standards.

TheOrderedItemBooklet,orOIB,containsoperationalandothertestitemsthatwere
administeredinspring2015 aswellasotheritemstofillininformationgaps,orderedfrom
easiesttomostdifficult.
OnceyouhavedevelopeddescriptionsofstudentswhojustbarelymeetthePLDsand
completeyourreviewoftheOIB,youwillbereadytorecommendperformancestandards
foreachoftheproficiencylevels.Youwillrecommendperformancestandardsinmultiple
rounds.

Modernstandardsettingapproachesgenerallyuseof anOrderedItemBooklet,orOIB,
withtestitemsorderedfromeasiesttomostdifficult tohelppanelistsandtostreamline
thestandardsettingprocess.
TheBookmarkmethodiswidelyusedformanyyearsinstatewideassessmentprograms,
andhasbeenusedtosetperformancestandardsforvariousotherstateassessments
includingtheAIMSsysteminArizona.

Althoughyouwillhaveplentyofopportunitiestodiscussbookmarkplacementswithour
fellowpanelists,yourbookmarkplacementsrepresentyouindividualrecommendations
andyouwillmakethoserecommendationsindependentlyfromtheotherpanelists.After
yourinitialbookmarkplacements,youwillhaveanopportunitytodiscussyour
recommendationsincontextofyourfellowpanelistsrecommendations,andaffirmor
reviseyourownrecommendationsinroundtwo.

C-6

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

Forexample,ingrade4math,thePLDdescribesaProficientstudent.Someskillsdescribed
inthePLDinclude:

PerformanceLevelDescriptorswillserveasyourguideforidentifyingperformance
standards.The PLDsdescribewhatskillsstudentsineachlevelareexpectedtohave.When
yourecommendastandard,youareassertingthatstudentswhomeetthatlevelof
performancefitthedescriptionofthestudentsabilitiesprovidedinthePLDs.Thislinkis
criticalbecauseitallowsteachersandfamiliestounderstandwhatastudentstestresults
mean.
TothoroughlyreviewthePLDs,itishelpfultoparsethestandards.Youmayfind,for
example,performancelevelsaredifferentiatedbytheverbsusedinthePLDs forexample,
studentsmayrecognize,identify,understand,explain,andsoon.
Youwillbeaskedtopaycarefulattentiontothecontentandskilldemandsrequiredateach
performancelevel.Itiscriticalforyoutounderstandthatwhenyourecommendacut
score,youareassertingthatstudentswhoperformatthatachievementlevelmeetthe
contentandskillrequirementsdescribedinthePLD.

C-7

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

OIBistheother primarytoolyouwilluse.TheOrderedItemBookletswillpresentyouwith
theitemsfromthespring2015assessments.Thecontentoftheitemsisproportionalto
thetestblueprint.

Wewillcomparetheperformanceleveldescriptors acrossperformancelevels.

TheOIBconsistsofall operationalitemsaswellas1520fieldtestitemstofillin
informationgap.Whatthatmeansisthatweaddeditemswheretherewasalargegapin
itemdifficultybetweenoperationalitemssothattheitemsintheOIBappearmorefluidin
termsofdifficulty.
Allitemswillbepresentedinorderofdifficulty;page1willpresenttheeasiestitem,and
thelastpagewillpresentthehardestitem.Someitemswillberepresentedmorethanonce.
Theseitemsaremoreengagingthanothersandareworthmorethanasinglescorepoint.
Eachpage intheordereditembookletwillrefertospecific scorepointoftheitem,In
eachroomyou willreviewtheseriesofitemsinanonlineenvironment.

C-8

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

Itemsareorderedfromeasiesttomostdifficult.TheOIBprovidesapictureofthe
rangeofknowledgeandskillsencompassedbytheitemsonthetest, andisa
vehicleusetomakecutscorejudgments.

WhenstudyingtheOIB,itisimportanttounderstandthedifferencebetweenitemsin
contextofthewholeOIB.Itemsthatappearearlierinthebookareeasier,despite
perceptionsotherwise,thanitemsthatappearlaterinthebook.

TheOIBshouldprovideusefulinformationaboutnaturalbreaksintheknowledge
andskillrequirementsnecessarytoconsistentlyperformsuccessfullyacrossa
rangeofitemcontent.

For eachitem,askyourselftwoquestions:
1) Whatdostudentsneedtoknowandbeabletodotorespondsuccessfullytothisitem?
2) Whyisthisitemmoredifficultthanthepreviousitems?

Foreachperformancestandard,you willplaceabookmarkonthepagethatdivides
theOIBintotwosetsofpageranges:pagesthatstudentsataparticularlevelcan
reliablyrespondcorrectlyto,andthepagesthatthestudentsatthatlevelcannot
respondcorrectlyto.

Whiletheitemswillbepresentedtoyouonline, youwillbeabletorecordyournotes
aboutthesequestionstorefertoasyoudecidewheretoplaceyourbookmark.

C-9

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

Thedifficultyoftheseitemsisbasedonstudents performanceduringthe2015
operationalassessment.Sowhileaparticularitemmayseemeasierorhardertoyou,the
placementintheordereditembookreflectshoweasyorharditactuallywasforstudents
toperformsuccessfullyontheseitems.
Ifanitemseemsoutofordertoyou,rememberthatanitemmaynotmeasurewhatyou
thinkitmeasures.Forexample,anitemmayintendforastudenttohavetoknowa
particularpieceofinformation,butperhapsthestudentswereabletoanswerusingrecall
fromalessonthatwastaughtrecently.
Insteadoffocusingononeitemthatmayseemoutofplacetoyou,trytoidentifynatural
breaksorthresholdsforgroupsofitems.

C-10

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

JustBarely
Evenwithin eachperformancelevel,studentsvaryinthedegreetowhichtheyhave
masteredtheArizonaStatestandards.Somestudentshavejustbarelycrossedtheline
betweenPartiallyProficientandProficient,whileothersaregettingreadytocrosstheline
betweenProficientandHighlyProficient.Ingeneral,thePLDsarewrittentodescribethe
performanceofstudentsinthemiddleofthecategory.However,weactuallywanttofocus
ourattentiononaspecificsubsetofstudentswithineachperformancelevel,thosewho
havejustbarelyentered intotheperformance level.Theyarenotthe typicalexampleof
aProficientstudent,andmaynotbewhatyoupicturewhenaskedtodescribeaProficient
student,buttheydostilljustmeetthecriteriadescribedbythePLD,andareaProficient
student.

There aretwoimportantconceptsthatyouwilldiscussinordertounderstandyour
bookmarktask:
1.Theideaofjustbarelymeetingastandard,orastudentthatisjustbarelydescribed
byaperformanceleveldescriptor,and
2.Acommonunderstandingofwhatmasterymeans.
Wewilldiscusseachoftheseinturn.

C-11

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

You willspendtimeinyourworkshoproomsreviewingthePLDs,andthinkingaboutwhat
knowledgeandskillsstudentsthatarejustbarelydescribedbyPLDhavetohave,andwhat
separatesthemfromstudentswhoarenotdescribedbythePLD.

Toplacebookmarks,youwillfindthelocationintheOIBthatdifferentiatesstudentswho
arejustbarelyfromthosethatarenot.Todothis,youwillevaluatewhetherjust
barelystudentscanrespondsuccessfullytoeachitemintheOIB.

Toframe this,wellthinkabout:
1. Studentswhofallneareachperformancestandardorlevel whatcharacterizesthese
students?
2. Whatdifferentiatesstudentswhojustmeettheperformancestandardfromthosethat
donot whatcantheydo,ornotdo,thatcategorizesthemoneithersideofthe
standard?

Inordertomakethisjudgment,weneedtodevelopacommonunderstandingofwhatit
meanstoperformsuccessfullyonanitem.Whenwesaythatjustbarelystudentscan
performsuccessfullyonanitem,dowereallymeanthatsuchstudentswillalwaysgetthe
itemcorrect?Wedonttypicallyoperateinabsolutes.Studentsdontalwaysgetitems
correct,foravarietyofreasons.Instead,wesaythatstudentsconsistentlyperform
successfullyonitemsortasks.Inasimilarvein,forthepurposeofthisworkshop,wewill
definesuccessfulperformanceasaresponseprobabilityof67%,whichisreferredtoas
RP67,meaningthatwewishtoidentifythelocationintheOIBwherestudentswhoarejust
barelyProficienthavea2/3chanceofrespondingcorrectlytotheitem.Youcanthink
aboutthisasawaytodefinewhatitmeanstosaythatastudentcanreliablyansweran
itemcorrectly theywontalwaysansweritcorrectly,buttheycanreliablyanswerit.

Eachroomwillproduceasummaryofjustbarelyskillsforeachperformancelevel.

Wecanthinkaboutthisconceptintwodifferentways ifyoupictureonejustbarely
student,theyhavea67%chanceofrespondingcorrectlytotheitem.Alternatively,ifyou
visualizeagroupof100justbarely students,twothirdsofthegroupwillrespondtothe
itemcorrectly.
Whenyouplacebookmarks,youwillworkthrougheachpageoftheOIBanddetermine
whether2/3ofjustbarelyProficientstudents,forexample,canrespondsuccessfullytothe

C-12

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

itemoneachpage.Thisjudgmentwillbethebasisforrecommendingabookmark.

IneachroomyouwillreviewtheOrderedItemBookwhichpresentsalongseriesof
items,orderedfromeasiesttomostdifficult.Whilereviewing,rememberyour
focuswillbetodeterminewhatstudentsneedtoknowandbeabletodoinorder
torespondtoeachitemsuccessfully,andwhyeachitemismoredifficultfor
studentsthantheitemsbefore.
Inthecomingdaysyouwillmakeperformancestandardsrecommendationsby
identifyingapagenumberoftheOIBthatwillserveasthecut.
Foreachperformancelevel,youwillworkthroughtheOIBandconsiderwhether
2/3ofjustbarelystudentscanrespondsuccessfullytotheitem.Youwillplace
yourbookmarkonthelastpagewhere2/3ofstudentswhojustbarelymeetthe
performancestandardwillanswercorrectly.Thismeansthatfewerthan2/3ofjust
barelymeetsstudentswouldbeexpectedtorespondsuccessfullytothenextitem
intheOIB.

C-13

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

C-14

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

Youwillalsobepresentedwithimpactdataforeachsubjectafterthe firstroundofcuts.
Thisisthepercentageofstudentswhowouldreachorexceedthestandardbasedonthe
item pageintheOrderedItemBooklet.Withthisinformation,youwillaskyourselfifthe
outcomeseemsreasonable.Whileimpactdatacanbeinformative,placementofyour
bookmarksshouldalwaysbeguidedbycontentconsiderationstoensurethatstudents
meetingtheperformancestandardareaccuratelydescribedbythePLDforeachlevel.

Afteryoucompleteyourinitialbookmarkplacement,youwillbeprovidedfeedbackabout
howotherpanelistsplacedtheirbookmarks.Youwillreceivefeedbackaboutthebookmark
placementsforyourtableandalsoseehowthebookmarkplacementsacrosstables
compare.
Thisdatacanserveasastartfordiscussionaboutbookmarkplacementsandhelppanelists
todevelopcommonunderstandingsofthe skills ajustbarely studenthas,takinginto
accountthevariedbackgroundsandexpertiseofyourfellowpanelists.Onceyouhavehad
achancetoreviewthefeedbackdatawithinyourtable,wewillexpandthediscussionto
othertablesinyourgrade.

Whenyoureceivetheimpactdata,youwillwanttoaskyourselfwhethertheimpactofthe
currentcutscoreplacementseemsreasonableandintheneighborhoodofwhatyouwere
expecting.Iftheimpactdataseemsoutoflinewithyourexpectations,considerwhythat
mightbe.Iftheimpactisthatfewer studentsmeetthestandards thanyouexpected,might
itbe,forexample,thatthenewacademiccontentstandardsaremorerigorousandrequire
studentstodemonstrategreaterknowledgeandskillsthanpreviously?Iftheimpactisthat
morestudents meetthestandardsthanyou expected,coulditbethatyouunderestimated
theknowledgeandskillsthatstudentscanreliablydemonstrate?

Fromthesediscussions,panelistsmayrevisetheirjudgmentsandchoosetomovetheir
bookmarkplacementinRound2,butthereisnoexpectationthatpanelistswillmove
bookmarks.Generally,wedoseeconvergencefromRound1toRound2,butconsensusis
notagoal.

Consideryourcutscore recommendationsinthelightoftheimpactdata,anddiscussthe
implicationsofthecurrentcutscoreplacementswithyourfellowpanelists.Remember,
whileyoumaychoosetomodifyyourcutscoresinlightoftheimpactdata,yourrationale
formakingeachcutscoreshouldbebasedoncontentconsiderations.

C-15

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

Whenwetalkaboutstandardsbeingarticulatedacrossgrades,werefertotheideathat
thereshouldnotbewidefluctuationsintheproportionofstudentsmeetingeach
performancestandardacrossgrades.Itisunlikely,forexample,thatif60%ofGrade4
studentsareconsideredtohaveachievedendofyearstandardsforreadingandare
academicallypreparedtobenefitfromGrade5readinginstruction,thatonly40%ofGrade
5studentsmeetendofyearstandardsinreading.
WhilethisverticalarticulationisincorporatedintothedevelopmentoftheArizona
AcademicContentStandardsaswellasthetestspecificationsfortheAzMERIT
assessments,maintainingthecrossgradearticulationinthesettingofmeaningful
performancestandardsisimportant,especiallyforreadingandmathematics,where
studentsareassessedannually.Lackofarticulationinthesesubjectscanresultin
confusion,withunreasonablylargeshiftsinstudentperformancelevelclassifications
occurringfromyeartoyear,resulting,forexample,inwidespreadmisidentificationofpoor
performingteachersacrossgradelevelswithinschools.
Forthisreason weconductmoderationsessions.

C-16

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

WewillcalculateproposedAdjacentGradePerformanceStandardscutscoreswillbe
presentedtobeforebeginningtheBookmarkplacementactivitiesfortheadjacentgrades.

Afteryourpanelsrecommendperformance standardsintheinitialgrade,tableleaderswill
beaskedtoparticipateinaModerationsession.Thisactivitygivesanopportunityfor
representativestoreviewtherecommendedstandardsacrossgradesinlightofthe
discussionofcontentdemandsandtherelativeimpactofstudentperformanceacross
grades.Allpanelistsareinvitedtositin,buttableleaderswillbeaskedtomake
recommendationformoderatingtherecommendations.Tableleaderswillrepresenttheir
tablesviewsinthediscussion.
Moderationwillservetwopurposes
1) Providingabroaderviewofrecommendationsandanopportunityforpaneliststo
benefitfromthedeliberationsandexperiencesofothergradelevelpanels
2) Produceasetofreadingperformancestandardsthatarearticulatedacrossgrades.
Similartocontentstandardsbeingarticulatedacrossgrades,werefertotheideathatthere
shouldnotbewidefluctuationsintheproportionofstudentsmeetingeachperformance
standardacrossgrades.Itisunlikely,forexample,thatif60%ofGrade4studentsare
consideredtohaveachievedendofyearstandardsforreadingandareacademically
preparedtobenefitfromGrade5readinginstruction,thatonly40%ofGrade5students
meetendofyearstandardsinreading.
Forthisreason,weconvenetableleaderstoreviewthestandardsinlightofeachindividual
panelsexpertjudgment,aswellasallpanelsjudgmentstogether.Wewillmoderate
standardsatthisstage,andagainattheendoftheworkshop.

C-17

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

C-18

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.1. Large Group Training Presentation and Script (Grades 3-8)

Thisconcludesourlargegroup trainingsession.Pleasebreakintoyourassignedgroups.
Yourpanelassignmentsshouldbeincludedinyourfoldersaswellasroomnumbers,which
arealsocurrentlydisplayedonthescreen.PleaselocateanAIRorADEemployee,as
indicatedbyourbadge,ifyourequireanyassistance.Thankyou.

C-19

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Note for facilitator: Introduce workshop staff, and have panelists introduce
themselves to the group. Encourage panelists to share names, school
district/region, and what grades and subjects each panelist works with.
Lets go around the room and introduce yourself, and share what school
district or area in the state you are coming from, and what students you
primarily work with.

C-20

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

On your table, there is a non-disclosure form. The first order of business is


to sign those and hand them in.

Facilitator Note: Please stress the importance of security. These are


operational items that will be used on future administrations. Absolutely no
picture taking, cell phones are to be put away and not out at the table, no
texting while working. We understand emergencies may happen, please take
all calls outside the meeting room. Do not surf other websites while using the
laptops.
We cant stress the importance of security enough. Well be working with
live test items that will be administered to students again in the future, and
its important from a test validity perspective that items remain secure. That
means that we ask that you keep cell phones, tablets, laptops away, and
step outside if you need to take a call.

C-21

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Notes to elaborate on:

You will now have the opportunity to take a test that was administered to
students this spring.

You will experience a subset of items administered in spring 2015


The interface is almost identical to the online test environment that the
student view and experience

Note: Secure browser should be deployed on each panelists computer.


Additional workshop staff will circulate rooms to expedite log-in process.

You cannot see your scores for hand scored items because they are
scored at a later time
There is only an hour reserved for experiencing the Online assessment
The purpose is not to complete the test, but to get an idea of what the
students experienced

C-22

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Select Grade 4 Math test


The AzMERIT test was administered in parts. If you complete part 1, you will
submit your test (remember this is just to get a feeling of what a student
would experience) then be brought to the login screen where you will follow
the same steps to access part 2.

C-23

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Once you are in the testing environment, work through the test, and take a
break as needed. At around 11am, we will move on to the next activity as a
group.

C-24

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

The High School panels have started and set standards for their grade
bands 9 - 11

The standard setting process that we will be engaged in during the next two
days is designed to translate the Arizona State standards in Math 3-4 into a
set of performance standards, or cut scores, on each of the assessments.
Two important documents, the Ordered Item Booklet and the Performance
Level Descriptors, will be your primary tools for translating the academic
content standards into performance standards.
Throughout this workshop, we will refer to different types of standards.
Arizona State standards specify what students should know and be able to
do by end of each academic year.
You will use two tools, OIB and PLDs

C-25

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

The Ordered Item Book, or OIB, is the other primary tool you will use. The
OIB contains operational test items administered in Spring 2015 ordered
from easiest to most difficult. Each page is a score point on an item.
Certain items appear multiple times in the OIB, once for each score point.
The number of pages in the book is equal to the number of points in the OIB,
not the number of items on the OIB.

Performance Level Descriptors will serve as your guide for identifying


performance standards. The PLD is a summary of what students within each
achievement level are expected to know and be able to do. The PLDs are a
link between content and performance standards. This means when you
recommend a standard, you are asserting that students who meet that level
of performance fit the description of the student provided in the PLDs. This
link is critical because it allows teachers and families to understand what a
students test results mean.

You will identify how much a student should know and be able to do to meet
the description for each performance level in the PLDs by placing a
bookmark in the OIB that divides the book into two groups: items that
students described by the performance level descriptor can respond
successfully to, and items that students in that performance level cannot
respond successfully to
You will have two different opportunities to make individual recommendations
youll make an initial judgment, and then you will receive feedback showing
the bookmarks of your fellow panelists. Well discuss everybodys
bookmarks, and well also look at the percentage of students in the state
who would meet or exceed each of the recommended standards, or impact.
Then, youll make individual recommendations again you can change your
bookmarks, but you dont have too.

C-26

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Performance levels are regions on the performance scale demarcated by the


performance standards. They classify students by how much of the content
standards they know and are able to do. The three performance standards
will result in four performance levels.

We can compare the Performance level descriptors across Performance


levels for each content standard.
Talk about content limits and verbs.

C-27

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Even within each performance level, students vary in the degree to which
they have mastered the Arizona State standards. Some students have just
barely crossed the line between Partially Proficient and Proficient, while
others are getting ready to cross the line between Proficient and Highly
Proficient. In general, the PLDs are written to describe the performance of
students in the middle of the category. However, we actually want to focus
our attention on a specific subset of students within each performance level,
those who have just barely entered into the performance level.
To frame this, well think about:
1. Students who fall near each performance standard what characterizes
these students?
2. What differentiates students who just meet the performance standard
from those that do not what can they do, or not do, that categorizes
them on either side of the standard?
3. Descriptions of how much of the content standards students who just
barely meet the performance standard have to know and be able to do in
order to be categorized in each performance level.

C-28

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

When thinking about what a just barely Proficient student can do, ensure you are
not describing a Partially Proficient student.

Facilitator start with the reporting category that you made examples for
and ask panelists to review that reporting category individually then as a
group write just barely statements
Choose a reporting category you feel comfortable with to do as a group
As we go through and review the PLDs we will think about what high level
skills are necessary for a student just entering each performance level.
Remember, the student has just barely crossed into the performance level.
They demonstrate just enough to be considered Partially Proficient,
Proficient or Highly Proficient.
Lets review the PLDs in the BLANK reporting category as a group to come
up with a summary of the overarching skills necessary for the just barely
students in each performance level.
Then we will assign different strands to each table to produce their own just
barely summary statements, then come back together as a group and share
what each table has produced and discuss any questions.

C-29

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

This activity will be completed by each individual table. Each table will be
assigned 1 to 2 reporting categories. The table leader will type a few
statements for each reporting category for each just barely performance
level.

C-30

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Discuss the just barely summaries across tables.


Encourage the tables to take notes on these documents and ask questions if
they do not understand or agree with the descriptions.
Everyone should be on the same page in understanding the skills of just
barely students.
**Ensure discussion is moving along and not stuck on one particular strand
Facilitator 15 minute break for panelist 2:30-2:45

C-31

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Now well turn our attention to your next task: review the Ordered Item Book.

OIB is the other primary tool you will use. The Ordered Item Booklets will
present you with the items from the spring 2015 assessments. The content
of the items is proportional to the test blueprint.

The Ordered Item Booklets will present you with a subset of items from the spring
2015 assessments. All items will be presented in order of difficulty; page 1 will
present the easiest item, and the last page will present the hardest item. Each
page in the ordered item booklet refers to an item.

The OIB consists of all operational items as well as 15-20 field test items to
fill in information gap. What that means is that we added items where there
was a large gap in item difficulty between operational items so that the items
in the OIB appear more fluid in terms of difficulty.

Typically, the OIB has been presented literally as a bound paper book. However,
because the AzMERIT includes technology enhanced items that cannot be
represented well on paper, you will use an electronic OIB. The OIB is ordered from
the easiest to most difficult item, so the first item is the easiest, and the last item is
the most difficult. You page forward to see increasingly more difficult items, and you
can page backward to see progressively easier items. You will page through the
OIB from easiest to the most difficult item. For each item, you will ask what students
need to know and be able to do to respond successfully to the item, and what
makes this item more difficult than the preceding items.

All items will be presented in order of difficulty; page 1 will present the
easiest item, and the last page will present the hardest item. Some items will
be represented more than once. These items are more engaging than others
and are worth more than a single score point. Each page in the ordered
item booklet will refer to specific score point of the item. We will review the
series of items in an online environment.

You can use the accompanying OIB map to keep notes.


Next, we will review:
1. What to consider when reviewing the OIB
2. How to log into the ITS and access the OIB, and how to navigate through each
page of the OIB
3. How your OIB Map corresponds to the OIB and your review of each item
Upon arrival tomorrow, you will get started on review of the OIB.

C-32

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

On the table are the public test blueprints which can also be found on ADEs
website. All tests administered to students met these blueprint requirements.
The OIB, which contained all operational items and were augmented with
field test items to fill in gaps, meets or very closely meets the public
blueprints.

When studying the OIB, it is important to understand the difference between


items in context of the whole OIB. Items that appear earlier in the book are
easier, despite possible perceptions otherwise, than items that appear later
in the book.
For each item, ask yourself two questions:
1) What do students need to know and be able to do to respond
successfully to this item?
2) Why is this item more difficult than the previous items?

C-33

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

The difficulty of these items is based on students performance during the


2015 operational assessment. So while a particular item may seem easier
or harder to you, the placement in the ordered item book reflects how easy
or hard it actually was for Arizona students to perform successfully on these
items.
If an item seems out of order to you, remember that an item may not
measure what you think it measures. For example, an item may intend for a
student to have to know a particular piece of information, but perhaps the
students were able to answer using recall from a lesson that was taught
recently.
Instead of focusing on one item that may seem out of place to you, try to
identify natural breaks or thresholds for groups of items.

C-34

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

C-35

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

The item plot shows a graphical representation of the difficulty of each page
in the OIB. This is just another way to view the OIB.

The item map will guide your review of the OIB.


Remember that items are presented in order from easiest to most difficult.
Each page number represents one item.

Point out NAEP benchmarks.

The ITS ID is shown on both the item map and in your online OIB. You
can jump to an OIB page by selecting the ITS ID from the dropdown in the
top right portion of the screen.

You can use this map to see where pages are clustered together, versus
spread out, in terms of the ability level they correspond to. The clustered
pages indicates that the difference in performance impact between adjacent
pages is smaller, whereas difference in bookmarks on non-clustered
adjacent pages may be more substantial. The item numbers correspond to
the page numbers that are included in the OIB map.

The AzCCRS column shows you each items alignment to the AzCCRS.
This corresponds to the content specified within the tests blueprint.
Item format displays whether an item is multiple choice, indicated by
MC, or an item that requires a student to construct a response, such as
GI meaning grid item or EQ meaning equation. As you review the
OIB, items that are not multiple choice will contain instructions on how
students are to answer.
Space for notes as you review each item in the OIB, remember to think
about two questions 1) What do students need to know and be able to
do in order to respond successfully to this question?, and 2) Why is this
question more difficult than the one before?

C-36

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

FACILITATORS: Move to step review of OIB

1. Panelist name should appear


2. You will see the title of the grade/subject you are working on

Facilitators username is firstname_lastname and your personal password


is written on your agendas.

3. Step should say Review of Ordered Item Booklet


4. Page forward/backward

Walk through the different Review Panel options


More about this item
Notes
Marks
Impact
Feedback
Moderation
Prior Feedback

C-37

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

(Leave slide up while panelists review OIB)


You have Writing Rubrics and Writing Anchor books that provide examples of
responses for each point and dimension.

C-38

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

C-39

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Panelist computers will be logged into the ITS and OIB prior to meeting start.
Once panelists arrive, they can immediately start re-reviewing the OIB,
checking their notes and filling in any notes they missed. As needed, assist
panelists with logging into ITS and display password at front of room. Once
all panelists arrive, review the activities for the morning.

To place bookmarks, you will find the location in the OIB that differentiates
students who are just barely from those that are not. To do this, you will
evaluate whether just barely students can respond successfully to each
item in the OIB.
In order to make this judgment, we need to develop a common
understanding of what it means to perform successfully on an item. When
we say that just barely students can perform successfully on an item, do
we really mean that such students will always get the item correct? We
dont typically operate in absolutes. Students dont always get items
correct, for a variety of reasons. Instead, we say that students consistently
perform successfully on items or tasks. In a similar vein, for the purpose of
this workshop, we will define successful performance as a response
probability of 67%, which is referred to as RP67, meaning that we wish to
identify the location in the OIB where students who are just barely Proficient
have a 2/3 chance of responding correctly to the item. You can think about
this as a way to define what it means to say that a student can reliably
answer an item correctly they wont always answer it correctly, but they
can reliably answer it.

We will spend the morning working through the Ordered Item Book.
Following review of the OIB, well discuss how to locate the just barely in
the OIB and recommend cut scores. We will recommend cut scores and
spend time reviewing feedback discussion our recommendations as a group
before making another round of recommendations. Follow the second round
of recommendations, you will receive performance information such as the
percent of students estimated to meet performance standards based on the
rooms median recommended bookmark page numbers. We will discuss the
implications of the performance information, and you will again make a final
round of recommendations.
Anchor Grade Moderation is required for Table Leaders but all other
panelists are invited to attend.

We can think about this concept in two different ways if you picture one

C-40

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

just barely student, they have a 67% chance of responding correctly to the item.
Alternatively, if you visualize a group of 100 just barely students, two thirds of the
group will respond to the item correctly.
When you place bookmarks, you will work through each page of the OIB and
determine whether 2/3 of just barely Proficient students, for example, can respond
successfully to the item on each page. This judgment will be the basis for
recommending a bookmark.

The OIB is ordered from easiest to most difficult. This fosters an integrated
concept of how the test reflects the performance standards. The OIB is the
vehicle to make cut score judgments and communicates how the trait
increases in difficulty as items ascend the scale.

C-41

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

We will use a Bookmark Placement Sheet to submit recommendations. You


will write the page number for the recommended cut score for all three
different performance standards Partially Proficient, Proficient, and Highly
Proficient

Yesterday,wereviewedtheOrderedItemBookwhichpresentedalongseriesof
items,orderedfromeasiesttomostdifficult.Whilereviewing,rememberour
focuswasdeterminingwhatstudentsneedtoknowandbeabletodoinorderto
respondtoeachitemsuccessfully,andwhyeachitemwasmoredifficultfor
studentsthantheitemsbefore.

In Round 1, you will make your initial recommendations, and write in your
panelist ID in the appropriate box on the form before turning it in. You will
receive the form back and use the same form to make recommendations in
subsequent rounds.

Today,wewillmakeperformancestandardsrecommendationsbyidentifyinga
pagenumberoftheOIBthatwillserveasthecut.

For example, in the starred box, you write your Round 1 recommended page
number for the Proficient cut. You will identify the last page in the OIB that
2/3 of just barely Proficient students can successfully respond to; fewer
than 2/3 of just barely students will be able to successfully respond to the
very next item.

Foreachperformancelevel,youwillworkthroughtheOIBandconsiderwhether
2/3ofjustbarelystudentscanrespondsuccessfullytotheitem.Youwillplace
yourbookmarkonthelastpagewhere2/3ofstudentswhojustbarelymeetthe
performancestandardwillanswercorrectly.Thismeansthatfewerthan2/3ofjust
barelymeetsstudentswouldbeexpectedtorespondsuccessfullytothenextitem
intheOIB.

In the Partially Proficient box, you will write the page number of the last page
in the OIB that 2/3 of students of just barely Partially Proficient students
can respond successfully to. Fewer than 2/3 of those students who just
barely approach Partially Proficient will be able to respond successfully to
the next item.

Recordthebookmarkedpagenumberinthebookmarkplacementsheet.

C-42

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

And finally, in the Highly Proficient box, you will write the page number of the last
page in the OIB that 2/3 of the students that just barely Highly Proficient students
can respond successfully to.
Remember that you will have an opportunity to discuss your recommendations
among your group after everyone has completed this task, and you will have a
chance to then change your recommendation.

It is important to understand that a page number in the OIB does not equate
to a number of items a student must get correct to meet a standard at that
bookmark. There is no relationship between the pages in the OIB and the
number of points needed to achieve a standard.

C-43

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Next, were going to complete a practice worksheet; the goal is to assess


whether the training so far has clearly explained the mechanics of the
bookmark procedure. This will help to identify which concepts that need
additional clarification before we place our bookmarks. Please take a few
minutes to review the worksheet, and then we will review it as a group.

As necessary, review that just barely means they are just barely
categorized as being described by the performance level descriptors. Key
idea: These are not the average student.

(Give panelists time to complete worksheet. Then walk through worksheet


and discuss results, and identify which concepts panelists do not grasp yet.)

C-44

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

The item on page 6 is MORE DIFFICULT than the item on page 5. Review
with panelists as necessary:
- Items are presented from easiest to most difficult.
- Difficulty is based on student performance on 2015 operational
assessments. It is not based on test developers judgments.

C-45

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Review that bookmark should be placed on the last page where 2/3 of
students described by the just barely PLD can be expected to respond
successfully to the item. On the following page, less than 2/3 of just barely
students would respond successfully. (Or just barely students would
respond successfully less than 2/3 of the time).

C-46

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Remember, we will focus our attention on a specific subset of students within


each performance level, those that just barely within the performance level.
Based on your parsing of the PLDs, you all have a list of the characteristics
that differentiate students who are just barely within the performance level
from those who are below the performance level, and a descriptor for
students that just barely meet the performance levels.
In addition, it is important that we define successful performance on test
items uniformly. For purposes of this workshop, we define successful
performance on an item as a response probability of 67%. We wish to
identify the location in the OIB where students who are just barely within
the performance level have a 2/3 chance of responding correctly to the item.
We can think about this concept in two different ways if you picture one
just barely student, they have a 67% chance of responding correctly to the
item. Alternatively, if you visualize a group of 100 just barely students, two
thirds of the group will respond to the item correctly.
Remember that the page numbers in the OIB have no relationship to the
number or percent of items that at student has to perform successfully on in
order to meet the recommended standard.

C-47

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Remove the readiness form from your folders. This form should say
Preparation for Round 1 Partially Proficient, Proficient, and Highly
Proficient near the top.
Review the specific bullets on the form, and please indicate whether you feel
you understand, and are ready to place your bookmark. If you answer No
to any questions, please notify a workshop staff member before continuing.
Lets take a minute to complete this form. Please turn them into your table
leaders.
(Note: In the event that a panelist indicates they are not ready, the facilitator
will work with the individual(s) to ensure they understand the procedures.)

C-48

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Note: Continue to display this slide while panelists are placing their bookmarks.

Remember, you are seeking to divide the OIB into two sets of items, so
dont get hung up if you find what appears to be a particularly difficult item in
the middle of otherwise relatively easy items. You are seeking to identify the
set of items that students who just barely within the performance level can
respond to successfully from those items that students who may not meet
the standards can also respond to successfully. You will identify the last page
in the OIB that 2/3 of the just barely students at the performance level can
successfully respond to; fewer than 2/3 of just barely students will be able
to successfully respond to the very next item.
It is important that everyone start with the Proficient standard, then move to
the Partially Proficient standard and finish your bookmark placement with the
Highly Proficient standard.
When you have completed placing your bookmarks and have initialed the
bookmark placement sheet, please hand it to your table leader. Table
leaders, when you have all of your tables sheets please alert myself or the
room assistant.
We will take a short 15 minute break after everyone has completed placing
their bookmarks.

C-49

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

C-50

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

NOTE: Facilitate discussion within the room. Panelists should share what
knowledge and skills required by the items or the PLDs led to
recommendations.

From these discussions, you may revise your judgment about the bookmark
placement and choose to move your bookmark placement in Round 2.
However, there is no requirement or expectation that you will move your
bookmarks. From experience, we do expect convergence from Round 1 to
Round 2, but consensus is not a goal.
This can serve as a good jumping off point for discussion; for example,
panelists who placed their bookmarks at the lowest and highest pages can
share why they felt those were the right cut points. After discussing the
feedback with your table members, you will discuss your bookmarks with the
room as a whole.
NOTE: Have panelists discuss their findings within their tables. They should
be able to see the cuts set by the other panelists at their table, all of the
table medians and room medians.

C-51

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

You will also be presented with impact data for each subject. This is the
percentage of students who would reach or exceed the standard based on
the item page in the Ordered Item Booklet. With this information, you will
ask yourself if the outcome seems reasonable. While impact data can be
informative, placement of your bookmarks should always be guided by
content considerations to ensure that students meeting the performance
standard are accurately described by the PLD for each level.

C-52

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

C-53

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

C-54

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

NOTE: Collect readiness forms with Round 2 initialed by panelists

C-55

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Each table will be assigned 1 to 2 reporting categories. The table leader


should type a few statements for each reporting category for each Just
Barely performance level.

Discuss the just barely summaries across tables.


Encourage the tables to take notes on these documents.
Everyone should be on the same page in understanding the skills of just
barely students.

C-56

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Only the Table Leaders are required to participate in the anchor grade
vertical moderation but all other panelists are invited to sit in.

C-57

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

(LeaveslideupwhilepanelistsreviewOIB)
Rememberthat
EachOIBconstitutesanaugmentedtest
Pagesareorderedbydifficulty
Eachpageisascorepointonanitem
Multipointitemsappearmultipletimes(onceforeachscorepoint)
Itemorderisbasedonstudentperformance
Itemsmayseemoutoforderbecausetheyareorderedbydifficultynotby
contentorcognitiveprocess
Ifyoubelievesomethingiswrongwithanitem,telltheworkshopleader,
thenskipovertheitemasyoureviewtherestoftheOIB

C-58

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Following recommendation of initial grade performance standards in anchor


grades table leaders were convened to engage in a review and moderation
of the initial recommendations. The purpose of the review was to ensure
that the standard setting workshops produce a system of cut scores that are
coherent across grade levels. Prior to moderation, all panelist deliberations
have been focused on placement of cut scores for a single grade. The
panelists who participated in the moderation were asked to recommend
adjustments. We will provide you with the recommendations resulting from
the moderation session.

C-59

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

NOTE: Debrief panelists on moderation activities and results: initial


recommendations, major discussion, and resulting changes in anchor grade
bookmarks

C-60

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

C-61

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

C-62

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

C-63

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

C-64

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Insert adjacent grade page numbers. Have panelists mark this page on their
Item Map.

C-65

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Note: Continue to display this slide while panelists are placing their
bookmarks.
After bookmark placement, we will break for lunch at 12:00

C-66

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

C-67

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

NOTE: Facilitate discussion within the room. Panelists should share what
knowledge and skills required by the items or the PLDs led to
recommendations.

From these discussions, you may revise your judgment about the bookmark
placement and choose to move your bookmark placement in Round 2.
However, there is no requirement or expectation that you will move your
bookmarks. From experience, we do expect convergence from Round 1 to
Round 2, but consensus is not a goal.

C-68

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

C-69

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

C-70

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Presentation C.2. Sample Workshop Presentation and Script (3-4 Math)

Following this meeting, another panel will convene that includes the table
leaders. The purpose of this final meeting is to allow table leaders to review
the system of standards as a whole so that they can review the
appropriateness of the recommended cut scores as they relate to the
performance level descriptors and their impact on students who meet each
performance standard.

Thank you for your participation and hard work these past two days. Please
remember to keep the content of the test items and the discussions about
specific recommendations secure, and please feel free to share information
about the standard setting process and your experience with colleagues and
other individuals.
Please take your time to fill out the workshop evaluation. We know that you
all often have to fill out evaluations to conclude meetings or trainings, but
please take the time to thoughtfully fill out the evaluation. The results of the
evaluation will be included in the Standard Setting Technical Report that will
be available to the public. Filling out the evaluation also provides us with
feedback to improve the standard setting process and to assure the validity
of the whole process.

C-71

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Appendix D Performance Level Descriptors

Appendix

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD

EnglishLanguageArts
Grade3

Standard Setting Technical Report

Standard

MinimallyProficient
Forgradeappropriate,low
complexitytexts,theMinimally
Proficientstudent

Detailed

3.RL.2

identifiesdetailsthatrecount
identifieskeydetailsthatrecount
stories;identifiesexplicitlystated stories;determinescentral
messages,lessons,ormoral.
centralmessages,lessons,or
moral.

Detailed

3.RL.3

identifiesbasicelements(e.g.,traits, describescharactersinastory(e.g., describescomplexelements(e.g.,


identifiesbasicelements(e.g.,
traits,motivations,orfeelings)of motivations,orfeelings)of
traits,motivations,orfeelings)and traits,motivations,orfeelings)of
charactersinastoryandexplains
charactersinastory.
explainshowtheiractionscontribute complexcharactersinastoryand
explainshowtheiractionscontribute
howtheseelementscontributeto tothesequenceofevents.
toacomplexsequenceofevents.
thestory.

Detailed

3.RL.4

Detailed

3.RL.5

useseasilylocated,explicitly
stateddetailsinorderto
determinethemeaningsof
familiarwordsandphrasesas
they are used in a text.
referstoeasilyidentifiedpartsof
stories,dramas,andpoems,
usingtermssuchaschapter,
scene,andstanza.

Detailed

PartiallyProficient
Proficient
HighlyProficient
Forgradeappropriate,lowto
Forgradeappropriate,moderateto Forgradeappropriate,high
moderatecomplexitytexts,the
highcomplexitytexts,theProficient complexitytexts,theHighly
Proficientstudent
PartiallyProficientstudent
student
Reading:Literature
asksandanswersexplicitquestions asksandanswersquestionsto
3.RL.1 asksandanswersquestionsto
asksandanswerscomplexquestions
demonstrateunderstandingofa todemonstrateunderstandingofa demonstrateunderstandingofatext, todemonstrateunderstandingofa
text,referringtothetextasthebasis referringexplicitlytothetextasthe text,referringexplicitlytothetextas
text.
foranswers.
thebasisforanswersandmaking
basisforanswers.
inferenceswherenecessary.

AzMERITPerformanceLevelDescriptors

recountsstories,includingfables,
folktales,andmythsfromdiverse
cultures;determinesthecentral
message,lesson,ormoraland
explainshowitisconveyedthrough
keydetailsinthetext.

usesdetailsfromthetextinorderto determinesthemeaningofwords
determinethemeaningofwordsand andphrasesastheyareusedina
text,distinguishingliteralfrom
phrasesastheyareusedinatext.
nonliterallanguage.

provideskeydetailsthatcompletely
recountstories;determinesimplicitly
statedcentralmessages,lessons,or
morals;andexplainshowtheseare
conveyedthroughkeydetailsinthe
text.

determinesthemeaningofunfamiliar
wordsandphrasesastheyareused
inatext,distinguishingliteralfrom
nonliterallanguage.

referstopartsofstories,dramas,
referstopartsofstories,dramas,
referstointricatepartsofstories,
andpoems,usingtermssuchas
andpoemswhenwritingorspeaking dramas,andpoemswhenwritingor
chapter,scene,andstanza;identifies aboutatext,usingtermssuchas
speakingaboutatext,usingterms
howonepartbuildsonanearlier
chapter,scene,andstanza;describes suchaschapter,scene,andstanza;
section.
howeachsuccessivepartbuildson explainshoweachsuccessivepart
earliersections.
buildsonearliersections.

D-1

AzMERIT - 2015

EnglishLanguageArts
Grade3

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
Detailed 3.RL.6 identifiesthepointsofviewof
distinguisheshisorherownpointof distinguisheshisorherownpointof distinguisheshisorherownpointof
viewfromimplicitlystatedpointsof
thenarratororcharacters.
viewfromexplicitlystatedpointsof viewfromthatofthenarratoror
thoseofthecharacters.
viewofthenarratororcharacters.
viewofthenarratororthoseofthe
characters.
Detailed

3.RL.7

Detailed
Detailed

3.RL.8
3.RL.9

usesspecificaspectsofatext's usesspecificaspectsofatext's
simpleillustrationstounderstand illustrationstounderstandthetext
thetextandidentifiesexplicit
andmakeslowerlevelinferences
detailsabouthowthe
abouthowtheillustrationsreflect
illustrationsreflectcharacters,
characters,setting,ormood.
setting,ormood.

analyzeshowspecificaspectsofa
explainshowspecificaspectsofa
text'sillustrationscontributetowhat complextext'sillustrationscontribute
isconveyedbythewordsinastory toamorethoroughunderstandingof
(e.g.,emphasizeaspectsofa
thetext;makeshigherlevel
characterorsetting,createmood). inferencesabouthowthe
illustrationsreflectcharacters,
setting or mood
N/A
N/A
N/A
N/A
describesexplicitlystatedthemes, comparesandcontraststhethemes, comparesandcontrastshighly
identifiessimpleandexplicit
settings,andplotsofstorieswritten settings,andplotsofstorieswritten complex,implicitlystatedthemes,
themes,settings,andplotsof
bythesameauthoraboutthesame bythesameauthoraboutthesame settings,andplotsofstorieswritten
storieswrittenbythesame
authoraboutthesameorsimilar orsimilarcharacters(e.g.,inbooks orsimilarcharacters(e.g.,inbooks bythesameauthoraboutthesame
orsimilarcharacters(e.g.,inbooks
fromaseries).
characters(e.g.,inbooksfroma fromaseries).
series).
fromaseries);makesinferencesto
identifysupportusedbyauthors.

AzMERITPerformanceLevelDescriptors

D-2

AzMERIT - 2015

EnglishLanguageArts
Grade3

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
Reading:InformationalText
asksandanswerscomplexquestions
asksandanswersexplicitquestions asksandanswersquestionsto
Detailed 3.RI.1 asksandanswersquestionsto
demonstrateunderstandingofa todemonstrateunderstandingofa demonstrateunderstandingofatext, todemonstrateunderstandingofa
text,referringtothetextasthebasis referringexplicitlytothetextasthe text,referringexplicitlytothetextas
text.
basisfortheanswers.
foranswers.
thebasisforanswersandmaking
inferenceswherenecessary.
Detailed

3.RI.2

Detailed

3.RI.3

Detailed

3.RI.4

Detailed

3.RI.5

Detailed

3.RI.6

determinesanimplicitlystatedmain
ideaofatext;recountskeydetails
andexplainshowtheysupportthe
main idea.
describestherelationshipbetweena analyzescomplexrelationships
describessimplerelationships
identifieshistoricalevents,
scientificideas,orsomestepsin betweenhistoricalevents,scientific seriesofhistoricalevents,scientific betweenaseriesofhistoricalevents,
ideasorconcepts,orstepsin
scientificideasorconcepts,orsteps
technicalproceduresinatext,
ideasorconcepts,orstepsin
usinglanguagewithanattempt technicalproceduresinatext,using technicalproceduresinatext,using intechnicalproceduresinatextwith
vaguelanguagethatpertainsto
immergingapplication,using
attimeorsequence.
languagethatpertainstotime,
time,sequence,andcause/effect.
sequence,andcause/effect.
academiclanguagethatpertainsto
time,sequence,andcause/effect.
determinesthemainideaofatext; determinesthemainideaofatext;
identifiesanexplicitlystated
mainideaofatext;identifieskey identifieskeydetailstorecountthe recountskeydetailsandexplains
howtheysupportthemainidea.
detailstorecountthemainidea. mainidea.

useseasilylocated,explicitly
stateddetailsinorderto
determinethemeaningofbasic
academicanddomainspecific
wordsandphrasesinatext
relevanttoagrade3topicor
subject area
usesbasictextfeaturesand
searchtools(e.g.,keywords,
sidebars,hyperlinks)tolocate
informationexplicitlystatedin
thetext.

usesdetailsfromthetextinorderto determinesthemeaningofgeneral
academicanddomainspecificwords
determinethemeaningofbasic
academicanddomainspecificwords andphrasesinatextrelevanttoa
andphrasesinatextrelevanttoa grade3topicorsubjectarea.
grade3topicorsubjectarea.
usesbasictextfeaturesandsearch
tools(e.g.,keywords,sidebars,
hyperlinks)tolocateinformation
relevanttoagiventopic.

determinesthemeaningofadvanced
academicanddomainspecificwords
andphrasesinatextrelevanttoa
grade3topicorsubjectarea.

usestextfeaturesandsearchtools usescomplextextfeaturesand
(e.g.,keywords,sidebars,hyperlinks) advancedsearchtools(e.g.,key
tolocateinformationrelevanttoa words,sidebars,hyperlinks)to
analyzeandinterpretinformation
giventopicefficiently.
relevanttoagiventopicefficiently.

identifiesthepointofviewofthe distinguisheshisorherownpointof distinguisheshisorherownpointof distinguisheshisorherownpointof


viewfromanimplicitlystatedpoint
authorofatext.
viewfromanexplicitlystatedpoint viewfromthatoftheauthorofa
text.
ofviewoftheauthorofatext.
ofviewoftheauthorofatext.

AzMERITPerformanceLevelDescriptors

D-3

AzMERIT - 2015

EnglishLanguageArts
Grade3

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
usesinformationgainedfromsimple
Detailed 3.RI.7 identifiesinformationgained
illustrations(e.g.,maps,
fromsimpleillustrations(e.g.,
photographs)andwordsinatextto
maps,photographs)andthe
explicitstatementswithinatext demonstrateunderstandingofthe
todemonstrateunderstandingof text(e.g.,where,when,why,and
howkeyeventsoccur).
thetext.

Proficient
HighlyProficient
usesinformationgainedfrom
analyzesinformationgainedfrom
complexillustrations(e.g.,maps,
illustrations(e.g.,maps,
photographs)andthewordsinatext photographs)andtheinferences
todemonstrateunderstandingofthe withinatexttodemonstrate
understandingofthetext.
text(e.g.,where,when,why,and
howkeyeventsoccur).
describesthelogicalconnections
betweenparticularsentencesand
paragraphsinatext(e.g.,
comparison,cause/effect,
first/second/thirdinasequence).

Detailed

3.RI.8

identifiesthesimpleconnections identifiesthelogicalconnections
betweenparticularsentencesina betweenparticularsentencesand
paragraphsinatext(e.g.,
text(e.g.,comparison,
cause/effect,first/second/thirdin comparison,cause/effect,
first/second/thirdinasequence).
asequence).

Detailed

3.RI.9

describesthemostimportantpoints comparesandcontraststhemost
identifiesthemostimportant
pointsandkeydetailspresented andkeydetailspresentedintwo
importantpointsandkeydetails
textsonthesametopic.
inatext.
presentedintwotextsonthesame
topic.

AzMERITPerformanceLevelDescriptors

D-4

Standard Setting Technical Report

describesthecomplexconnections
betweenparticularsentencesand
paragraphsinatextusingtextual
evidence(e.g.,comparison,
cause/effect,first/second/thirdina
sequence).
comparesandcontraststhemost
importantpointsandkeydetails
presentedintwotextsonthesame
topicandprovidestextualevidence
tosupportthesecomparisons.

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Writing
Detailed 3.W.1
writesanopinionpiecethatlacks
organization,doesnotincludean writesalooselyorganizedopinion
piecewithasimpleintroductionand
introductionorconclusionor
includesanineffectiveone,and conclusion,andprovideslimited
reasonstosupporttheopinion.
providesirrelevantreasonsto
supporttheopinion.
Detailed

Detailed

Detailed

Standard Setting Technical Report

EnglishLanguageArts
Grade3
Proficient

HighlyProficient

writesawellorganized,multi
writesawellorganizedopinionpiece
paragraphopinionpiecethat
thatintroducesthetopic,provides
effectivelyintroducesthetopic,
reasonsthatsupportthe
providesevidencethateffectively
opinion,useslinkingwordsand
supportstheopinion,useslinking
phrases,andprovidesaconcluding
wordsandphrases,andprovidesan
statement.
effectiveconcludingstatement.
writesawellorganized,multi
3.W.2 writesanexplanatorypiecethat
writesawellorganizedexplanatory paragraphexplanatorypiecethat
writesalooselyorganized
lacksorganization;doesnot
piecethatintroducesthetopic;
effectivelyintroducesthetopic;
explanatorypiecewithasimple
includeanintroductionor
providesfacts,definitions,and
providesfacts,definitions,anddetails
introductionandconclusion;and
conclusionorincludesan
detailstosupportthetopic;uses
thateffectivelysupportthetopic;
provideslimitedfacts,definitions,
ineffectiveone;andprovides
linkingwordsandphrases;and
useslinkingwordsandphrases;and
irrelevantfacts,definitions,and anddetailstosupportthetopic.
providesaconcludingstatement.
providesaneffectiveconcluding
detailstosupportthetopic.
statement.
3.W.46 produceswritingwithguidance produceswritingwithguidanceand produceswritingwithguidanceand produceswritingwithguidanceand
supportthatincludesandexhibits
andsupportthatincludes
supportthatincludesincompleteor supportthatincludesandexhibits
complexdevelopment,concise
incompleteandinsufficient
insufficientdevelopment,minimal development,revision,and
revision,andcollaborativeelements.
development,incomplete
revision,andcollaborativeelements. collaborativeelements.
revision,andcollaborative
elements
3.W.78 conductsminimalresearchand conductssomeresearchandrecalls conductsresearchandrecalls
conductsfocusedresearchandrecalls
recallssomeinformationfrom
someinformationfromexperiences informationfromexperiencesand applicableinformationfrom
experiencesandsources,sorting
experiencesandsources,sorting andsources,sortingevidenceinto sources,sortingrelevantevidence
relevantevidenceintoprovided
evidenceintoprovidedcategories providedcategorieswhileproviding intoprovidedcategories.
categories.
whileprovidingevidencethatis someevidencethatmaynotbe
sortedintotheprovidedcategories.
notrelevanttothecategories.

AzMERITPerformanceLevelDescriptors

D-5

AzMERIT - 2015

EnglishLanguageArts
Grade3

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Proficient
Listening
determinesthemainideasand
identifiesthemainideasand
identifiesdetailsofatextread
supportingdetailsofatextread
aloudorinformationpresented supportingdetailsofatextread
aloudorinformationpresentedin aloudorinformationpresentedin
indiversemediaandformats,
includingvisually,quantitatively, diversemediaandformats,including diversemediaandformats,including
visually,quantitatively,andorally.
visually,quantitatively,andorally.
andorally.
Detailed 3.SL.2
asksandanswersexplicitquestions asksandanswersquestionsabout
asksandanswerssimple
aboutinformationfromaspeaker. informationfromaspeaker,offering
questionsaboutinformation
appropriateelaborationanddetail.
fromaspeaker.
Detailed
Detailed

3.SL.3
3.L.1

demonstratescommandof
grammarinsimplesentences.

AzMERITPerformanceLevelDescriptors

Language
demonstratescommandofgrammar
insimpleandcompoundsentences
includingalimitedunderstandingof
thefunctionofcommonand
straightforwardnouns,pronouns,
adjectives,adverbs,and
conjunctions.

D-6

demonstratescommandofgrammar
insimple,compound,andcomplex
sentences,includingthefunctionof
nouns(pluralandabstract),
pronouns,adjectives(comparative
andsuperlative),adverbs
(comparativeandsuperlative),
conjunctions(coordinatingand
subordinating),verbs(regularand
irregular)andsimpleverbtenses,
andsubjectverbandpronoun
antecedentagreement.

HighlyProficient
accuratelysummarizesthemain
ideasandsupportingdetailsofatext
readaloudorinformationpresented
indiversemediaandformats,
includingvisually,quantitatively,and
orally
asksandanswerscomplexquestions
aboutinformationfromaspeaker,
offeringrelevantandeffective
elaborationanddetail.

demonstratesstrongcommandof
grammarinsimple,compound,and
complexsentences,includingthe
functionofnouns(pluraland
abstract),pronouns,adjectives
(comparativeandsuperlative),
adverbs(comparativeand
superlative),conjunctions
(coordinatingandsubordinating),
verbs(regularandirregular)andverb
tenses,andsubjectverband
pronounantecedentagreement.

AzMERIT - 2015

EnglishLanguageArts
Grade3

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Detailed 3.L.2 demonstrateslimitedcommand demonstratescommandof
ofcapitalizationconventionsin capitalizationconventionsintitles
andofcommasinaddresses;spells
titlesandofcommasin
addresses;spellshighfrequency highfrequencywordscorrectlyand
usesspellingpatternsand
wordscorrectly.
generalizationsinwritingunknown
words.

Proficient
demonstratescommandof
capitalizationconventionsintitles,
commasinaddresses,commasand
quotationmarksindialogue,and
howtoformandusepossessives;
spellshighfrequencywords
correctly;usesspellingpatternsand
generalizationsinwritingunknown
wordsandforaddingsuffixesto
bases.

Standard Setting Technical Report


HighlyProficient
demonstratesstrongcommandof
capitalizationconventionsintitles,
commasinaddresses,commasand
quotationmarksindialogue,and
howtoformandusepossessives;
spellsmostwordscorrectly;uses
spellingpatternsandgeneralizations
inwritingunknownwordsandfor
addingsuffixestobases,including
useofcomplexpatternsand
irregularlyspelledwords.

Detailed

3.L.3

chooseswords/phraseswithout chooseswords/phrasesforeffect
chooseswords/phrasesforeffect
concernforeffect.
andrecognizesthedifferences
andrecognizesandobservesthe
betweenspokenandwrittenEnglish. differencesbetweenspokenand
writtenEnglish.

carefullychooseswords/phrasesfor
effectandtostrengthenthemessage
ofthewriting;recognizesand
observesthedifferencesbetween
spokenandwrittenEnglish.

Detailed

3.L.4

clarifiesthemeaningofunknown clarifiesthemeaningofmultiple
clarifiesthemeaningofunknown
wordsusingimmediate,explicit meaningwordsusingsentencelevel andmultiplemeaningwordsusing
contextclues.
contextclues;clarifiesthemeaning sentencelevelcontextclues,
ofunknownwordsusingmorphology morphology(gradelevelrootsand
(gradelevelrootsandaffixes)and/or affixes),and/orreferenceresources.
referenceresources.

clarifiesthemeaningofunknownand
multiplemeaningwordsusing
sentenceandparagraphlevel
contextclues,morphology(rootsand
affixes),and/orreferenceresources.

AzMERITPerformanceLevelDescriptors

D-7

AzMERIT - 2015

EnglishLanguageArts
Grade3

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
demonstratesunderstandingof
demonstratesunderstandingof
demonstratesunderstandingof
Detailed 3.L.5
recognizessimplefigurative
simplefigurativelanguage,simple figurativelanguage,word
complexfigurativelanguage,complex
language,simpleword
wordrelationships,andnuancesin relationships,andnuancesinword wordrelationships,andsubtle
relationships,andnuancesin
nuancesinwordmeanings;
meanings;distinguishestheliteral
wordmeanings;identifiesexplicit wordmeanings;recognizesthe
literalandnonliteraluseofwords
andnonliteralmeaningsofwords
reallifeconnectionsbetween
distinguishestheliteralandnonliteral
andphrasesincontext(e.g.,take
wordsandtheiruse(e.g.,
meaningsofwordsandphrasesin
andphrasesincontext(e.g.,take
describepeoplewhoarefriendly steps);identifiesreallifeconnections steps);identifiesreallifeconnections context(e.g.,takesteps);identifies
orhelpful).
betweenwordsandtheiruse(e.g., betweenwordsandtheiruse(e.g., subtleorcomplexreallife
describepeoplewhoarefriendlyor describepeoplewhoarefriendlyor connectionsbetweenwordsandtheir
use(e.g.,describepeoplewhoare
helpful).
helpful);distinguishesshadesof
meaningamongrelatedwordsthat friendlyorhelpful);distinguishes
describestatesofmindordegreesof subtleshadesofmeaningamong
relatedwordsthatdescribestatesof
certainty(e.g.,knew,believed,
mindordegreesofcertainty(e.g.,
suspected,heard,wondered).
knew,believed,suspected,heard,
wondered).

AzMERITPerformanceLevelDescriptors

D-8

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD

EnglishLanguageArts
Grade4

Standard Setting Technical Report

Standard

MinimallyProficient
Forgradeappropriate,low
complexitytexts,theMinimally
Proficientstudent

PartiallyProficient
Forgradeappropriate,lowto
moderatecomplexitytexts,the
PartiallyProficientstudent

Detailed

4.RL.1

identifiesdetailsandexamples
fromthetextanddrawssimple
inferences.

Reading:Literature
explainswhatthetextsaysexplicitly referstodetailsandexamplesina
textwhenexplainingwhatthetext
anddrawssimpleinferences;
identifieskeydetailsandexamplesin saysexplicitlyandwhendrawing
inferencesfromthetext.
thetext.

Detailed

4.RL.2

identifiesanexplicitlystated
themeinastory,drama,or
poem;identifiessomedetails
fromthetext.

Detailed

4.RL.3

identifiesaspectsofacharacter, describesacharacter,setting,or
eventinastoryordrama,using
setting,oreventinastoryor
explicitdetailsinthetext.
drama,drawingonexplicitly
stateddetailsinthetext.

Detailed

4.RL.4

Detailed

4.RL.5

Detailed

4.RL.6

determinesthemeaningofwords
identifiesthemeaningoffamiliar usesdetailsfromthetextto
understandthegeneralmeaningof andphrasesastheyareusedina
wordsandphrasesastheyare
wordsandphrasesastheyareused text,includingthosethatalludeto
usedinatext.
inatext,recognizingthosethat
significantcharactersfoundin
alludetosignificantcharactersfound mythology.
in mythology
identifiesbasicdifferences
explainsmajordifferencesbetween
describesdifferencesbetween
betweenpoems,drama,and
poems,drama,andprose,andrefers
poems,drama,andprose,and
prose,andidentifiescommon
recognizesthestructuralelements. tothestructuralelements.
structural elements.
identifiesthenarrator'spointof determinesthepointofviewfrom comparesandcontraststhepointof
whichdifferentstoriesarenarrated, viewfromwhichdifferentstoriesare
viewinastory;identifiesfirst
narrated,includingthedifference
includingdistinguishingbetween
andthirdpersonnarrations.
firstandthirdpersonnarrations. betweenfirstandthirdperson
narrations.

AzMERITPerformanceLevelDescriptors

Proficient
HighlyProficient
Forgradeappropriate,moderateto Forgradeappropriate,high
highcomplexitytexts,theProficient complexitytexts,theHighly
Proficientstudent
student
quotesaccuratelyfromatextand
referstokeydetailsandexamples
whenexplainingwhatthetextsays
explicitlyandwhendrawingcomplex
inferencesfromthetext.

determinesanimplicitlystated
theme,ormultiplethemes,ofa
story,drama,orpoem;
comprehensivelysummarizesthe
text.
describesindepthandanalyzesa
describesindepthacharacter,
setting,oreventinastoryordrama, complexcharacter,setting,orevent
inastoryordrama,drawingon
drawingonspecificdetailsinthe
implicit,specificdetailsinthetext.
text.

recognizesastatedthemeofastory, determinesthethemeofastory,
drama,orpoem;determinesthekey drama,orpoem;summarizesthe
text.
detailsinthetext.

D-9

analyzesthemeaningofunfamiliar
wordsandphrasesastheyareused
inatext,includingthosethatallude
tosignificantcharactersfoundin
mythology.
analyzesmajorhowdifferences
betweenpoems,drama,andprose
affectmeaning,andrefersto
complex structural elements.
comparesandcontrasts,then
analyzes,thepointofviewfrom
whichdifferentstoriesarenarrated,
includingthedifferencebetweenfirst
andthirdpersonnarrations.

AzMERIT - 2015

EnglishLanguageArts
Grade4

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
makessimpleconnectionsbetween
Detailed 4.RL.7 identifiesobvioussimilarities
thetextofastoryordramaandthe
betweenthetextofastoryor
visualororalpresentationofthe
dramaandthevisualororal
text.
presentationofthetext.

Detailed

4.RL.9

describesthetreatmentofsimilar
identifiessimilarthemesand
topicsandpatternsofeventsin themesandtopicsandpatternsof
eventsinstories,myths,and
stories,myths,andtraditional
literaturefromdifferentcultures. traditionalliteraturefromdifferent
cultures.

AzMERITPerformanceLevelDescriptors

D-10

Standard Setting Technical Report

Proficient
makesconnectionsbetweenthetext
ofastoryordramaandthevisualor
oralpresentationofthetext,
identifyingwhereeachversion
reflectsspecificdescriptionsand
directionsinthetext.

HighlyProficient
analyzescomplexconnections
betweenthetextofastoryordrama
andthevisualororalpresentationof
thetext,determiningwhereeach
versionreflectsspecificdescriptions
anddirectionsinthetext.

comparesandcontraststhe
treatmentofsimilarthemesand
topicsandpatternsofeventsin
stories,myths,andtraditional
literature from different cultures.

analyzesthedifferenttreatmentof
similarthemesandtopicsand
patternsofeventsinstories,myths,
andtraditionalliteraturefrom
different cultures.

AzMERIT - 2015

EnglishLanguageArts
Grade4

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Proficient
ReadingInformationalText
Detailed 4.RI.1 identifiesdetailsandexamples identifieskeydetailsandexamplesin referstodetailsandexamplesina
fromthetextanddrawssimple thetext;explainswhatthetextsays textwhenexplainingwhatthetext
saysexplicitlyandwhendrawing
explicitlyanddrawssimple
inferences.
inferencesfromthetext.
inferences.
Detailed

4.RI.2

identifiesanexplicitlystated
mainideaandkeydetailsofa
text.

Detailed

4.RI.3

identifiesevents,procedures,
ideas,orconceptsinahistorical,
scientific,ortechnicaltextbased
onspecificinformationinthe
text.

Detailed

4.RI.4

Detailed

4.RI.5

Detailed

4.RI.6

recognizesastatedmainideaofa
textanddetermineskeydetails;
providesasimplesummaryofthe
text.

determinesthemainideaofatext
andexplainshowitissupportedby
keydetails;summarizesthetext.

Standard Setting Technical Report


HighlyProficient
quotesaccuratelyfromatextand
referstokeydetailsandexamples
whenexplainingwhatthetextsays
explicitlyandwhendrawingcomplex
inferencesfromthetext.
determinesanimplicitlystatedmain
ideaofatextandexplains,using
textualevidence,howitissupported
bykeydetails;comprehensively
summarizesthetext.

explainsevents,procedures,ideas, analyzesevents,procedures,ideas,or
orconceptsinahistorical,scientific, conceptsinahistorical,scientific,or
technicaltext,includingwhat
ortechnicaltext,includingwhat
happenedandwhy,basedonspecific happenedandwhy,usingevidence
fromthetexttojustifythe
informationinthetext.
explanation
identifiestheloosemeaningof determinestheapproximate
determinesthemeaningofgeneral determinesandanalyzesthe
frequentlyusedacademicand
meaningofbasicacademicand
academicanddomainspecificwords meaningofacademicanddomain
specificwordsorphrasesinatext.
domainspecificwordsand
domainspecificwordsorphrasesin orphrasesinatext.
phrases in a text.
a text.
describestheoverallstructure(e.g., analyzestheoverallstructure(e.g.,
determinestheoverallstructure
identifiesthestructure(e.g.,
chronology,comparison,
chronology,comparison,
(e.g.,chronology,comparison,
chronology,comparison,
cause/effect,problem/solution) cause/effect,problem/solution)of cause/effect,problem/solution)of cause/effect,problem/solution)of
events,ideas,concepts,or
events,ideas,concepts,or
events,ideas,concepts,or
ofevents,ideas,concepts,or
informationinatextorpartofatext. informationinatextorpartofatext,
informationinatextorpartofa
informationinpartofatext.
includinghowitcontributestothe
text.
meaning of the text
identifieswhethertextswritten determinesthedifferencesbetween comparesandcontrastsafirsthand comparesandcontrasts,then
onthesameeventortopicarea afirsthandandsecondhandaccount andsecondhandaccountofthesame analyzes,afirsthandandsecondhand
accountofthesameeventortopic,
eventortopic;describesthe
firsthandorsecondhandaccount; ofthesameeventortopic;
includingthedifferenceinfocusand
differenceinfocusandthe
recognizesthedifferenceinfocus
determinesthefocusofthe
theinformationprovided.
informationprovided.
andtheinformationprovided.
account.

AzMERITPerformanceLevelDescriptors

describesevents,procedures,ideas,
orconceptsinahistorical,scientific,
ortechnicaltext,includingwhat
happenedandwhy,basedon
specificinformationinthetext.

D-11

AzMERIT - 2015

EnglishLanguageArts
Grade4

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
identifiesordescribesinformation
Detailed 4.RI.7 identifiesordescribes
informationpresentedvisually, presentedvisually,orally,or
quantitativelyandrecognizeshow
orally,orquantitatively.
theinformationcontributestoan
understandingofthetextinwhichit
appears
Detailed 4.RI.8 identifiesreasonsandevidence describeshowanauthoruses
anauthorincludesinatext.
reasonsandevidencetosupportthe
overallpointinatext.
Detailed 4RI.9 identifiesinformationfromtwo utilizesinformationfromtwotexts
textsonthesametopicinorder onthesametopictowriteorspeak
toanswerquestions,orallyorin aboutthesubjectknowledgeably.
writing,aboutthesubject.

AzMERITPerformanceLevelDescriptors

D-12

Proficient
interpretsinformationpresented
visually,orally,orquantitativelyand
explainshowtheinformation
contributestoanunderstandingof
thetextinwhichitappears.

Standard Setting Technical Report

HighlyProficient
analyzesinformationpresented
visually,orally,orquantitativelyand
explainshowtheinformation
contributestotheoverall
understandingofthetextinwhichit
appears
explainshowanauthorusesreasons analyzeshowanauthorusesreasons
andevidencetosupportparticular andevidencetosupportparticular
pointsinatext.
pointsinatext.
synthesizescomplexinformationand
integratesinformationfromtwo
textsonthesametopicinorderto textualevidencefromtwotextson
thesametopicinordertowriteor
writeorspeakaboutthesubject
speakaboutthesubject
knowledgeably.
knowledgeably.

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Writing
Detailed 4.W.1 writesopinionpiecesthatlack
writesmoderatelyorganizedopinion
organizationandaclearpointof piecesontopicsortexts.
view.
a.introducesatopicortextby
a.statesanopinionbutusesan statinganopinion,andgenerally
ineffectiveorinappropriate
groupsideastogetherinawaythat
organizationalstructureto
supportsthewriter'spurpose.
presentideas.
b.providesbothrelevantand
b.providesfactsanddetailsthat irrelevantfactsanddetails.
arenotrelevanttothetopic.
c.linksopinionandreasonsusing
c.opinionandreasonsarenot
basictransitionalwords.
linkedwithtransitions.
d.providesaconcludingstatement.
d.includesanineffective
concludingstatement.

AzMERITPerformanceLevelDescriptors

Standard Setting Technical Report

EnglishLanguageArts
Grade4

D-13

Proficient

HighlyProficient

writeswellorganizedopinionpieces
writesopinionpiecesontopicsor
texts,supportingapointofviewwith ontopicsortexts,fullysupportinga
pointofviewwithreasonsand
reasonsandinformation.
information.
a.introducesatopicortextclearly,
a.effectivelyintroducesatopicor
statesanopinion,andcreatesan
textclearly,statesanopinion,and
organizationalstructureinwhich
relatedideasaregroupedtosupport createsanorganizationalstructurein
whichrelatedideasarelogically
thewriter'spurpose.
groupedtosupportthewriter's
purpose.
b.providesreasonsthatare
supportedbyfactsanddetails.
b.provideslogicallyorderedreasons
thataresupportedbyfactsand
c.linksopinionandreasonsusing
details.
wordsandphrases(e.g., for
instance,inorderto,inaddition).
c.smoothlylinksopinionandreasons
d.providesaconcludingstatement usingwordsandphrases(e.g.,for
instance,inorderto,inaddition).
orsectionrelatedtotheopinion
presented.
d.providesarelevantandeffective
concludingstatementorsection
relatedtotheopinionpresented.

AzMERIT - 2015

EnglishLanguageArts
Grade4

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Detailed 4.W.2 writesinformative/explanatory writesmoderatelyorganized
informative/explanatorytextsto
textstodiscussatopic.
discussatopicandconveyideasand
information.
a.statesthetopicandgroups
informationinanillogicalor
a.introducesthetopicandgroups
unrelatedmanner;includes
relatedinformationlogically;
irrelevantordistracting
includesformatting(e.g.,headings),
formatting,illustrations,and
illustrations,andmultimedia.
multimedia.
b.providesirrelevantor
unreliablefacts,definitions,
details,quotations,orother
informationandexamples.
c.ideasarenotclearlyor
effectivelylinked.
d.usessimplevocabularywhen
explainingthetopic.
e.providesanincomplete
concludingstatement.

Proficient
HighlyProficient
writesinformative/explanatorytexts writesinformative/explanatorytexts
toexamineatopicandconveyideas tothoroughlyexamineatopicand
conveyideasandinformationclearly
andinformationclearly.
andcompletely.
a.introducesatopicclearlyand
a.clearlyandeffectivelyintroduces
grouprelatedinformationin
thetopicandgroupsrelated
paragraphsandsections;includes
informationlogicallyinparagraphs
formatting(e.g.,headings),
illustrations,andmultimediawhen andsections;includeseffective
formatting(e.g.,headings),
usefultoaidcomprehension.
illustrations,andmultimediathat
b.supportsthetopicwithfacts,
enhancecomprehension.
b.developsthetopicwithfacts,
definitions,concretedetails,
quotations,orotherinformationand definitions,concretedetails,
quotations,orotherinformationand b.fullydevelopsthetopicwith
examples.
relevantfacts,definitions,concrete
examplesrelatedtothetopic.
details,quotations,orother
c.linksideaswithincategoriesof
informationandexamplesrelatedto
informationusingsimpletransitional c.linksideaswithincategoriesof
informationusingwordsandphrases thetopic.
wordsorphrases.
(e.g.,another,forexample,also,
c.smoothlylinksideaswithin
d.usesdomainspecificvocabularyin because).
categoriesofinformationusing
anattempttoexplainthetopic.
d.usespreciselanguageanddomain purposefultransitionalwordsand
e.providesaconcludingstatement. specificvocabularytoinformabout phrases.
orexplainthetopic.
d.usespreciselanguageanddomain
e.providesaconcludingstatement specificvocabularyefficientlyand
orsectionrelatedtotheinformation effectivelytoinformorexplainabout
thetopic.
orexplanationpresented.
e.providesarelevantandeffective
l d
l d
h

AzMERITPerformanceLevelDescriptors

D-14

AzMERIT - 2015

EnglishLanguageArts
Grade4

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
Detailed 4.W.46 produceswritingwithguidance producesclearwritinginwhichthe producesclearandcoherentwriting producesclearandwelldeveloped
writinginwhichthedevelopment,
development,organization,andstyle inwhichthedevelopment,
andsupportinwhichthe
organization,andstyleare
organization,andstyleare
development,organization,and arelargelyappropriatetotask,
appropriatetotask,purpose,and
appropriatetotask,purpose,and
purpose,andaudience;with
styleareevident;develops
audience;developsandstrengthens
audience;withguidanceand
guidanceandsupport,develops
writingwithsomeplanning,
support,developsandstrengthens writingonanongoingbasisby
writingbyplanning,revising,and
revising,andediting,including
planning,revising,andediting,
writingbyplanning,revising,and
editing,includingeditingfor
editingforconventions;
includingeditingforconventions;
demonstratesbasiccommandof conventions;demonstratessufficient editing,includingeditingfor
conventions;demonstratessufficient demonstratessufficientcommandof
commandofkeyboardingskillsto
keyboardingskills.
keyboardingskillstotypeoneor
commandofkeyboardingskillsto
typeuptoonepageinasingle
morepagesinasinglesitting.
typeaminimumofonepageina
sitting.
singlesitting.
Detailed

4.W.78 conductsshortresearchprojects
aboutatopic;recallssome
informationfromexperiences
andsources;providesnotes
regardinginformation.

AzMERITPerformanceLevelDescriptors

conductsshortresearchprojects
thatuseseveralsourcestodiscussa
topic;recallssomeinformationfrom
experiencesandgathersinformation
fromsources;providesbriefnotes
aboutinformation.

D-15

conductsshortresearchprojectsthat conductsresearchprojectsthatuse
severalhighqualitysourcestobuild
buildknowledgethrough
investigationofdifferentaspectsofa knowledgebyfullyinvestigatinga
topic;usesrelevantinformationfrom
topic;recallsrelevantinformation
fromexperiencesorgathersrelevant experiencesandgatheredfromprint
anddigitalsources;fullysummarizes
informationfromprintanddigital
sources;takesnotesandcategorizes orparaphrasesinformationinnotes
andefficientlycategorizes
information.
information.

AzMERIT - 2015

EnglishLanguageArts
Grade4

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Proficient
Listening
paraphrasesportionsofatextread
Detailed 4.SL.2 identifieskeydetailsfromatext describeskeydetailsfromatext
readaloudorinformationpresented aloudorinformationpresentedin
readaloudorinformation
diversemediaandformats,including
indiversemediaandformats,
presentedinasinglemedia
includingvisually,quantitatively,and visually,quantitatively,andorally.
format,includingvisually,
orally.
quantitatively,andorally.
Detailed

4.SL.3

Detailed

4.L.1

HighlyProficient
clearly,coherently,andefficiently
paraphrasesportionsofatextread
aloudorinformationpresentedin
diversemediaandformats,including
visually,quantitatively,andorally.

identifiesthepointsaspeakermakes identifiesthereasonsandevidencea evaluatesthereasonsandevidencea


speakerprovidestosupport
speakerprovidestosupport
andkeydetailsaboutthetopic.
particularpoints.
particularpoints.
Language
demonstratesstrongcommandof
demonstratesanunderstandingof demonstratescommandofthe
demonstratesabasic
theconventionsofstandardEnglish
understandingoftheconventions theconventionsofstandardEnglish conventionsofstandardEnglish
grammarandusagewhenwriting,
ofstandardEnglishgrammarand grammarandusagewhenwriting, grammarandusagewhenwriting,
includingusingrelativepronounsand includingusingrelativepronounsand
usagewhenwriting;formsand includingusingrelativepronouns
relativeadverbs,formingandusing relativeadverbs,formingandusing
andrelativeadverbsandforming
usessimpleprepositional
theprogressiveverbtenses,and
andusingtheprogressiveverbtense; theprogressiveverbtenses,and
phrases.
usingmodalauxiliaries(e.g.,can,
ordersadjectiveswithinsentences usingmodalauxiliaries(e.g.,can,
may,must)toconveyvarious
accordingtoconventionalpatterns; may,must)toconveyvarious
formsandusessimpleprepositional conditions;ordersadjectiveswithin conditions;ordersadjectiveswithin
sentencesaccordingtoconventional sentencesaccordingtoconventional
phrases;producescomplete
patterns;formsandusescomplex
patterns;formsanduses
sentences,recognizingand
prepositionalphrases;produces
correctinginappropriatefragments prepositionalphrases;produces
completesentences,recognizingand completesentenceswithvarying
andrunons.
correctinginappropriatefragments complexity,recognizingand
correctinginappropriatefragments
andrunons;correctlyuses
frequentlyconfusedwords(e.g.,to, andrunons;correctlyuses
frequentlyconfusedwords(e.g.,to,
too,two;there,their).
too,two;there,their).
identifiesthepointsaspeaker
makes.

AzMERITPerformanceLevelDescriptors

D-16

AzMERIT - 2015

EnglishLanguageArts
Grade4

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
demonstratesunderstandingofthe
Detailed 4.L.2
demonstratesabasic
understandingoftheconventions conventionsofstandardEnglish
ofstandardEnglishcapitalization, capitalization,punctuation,and
punctuation,andspellingwhen spellingwhenwriting;usescommas
and/orquotationmarkstomark
writing;usescommasand/or
quotationmarkstomarkdirect directspeechandquotationsfroma
text;spellsmostwordscorrectly,
speechandquotationsfroma
text;spellsmostwordscorrectly, consultingreferencesasneeded.
consultingreferencesasneeded.

Detailed

4.L.3

usesabasicknowledgeof
languageanditsconventions
whenwriting,speaking,reading,
orlistening;chooseswordsand
phrasestoformsentences;uses
somepunctuation.

AzMERITPerformanceLevelDescriptors

usesabasicknowledgeoflanguage
anditsconventionswhenwriting,
speaking,reading,orlistening;
chooseswordsandphrasesto
conveyideas;usesappropriate
punctuation;usesaconsistently
formalorinformaltone.

D-17

Proficient
demonstratescommandofthe
conventionsofstandardEnglish
capitalization,punctuation,and
spellingwhenwriting;usescommas
andquotationmarkstomarkdirect
speechandquotationsfromatext;
usesacommabeforeacoordinating
conjunctioninacompound
sentence;spellswordscorrectly,
consultingreferencesasneeded.

Standard Setting Technical Report


HighlyProficient
demonstratesstrongcommandof
theconventionsofstandardEnglish
capitalization,punctuation,and
spellingwhenwriting;usescommas
andquotationmarkstomarkdirect
speechandquotationsfromatext;
usesacommabeforeacoordinating
conjunctioninacompoundsentence;
spellslowfrequencyandabove
gradelevelwordscorrectly,
consultingreferencesasneeded.

usesknowledgeoflanguageandits usesdeepknowledgeoflanguage
conventionswhenwriting,speaking, anditsconventionswhenwriting,
reading,orlistening;chooseswords speaking,reading,orlistening;
chooseswordsandphrasestoconvey
andphrasestoconveyideas
precisely;choosespunctuationfor ideasprecisely;choosespunctuation
foreffect;differentiatesbetween
effect;differentiatesbetween
contextsthatcallforformalEnglish contextsthatcallforformalEnglish
(e.g.,presentingideas)andsituations (e.g.,presentingideas)andsituations
whereinformaldiscourseis
whereinformaldiscourseis
appropriate(e.g.,smallgroup
appropriate(e.g.,smallgroup
discussion).
discussion).

AzMERIT - 2015

EnglishLanguageArts
Grade4

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Proficient
Detailed L.4.4
clarifiesthemeaningofunknown determinesorclarifiesthemeaning determinesorclarifiesthemeaning
ofunknownandmultiplemeaning ofunknownandmultiplemeaning
wordsandphrases,choosing
wordsandphrases,choosingfroma wordsandphrases,choosingflexibly
fromalimitedrangeof
strategies;usesimmediateand rangeofstrategies;usesimmediate fromarangeofstrategies;uses
explicitcontextasacluetothe contextasacluetothemeaningofa contextasacluetothemeaningofa
wordorphrase;usescommongrade
wordorphrase;recognizesGreek
meaningofawordorphrase;
consultsreferencematerials(e.g., andLatinaffixesandroots;consults appropriateGreekandLatinaffixes
androotsascluestothemeaningof
referencematerials(e.g.,
dictionaries,glossaries,
dictionaries,glossaries,thesauruses), aword;consultsreferencematerials
thesauruses),bothprintand
(e.g.,dictionaries,glossaries,
digital,todeterminethemeaning bothprintanddigital,tofindthe
thesauruses),bothprintanddigital,
pronunciationanddetermineor
ofwordsandphrases.
clarifythemeaningofkeywordsand tofindthepronunciationand
determineorclarifytheprecise
phrases.
meaningofkeywordsandphrases.

Detailed

4.L.5

recognizessimplefigurative
language,simpleword
relationships,andnuancesin
wordmeanings;recognizes
simplesimilesandmetaphors;
recognizescommonidioms,
adages,andproverbs;
understandsthatwordshave
directopposites(antonyms)and
somewordshavesimilarbutnot
identicalmeanings(synonyms).

AzMERITPerformanceLevelDescriptors

demonstratesunderstandingof
demonstratesunderstandingof
simplefigurativelanguage,simple figurativelanguage,word
wordrelationships,andnuancesin relationships,andnuancesinword
meanings;explainsthemeaningof
wordmeanings;identifiesthe
simplesimilesandmetaphors(e.g.,
meaningofsimplesimilesand
asprettyasapicture)incontext;
metaphors(e.g.,asprettyasa
picture)incontext;recognizesand recognizesandexplainsthemeaning
identifiesthemeaningofcommon, ofcommonidioms,adages,and
simpleidioms,adages,andproverbs; proverbs;demonstrates
understandingofwordsbyrelating
demonstratesalimited
understandingofwordsbyrelating themtotheiropposites(antonyms)
themtotheiropposites(antonyms) andtowordswithsimilarbutnot
identicalmeanings(synonyms).
andtowordswithsimilarbutnot
identicalmeanings(synonyms).

D-18

HighlyProficient
determinesorclarifiesandapplies
themeaningofunknownand
multiplemeaningwordsandphrases,
choosingstrategicallyfromarangeof
strategies;usessentenceand
paragraphlevelcontextasaclueto
themeaningofawordorphrase;
usesGreekandLatinaffixesandroots
ascluestothemeaningofaword;
consultsreferencematerials(e.g.,
dictionaries,glossaries,thesauruses),
bothprintanddigital,tofindthe
pronunciationanddetermineor
clarifytheprecisemeaningofkey
wordsandphrases.
demonstratesunderstandingof
complexfigurativelanguage,complex
wordrelationships,andsubtle
nuancesinwordmeanings;explains
themeaningofcomplexandimplicit
similesandmetaphorsincontext;
recognizesandexplainsthemeaning
ofidioms,adages,andproverbs;
demonstratesdeepunderstandingof
wordsbyrelatingthemtotheir
opposites(antonyms)andtowords
withsimilarbutnotidentical
meanings(synonyms).

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD

EnglishLanguageArts
Grade5

Standard Setting Technical Report

Standard

MinimallyProficient
Forgradeappropriate,low
complexitytexts,theMinimally
Proficientstudent

Detailed

5.RL.1

explainswhatthetextsays
explicitlyanddrawssimple
inferences.

Detailed

5.RL.2

identifiesanexplicitlystated
themeofastory,drama,or
poem;providesabasiclistof
eventsinatext.

identifiesathemeofastory,drama, determinesathemeofastory,
determinesimplicitlystatedthemes
orpoem;identifiesthekeyeventsor drama,orpoemfromdetailsinthe ofastory,drama,orpoem,including
detailsinatext.
text,includinghowcharactersina howcharactersinastoryordrama
storyordramarespondtochallenges respondtochallengesorhowthe
orhowthespeakerinapoem
speakerinapoemreflectsupona
reflectsuponatopic;summarizesthe topic;comprehensivelysummarizes
text.
thetext.

Detailed

5.RL.3

identifiesdifferencesor
similaritiesbetweentwo
characters,settings,oreventsin
astoryordrama,drawingon
simple,explicitdetailsinthetext.

determinesdifferencesorsimilarities comparesandcontraststwoormore
betweentwoormorecharacters,
characters,settings,oreventsina
storyordrama,drawingonspecific
settings,oreventsinastoryor
drama,usingexplicitdetailsinthe detailsinthetext(e.g.,how
charactersinteract).
text.

analyzesthesimilaritiesand
differencesbetweentwoormore
characters,settings,oreventsina
storyordrama,drawingonimplicitly
stateddetailsinthetext(e.g.,how
charactersinteract).

Detailed

5.RL.4

identifiestheliteralmeaningof
familiarwordsandphrasesas
theyareusedinatext.

determinesthemeaningofwords
andphrasesastheyareusedina
text,includingfigurativelanguage
suchasmetaphorsandsimiles.

analyzesthemeaningofunfamiliar
wordsandphrasesastheyareused
inatext,includingfigurative
languagesuchasmetaphorsand
similes.

Detailed

5.RL.5

identifiesaparticularchapter,
scene,orstanzathatprovides
structuretoaparticularstory,
drama,orpoem.

distinguishesbetweenliteraland
figurativemeaningsofwordsand
phrasesastheyareusedinatext,
includingrecognizingfigurative
languagesuchasmetaphorsand
similes
explainshowaseriesofchapters,
scenes,orstanzasaffectsthebasic
structureofaparticularstory,
drama,orpoem.

explainshowaseriesofchapters,
scenes,orstanzasfitstogetherto
providetheoverallstructureofa
particularstory,drama,orpoem.

analyzeshowaseriesofchapters,
scenes,orstanzasfitstogetherand
interactstoprovidetheoverall
structureofaparticularstory,drama,
orpoem.

AzMERITPerformanceLevelDescriptors

PartiallyProficient
Proficient
Forgradeappropriate,moderateto
Forgradeappropriate,lowto
highcomplexitytexts,theProficient
moderatecomplexitytexts,the
student
PartiallyProficientstudent
Reading:Literature
quotesaccuratelyfromatextwhen
paraphrasespartsofthetextto
explainwhatthetextsaysexplicitly explainingwhatthetextsays
andwhendrawinginferencesfrom explicitlyandwhendrawing
inferencesfromthetext.
thetext.

D-19

HighlyProficient
Forgradeappropriate,high
complexitytexts,theHighly
Proficientstudent
accuratelyquotesstrongtextual
evidencewhenexplainingwhatthe
textsaysexplicitlyandwhendrawing
complexinferencesfromthetext.

AzMERIT - 2015

EnglishLanguageArts
Grade5

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Proficient
Detailed RL.5.6 identifiesanarrator'sor
describeshowanarratororspeaker describeshowanarrator'sor
speaker'spointofview.
describeseventsinatext.
speaker'spointofviewinfluences
howeventsaredescribed.
Detailed

5.RL.7

Detailed
Detailed

RL.5.8
5.RL.9

identifieshowvisualand
describeshowvisualandmultimedia
multimediaelementssupportthe elementscontributetothemeaning
meaningofaportionofthetext ofatext(e.g.,graphicnovel,
(e.g.,graphicnovel,multimedia multimediapresentationoffiction,
presentationoffiction,folktale, folktale,myth,poem).
myth poem)
N/A
N/A
identifiesvariousgenrespecific determinesvariousgenrespecific
characteristicsofstoriesinthesame
characteristicsofstoriesinthe
samegenre(e.g.,mysteriesand genre(e.g.,mysteriesandadventure
adventurestories),butwithlittle stories)withsimilarthemesand
ornoconnectiontothethemes topics.
andtopics.

AzMERITPerformanceLevelDescriptors

D-20

Standard Setting Technical Report


HighlyProficient
analyzeshowanarrator'sor
speaker'spointofviewinfluences
howcomplexeventsaredeveloped.

analyzeshowvisualandmultimedia analyzes,thenevaluates,howvisual
elementscontributetothemeaning, andmultimediaelementscontribute
tone,orbeautyofatext(e.g.,
tothemeaning,tone,orbeautyofa
graphicnovel,multimedia
text(e.g.,graphicnovel,multimedia
presentationoffiction,folktale,
presentationoffiction,folktale,
myth poem)
myth poem)
N/A
N/A
comparesandcontrastsstoriesinthe compares,contrasts,and
samegenre(e.g.,mysteriesand
analyzes/evaluatesstoriesinthe
samegenre(e.g.,mysteriesand
adventurestories)ontheir
adventurestories)ontheir
approachestosimilarthemesand
approachestosimilarthemesand
topics.
topics.

AzMERIT - 2015

EnglishLanguageArts
Grade5

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Proficient
Reading:InformationalText
quotesaccuratelyfromatextwhen
paraphrasespartsofthetextto
Detailed 5.RI.1 explainswhatthetextsays
explainwhatthetextsaysexplicitly explainingwhatthetextsays
explicitlyanddrawssimple
andwhendrawinginferencesfrom explicitlyandwhendrawing
inferences.
inferencesfromthetext.
thetext.

HighlyProficient
accuratelyquotesstrongtextual
evidencewhenexplainingwhatthe
textsaysexplicitlyandwhendrawing
complexinferencesfromthetext.

Detailed

5.RI.2

identifiestwoormoreexplicitly
statedmainideasofatext;
identifiesrelevantdetailsfrom
thetext;providesabasiclistof
eventsorfactsfromthetext.

determinestwoormoreexplicitly
statedmainideasofatextand
explainshowtheyarerelatedto
relevantdetails;providesasimple
summaryofthetext.

determinestwoormoremainideas
ofatextandexplainshowtheyare
supportedbykeydetails;
summarizesthetext.

analyzestherelationshipbetween
twoormoremainideasofatextand
explainshowtheyaresupportedby
keydetails;providesa
comprehensivesummaryofthetext.

Detailed

5.RI.3

identifiesstraightforward
relationshipsorinteractions
betweentwoindividuals,events,
ideas,orconceptsinahistorical,
scientific,ortechnicaltext.

describestherelationshipsor
interactionsbetweentwo
individuals,events,ideas,or
conceptsinahistorical,scientific,or
technicaltext,relyingonageneral
understandingofthetext.

explainstherelationshipsor
interactionsbetweentwoormore
individuals,events,ideas,or
conceptsinahistorical,scientific,or
technicaltextbasedonspecific
informationinthetext.

analyzescomplexrelationshipsor
interactionsbetweentwoormore
individuals,events,ideas,orconcepts
inahistorical,scientific,ortechnical
text,providingevidencebasedon
specificinformationinthetext.

Detailed

5.RI.4

Detailed

5.RI.5

identifiestheloosemeaningof
frequentlyusedacademicand
domainspecificwordsand
phrases in a text.
identifiestheoverallstructure
(e.g.,chronology,comparison,
cause/effect,problem/solution)
ofevents,ideas,concepts,or
informationintwoormoretexts.

determinestheapproximate
determinesthemeaningofgeneral
meaningofbasicacademicand
academicanddomainspecificwords
domainspecificwordsandphrases andphrasesinatext.
in a text.
explainstheoverallstructure(e.g., comparesandcontraststheoverall
structure(e.g.,chronology,
chronology,comparison,
cause/effect,problem/solution)of comparison,cause/effect,
problem/solution)ofevents,ideas,
events,ideas,concepts,or
informationintwoormoretexts.
concepts,orinformationintwoor
moretexts.

Detailed

5.RI.6

identifiesthepointofviewin
multipleaccountsofthesame
eventortopic.

determinessimilaritiesand
differencesinthepointsofviewin
multipleaccountsofthesameevent
ortopic.

AzMERITPerformanceLevelDescriptors

D-21

determinesandanalyzesthe
meaningandeffectofadvanced
academicanddomainspecificwords
and phrases in a text.
comparesandcontrasts,then
analyzes,theoverallstructure(e.g.,
chronology,comparison,
cause/effect,problem/solution)of
events,ideas,concepts,or
informationintwoormoretexts,
includinghowthatstructure
contributestotheoverallmeaning.

analyzesmultipleaccountsofthe
analyzesmultipleaccountsofthe
sameeventortopic,explains
sameeventortopic,noting
importantsimilaritiesanddifferences importantsimilaritiesanddifferences
inthepointofviewtheyrepresent. inthepointofviewtheyrepresent,
andevaluatestheeffectivenessofthe
accounts.

AzMERIT - 2015

EnglishLanguageArts
Grade5

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Proficient
drawsoninformationfrommultiple drawsoninformationfrommultiple
Detailed 5.RI.7 identifiesexplicitinformation
withinprintordigitalsourcesin printordigitalsources,
printordigitalsources,
demonstratingtheabilitytolocatea demonstratingtheabilitytolocate
ordertolocateananswertoa
simpleanswertoanexplicitquestion ananswertoaquestionquicklyorto
basicquestionorsolveabasic
ortosolveanexplicitproblem.
problem.
solveaproblemefficiently.

HighlyProficient
drawsonrelevantinformationfrom
reliablemultipleprintordigital
sources,demonstratingtheabilityto
fullyanswercomplexquestionsorto
solveacomplexproblemefficiently.

Detailed

5.RI.8

identifieswhichreasonsor
evidencesupportaparticular
pointinatext.

explainshowanauthorusesreasons
describeshowanauthoruses
andevidencetosupportparticular
reasonsandevidencetosupport
particularpointsinatext,identifying pointsinatext,identifyingwhich
reasonsandevidencesupportwhich
relevantsupportingdetails.
point(s).

evaluatesthestrengthofthereasons
andevidenceanauthorusesto
supportparticularpointsinatext,
explaininghowthereasonsand
evidencesupportthepoint(s).

Detailed

5.RI.9

identifiesinformationfromone
ortwotextsandprovidesan
incompleteresponsewhen
writingorspeakingaboutthe
subject.

findsrelevantinformationfrom
severaltextsonthesametopicin
ordertowriteorspeakaboutthe
subject.

integratescomplexorinferred
informationfromseveraltextsonthe
sametopicinordertowriteorspeak
aboutthesubjectknowledgably,
usingtextualevidenceassupport.

AzMERITPerformanceLevelDescriptors

D-22

integratesinformationfromseveral
textsonthesametopicinorderto
writeorspeakaboutthesubject
knowledgably.

AzMERIT - 2015

EnglishLanguageArts
Grade5

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
Writing
writeswellorganized,multi
Detailed 5.W.1 writesopinionpiecesthatlack
writesmoderatelyorganizedopinion writesopinionpiecesontopicsor
organizationandaclearpointof piecesontopicsortexts,providinga texts,supportingapointofviewwith paragraphopinionpieces,supporting
apointofviewwitheffectivereasons
view.
clearpointofview.
reasonsandinformation.
andrelevantinformation.
a.statesanopinionbutusesan a.introducesatopicortextby
a.introducesatopicortextclearly,
a.effectivelyintroducesatopicor
ineffectiveorinappropriate
statinganopinionandorganizes
statesanopinion,andcreatesan
organizationalstructureto
ideasinagenerallyeffective
organizationalstructureinwhich
textclearly,statesanopinion,and
presentideas.
organizationalstructure.
ideasarelogicallygroupedto
createsaneffectiveorganizational
supportthewriter'spurpose.
structureinwhichideasarelogically
andeffectivelygrouped,emphasizing
b.providesfactsanddetailsthat b.providesbothrelevantand
arenotrelevanttothetopic.
irrelevantreasonsthatarelogically b.provideslogicallyorderedreasons thewriter'spurpose.
ordered.
thataresupportedbyfactsand
details.
b.provideseffective,relevant
c.opinionsandreasonsarenot
reasonsthatarelogicallyand
linkedwithtransitions.
c.linksopinionsandreasonsusing
purposefullyorderedandsupported
basictransitionalwords.
c.linksopinionandreasonsusing
byfactsanddetails.
words,phrases,andclauses(e.g.,
d.includesanineffective
concludingstatement.
d.providesaconcludingstatement. consequently,specifically).
c.smoothlylinksopinionsand
d.providesaconcludingstatement reasonsusingwords,phrases,and
orsectionrelatedtotheopinion
clauses(e.g.,consequently,
presented.
specifically)
d.providesarelevantandeffective
concludingstatementorsection
relatedtotheopinionpresented.

AzMERITPerformanceLevelDescriptors

D-23

AzMERIT - 2015

EnglishLanguageArts
Grade5

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Detailed 5.W.2 writesinformative/explanatory writesinformative/explanatorytexts
textstodiscussatopic.
todiscussatopicandconveyideas
andinformation.
a.statesthetopic,writeswith
littlefocus,andgroups
a.introducesthetopic,providesa
informationinanillogicalor
generalobservationwithaloose
unrelatedmanner;includes
focus,andgroupsrelated
irrelevantordistracting
informationlogically;includes
formatting,illustrations,and
formatting(e.g.,headings),
multimedia.
illustrations,andmultimedia.
b.providesirrelevantor
unreliablefacts,definitions,
details,quotations,orother
informationandexamples.
c.ideasarenotclearlyor
effectivelylinked.
d.usessimplevocabularywhen
explainingthetopic.
e.providesanincomplete
concludingstatement.

Proficient
HighlyProficient
writesinformative/explanatorytexts writesinformative/explanatorytexts
toexamineatopicandconveyideas tothoroughlyexamineatopicand
conveycomplexideasand
andinformationclearly.
informationclearly.
a.introducesatopicclearly,provides
ageneralobservationandfocus,and a.clearlyandeffectivelyintroduces
groupsrelatedinformationlogically; thetopic,providesaspecific
includesformatting(e.g.,headings), observationandclearfocus,and
illustrations,andmultimediawhen groupsrelatedinformationlogically;
usefultoaidingcomprehension.
includeseffectiveandpurposeful
formatting(e.g.,headings),
illustrations,andmultimediato
b.developsthetopicwithfacts,
b.supportsthetopicwithfacts,
enhancecomprehension.
definitions,concretedetails,
definitions,concretedetails,
quotations,orotherinformationand quotations,orotherinformationand
examplesrelatedtothetopic.
b.fullydevelopsthetopicwith
examples.
relevantfacts,definitions,concrete
details,quotations,orother
c.linksideaswithinandacross
c.linksideaswithincategoriesof
informationusingsimpletransitional categoriesofinformationusing
informationandexamplesrelatedto
wordsorphrases.
words,phrases,andclauses(e.g.,in thetopic.
contrast,especially).
d.usesdomainspecificvocabularyin
c.smoothlylinkssupportedideas
anattempttoexplainthetopic.
d.usespreciselanguageanddomain withinandacrosscategoriesof
specificvocabularytoinformabout informationusingpurposeful
transitionalphrasesandclauses,
e.providesaconcludingstatement. orexplainthetopic.
e.providesaconcludingstatement d.usespreciselanguageanddomain
orsectionrelatedtotheinformation specificvocabularyefficientlyand
effectivelytoinformorexplainabout
orexplanationpresented.
thetopic.
d

AzMERITPerformanceLevelDescriptors

D-24

d ff

AzMERIT - 2015

EnglishLanguageArts
Grade5

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
Detailed 5.W.46 producesclearwritinginwhich producesclearwritinginwhichthe producesclearandcoherentwriting producesclearandwelldeveloped
writinginwhichthedevelopment,
thedevelopment,organization, development,organization,andstyle inwhichthedevelopment,
organization,andstyleare
organization,andstyleare
andstyleareevident;develops arelargelyappropriatetotask,
purpose,andaudience;develops
appropriatetotask,purpose,and
appropriatetotask,purpose,and
writingwithsomeplanning,
writingbyplanning,revising,editing, audience;withguidanceand
revising,andediting,including
audience;developsandstrengthens
editingforconventions;
rewriting,ortryinganewapproach, support,developsandstrengthens writingonanongoingbasisby
demonstratesbasiccommandof includingeditingforconventions;
writingbyplanning,revising,editing, planning,revising,editing,rewriting,
keyboardingskills.
demonstratessufficientcommandof rewriting,ortryinganewapproach, ortryinganewapproach,including
editingforconventions;
keyboardingskillstotypeuptotwo includingeditingforconventions;
pagesinasinglesitting.
demonstratessufficientcommandof demonstratessufficientcommandof
keyboardingskillstotypeaminimum keyboardingskillstotypetwoor
morepagesinasinglesitting.
oftwopagesinasinglesitting.

Detailed

5.W.78 conductsshortresearchprojects
aboutatopic;recallssome
informationfromexperiences
andsources;providesan
incompletesummaryorlistof
informationinnotes.

AzMERITPerformanceLevelDescriptors

conductsshortresearchprojectsthat conductsresearchprojectsthatuse
conductsshortresearchprojects
thatuseseveralsourcestodiscussa useseveralsourcestoinvestigatea severalhighqualitysourcestofully
investigateatopic;usesrelevant
topic;recallssomeinformationfrom topic;recallsrelevantinformation
informationfromexperiencesand
experiencesandgathersinformation fromexperiencesandgathers
fromsources;providesabrief
relevantinformationfromsources; gatheredfromsources;fully
summaryofinformationinnotesand summarizesorparaphrases
summarizesorparaphrases
finishedwork.
informationinnotesandfinished
informationinnotesandfinished
work.
work.

D-25

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Listening
Detailed 5SL.2 identifiesdetailsofawrittentext determinesthekeydetailsofa
writtentextreadaloudor
readaloudorinformation
presentedindiversemediaand informationpresentedindiverse
mediaandformats,including
formats,includingvisually,
visually,quantitatively,andorally.
quantitatively,andorally.
Detailed

5SL.3

Detailed

5.L.1

identifiesthepointsaspeaker
makes.

Proficient

HighlyProficient

summarizeawrittentextreadaloud
orinformationpresentedindiverse
mediaandformats,including
visually,quantitatively,andorally.

clearlyandcoherentlysummarizesa
writtentextreadaloudor
informationpresentedindiverse
mediaandformats,includingvisually,
quantitatively,andorally.

determinesthepointsaspeaker
summarizesthepointsaspeaker
makesandidentifieskeydetailsthat makesandexplainshoweachclaim
supportthepoints.
issupportedbyreasonsand
evidence.

Language
demonstratesabasic
demonstratesanunderstandingof
understandingoftheconventions theconventionsofstandardEnglish
ofstandardEnglishgrammarand grammarandusagewhenwritingor
usagewhenwritingorspeaking; speaking,understandingthe
functionofconjunctions,
attemptstoformandusethe
perfectverbtenses;attemptsto prepositions,andinterjectionsin
usecorrelativeconjunctions(e.g., generalandtheirfunctionin
particularsentences;formsanduses
either/or,neither/nor).
theperfectverbtenses,usesverb
tensetoconveyvarioustimes,
sequences,states,andconditions,
andrecognizesinappropriateshifts
inverbtense;usescorrelative
conjunctions(e.g.,either/or,
neither/nor).

AzMERITPerformanceLevelDescriptors

Standard Setting Technical Report

EnglishLanguageArts
Grade5

D-26

providesacomprehensivesummary
ofthepointsaspeakermakesand
evaluateshoweachclaimis
supportedbyreasonsandevidence.

demonstratesstrongcommandof
demonstratescommandofthe
theconventionsofstandardEnglish
conventionsofstandardEnglish
grammarandusagewhenwritingor grammarandusagewhenwritingor
speaking,explainingthefunctionof speaking,explainingthefunctionof
conjunctions,prepositions,and
conjunctions,prepositions,and
interjectionsingeneralandtheir
interjectionsingeneralandtheir
functioninparticularsentences;
functioninparticularsentences;
formsandusestheperfectverb
formsandusestheperfectverb
tenses,usesverbtensetoconvey
tenses,usesverbtensetoconvey
varioustimes,sequences,states,and variousspecifictimes,sequences,
states,andconditions,and
conditions,andrecognizesand
correctsinappropriateshiftsinverb recognizesandcorrectsinappropriate
tense;usescorrelativeconjunctions shiftsinverbtense;usescorrelative
(e.g.,either/or,neither/nor).
conjunctions(e.g.,either/or,
neither/nor).

AzMERIT - 2015

EnglishLanguageArts
Grade5

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
demonstratesanunderstandingof demonstratescommandofthe
demonstratesstrongcommandof
Detailed 5.L.2
demonstrateslimited
theconventionsofstandardEnglish
understandingoftheconventions theconventionsofstandardEnglish conventionsofstandardEnglish
capitalization,punctuation,and
ofstandardEnglishcapitalization, capitalization,punctuation,and
capitalization,punctuation,and
spellingwhenwriting;uses
punctuation,andspellingwhen spellingwhenwriting;uses
spellingwhenwriting;uses
punctuationtoseparateitemsina punctuationtoseparateitemsina punctuationtoseparateitemsina
writing;usespunctuationto
separateitemsinaseries;spells series;usesacommatoseparatean series;usesacommatoseparatean series;usesacommatoseparatean
wordscorrectly,consulting
introductoryelementfromtherest introductoryelementfromtherest introductoryelementfromtherestof
referencesasneeded.
ofthesentence;usesacommatoset ofthesentence;usesacommatoset thesentence;usesacommatosetoff
offthewordsyesandno,tosetoffa offthewordsyesandno,tosetoffa thewordsyesandno,tosetoffatag
tagquestionfromtherestofthe
tagquestionfromtherestofthe
questionfromtherestofthe
sentence,andtoindicatedirect
sentence,andtoindicatedirect
sentence,andtoindicatedirect
address;spellswordscorrectly,
address;usesunderlining,quotation address;usesunderlining,quotation
consultingreferencesasneeded.
marks,oritalicstoindicatetitlesof marks,oritalicstoindicatetitlesof
works;spellswordscorrectly,
works;spellswordscorrectly,
consultingreferencesasneeded.
consultingreferencesasneeded.

Detailed

5.L.3

usesknowledgeoflanguageandits usesknowledgeoflanguageandits usesdeepknowledgeoflanguage


usesabasicknowledgeof
conventionswhenwriting,speaking, conventionswhenwriting,speaking, anditsconventionswhenwriting,
languageanditsconventions
reading,orlistening;expands,
speaking,reading,orlistening;
whenwriting,speaking,reading, reading,orlistening;expands,
orlistening;expandsandreduces combines,andreducessentencesfor combines,andreducessentencesfor effectivelyexpands,combines,and
reducessentencesformeaning,
sentencesformeaning;identifies meaning;recognizesthevarietiesof meaning,reader/listenerinterest,
thetypeoflanguageusedin
English(e.g.,dialects,registers)used andstyle;comparesandcontrasts reader/listenerinterest,andstyle;
stories,dramas,orpoems.
instories,dramas,orpoems.
thevarietiesofEnglish(e.g.,dialects, comparesandcontrasts,then
registers)usedinstories,dramas,or analyzes,thevarietiesofEnglish
(e.g.,dialects,registers)usedin
poems.
stories,dramas,orpoems.

AzMERITPerformanceLevelDescriptors

D-27

AzMERIT - 2015

EnglishLanguageArts
Grade5

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
Detailed 5.L.4
clarifiesthemeaningofunknown determinesorclarifiesthemeaning determinesorclarifiesthemeaning determinesorclarifiesandapplies
wordsandphrases,choosing
ofunknownandmultiplemeaning ofunknownandmultiplemeaning themeaningofunknownand
fromalimitedrangeof
wordsandphrases,choosingflexibly wordsandphrases,choosingflexibly multiplemeaningwordsandphrases,
fromarangeofstrategies;uses
strategies;usesimmediateand fromarangeofstrategies;uses
choosingstrategicallyfromarangeof
explicitcontextasacluetothe immediatecontextasacluetothe contextasacluetothemeaningofa strategies;usessentenceand
wordorphrase;usescommon,grade paragraphlevelcontextasaclueto
meaningofawordorphrase;
meaningofawordorphrase;
consultsreferencematerials(e.g., recognizesGreekandLatinaffixes appropriateGreekandLatinaffixes themeaningofawordorphrase;
dictionaries,glossaries,
androots;consultsreference
androotsascluestothemeaningof usesGreekandLatinaffixesandroots
thesauruses),bothprintand
materials(e.g.,dictionaries,
aword;consultsreferencematerials ascluestothemeaningofaword;
digital,todeterminethemeaning glossaries,thesauruses),bothprint (e.g.,dictionaries,glossaries,
consultsreferencematerials(e.g.,
ofkeywordsandphrases.
anddigital,tofindthepronunciation thesauruses),bothprintanddigital, dictionaries,glossaries,thesauruses),
tofindthepronunciationand
bothprintanddigital,tofindthe
anddetermineorclarifythe
pronunciationanddetermineor
meaningofkeywordsandphrases. determineorclarifytheprecise
meaningofkeywordsandphrases. clarifytheprecisemeaningofkey
wordsandphrases.
Detailed

5.L.5

recognizesfigurativelanguage,
basicwordrelationships,and
nuancesinwordmeanings;
recognizescommonidioms,
adages,andproverbs;
understandstherelationship
betweenparticularwords(e.g.,
synonyms,antonyms,
homographs).

AzMERITPerformanceLevelDescriptors

demonstratesunderstandingofbasic demonstratesunderstandingof
figurativelanguage,basicword
figurativelanguage,word
relationships,andnuancesinword relationships,andnuancesinword
meanings;interpretsbasicfigurative meanings;interpretsfigurative
language,includingsimilesand
language,includingsimilesand
metaphors,incontext;recognizes
metaphors,incontext;recognizes
commonidioms,adages,and
andexplainsthemeaningof
proverbs;recognizesthatthe
commonidioms,adages,and
relationshipbetweenparticular
proverbs;usestherelationship
words(e.g.,synonyms,antonyms, betweenparticularwords(e.g.,
homographs)canincrease
synonyms,antonyms,homographs)
understandingofeachofthewords. tobetterunderstandeachofthe
words.

D-28

demonstratesastrongunderstanding
ofcomplexfigurativelanguage,
complexwordrelationships,and
subtlenuancesinwordmeanings;
interpretscomplexfigurative
language,includingsimilesand
metaphors,incontext;recognizes
andanalyzesthemeaningofidioms,
adages,andproverbs;usesthe
relationshipbetweenparticular
words(e.g.,synonyms,antonyms,
homographs)tofullyunderstand
eachofthewords.

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD

Standard

MinimallyProficient
Forgradeappropriate,low
complexitytexts,theLevel1
student

EnglishLanguageArts
Grade6

Standard Setting Technical Report

Proficient
HighlyProficient
PartiallyProficient
Forgradeappropriate,moderateto Forgradeappropriate,high
Forgradeappropriate,lowto
complexitytexts,theLevel4student
moderatecomplexitytexts,theLevel highcomplexitytexts,theLevel3
student
2student

Reading:Literature
looselyreferstothetextto
citestextualevidencetosupport
identifiestextualevidencethat
analysisofwhatthetextsays
supportanalysisofwhatthetext supportsanalysisofwhatthetext
explicitlyaswellasinferencesdrawn
saysexplicitly.
saysexplicitly.
from the text.
identifiesathemeorcentralidea identifiesathemeorcentralideaof determinesathemeorcentralidea
ofatext;providesabasiclistof atext;providesasimplesummaryof ofatextandhowitisconveyed
atextdistinctfrompersonal
eventsinatext.
throughparticulardetails;providesa
opinionsorjudgments.
summaryofthetextdistinctfrom
personalopinionsorjudgments.

Detailed

6.RL.1

Detailed

6.RL.2

Detailed

6.RL.3

identifiesabasicplotofa
particularstoryordramaand
recognizesthatthecharacters
changeduringthestory.

Detailed

6.RL.4

identifiestheliteralmeaningof distinguishesbetweenliteral,
determinesthemeaningofwords
simplewordsandphrasesasthey figurative,andconnotativemeanings andphrasesastheyareusedina
ofwordsandphrasesastheyare
areusedinatext.
text,includingfigurativeand
usedinatext;identifiestheimpact connotativemeanings;analyzesthe
ofspecificwordchoiceonmeaning impactofspecificwordchoiceon
meaningandtone.
andtone.

AzMERITPerformanceLevelDescriptors

describeshowtheplotofa
particularstoryordramaunfolds
andhowthecharacterschange
overall.

D-29

appliesstrongtextualevidencein
supportingacomplexinferenceor
analysisofthetext.
evaluatesthemesorcentralideasin
regardtomajor/minorthemesand
howtheyareconveyedthrough
particulardetails;providesa
comprehensivesummaryofatext
distinctfrompersonalopinionsor
judgments.

describeshowtheplotofaparticular analyzeshowtheplotofaparticular
storyordramaunfoldsinaseriesof storyordramaunfoldsinaseriesof
episodes,aswellashowthe
episodes,aswellashowthe
charactersrespondorchangeasthe responsesandchangesofcomplex
plotmovestowardaresolution.
characterscontributetotheplotasit
movestowardaresolution.
analyzesthemeaningofwordsand
phrasesastheyareusedinatext,
includingfigurativeandconnotative
meanings,andassessestheir
effectiveness;evaluatestheimpactof
specificwordchoiceonmeaningand
tone.

AzMERIT - 2015

EnglishLanguageArts
Grade6

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Detailed 6.RL.5 identifiesaparticularsentence, describeshowaparticularsentence, analyzeshowaparticularsentence, articulateswhytheauthorincludesa
chapter,scene,orstanzacontributes chapter,scene,orstanzafitsintothe particularsentence,chapter,scene,
chapter,scene,orstanzathat
overallstructureofatextand
totheoverallstructureand
orstanza,andanalyzeshowitaffects
contributestotheoverall
developmentofatext.
structureofatext.
contributestothedevelopmentof theoverallstructureofatextand
contributestothedevelopmentof
thetheme,setting,orplot.
thetheme,setting,orplot
throughoutthetext.
Detailed

6.RL.6

identifiesthepointofviewofthe describesthepointofviewofthe
narratororspeakerinatext.
narratororspeakerinatext.

explainshowanauthordevelopsthe analyzeshowanauthordevelopsthe
pointofviewofthenarratoror
pointofviewofthenarratoror
speakerinatext,citingevidenceto
speakerinatext.
supporttheanalysis.

Detailed

6.RL.7

determinesthesimilaritiesinthe comparesandcontraststhe
experienceofreadingastory,
experienceofreadingastory,
drama,orpoemandlisteningto drama,orpoemtolisteningtoor
orviewinganaudio,video,orlive viewinganaudio,video,orlive
versionofthetext.
versionofthetext.

comparesandcontrasts,then
comparesandcontraststhe
experienceofreadingastory,drama, analyzes,theexperienceofreadinga
orpoemtolisteningtoorviewingan story,drama,orpoemtolisteningto
orviewinganaudio,video,orlive
audio,video,orliveversionofthe
text,includingcontrastingwhats/he versionofthetext.Analyzeswhat
seesandhearswhenreadingthe s/he"sees"and"hears"whenreading
thetextcomparedtowhats/he
texttowhats/heperceiveswhen
perceiveswhenlisteningorwatching.
listeningorwatching.

Detailed

6.RL.9

identifiesvarioustextual
elementsindifferentformsor
genreswithsimilarthemesor
topics.

AzMERITPerformanceLevelDescriptors

comparesandcontraststextsin
determinesdifferingtextual
elementsindifferentformsorgenres differentformsorgenres(e.g.,
(e.g.,storiesandpoems;historical storiesandpoems;historicalnovels
novelsandfantasystories)with
andfantasystories)intermsoftheir
similarthemesortopics.
approachestosimilarthemesand
topics.

D-30

compares,contrasts,and
analyzes/evaluatestextsindifferent
formsorgenres(e.g.,storiesand
poems;historicalnovelsandfantasy
stories)intermsoftheirapproaches
tosimilarthemesandtopics.

AzMERIT - 2015

EnglishLanguageArts
Grade6

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
MinimallyProficient
PartiallyProficient
Reading:InformationalText
citestextualevidencetosupport
identifiestextualevidencethat
Detailed 6RI.1 looselyreferstothetextto
analysisofwhatthetextsays
supportanalysisofwhatthetext supportsanalysisofwhatthetext
explicitlyaswellasinferencesdrawn
saysexplicitly.
saysexplicitly.
from the text.
determinesacentralideaofatext
Detailed 6.RI.2 identifiesacentralideaofatext; identifiesacentralideaofatext;
providesabasiclistofeventsina providesasimplesummaryofatext andhowitisconveyedthrough
distinctfrompersonalopinionsor particulardetails;providesa
text.
judgments.
summaryofthetextdistinctfrom
personalopinionsorjudgments.

HighlyProficient
appliesstrongtextualevidencein
supportingacomplexinferenceor
analysisofthetext.
evaluatescentralideasandhowthey
areconveyedthroughparticular
details;providesacomprehensive
summaryofatextdistinctfrom
personalopinionsorjudgments.

identifieshowakeyindividual, explainshowakeyindividual,event, analyzesindetailhowakey


event,orideaisintroducedand orideaisintroduced,illustrated,and individual,event,orideais
introduced,illustrated,and
elaboratedinatext(e.g.,through
illustratedinatext.
elaboratedinatext(e.g.,through
examplesoranecdotes).
examplesoranecdotes).

Detailed

6.RI.3

Detailed

6.RI.4

Detailed

6.RI.5

explainshowaparticularsentence,
locatesaparticularsentence,
paragraph,chapter,orsection
paragraph,chapter,orsection
thatcontributestothe
contributestotheoverallstructure
developmentofthekeyideasofa ofatextandcontributestothe
text.
developmentoftheideas.

Detailed

6.RI.6

identifiesanauthorsexplicit
identifiesanauthorspointofview determinesanauthorspointofview analyzesanauthorspointofview
and purposeinatext;provides
pointofvieworpurposeinatext. orpurposeinatextandidentifies orpurposeinatextandexplains
textualevidencetoshowhowthe
anexampleofwhereitisconveyed howitisconveyedinthetext.
author'spointofviewandpurpose
inthetext.
areconveyedinthetext.

analyzesindetailhowakey
individual,event,orideais
introduced,illustrated,and
elaboratedinatext(e.g.,through
examplesoranecdotes)andanalyzes
relationshipsamongkeyindividuals,
events or ideas
analyzesthemeaningofwordsand
identifiestheliteralmeaningof distinguishesbetweensomeliteral, determinesthemeaningofwords
phrasesastheyareusedinatext,
simplewordsandphrasesasthey figurative,andconnotativemeanings andphrasesastheyareusedina
text,includingfigurative,
ofwordsandphrasesastheyare
includingfigurative,connotative,and
areusedinatext.
usedinatext.
connotative,andtechnicalmeanings. technicalmeanings;evaluatesthe
impactofspecificwordchoice.

AzMERITPerformanceLevelDescriptors

D-31

analyzeshowaparticularsentence,
paragraph,chapter,orsectionfits
intotheoverallstructureofatext
andcontributestothedevelopment
oftheideas.

articulateswhytheauthorusesa
particularsentence,paragraph,
chapter,orsection,andanalyzeshow
itaffectstheoverallstructureofa
textandcontributestothe
developmentoftheideas.

AzMERIT - 2015

EnglishLanguageArts
Grade6

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
integratesinformationpresentedin
Detailed 6.RI.7 identifieskeyinformation
presentedindifferentmediaor differentmediaorformats(e.g.,
visually,quantitatively)aswellasin
formats(e.g.,visually,
wordstoshowapartiallydeveloped
quantitatively)aswellasin
understandingofatopicorissue.
words.

Standard Setting Technical Report

Proficient
integratesinformationpresentedin
differentmediaorformats(e.g.,
visually,quantitatively)aswellasin
wordstodevelopacoherent
understandingofatopicorissue.

HighlyProficient
evaluatesandsynthesizes
informationpresentedindifferent
mediaorformats(e.g.,visually,
quantitatively)aswellasinwordsto
developacomprehensive
understandingofatopicorissue.

tracesandevaluatestheargument
andspecificclaimsinatext,
distinguishingclaimsthatare
supportedbyreasonsandevidence
fromclaimsthatarenot.

tracesandevaluatestheargument
andspecificclaimsinatext,analyzing
howthereasoningandevidence
supportordonotsupporttheclaim.

Detailed

6.RI.8

identifiesspecificclaims,
determinestheargumentand
reasoning,andevidenceinatext. specificclaims,reasoning,and
evidenceinatext.

Detailed

6.RI.9

identifiesexplicitsimilaritiesor comparesandcontraststhewaysin comparesandcontrastsoneauthors comparesandcontrastsoneauthors


differencesbetweentwoauthors' whichtwoauthorspresentevents presentationofeventswiththatof presentationofeventswiththatof
another(e.g.,amemoir
differently.
another(e.g.,amemoirbyone
presentationofevents.
byonepersonandabiographyof
personandabiographyofthat
person);evaluatestheeffectand
thatperson).
impactofthedifferentpresentations.

AzMERITPerformanceLevelDescriptors

D-32

AzMERIT - 2015

EnglishLanguageArts
Grade6

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Writing
Detailed 6.W.1 writesargumentstosupport
writesargumentstosupportclaims writesargumentstosupportclaims writesargumentstosupportclaims
withclearreasonsandrelevant
withclearreasonsandevidence.
claims.
withclearreasonsandrelevant
evidence.
evidence.
a.introducesclaim(s)andorganizes
a.introducesclaim(s).
thereasonsandevidencewith
a.introducesclaim(s)andorganizes a.introducessolidclaim(s)and
organizesthereasonsandevidence
thereasonsandevidenceclearly.
b.supportsclaim(s)withreasons, purpose.
clearlyandlogically.
usingsourcesornontextual
b.supportsclaim(s)withreasonsand b.supportsclaim(s)withclear
evidenceanddemonstratinga
basicunderstandingofthetopic evidence,usingappropriatesources reasonsandrelevantevidence,using b.supportsclaim(s)withclear
anddemonstratingageneral
ortext.
crediblesourcesanddemonstrating reasonsandrelevantevidence,using
crediblesourcesanddemonstratinga
understandingofthetopicortext. anunderstandingofthetopicor
thoroughunderstandingofthetopic
text.
c.useswords,phrases,and
ortext.
clausestostatetheclaim(s)and c.useswords,phrases,andclauses
tostatetherelationshipsamong
reasons.
c.useswords,phrases,andclauses
c.useswords,phrases,andclausesto
claim(s)andreasons.
toclarifytherelationshipsamong
claim(s)andreasons.
d.usesaninformalstyle.
clarifyandelaborateonthe
d.establishesaformalstylebutdoes
relationshipsamongclaim(s)and
notconsistentlymaintainit.
d.establishesandmaintainsaformal reasons.
e.providesaconcluding
style.
statementorsectionthat
illogicallyfollowsfromthe
e.providesaconcludingstatement
d.establishesandmaintainsaformal
argumentpresented.
orsectionthatpartiallyfollowsfrom e.providesaconcludingstatement style.
theargumentpresented.
orsectionthatfollowsfromthe
argumentpresented.
e.providesawelldeveloped
concludingsectionthatclearlyand
logicallyfollowsfromtheargument
presented.

AzMERITPerformanceLevelDescriptors

D-33

AzMERIT - 2015

EnglishLanguageArts
Grade6

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Detailed 6.W.2 writesinformative/explanatory writesinformative/explanatorytexts
textstorestateatopicand
toexplainatopicandconveyideas,
conveyideas,concepts,and
concepts,andinformationthrough
informationthroughthe
theselectionandorganizationof
selection,organizationof
relevantcontent.
content.
a.introducesatopic;organizes
a.partiallyintroducesatopic;
ideas,concepts,andinformation,
organizesideas,concepts,and
usingstrategiessuchasdefinition,
information,butinconsistently classification,comparison/contrast,
appliesstrategiessuchas
andcause/effect;includes
definition,classification,
formatting(e.g.,headings),graphics
comparison/contrast,and
(e.g.,charts,tables)whenusefulto
cause/effect.
aidingcomprehension.
b.developsthetopicwithfacts.
c.usesbasictransitionsto
connectideasandconcepts.
d.usessomedomainspecific
vocabularytoinformaboutor
explainthetopic.
e.usesaninformalstyle.
f.providesaconcluding
statementorsectionthat
illogicallyfollowsfromthe
informationorexplanation
d

AzMERITPerformanceLevelDescriptors

Standard Setting Technical Report

Proficient
writesinformative/explanatorytexts
toexamineatopicandconveyideas,
concepts,andinformationthrough
theselection,organization,and
analysisofrelevantcontent.

HighlyProficient
writesinformative/explanatorytexts
toexamineatopicandconveyideas,
concepts,andinformationthrough
theselection,organization,and
analysisofrelevantcontent.

a.introducesatopic;organizesideas, a.clearlyintroducesatopic;logically
concepts,andinformation,using
organizesideas,concepts,and
strategiessuchasdefinition,
information,usingstrategiessuchas
classification,comparison/contrast, definition,classification,
andcause/effect;includesformatting comparison/contrast,and
(e.g.,headings)andgraphics(e.g.,
cause/effect;includesformatting
charts,tables)andmultimediawhen (e.g.,headings)andgraphics(e.g.,
usefultoaidingcomprehension.
charts,tables)inawaythatenhances
theexplanation.
b.developsthetopicwithfacts,
b.developsthetopicwithrelevant
definitions,details,quotations,or
facts,definitions,concretedetails, b.developsthetopicwithsignificant
quotations,orotherinformationand facts,definitions,concretedetails,
otherinformationandexamples.
examples.
insightfulquotations,orother
informationandexamples.
c.usesappropriatetransitionsto
c.usesappropriatetransitionsto
connectideasandconcepts.
clarifytherelationshipsamongideas c.usesappropriatetransitionsto
clarifyandelaborateonthe
d.usessomepreciselanguageand andconcepts.
relationshipsamongideasand
domainspecificvocabularytoinform
d.usespreciselanguageanddomain concepts.
aboutorexplainthetopic.
specificvocabularytoinformabout
d.usespreciselanguageanddomain
e.establishesaformalstylebutdoes orexplainthetopic.
specificvocabularytoenhancethe
notconsistentlymaintainit.
e.establishesandmaintainsaformal explanationofthetopic.
style.
f.providesabasicconcluding
i
h
i ll
bli h
d
i i
f
l

D-34

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
presented.
statementorsectionthatpartially
followsfromtheinformationor
explanationpresented.

Proficient

HighlyProficient
e.establishesandmaintainsaformal
f.providesaconcludingstatementor style.
sectionthatfollowsfromthe
informationorexplanation
f.providesawelldeveloped
presented.
concludingstatementorsectionthat
clearlyandlogicallyfollowsfromthe
informationorexplanation
presented.

producesclearwritinginwhichthe producesclearandcoherentwriting producesclearandwelldeveloped


development,organization,andstyle inwhichthedevelopment,
writinginwhichthedevelopment,
arelargelyappropriatetotask,
organization,andstyleare
organization,andstyleare
purpose,andaudience;develops
appropriatetotask,purpose,and
appropriatetotask,purpose,and
writingbyplanning,revising,editing, audience;developsandstrengthens audience;developsandstrengthens
rewriting,ortryinganewapproach, writingbyplanning,revising,editing, writingonanongoingbasisby
includingeditingforconventions;
rewriting,ortryinganewapproach, planning,revising,editing,rewriting,
ortryinganewapproach,including
demonstratessufficientcommandof includingeditingforconventions;
keyboardingskillstotypeuptothree demonstratessufficientcommandof editingforconventions;
pagesinasinglesitting.
keyboardingskillstotypeaminimum demonstratessufficientcommandof
keyboardingskillstotypethreeor
ofthreepagesinasinglesitting.
morepagesinasinglesitting.

Detailed

6.W.46 producesclearwritinginwhich
thedevelopment,organization,
andstyleareevident;develops
writingwithsomeplanning,
revising,andediting,including
editingforconventions;
demonstratesbasiccommandof
keyboardingskills.

Detailed

6.W.78 conductsshortresearchprojects conductsshortresearchprojectsto


toansweraquestion,drawingon answeraquestion,drawingon
oneortwosources;uses
severalsources;usesinformation
frommultiplesources;assessesthe
informationfromoneortwo
credibilityofsomesources;
sources;paraphrasesthe
paraphrasesthedataand
conclusionsofotherswhile
conclusionsofotherswhileavoiding
avoidingplagiarism.
plagiarism.

AzMERITPerformanceLevelDescriptors

Standard Setting Technical Report

EnglishLanguageArts
Grade6

D-35

conductsshortresearchprojectsto
answeraquestion,drawingon
severalsourcesandrefocusingthe
inquirywhenappropriate;gathers
relevantinformationfrommultiple
sources;assessesthecredibilityof
sourcesasappropriate;quotesor
paraphrasesthedataand
conclusionsofotherswhileavoiding
plagiarism.

conductsresearchprojectstoanswer
animportantquestion,drawingon
severalsourcesandrefocusingthe
inquirywhenappropriate;gathers
relevant,highqualityinformation
frommultiplesources;assessesthe
credibilityofsourcesasappropriate;
citesthedataandconclusionsof
otherswhileavoidingplagiarismand
usingstandardformatforcitation.

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Listening
Detailed 6.SL.2 recallsinformationpresentedin recallsinformationpresentedin
diversemediaandformatsand diversemediaandformatsand
identifiesatopic,text,orissue describesdetailsrelatedtoatopic,
text,orissueunderstudy.
understudy.
Detailed

Detailed

6.SL.3

6.L.1

identifiesaspeaker'sargument
andspecificclaims.

Proficient

HighlyProficient

interpretsinformationpresentedin
diversemediaandformatsand
explainshowitcontributestoa
topic,text,orissueunderstudy.

interpretsandevaluatesinformation
presentedindiversemediaand
formatsandexplainshowit
contributestoatopic,text,orissue
under study.
identifiesaspeaker'sargumentand delineatesaspeaker'sargumentand delineatesaspeaker'sargumentand
specificclaimsandrecognizesthat specificclaims,distinguishingclaims specificclaims,critiquingclaimsand
someclaimsarenotsupportedby thataresupportedbyreasonsand evaluatingwhetherornottheyare
evidencefromclaimsthatarenot.
reasonsandevidence.
supportedbyreasonsandevidence.

Language
demonstratesunderstandingofthe
demonstratesbasic
understandingoftheconventions conventionsofstandardEnglish
ofstandardEnglishgrammarand grammarandusagewhenwritingor
usagewhenwritingorspeaking: speaking:ensuresthatpronounsare
inconsistentlyusespronounsin inthepropercase;usesintensive
thecorrectcase;inconsistently pronouns;recognizesinappropriate
recognizesinappropriateshiftsin shiftsinpronounnumberand
person;recognizesvaguepronouns;
pronounnumberandperson;
andidentifiesvariationsfrom
andidentifiessomevariations
fromstandardEnglish,using
standardEnglishandusesstrategies
basicstrategiestoimprove
toimproveexpressionin
expressioninconventional
conventionallanguage.
language.

AzMERITPerformanceLevelDescriptors

Standard Setting Technical Report

EnglishLanguageArts
Grade6

D-36

demonstratescommandofthe
conventionsofstandardEnglish
grammarandusagewhenwritingor
speaking:ensuresthatpronounsare
inthepropercase;usesintensive
pronouns;recognizesandcorrects
inappropriateshiftsinpronoun
numberandperson;recognizesand
correctsvaguepronouns;and
recognizesvariationsfromstandard
Englishandusesstrategiesto
improveexpressioninconventional
language.

demonstratesstrongcommandof
theconventionsofstandardEnglish
grammarandusagewhenwritingor
speaking:ensuresthatpronounsare
inthepropercase;usesintensive
pronouns;recognizesandcorrects
inappropriateshiftsinpronoun
numberandperson;andrecognizes
andcorrectsvaguepronouns;and
identifiesvariationsfromstandard
Englishandusesspecificstrategiesto
significantlyimproveexpressionin
conventionallanguage.

AzMERIT - 2015

EnglishLanguageArts
Grade6

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
demonstratesanunderstandingof
Detailed 6.L.2
demonstratesalimited
understandingoftheconventions theconventionsofstandardEnglish
ofstandardEnglishcapitalization, capitalization,punctuation,and
punctuation,andspellingwhen spellingwhenwriting:generallyuses
punctuation(commas,parentheses,
writing:inconsistentlyuses
dashes)tosetoff
punctuation(commas,
parentheses,dashes)tosetoff nonrestrictive/parenthetical
nonrestrictive/parenthetical
elements;spellscorrectly.
elements;spellscorrectly.

Standard Setting Technical Report

Proficient
demonstratescommandofthe
conventionsofstandardEnglish
capitalization,punctuation,and
spellingwhenwriting:uses
punctuation(commas,parentheses,
dashes)tosetoff
nonrestrictive/parenthetical
elements;spellscorrectly.

Detailed

6.L.3

usesbasicknowledgeoflanguage usesknowledgeoflanguageandits
anditsconventionswhenwriting, conventionswhenwriting,speaking,
speaking,reading,orlistening, reading,orlistening,generally
applyingbasicvariationsin
varyingsentencepatternsfor
sentencepatternsformeaning, meaning,interest,reader/listener
interest,reader/listenerinterest, interest,andstylewhile
andstylewhileattemptingsome demonstratingsomeconsistencyin
consistencyinstyleandtone.
styleandtone.

Detailed

6.L.4

withstrongsupport,determines generallydeterminesorclarifiesthe determinesorclarifiesthemeaning


orclarifiestheexplicitmeaningof meaningofunknownandmultiple ofunknownandmultiplemeaning
basicwordsandphrases,using meaningwordsandphrases,
wordsandphrases,choosingfroma
contextandGreekandLatin
choosingfromarangeofstrategies; rangeofstrategies;usescontextasa
affixesandrootsascluestothe usesimmediatecontextasaclueto cluetothemeaningofawordor
themeaningofawordorphrase;
meaning,consultingreference
phrase;usescommonGreekand
usescommon,simpleGreekand
materialsasneeded.
Latinaffixesandrootsascluestothe
Latinaffixesandrootsascluestothe meaningoftheword;consults
referencematerialsasneeded;and
meaningoftheword;consults
verifiesthepreliminary
referencematerialsasneeded.
determinationofthemeaningofa
wordorphrase.

AzMERITPerformanceLevelDescriptors

D-37

HighlyProficient
demonstratesstrongandstrategic
commandoftheconventionsof
standardEnglishcapitalization,
punctuation,andspellingwhen
writing:usespunctuation(commas,
parentheses,dashes)tosetoff
nonrestrictive/parenthetical
elements;spellscorrectly.

usesknowledgeoflanguageandits strategicallyusesknowledgeof
conventionswhenwriting,speaking, languageanditsconventionswhen
reading,orlistening,varying
writing,speaking,reading,or
sentencepatternsformeaning,
listening,varyingsentencepatterns
interest,reader/listenerinterest,and formeaning,interest,reader/listener
stylewhilemaintainingconsistency interest,andstylewhilemaintaining
instyleandtone.
strongconsistencyinstyleandtone.

definitivelydeterminesorclarifiesthe
meaningofunknownandmultiple
meaningwordsandphrases,
choosingfromarangeofstrategies;
usessentenceandpassagelevel
contextasacluetothemeaningofa
wordorphrase;usescommonGreek
andLatinaffixesandrootsascluesto
themeaningoftheword;consults
specificandappropriatereference
materialsasneeded;andverifiesthe
preliminarydeterminationofthe
meaningofawordorphrase.

AzMERIT - 2015

EnglishLanguageArts
Grade6

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
MinimallyProficient
PartiallyProficient
demonstratesabasicunderstanding demonstratesunderstandingof
Detailed 6.L.5
demonstratesalimited
offigurativelanguage,word
figurativelanguage,word
understandingoffigurative
languageandwordrelationships relationships,andnuancesinword relationships,andnuancesinword
meanings,includinginterpreting
inwordmeanings,includingin
meanings,includingidentifying
identifyingfiguresofspeechand figuresofspeechincontext,using figuresofspeechincontext,using
usingtherelationshipbetween therelationshipbetweenparticular therelationshipbetweenparticular
particularwordstobetter
wordstobetterunderstandeachof wordstobetterunderstandeachof
understandeachofthewords, thewords,anddistinguishingamong thewords,anddistinguishingamong
andininconsistently
theconnotationsofwordswith
theconnotationsofwordswith
distinguishingamongthe
similardenotations.
similardenotations.
connotationsofwordswith
similardenotations.

AzMERITPerformanceLevelDescriptors

D-38

HighlyProficient
demonstratescommandoffigurative
language,wordrelationships,and
nuancesinwordmeanings,including
interpretingcomplexfiguresof
speechincontext,evaluatingthe
relationshipbetweenparticular
wordstobetterunderstandeachof
thewords,anddistinguishingamong
theconnotationsofwordswith
similardenotationsandapplying
theminspeakingandwriting.

AzMERIT - 2015

EnglishLanguageArts
Grade7

Appendix D. Performance Level Descriptors (PLDs)


Proficient
Standard
MinimallyProficient
PartiallyProficient
Forgradeappropriate,lowto
Forgradeappropriate,moderateto
Forgradeappropriate,low
highcomplexitytexts,theProficient
complexitytexts,theMinimally moderatecomplexitytexts,the
PartiallyProficientstudent
student
Proficientstudent
Reading:Literature
identifiessometextualevidencethat citesseveralpiecesoftextual
Detailed 7.RL.1 looselyreferstothetextto
evidencetosupportanalysisofwhat
supportanalysisofwhatthetext supportsanalysisofwhatthetext
thetextsaysexplicitlyaswellas
saysexplicitly.
saysexplicitly.
inferencesdrawnfromthetext.
PLD

Detailed

7.RL.2

identifiesathemeorcentralidea identifiesathemeorcentralideaof determinesathemeorcentralidea


ofatext;providesasequenceof atext;providesasimpleobjective ofatextandanalyzesits
developmentoverthecourseofa
summaryofatext.
eventsinatext.
text;providesanobjectivesummary
ofatext.

Detailed

7.RL.3

identifiesparticularelementsofa explainshowparticularelementsof
astoryordramainteract(e.g.,how
storyordrama(e.g.,settingor
settingshapesthecharactersor
characters).
plot).

Detailed

7.RL.4

identifiestheliteralorfigurative
meaningofwordsandphrasesas
theyareusedinatext;identifies
rhymesandotherrepetitionsof
soundsinaspecificverseor
stanzaofapoemorsectionofa
storyordrama.

Detailed

7.RL.5

describesadramasorpoemsform analyzeshowadramasorpoems
identifiesadramasorpoems
formorstructure(e.g.,soliloquy, orstructure(e.g.,soliloquy,sonnet) formorstructure(e.g.,soliloquy,
sonnet)contributestoitsmeaning.
andhowitcontributestothe
sonnet).
meaningofthetext.

AzMERITPerformanceLevelDescriptors

Standard Setting Technical Report


Highly Proficient

Forgradeappropriate,high
complexitytexts,theHighly
Proficientstudent
appliesnumerous,strongpiecesof
textualevidenceinsupportinga
complexinferenceoranalysisofthe
text.
evaluatesthemesorcentralideasof
atextandanalyzestheir
developmentoverthecourseofa
text;providesacomprehensive,
objectivesummaryofatext.

evaluatestherelationshipsbetween
particularelementsofastoryor
drama(e.g.,howsettingshapesthe
charactersorplot)andanalyzesthe
impact.
determinesthemeaningand
determinesthemeaningofwords
distinguishesbetweenliteral,
analyzestheimpactofwordsand
figurative,andconnotativemeanings andphrasesastheyareusedina
phrasesastheyareusedinatext,
text,includingfigurativeand
ofwordsandphrasesastheyare
usedinatext;describestheimpact connotativemeanings;analyzesthe includingfigurativeandconnotative
meanings,andassessestheir
ofrhymesandotherrepetitionsof impactofrhymesandother
effectiveness;analyzesandevaluates
repetitionsofsounds(e.g.,
sounds(e.g.,alliteration)ona
theimpactofrhymesandother
specificverseorstanzaofapoemor alliteration)onaspecificverseor
repetitionsofsounds(e.g.,
stanzaofapoemorsectionofa
sectionofastoryordrama.
alliteration)onaspecificverseor
storyordrama.
stanzaofapoemorsectionofastory
ordrama.

D-39

analyzeshowparticularelementsof
astoryordramainteract(e.g.,how
settingshapesthecharactersor
plot).

analyzesandevaluateshowa
dramasorpoemsformorstructure
(e.g.,soliloquy,sonnet)contributes
toitsmeaningandimpact.

AzMERIT - 2015

EnglishLanguageArts
Grade7

Appendix D. Performance Level Descriptors (PLDs)


Proficient
Standard
MinimallyProficient
PartiallyProficient
explainsthedifferencesinpointsof analyzeshowanauthordevelops
Detailed 7.RL.6 identifiesthepointsofviewof
andcontraststhepointsofviewof
differentcharactersornarrators viewofdifferentcharactersor
differentcharactersornarratorsina
narratorsinatext.
inatext.
text.
PLD

Detailed

7.RL.7

identifiessimilaritiesor
differencesbetweenawritten
story,drama,orpoemtoits
audio,filmed,staged,or
multimediaversion.

Detailed

7.RL.9

comparesandcontrastsafictional
identifiessimilaritiesor
differencesbetweenafictional portrayalofatime,place,or
characterandahistoricalaccountof
portrayalofatime,place,or
characterandahistoricalaccount thesameperiod;identifieshowan
authoroffictionaltershistory.
ofthesameperiod.

AzMERITPerformanceLevelDescriptors

comparesandcontrastsawritten
story,drama,orpoemtoitsaudio,
filmed,staged,ormultimedia
version,andidentifiesthe
techniquesuniquetoeachmedium
(e.g.,lighting,sound,color,or
camerafocusandanglesinafilm).

D-40

comparesandcontrastsawritten
story,drama,orpoemtoitsaudio,
filmed,staged,ormultimedia
version,analyzingtheeffectsof
techniquesuniquetoeachmedium
(e.g.,lighting,sound,color,or
camerafocusandanglesinafilm).

Standard Setting Technical Report


Highly Proficient

analyzesandevaluatesthe
effectivenessofhowanauthor
developsandcontraststhepointsof
viewofdifferentcomplexcharacters
ornarratorsinatext.
analyzesandcritiquesanaudio,
filmed,staged,ormultimediaversion
ofawrittenstory,dramaorpoemas
comparedtoitswrittenversion;
evaluatestheimpactand
effectivenessoftechniquesuniqueto
eachmedium(e.g.,lighting,sound,
color,orcamerafocusandanglesina
film)

comparesandcontrastsafictional comparesandcontrasts,then
analyzes,afictionalportrayalofa
portrayalofatime,place,or
characterandahistoricalaccountof time,place,orcharacteranda
historicalaccountofthesameperiod
thesameperiodasameansof
understandinghowauthorsoffiction tounderstandandevaluatehow
authorsoffictionuseoralterhistory.
useoralterhistory.

AzMERIT - 2015

EnglishLanguageArts
Grade7

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


Proficient
Standard
MinimallyProficient
PartiallyProficient
Reading:InformationalText
identifiessometextualevidencethat citesseveralpiecesoftextual
Detailed 7.RI.1 looselyreferstothetextto
evidencetosupportanalysisofwhat
supportanalysisofwhatthetext supportsanalysisofwhatthetext
thetextsaysexplicitlyaswellas
saysexplicitly.
saysexplicitly.
inferencesdrawnfromthetext.
PLD

Highly Proficient

appliesnumerous,strongpiecesof
textualevidenceinsupportinga
complexinferenceoranalysisofthe
text.

Detailed

7.RI.2

identifiestwoormorecentralideas determinestwoormorecentral
identifiesacentralideaofthe
text;providesabasicsequenceof ofatext;providesasummaryofa ideasinatextandanalyzestheir
developmentoverthecourseofthe
text.
eventsorideasinatext.
text;providesanobjectivesummary
ofatext.

evaluatestwoormorecentralideas
andanalyzestheirdevelopmentover
thecourseofthetext;providesa
comprehensive,objectivesummary
ofatext.

Detailed

7.RI.3

analyzestheinteractionsbetween
individuals,events,andideasina
text(e.g.,howideasinfluence
individualsorevents,orhow
individualsinfluenceideasorevents).

analyzesandevaluatescomplex
relationshipsbetweenindividuals,
events,andideasinatext(e.g.,how
ideasinfluenceindividualsorevents,
orhowindividualsinfluenceideasor
events).

Detailed

7.RI.4

identifiessomeofthe
relationshipsbetween
individuals,events,andideasina
text(e.g.,howideasinfluence
individualsorevents,orhow
individualsinfluenceideasor
events).
identifiestheliteralorfigurative
meaningofwordsandphrasesas
theyareusedinatext;
recognizesthataspecificword
choicehasanimpactonmeaning
andtone.

Detailed

7.RI.5

determinestherelationships
betweenindividuals,events,and
ideasinatext(e.g.,howideas
influenceindividualsorevents,or
howindividualsinfluenceideasor
events).

analyzesthemeaningofwordsand
determinesthemeaningofwords
distinguishesbetweenliteral,
phrasesastheyareusedinatext,
figurative,connotative,andtechnical andphrasesastheyareusedina
includingfigurative,connotative,and
meaningsofwordsandphrasesas text,includingfigurative,
theyareusedinatext;describesthe connotative,andtechnicalmeanings; technicalmeanings;evaluatesthe
effectofaspecificwordchoiceon
impactofaspecificwordchoiceon analyzestheimpactofaspecific
wordchoiceonmeaningandtone. meaningandtone.
meaningandtone.

describesthestructureanauthor determinesthestructureanauthor analyzesthestructureanauthoruses evaluatestheeffectivenessofthe


toorganizeatext,includinghowthe structureanauthorusestoorganize
usestoorganizeatext;identifies usestoorganizeatext;describes
atextandanalyzeshowthemajor
howthemajorsectionscontributeto majorsectionscontributetothe
themajorsectionsofthetext.
thestructureofthewholetextorto wholeandtothedevelopmentofthe sectionscontributetothewholeand
tothedevelopmentoftheideas;can
ideas.
thedevelopmentoftheideas.
articulatehowadifferenttext
structuremightimpactthemeaning
ofthetext.

AzMERITPerformanceLevelDescriptors

D-41

AzMERIT - 2015

Standard Setting Technical Report

EnglishLanguageArts
Grade7

Appendix D. Performance Level Descriptors (PLDs)


Standard
MinimallyProficient
PartiallyProficient
Detailed 7.RI.6 identifiesanauthorspurposein identifiesanauthorspointofview
atextandwhatdistinguisheshis orpurposeinatextanddescribes
howtheauthordistinguisheshisor
orherpositionfromthatof
herpositionfromthatofothers.
others.
PLD

Proficient

Highly Proficient

determinesanauthorspointofview
orpurposeinatextandanalyzes
howtheauthordistinguisheshisor
herpositionfromthatofothers.

analyzesanauthorspointofview
andpurposeinatext;evaluateshow
effectivelytheauthordistinguishes
hisorherpositionfromthatofothers
toaccomplishhisorherpurpose.

comparesandcontrastsatexttoan
audio,video,ormultimediaversion
ofthetext,analyzingeachmediums
portrayalofthesubject(e.g.,how
thedeliveryofaspeechaffectsthe
impactofthewords).

evaluatestheeffectivenessand
impactofatextascomparedtoan
audio,filmed,staged,ormultimedia
version,analyzingeachmediums
portrayalofthesubject(e.g.,howthe
deliveryofaspeechaffectsthe
impactofthewords).

Detailed

7.RI.7

identifiessimilarities or
differencesbetweenatextand
anaudio,video,ormultimedia
versionofthetext.

Detailed

7.RI.8

Detailed

7.RI.9

explicatesandevaluatesthe
argumentandspecificclaimsina
complextext;citesspecificlanguage
orexamplesinthetextinan
assessmentofwhetherornotthe
reasoningissoundandtheevidence
isrelevantandsufficienttosupport
the claims
describeshowtwoormoreauthors analyzeshowtwoormoreauthors citestextualevidenceinan
describeshowtwoormore
authorswritingaboutthesame writingaboutthesametopicshape writingaboutthesametopicshape evaluationofthedifferentrhetorical
effectsusedbytwoormoreauthors
theirpresentationsofkey
topicshapetheirpresentationsof theirpresentationsofkey
informationbyemphasizingdifferent informationbyemphasizingdifferent writingaboutthesametopicshape
keyinformation.
theirpresentationsofkey
evidenceoradvancingdifferent
evidence.
informationbyemphasizingdifferent
interpretationsoffacts.
evidenceoradvancingdifferent
interpretationsoffacts.
tracestheargumentandaclaim
inatext,identifyingthe
reasoningandevidenceusedto
supporttheclaim.

AzMERITPerformanceLevelDescriptors

comparesandcontrastsatexttoan
audio,video,ormultimediaversion
ofthetext,identifyinghoweach
mediumportraysthesubject(e.g.,
howthedeliveryofaspeechaffects
theimpactofthewords).
tracesandevaluatestheargument
andclaimsinatext,describingthe
reasoningandevidenceusedto
supporttheclaims.

D-42

tracesandevaluatestheargument
andspecificclaimsinatext,
assessingwhetherthereasoningis
soundandtheevidenceisrelevant
andsufficienttosupporttheclaims.

AzMERIT - 2015

EnglishLanguageArts
Grade7

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


Proficient
Highly Proficient
Standard
MinimallyProficient
PartiallyProficient
Writing
Detailed 7.W.1 writesargumentsthatincludea writesargumentstosupportclaims writesargumentstosupportclaims writesclearargumentstosupport
claimswithlogicalreasoningand
withclearreasonsandrelevant
claimsupportedbyextratextual withreasonsandevidenceto
relevantevidence.
evidence.
evidence.
supportaclaim.
PLD

a.introducesclaim(s)andorganizes a.introducesclaim(s),acknowledges a.introducessupportableclaim(s),


acknowledgesandevaluatesalternate
thereasonsandevidencelogically. alternateoropposingclaims,and
organizesthereasonsandevidence oropposingclaim(s),andorganizesthe
reasonsandevidencelogically.
b.supportsclaim(s)withreasoning logically.
b.supportsclaim(s),
andevidencefromthetext
b.supportsclaim(s)withlogical
b.supportsclaim(s)withlogical
demonstratingabasic
(extratextualevidencemay
reasoningandspecificevidence,using
reasoningandrelevantevidence,
understandingofthetopicor
occasionallybepresent)that
accurate,crediblesourcesand
text.
demonstratesanunderstandingof usingaccurate,crediblesourcesand
demonstratinganacuteunderstanding
demonstratinganunderstandingof ofthetopicortext.
thetopicortext.
thetopicortext.
c.usestransitionalwordstolink
claim(s),reasons,andevidence. c.useswords,phrases,andclauses
c.usesprecisewords,phrases,and
c.useswords,phrases,andclauses clausestocreatecohesivelinksamong
tolinkclaim(s),reasons,and
tocreatecohesionandclarifythe
evidence.
d.writesinaninformalstyle.
majorsectionsoftheessayandclarify
relationshipsamongclaim(s),
therelationshipsamongclaim(s),
d.establishesformalstyle,butdoes reasons,andevidence.
e.providesaconcluding
reasons,andevidence.
notconsistentlymaintainit.
statementorsection.
d.establishesandmaintainsaformal d.establishesandmaintainsaformal
styleandobjectivetone.
e.providesaconcludingstatement style.
orsectionthatfollowsfromthe
e.providesaconcludingstatement e.providesacompellingconcluding
argumentpresented.
statementorsectionthatincludes
orsectionthatfollowsfromand
analysisoftheevidenceandfollows
supportstheargumentpresented.
andsupportstheargumentpresented.

a.introducesclaim(s)and
organizesthereasonsand
evidence.

AzMERITPerformanceLevelDescriptors

D-43

AzMERIT - 2015

Standard Setting Technical Report

EnglishLanguageArts
Grade7

Appendix D. Performance Level Descriptors (PLDs)


Standard
MinimallyProficient
PartiallyProficient
Detailed 7.W.2 writesinformative/explanatory writesinformative/explanatorytext
texttodescribeatopicthrough toexplainatopicthroughthe
theselectionandorganizationof selectionandorganizationof
relevantcontent.
content.
PLD

Proficient

Highly Proficient

writesinformative/explanatorytexts
toexamineatopicandconveyideas,
concepts,andinformationthrough
theselection,organization,and
analysisofrelevantcontent.

writesinformative/explanatorytexts
toexamineatopicandconvey
complexideas,concepts,and
informationwithastrongly
developedfocusthroughthe
selection,organization,andanalysis
ofrelevantcontent.

a.introducesatopic;attemptsan a.introducesatopicclearly;
a.introducesatopicclearly,
organizationofideas,concepts, organizesideas,concepts,and
andinformationusingstrategies information,usingstrategiessuchas previewingwhatistofollow;
suchasdefinition,classification, definition,classification,
organizesideas,concepts,and
comparison/contrast,and
comparison/contrast,and
information,usingstrategiessuchas
cause/effect;includesformatting
cause/effect.
definition,classification,
(e.g.,headings)andgraphics(e.g., comparison/contrast,and
cause/effect;includesformatting
b.describesthetopicwithfacts, charts,tables)whenusefultoaid
comprehension.
(e.g.,headings)andgraphics(e.g.,
definitions,concretedetails,
charts,tables)whenusefultoaiding
quotations,orotherinformation
b.developsthetopicwithfacts,
comprehension.
andexamples.
definitions,concretedetails,
quotations,orotherinformationand b.developsthetopicwithrelevant
c.usesbasictransitionstolink
examples.
facts,definitions,concretedetails,
ideasandconcepts.
quotations,orotherinformationand
c.usesappropriatetransitionsto
examples.
d.usestopicappropriate
createcohesion.
languageandvocabularyto
c.usesappropriatetransitionsto
informaboutordescribethe
d.usestopicappropriatelanguage createcohesionandclarifythe
topic.
andvocabularytoinformaboutor relationshipsamongideasand
explainthetopic.
concepts.
e.usesaninformalstyle.
f.providesaconcluding
statementorsection.

AzMERITPerformanceLevelDescriptors

a.introducesatopicwithastrongly
developedfocususingappropriate
strategiessuchasdefinition,
classification,comparison/contrast,
andcauseandeffect;includesformal
formatting(e.g.,headings)and
graphics(e.g.,charts,tables)to
enhancecomprehension.
b.developsthetopicwithanalysisof
relevantfacts,complexideas,
definitions,concretedetails,
quotations,orotherinformationand
examplesappropriatetothe
audience'sknowledgeofthetopic.

c.usesappropriateandvaried
transitionstocreatecohesionand
clarifytherelationshipsamongideas
e.establishesformalstyle,butdoes d.usespreciselanguageanddomain andconcepts.
notconsistentlymaintainit.
specificvocabularytoinformabout
d.usespreciselanguageanddomain
orexplainthetopic.
f
d
l d
f
b l
h

D-44

AzMERIT - 2015

EnglishLanguageArts
Grade7

Appendix D. Performance Level Descriptors (PLDs)


Proficient
Standard
MinimallyProficient
PartiallyProficient
producesclearwritinginwhichthe producesclearandcoherentwriting
Detailed 7.W.46 produceswritinginwhichthe
development,organization,and development,organization,andstyle inwhichthedevelopment,
styleareappropriatetothetask; areappropriatetotaskandpurpose; organization,andstyleare
developsandstrengthenswritingas appropriatetotask,purpose,and
developswritingbyapplying
audience;developsandstrengthens
neededbyplanning,revising,
planning,revising,editing,or
writingasneededbyplanning,
editing,rewriting,ortryinganew
rewriting;editingshould
revising,editing,rewriting,ortrying
demonstratebasiccommandof approach,focusingonhowwell
purposehasbeenaddressed;editing anewapproach,focusingonhow
Languagestandards13upto
shoulddemonstratebasiccommand wellpurposeandaudiencehave
andincludinggrade7;uses
technologytoproducewriting. ofLanguagestandards13uptoand beenaddressed;editingshould
includinggrade7;usestechnologyto demonstratecommandofLanguage
standards13uptoandincluding
producewritingandreferto
grade7;usestechnologytoproduce
sources.
writingandcitesources.
PLD

AzMERITPerformanceLevelDescriptors

D-45

Standard Setting Technical Report


Highly Proficient

produceswelldevelopedand
cohesivewritinginwhichthe
development,organization,andstyle
areappropriatetotask,purpose,and
audience;developsandstrengthens
writingasneededbyplanning,
revising,editing,rewriting,ortryinga
newapproach,successfully
addressingtheintendedpurposeand
audience;editingshould
demonstrateskillfulcommandof
Languagestandards13uptoand
includinggrade7;usestechnologyto
producewritingandcitesourcesas
wellasconnectideasefficiently.

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


Standard
MinimallyProficient
PartiallyProficient
Detailed 7.W.78 conductsshortresearchprojects conductsshortresearchprojectsto
toansweraquestion,drawingon answeraquestion,drawingon
severalsources;gathersrelevant
minimalsources;gathers
informationfromafewsources; informationfrommultiplesources
andredirectsinquiryasappropriate;
assessesthecredibilityof
assessesthecredibilityandaccuracy
sources;paraphrasesthedata
andconclusionsofotherswhile ofeachsource;andquotesor
paraphrasesthedataand
avoidingplagiarism.
conclusionsofotherswhileavoiding
plagiarism.
PLD

Detailed

7.SL.2

identifiesthemainideasand
supportingdetailspresentedin
diversemediaandformats.

Detailed

7.SL.3

identifiesaspeaker'sargument
andspecificclaims.

AzMERITPerformanceLevelDescriptors

Standard Setting Technical Report

EnglishLanguageArts
Grade7

Listening
explainsthemainideasand
supportingdetailspresentedin
diversemediaandformatsandhow
theyrelatetothetopic.

explainsaspeakersargumentand
specificclaims,identifyingthe
relevanceoftheevidence
introduced.

D-46

Proficient

conductsshortresearchprojectsto
answeraquestion,drawingon
severalsourcesandgenerating
additionalrelated,focusedideas;
gathersrelevantinformationfrom
multiplesources;assessesthe
credibilityandaccuracyofeach
source;andquotesorparaphrases
thedataandconclusionsofothers
whileavoidingplagiarismand
followingastandardformatfor
citation.

Highly Proficient

conductsshortresearchprojectsto
answeranimportantquestion,
drawingonseveralsourcesand
generatingadditionalrelated,
focused,andevaluativeideas;
gathersrelevantinformationfrom
multiplesources;evaluatesthe
credibilityandaccuracyofeach
source;andjudiciouslyquotesor
paraphrasesthedataandconclusions
ofotherswhileavoidingplagiarism
andfollowingastandardformatfor
citation.

analyzesthemainideasand
analyzesthemainideasand
supportingdetailspresentedin
supportingdetailspresentedin
diversemediaandformatsand
diversemediaandformats(e.g.,
visually,quantitatively,orally)and evaluateshowwelltheideasclarifya
explainshowtheideasclarifyatopic, topic,text,orissueunderstudy.
text,orissueunderstudy.

delineatesaspeaker'sargumentand
specificclaims,evaluatingthe
soundnessofthereasoningandthe
relevanceandsufficiencyofthe
evidence.

delineatesaspeaker'sargumentand
specificclaims,evaluatingthe
soundnessofreasoningandthe
relevanceandsufficiencyofthe
evidenceusingrealworldapplication,
rhetoricalanalysis,orexaminationof
discoursestyle.

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


Standard
MinimallyProficient
PartiallyProficient
Language
demonstratesunderstandingofthe
Detailed 7.L.1
demonstratesbasic
understandingoftheconventions conventionsofstandardEnglish
ofstandardEnglishgrammarand grammarandusagewhenwritingor
usagewhenwritingorspeaking speakinginthefollowingareas:
inthefollowingareas:
a.identifiesthefunctionofphrases
andclausesingeneralandtheir
a.recognizesthefunctionof
functioninspecificsentences.
phrasesandclausesingeneral
andtheirfunctioninspecific
b.choosesamongsimple,
sentences.
compound,complex,andcompound
b.reliesonsimple,compound, complexsentencestosignal
andcomplexsentencestosignal relationshipsamongideas.
differingrelationshipsamong
ideas.
c.placesphrasesandclauseswithin
asentence,avoidingmisplacedand
danglingmodifiers.
c.placesphrasesandclauses
withinasentences.
PLD

AzMERITPerformanceLevelDescriptors

Standard Setting Technical Report

EnglishLanguageArts
Grade7

D-47

Proficient

Highly Proficient

demonstratescommandofthe
conventionsofstandardEnglish
grammarandusagewhenwritingor
speaking:

demonstratescorrectapplicationand
commandoftheconventionsof
standardEnglishgrammarandusage
whenwritingorspeaking:

a.explainsthefunctionofphrases
andclausesingeneralandtheir
functioninspecificsentences.

a.analyzesthefunctionofphrases
andclausesingeneralandexplains
theirfunctioninspecificsentences.

b.choosesamongsimple,
compound,complex,andcompound
complexsentencestosignaldiffering
relationshipsamongideas.

b.makesinformedchoicesamong
simple,compound,complex,and
compoundcomplexsentencesto
signaldifferingrelationshipsamong
ideas.

c.placesphrasesandclauseswithina
sentence,recognizingandcorrecting c.effectivelyplacesphrasesand
clauseswithinasentence,
misplacedanddanglingmodifiers.
recognizingandcorrectingmisplaced
anddanglingmodifiers.

AzMERIT - 2015

Standard Setting Technical Report

EnglishLanguageArts
Grade7

Appendix D. Performance Level Descriptors (PLDs)


Standard
MinimallyProficient
PartiallyProficient
demonstratesunderstandingofthe
Detailed 7.L.2
demonstratesbasic
understandingoftheconventions conventionsofstandardEnglish
ofstandardEnglishcapitalization, capitalization,punctuation,and
punctuation,andspellingwhen spellingwhenwriting:
writing:
a.usesacommatoseparate
a.inconsistentlyusesacommato coordinateadjectives(e.g.,Itwasa
separatecoordinateadjectives fascinating,enjoyablemovie;but
not:Heworeanold[,]greenshirt).
(e.g.,Itwasafascinating,
enjoyablemovie;butnot:He
woreanold[,]greenshirt).
b.spellsgradelevelwordscorrectly.
PLD

Proficient

demonstratescommandofthe
conventionsofstandardEnglish
capitalization,punctuation,and
spellingwhenwriting:
a.usesacommatoseparate
coordinateadjectives(e.g.,Itwasa
fascinating,enjoyablemovie;but
not:Heworeanold[,]greenshirt).

7.L.3

a.usesacommatoseparate
coordinateadjectives(e.g.,Itwasa
fascinating,enjoyablemovie;butnot:
Heworeanold[,]greenshirt).
b.spellsabovegradelevelwords
correctly.

usesknowledgeoflanguageandits usesknowledgeoflanguageandits
usesabasicknowledgeof
conventionswhenwriting,speaking, conventionswhen
languageanditsconventions
writing,speaking,reading,or
whenwriting,speaking,reading, reading,orlistening:
listening:
orlistening:
a.chooseslanguagethatexpresses
a.chooseslanguagethatexpresses
ideaspreciselyandconcisely,
a.inconsistentlychooses
ideaspreciselyandconcisely,
occasionallyrecognizingand
languagethatexpressesideas
recognizingandeliminating
eliminatingwordinessand
withoutwordinessand
wordinessandredundancy.
redundancy.
redundancy.

AzMERITPerformanceLevelDescriptors

demonstratescorrectapplicationand
commandoftheconventionsof
standardEnglishcapitalization,
punctuation,andspellingwhen
writing:

b.spellscorrectly.

b.spellsgradelevelwords
correctly.
Detailed

Highly Proficient

D-48

usescomprehensiveknowledgeof
languageanditsconventionswhen
writing,speaking,reading,or
listening:
a.strategicallychooseslanguagethat
expressesideaspreciselyand
concisely,consciouslyrecognizing
andeliminatingwordinessand
redundancy.

AzMERIT - 2015

EnglishLanguageArts
Grade7

Appendix D. Performance Level Descriptors (PLDs)


Proficient
Standard
MinimallyProficient
PartiallyProficient
inconsistentlydeterminesor
determinesorclarifiesthemeaningof determinesorclarifiesthemeaningof
Detailed 7.L.4
PLD

Standard Setting Technical Report


Highly Proficient
authoritativelydeterminesorclarifies
themeaningofunknownandmultiple
meaningwordsandphrases,choosing
flexiblyfromarangeofstrategies:

clarifiesthemeaningofunknown unknownandmultiplemeaningwords unknownand


andmultiplemeaningwordsand
andphrases,usingoneormore
multiplemeaningwordsandphrases,
phrases,usingatleastonestrategy: strategies:
choosingflexiblyfromarangeof
strategies:
a.usescontext(e.g.,theoverall
a.usescontext(e.g.,theoverall
a.usescontext(e.g.,theoverallmeaning
meaningofasentenceor
meaningofasentenceorparagraph;a a.usescontext(e.g.,theoverall
ofasentenceorparagraph;awords
paragraph;awordspositionor
wordspositionorfunctionina
meaningofasentenceorparagraph;a positionorfunctioninasentence)asa
functioninasentence)asaclueto sentence)asacluetothemeaningofa wordspositionorfunctionina
cluetothemeaningofawordorphrase.
themeaningofawordorphrase.
wordorphrase.
sentence)asacluetothemeaningofa
wordorphrase.
b.usescommon,gradeappropriate
b.usescommon,belowgradeGreek b.usescommon,gradeappropriate
GreekorLatinaffixesandrootsasclues
orLatinaffixesandrootsascluesto GreekorLatinaffixesandrootsasclues b.usescommon,gradeappropriate
tothemeaningofaword(e.g.,
themeaningofaword
tothemeaningofaword(e.g.,
GreekorLatinaffixesandrootsasclues belligerent,bellicose,rebel).
belligerent,bellicose,rebel).
tothemeaningofaword(e.g.,
c.consultsgeneralandspecialized
belligerent,bellicose,rebel).
c.consultsgeneralandspecialized
referencematerials(e.g.,
c.consultsgeneralandspecialized
referencematerials(e.g.,dictionaries,
dictionaries,glossaries,
referencematerials(e.g.,dictionaries, c.consultsgeneralandspecialized
glossaries,thesauruses),bothprintand
thesauruses),bothprintanddigital, glossaries,thesauruses),bothprintand referencematerials(e.g.,dictionaries, digital,tofindthepronunciationofa
tofindthepronunciationofaword digital,tofindthepronunciationofa
glossaries,thesauruses),bothprintand wordordetermineorclarifyitsprecise
ordetermineorclarifyitsprecise
wordordetermineorclarifyitsprecise digital,tofindthepronunciationofa
meaningoritspartofspeech.
meaningoritspartofspeech.
meaningoritspartofspeech.
wordordetermineorclarifyitsprecise
meaningoritspartofspeech.
d.verifiesthepreliminarydetermination
d.verifiesthepreliminary
d.verifiesthepreliminary
ofthemeaningofawordorphrase(e.g.,
determinationofthemeaningofa determinationofthemeaningofaword d.verifiesthepreliminarydetermination bycheckingtheinferredmeaningin
wordorphrase(e.g.,bychecking
orphrase(e.g.,bycheckingtheinferred ofthemeaningofawordorphrase
contextorinadictionary).
theinferredmeaningincontextor meaningincontextorinadictionary). (e.g.,bycheckingtheinferredmeaning
inadictionary).
incontextorinadictionary).

AzMERITPerformanceLevelDescriptors

D-49

AzMERIT - 2015

EnglishLanguageArts
Grade7

Appendix D. Performance Level Descriptors (PLDs)


Proficient
Standard
MinimallyProficient
PartiallyProficient
demonstratesbasicunderstandingof demonstratesunderstandingof
Detailed 7.L.5
demonstrateslimited
figurativelanguage,word
figurativelanguage,word
understandingoffigurative
language,wordrelationships,and relationships,andnuancesinword relationships,andnuancesinword
meanings:
meanings:
nuancesinwordmeanings:
PLD

a.inconsistentlyidentifiesfigures a.identifiesfiguresofspeech(e.g.,
ofspeech(e.g.,literary,biblical, literary,biblical,mythological
allusions)incontext.
mythologicalallusions)in
context.
b.identifiestherelationship
betweenparticularwords(e.g.,
b.inconsistentlyidentifiesthe
relationshipbetweenparticular synonym/antonym,analogy)to
betterunderstandeachofthe
basicwords(e.g.,
synonym/antonym,analogy)to words.
betterunderstandeachofthe
c.identifiestheconnotations
words.
(associations)ofwordswithsimilar
denotations(definitions)(e.g.,
c.inconsistentlyidentifiesthe
refined,respectful,polite,
connotations(associations)of
wordswithsimilardenotations diplomatic,condescending).
(definitions)(e.g.,refined,
respectful,polite,diplomatic,
condescending).

AzMERITPerformanceLevelDescriptors

D-50

a.interpretsfiguresofspeech(e.g.,
literary,biblical,andmythological
allusions)incontext.

Standard Setting Technical Report


Highly Proficient

demonstratesdeepunderstandingof
figurativelanguage,word
relationships,andnuancesinword
meanings:
a.interpretsfiguresofspeech(e.g.,
literary,biblical,mythological
allusions)incontexttoevaluatethe
effectofdictionuponthetext.

b.usestherelationshipbetween
b.usestherelationshipbetween
particularwords(e.g.,
particularwords(e.g.,
synonym/antonym,analogy)to
betterunderstandeachofthewords. synonym/antonym,analogy)to
evaluatetheeffectofdictionupon
thetext.
c.distinguishesamongthe
connotations(associations)ofwords
withsimilardenotations(definitions) c.distinguishesamongthe
connotations(associations)ofwords
(e.g.,
withsimilardenotations(definitions)
refined,respectful,polite,
(e.g.,refined,respectful,polite,
diplomatic,condescending).
diplomatic,condescending)to
evaluatetheeffectofdictionupon
thetext.

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD

Standard

EnglishLanguageArts
Grade8

Proficient
MinimallyProficient
PartiallyProficient
Forgradeappropriate,lowto
Forgradeappropriate,moderateto
Forgradeappropriate,low
complexitytexts,theMinimally moderatecomplexitytexts,the highcomplexitytexts,theProficient
PartiallyProficientstudent
student
Proficientstudent
Reading:LiteraryText
citestextualevidencetosupport citestextualevidencetosupport citesthetextualevidencethatmost
ananalysisofwhatthetextsays ananalysisofwhatthetextsays stronglysupportsananalysisofwhat
thetextsaysexplicitlyaswellas
explicitlyaswellasinferences
explicitly.
inferencesdrawnfromthetext.
drawnfromthetext.

Standard Setting Technical Report

HighlyProficient
Forgradeappropriate,highcomplexity
texts,theHighlyProficientstudent

appliesthoroughtextualevidenceto
stronglysupportadeepanalysisofthe
textaswellascomplexinferences
drawnfromthetext.

Detailed

8.RL.1

Detailed

8.RL.2

identifiesathemeorcentral
ideaofatext;identifies
characters,setting,andplot;
providesalistofeventsfrom
thetext.

Detailed

8.RL.3

Detailed

8.RL.4

analyzesandevaluatesthe
effectivenessofanauthor'suseof
dialogueorincidentsinastoryor
dramatopropeltheaction,reveal
aspectsofthecharacter,orprovokea
decision
determinesthemeaningofwordsand determinesthemeaningandevaluates
identifiestheliteralorfigurative distinguishesbetweenliteral,
theimpactofwordsandphrases,
phrases,includingfigurativeand
meaningofwordsandphrases figurative,andconnotative
meaningsofwordsandphrases connotativemeanings;analyzesthe includingfigurativeandconnotative
astheyareusedinatext;
meanings;analyzesandevaluatesthe
impactofspecificwordchoiceson
astheyareusedinatext;
identifieswordsthatimpact
determinestheeffectofspecific meaningandtone,includinganalogies impactofspecificwordchoiceson
meaningandtone.
meaningandtone,includinganalogies
orallusionstoothertexts.
wordchoicesonmeaningand
orallusionstoothertexts.
tone,includinganalogiesor
allusionstoothertexts.
identifiesspecificlinesof
dialogueorincidentsinastory
ordramathatpropeltheaction
andrevealaspectsofthe
character.

AzMERITPerformanceLevelDescriptors

identifiesathemeorcentralidea determinesathemeorcentralideaof
ofatextanddeterminesdetails atextandanalyzesitsdevelopment
oreventsthatdevelopit;explains overthecourseofatext,includingits
relationshiptothecharacters,setting,
characters,setting,andplot;
andplot;providesanobjective
providesasimple,objective
summaryofthetext.
summaryofthetext.

describeshowspecificlinesof
dialogueorincidentsinastoryor
dramapropeltheactionand
revealaspectsofthecharacter.

D-51

analyzeshowspecificlinesofdialogue
orincidentsinastoryordramapropel
theaction,revealaspectsofthe
character,orprovokeadecision.

determinestwoormorethemesor
centralideasandanalyzestheir
developmentoverthecourseofatext;
evaluatesthetheme(s)orcentral
idea(s)andtherelationshiptonarrative
elements;providesaconciseand
comprehensiveobjectivesummaryof
thetext.

AzMERIT - 2015

EnglishLanguageArts
Grade8

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
MinimallyProficient
PartiallyProficient
comparesandcontraststhestructure
Detailed 8.RL.5 comparesandcontraststhe
comparesandcontraststhe
structureoftwotexts.
structureoftwoormoretexts, oftwoormoretexts,analyzinghow
describingtheconnectiontotheir thedifferingstructureofeachtext
contributestoitsmeaningandstyle.
meaningandstyle.

Standard Setting Technical Report


HighlyProficient
comparesandcontrasts,thenevaluates
foreffectiveness,thestructureoftwo
ormoretexts,analyzinghowthe
differingstructureofeachtext
contributestomeaningandstyle.

Detailed

8.RL.6

identifiesthatdifferencesinthe describeshowdifferencesinthe
pointsofviewofthecharacters pointsofviewofthecharacters
orthereaderaffectthemeaning orthereadercontributetoan
understandingofthetext.
ofthetext.

analyzeshowdifferencesinthepoints evaluatestheimpactofdifferencesin
ofviewofthecharactersorthereader thepointsofviewofthecharactersor
thereader(e.g.,createdthroughthe
(e.g.,createdthroughtheuseof
dramaticirony)createsucheffectsas useofdramaticirony)andthe
effectivenessofcreatingsuspenseor
suspenseorhumorinthetext.
humorinthetext.

Detailed

8.RL.7

Detailed

8.RL.9

describestheextenttowhicha
filmofastoryordramastays
faithfultoordepartsfromthe
textorscript,identifyingthe
choicesmadebythedirectoror
actors
determineshowamodernwork
identifiesarelationship
offictiondrawsonexplicit
betweenamodernworkof
fictionandpatternsofeventsor themes,patternsofevents,or
charactertypesfrommyths,
charactertypesfrommyths,
traditionalstories,orreligious traditionalstories,orreligious
works,describinghowthe
works.
materialisrenderednew.

evaluatestheextenttowhichafilmof
astoryordramastaysfaithfultoor
departsfromthetextorscript;
critiquesthechoicesmadebythe
directororactorsandproposes
alternate treatments
analyzeshowamodernworkoffiction evaluateshowamodernworkoffiction
drawsonthemes,patternsofevents, drawsonthemes,patternsofevents,
orcharactertypesfrommyths,
orcharactertypesfrommyths,
traditionalstories,orreligiousworks, traditionalstories,orreligiousworks;
includinghowthematerialisrendered evaluatestheimpactofthenewly
renderedmaterial.
new.

identifiestheextenttowhicha
filmofastoryordramastays
faithfultoordepartsfromthe
textorscript.

AzMERITPerformanceLevelDescriptors

D-52

analyzestheextenttowhichafilmof
astoryordramastaysfaithfultoor
departsfromthetextorscript,
evaluatingthechoicesmadebythe
directororactors.

AzMERIT - 2015

EnglishLanguageArts
Grade8

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
MinimallyProficient
PartiallyProficient
Reading:InformationalText
Detailed 8.RI.1 citestextualevidencetosupport citestextualevidencetosupport citesthetextualevidencethatmost
ananalysisofwhatthetextsays ananalysisofwhatthetextsays stronglysupportsananalysisofwhat
thetextsaysexplicitlyaswellas
explicitlyaswellasinferences
explicitly.
inferencesdrawnfromthetext.
drawnfromthetext.
Detailed

8.RI.2

Detailed

8.RI.3

Detailed

8.RI.4

Detailed

8.RI.5

Standard Setting Technical Report


HighlyProficient
appliesthoroughtextualevidenceto
stronglysupportadeepanalysisofthe
textaswellascomplexinferences
drawnfromthetext.

identifiesacentralideaofatext determinesacentralideaofatextand determinesandanalyzesthecentral


identifiesacentralideaofa
ideasofatextandanalyzestheir
analyzesitsdevelopmentoverthe
text;providesalistofeventsor anddescribesitsdevelopment
developmentoverthecourseofatext;
courseofatext,includingits
overthecourseofatext;
detailsfromthetext.
evaluatesthestrengthofthe
relationshiptosupportingideas;
providesasimple,objective
providesanobjectivesummaryofthe supportingideas;providesa
summaryofthetext.
comprehensiveobjectivesummaryof
text.
the text
evaluateshowatextmakes
identifiesthatatextmakes
describeshowatextmakes
analyzeshowatextmakes
explicitconnectionsamongand explicitconnectionsamongand connectionsamonganddistinctions connectionsamonganddistinctions
distinctionsbetweenindividuals, distinctionsbetweenindividuals, betweenindividuals,ideas,orevents betweenindividuals,ideas,orevents
ideas,orevents(e.g.,through ideas,orevents(e.g.,through
(e.g.,throughcomparisons,analogies, (e.g.,throughcomparisons,analogies,
orcategories).
comparisons,analogies,or
comparisons,analogies,or
orcategories).
categories)
categories)
identifiestheliteralorfigurative determinesthemeaningofbasic determinesthemeaningofwordsand analyzesthemeaningofwordsand
phrasesastheyareusedinatext,
phrasesastheyareusedinatext,
meaningofwordsandphrases wordsandphrasesastheyare
usedinatext,includingcommon includingfigurative,connotative,and includingfigurative,connotative,and
astheyareusedinatext;
technicalmeanings;evaluatesthe
technicalmeanings;analyzesthe
identifiestheimpactofspecific figurative,connotative,and
rhetoricaleffectofspecificword
wordchoicesonmeaningand technicalmeanings;describesthe impactofspecificwordchoiceson
impactofspecificwordchoices meaningandtone,includinganalogies choicesonmeaningandtone,including
tone.
analogiesorallusionstoothertexts.
onmeaningandtone,including orallusionstoothertexts.
analogiesorallusionstoother
texts.
describesthestructureofa
specificparagraphinatext;
describestheroleofparticular
sentencesincreatingthat
structure.

AzMERITPerformanceLevelDescriptors

identifiesthestructureofa
specificparagraphinatextand
describesitseffectonatext;
describestheroleofparticular
sentencesindevelopingand
refiningakeyconcept.

D-53

evaluatestheeffectofthestructureof
analyzesindetailthestructureofa
specificparagraphinatext,including aspecificparagraphinatextandits
roleinthetextasawhole,including
theroleofparticularsentencesin
developingandrefiningakeyconcept. theroleofparticularsentencesin
developingandrefiningakeyconcept.

AzMERIT - 2015

EnglishLanguageArts
Grade8

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Detailed 8.RI.6 identifiesanauthorspointof identifiesanauthorspointof
vieworpurposeinatext;
vieworpurposeinatextand
identifiesexampleswherethe describeshowtheauthor
authoracknowledgesor
acknowledgesandrespondsto
respondstoconflictingevidence conflictingevidenceor
or viewpoints
viewpoints
comparesandcontraststheuse
Detailed 8.RI.7 identifiesdifferencesor
similaritiesinthepresentation ofdifferentmedia(e.g.,printor
ofaparticulartopicorideaas digitaltext,video,multimedia)in
presentingaparticulartopicor
presentedindifferentmedia
(e.g.,printordigitaltext,video, idea.
multimedia).

Proficient
determinesanauthorspointofview
orpurposeinatextandanalyzeshow
theauthoracknowledgesand
respondstoconflictingevidenceor
viewpoints.

Standard Setting Technical Report


HighlyProficient
analyzesanauthorspointofviewor
purposeinatextandevaluatesthe
effectofhowtheauthoracknowledges
andrespondstoconflictingevidenceor
viewpoints.

evaluatesandcritiquestheuseof
evaluatestheadvantagesand
disadvantagesofusingdifferentmedia differentmedia(e.g.,printordigital
text,video,multimedia)topresenta
(e.g.,printordigitaltext,video,
particulartopicoridea,providing
multimedia)topresentaparticular
specificevidenceassupport.
topicoridea.

Detailed

8.RI.8

identifiestheargumentor
specificclaimsinatext,
describingthereasoningand
evidenceusedtosupportthe
argumentorclaims.

describestheargumentand
specificclaimsinatext,
discussingwhetherthereasoning
issoundandtheevidenceis
relevantandsufficient.

delineatesandevaluatesthe
argumentandspecificclaimsinatext,
assessingwhetherthereasoningis
soundandtheevidenceisrelevant
andsufficient;recognizewhen
irrelevantevidenceisintroduced.

Detailed

8.RI.9

identifiesacaseinwhichtwoor
moretextsprovideconflicting
informationonthesametopic,
andidentifieswherethetexts
disagree.

describesacaseinwhichtwoor
moretextsprovideconflicting
informationonthesametopic,
andidentifieswherethetexts
disagreeonmattersoffact.

analyzesacaseinwhichtwoormore analyzesandevaluatesacaseinwhich
textsprovideconflictinginformation twoormoretextsprovideconflicting
informationonthesametopic,and
onthesametopic,andidentifies
wherethetextsdisagreeonmattersof identifieswherethetextsdisagreeon
mattersoffactorinterpretation,
factorinterpretation.
evaluatingthestrengthorreliabilityof
each.

AzMERITPerformanceLevelDescriptors

D-54

synthesizestheargumentandspecific
claimsinatext,citingspecificlanguage
toevaluatewhetherthereasoningis
soundandtheevidenceisrelevantand
sufficient;recognizesirrelevant
evidenceandprovesitsirrelevancy.

AzMERIT - 2015

EnglishLanguageArts
Grade8

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Detailed

8.W.1

writesargumentstosupport
claimswithreasonsand
evidence.

Proficient
Writing
writesargumentstosupportclaims
writesargumentstosupport
claimswithreasonsandrelevant withclearreasonsandrelevant
evidence.
evidence.

a.introducesclaim(s),states
a.introducesclaim(s),states
opposingclaims,andorganizes alternateoropposingclaims,and
organizesthereasonsand
reasonsandevidence.
evidencelogically.
b.supportsclaimswith
b.supportsclaimswithreasoning
extratextualevidence,and
andevidence,usingsourcesand
demonstratingabasic
demonstratinganunderstanding
understandingofthetopicor
ofthetopicortext.
text.

a.introducesclaim(s),acknowledges
anddistinguishestheclaim(s)from
alternateoropposingclaims,and
organizesthereasonsandevidence
logically.
b.supportsclaim(s)withlogical
reasoningandrelevantevidence,
usingaccurate,crediblesourcesand
demonstratinganunderstandingof
thetopicortext.

c.usestransitionwordstolink c.useswords,phrases,and
claim(s),counterclaims,reasons, clausestoclarifytherelationships
amongclaim(s),counterclaims, c.useswords,phrases,andclausesto
andevidence.
createcohesionandclarifythe
reasons,andevidence.
relationshipsamongclaim(s),
d.attemptstoestablishaformal
counterclaims,reasons,andevidence.
d.establishesaformalstyle.
style.
e.providesaconcluding
statementorsection.

AzMERITPerformanceLevelDescriptors

Standard Setting Technical Report


HighlyProficient
writesargumentstosupportclaimswith
clearreasonsandanalysisofrelevant
evidence.
a.introducesclaims,acknowledgesand
distinguishestheclaimsfromalternateor
opposingclaims,evaluatingtheirvalidity,
andorganizesthereasonsandevidence
logically.
b.supportsclaimswithaclearposition
basedonlogicalreasoningandrelevant
evidenceusingaccurate,crediblesources
anddemonstratingadeepunderstanding
ofthetopicortext.
c.usesavarietyofwords,phrases,and
clausestocreatecohesionandclarifythe
relationshipsamongclaim(s),
counterclaims,reasons,andevidence.

d.establishesandmaintainsaformal
d.establishesandmaintainsaformal styleandobjectivetonethatenhances
theargument.
style.

e.providesaconcluding
statementorsectionthat
supportstheargument
presented.

e.providesaconcludingstatementor e.providesacompellingconcluding
sectionthatfollowsfromandsupports statementorsectionthatfollowsfrom
andsupportstheargumentpresented.
theargumentpresented.

D-55

AzMERIT - 2015

EnglishLanguageArts
Grade8

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Detailed 8.W.2 writesinformative/explanatory writesinformative/explanatory writesinformative/explanatorytexts writesinformative/explanatorytextsto
texttodescribeatopicthrough textstoexplainatopicand
toexamineatopicandconveyideas, examineatopicandconveyideas,
theselectionandorganization conveyideas,concepts,and
concepts,andinformationthroughthe concepts,andinformationwitha
informationthroughtheselection selection,organization,andanalysisof stronglydevelopedfocusthroughthe
ofcontent.
selection,organization,andanalysisof
andorganizationofcontent.
relevantcontent.
highlyrelevantcontent.
a.introducesatopic;attempts
a.introducesatopicclearly,
a.introducesatopicclearly,
anorganizationofideas,
previewingwhatistofollow;
concepts,andinformation.
previewingwhatistofollow;organizes a.introducesacomplextopicclearly,
organizesideas,concepts,and
ideas,concepts,andinformationinto previewingwhatistofollow;organizes
ideas,concepts,andinformationinto
informationintobroader
broadercategories;includes
b.summarizesthetopicwith
categories.
formatting(e.g.,headings),graphics broadercategories;includesformatting
facts,definitions,concrete
(e.g.,charts,tables),whenusefulto (e.g.,headings),andgraphics(e.g.,
details,quotations,orother
charts,tables)whenusefultoenhance
b.developsthetopicwithfacts, aidcomprehension.
informationandexamples.
comprehension.
definitions,concretedetails,
c.usesappropriatetransitions quotations,orotherinformation b.developsthetopicwithrelevant,
b.developsandanalyzesthetopicwith
andexamples.
wellchosenfacts,definitions,
tocreatecohesion.
concretedetails,quotations,orother relevant,wellchosenfacts,definitions,
concretedetails,quotations,orother
c.usesappropriatetransitionsto informationandexamples.
d.usestopicappropriate
informationandexamplesappropriate
createcohesionandclarifythe
languageandvocabularyto
totheaudience'sknowledgeofthe
relationshipsamongideasand
c.usesappropriateandvaried
inform.
topic.
concepts.
transitionstocreatecohesionand
clarifytherelationshipsamongideas
e.attemptsaformalstyle.
c.effectivelyusesappropriateand
andconcepts.
d.usestopicappropriate
variedtransitionstocreatecohesion
languageanddomainspecific
f.providesaconcluding
vocabularytoinformaboutor
d.usespreciselanguageanddomain andclarifytherelationshipsamong
statementorsection.
explainthetopic.
specificvocabularytoinformaboutor complexideasandconcepts.
explainthetopic.
d.usespreciselanguageanddomain
e.establishesaformalstyle.
e.establishesandmaintainsaformal specificvocabularytomanagethe
complexityofthetopic.
style.
f.providesaconcluding
h f ll

AzMERITPerformanceLevelDescriptors

D-56

AzMERIT - 2015

EnglishLanguageArts
Grade8

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
producesclearandcoherentwritingin producesclearandcoherentwritingin
Detailed 8.W.46 produceswritinginwhichthe producesclearandcoherent
whichthedevelopment,organization, whichthedevelopment,organization,
development,organization,and writinginwhichthe
andstyleareappropriatetotask,
styleareappropriatetotaskand development,organization,and andstyleareappropriatetotask,
purpose,andaudience;developsand purpose,andaudience;developsand
styleareappropriatetotask,
purpose;developsand
strengthenswritingasneededby
strengthenswritingasneeded purpose,andaudience;develops strengthenswritingasneededby
planning,revising,editing,rewriting, planning,revising,editing,rewriting,or
andstrengthenswritingas
byplanning,revising,editing,
ortryinganewapproach,focusingon tryinganewapproach,focusingon
neededbyplanning,revising,
rewriting,ortryinganew
approach,focusingonpurpose editing,rewriting,ortryinganew howwellpurposeandaudiencehave howwellpurposeandaudiencehave
beenaddressed;editingfor
approach,focusingonhowwell beenaddressed;editingfor
andaudience;editingshould
conventionsshoulddemonstrateskillful
demonstratebasiccommandof purposeandaudiencehavebeen conventionsshoulddemonstrate
commandofLanguagestandards13 commandofLanguagestandards13
Languagestandards13upto addressed;editingshould
uptoandincludinggrade8;uses
uptoandincludinggrade8;uses
demonstratecommandof
andincludinggrade8;uses
technologytoproducewritingand
technologytoproducewriting. Languagestandards13uptoand technologytoproducewritingand
presenttherelationshipsbetween
presenttherelationshipsbetween
includinggrade8;uses
informationandideasinadynamic
informationandideasefficiently.
technologytoproducewriting
way.
andpresenttherelationships
betweeninformationandideas.

Detailed

8.W.78 conductsshortresearchprojects conductsshortresearchprojects conductsshortresearchprojectsto


toansweraquestion,drawing toansweraquestion(includinga answeraquestion(includingaself
selfgeneratedquestion),drawing generatedquestion),drawingon
onminimalsources;gathers
severalsourcesandgenerating
onseveralsourcesand
relevantinformationfrom
additionalrelated,focusedquestions
sourcesandredirectsinquiryas generatingadditionalideas;
thatallowformultipleavenuesof
gathersrelevantinformation
appropriate;assessesthe
frommultiplesources;assesses exploration;gathersrelevant
credibilityofeachsource;
informationfrommultiplesources;
quotesorparaphrasesthedata thecredibilityandaccuracyof
assessesthecredibilityandaccuracy
andconclusionsofotherswhile eachsource;quotesor
ofeachsource;andquotesor
avoidingplagiarism.Attemptsto paraphrasesthedataand
paraphrasesthedataandconclusions
conclusionsofotherswhile
followastandardformatfor
avoidingplagiarismandfollowing ofotherswhileavoidingplagiarism
citation.
andfollowingastandardformatfor
astandardformatforcitation.
citation.

AzMERITPerformanceLevelDescriptors

D-57

conductsshortresearchprojectsto
answeraquestion(includingaself
generatedquestion),drawingon
severalsourcesandgenerating
additionalrelated,focusedquestions
thatallowformultipleavenuesof
explorationandevaluation;gathers
andsynthesizesrelevantinformation
frommultiplesources;assessesthe
credibilityandaccuracyofeachsource;
andjudiciouslyquotesorparaphrases
thedataandconclusionsofothers
whileavoidingplagiarismandfollowing
astandardformatforcitation.

AzMERIT - 2015

EnglishLanguageArts
Grade8

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
MinimallyProficient
PartiallyProficient
Listening
determinesthepurposeof
Detailed
identifiesthepurposeof
analyzesthepurposeofinformation
informationpresentedindiverse presentedindiversemediaand
informationpresentedin
8.SL.2 diversemediaandformats.
mediaandformatsanddescribes formatsandevaluatesthemotives
themotivesbehindits
behinditspresentation.
presentation.
delineatesaspeakersargumentand
Detailed
identifiesaspeaker'sargument explainsaspeakersargument
specificclaims,evaluatingthe
andspecificclaims.
andspecificclaims,identifying
whetherthereasoningissound. soundnessofthereasoningandthe
relevanceandsufficiencyofthe
8.SL.3
evidenceandidentifyingwhen
irrelevantevidenceisintroduced.

AzMERITPerformanceLevelDescriptors

D-58

Standard Setting Technical Report


HighlyProficient
analyzesandevaluatestheinformation
presentedindiversemediaandformats
tocritiquethemotivesandevaluate
theimpactofthepresentation.
tracesandanalyzestheargumentand
specificclaimsofaspeaker,citing
specificexamplestoevaluatewhether
thereasoningissoundandthe
evidenceisrelevantandsufficient;
recognizesirrelevantevidenceand
provesitsirrelevancy.

AzMERIT - 2015

EnglishLanguageArts
Grade8

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Language
demonstratesstrongcommandofthe
demonstratesunderstandingof demonstratescommandofthe
Detailed 8.L.1
demonstratesbasic
conventionsofstandardEnglish
conventionsofstandardEnglish
theconventionsofstandard
understandingofthe
conventionsofstandardEnglish Englishgrammarandusagewhen grammarandusagewhenwritingor grammarandusagewhenwritingor
speaking:
speaking:
writingorspeaking:
grammarandusagewhen
writingorspeaking:
a.explainsthefunctionofverbals
a.explainsthefunctionofverbals
a.describesthefunctionof
(gerunds,participles,infinitives)in
(gerunds,participles,infinitives)in
verbals(gerunds,participles,
a.recognizesthefunctionof
generalandtheirfunctioninparticular generalandevaluatestheirfunctionin
infinitives)ingeneralandtheir
verbals(gerunds,participles,
particularsentences.
infinitives)ingeneralandtheir functioninparticularsentences. sentences.
functioninparticularsentences.
b.formsandusesverbsintheactive b.intentionallyformsandusesverbsin
b.formsandusesverbsinthe
theactiveandpassivevoicetoachieve
andpassivevoice.
b.inconsistentlyformsanduses activeandpassivevoice.
adesiredstyle.
verbsintheactiveandpassive
c.generallyformsandusesverbs c.formsandusesverbsinthe
voice.
indicative,imperative,interrogative, c.strategicallyformsandusesverbsin
intheindicative,imperative,
conditional,andsubjunctivemood. theindicative,imperative,
c.inconsistentlyformsanduses interrogative,conditional,and
interrogative,conditional,and
subjunctivemood.
verbsintheindicative,
subjunctivemood.
d.recognizesandcorrects
imperative,interrogative,
inappropriateshiftsinverbvoiceand
d.recognizesandoccasionally
conditional,andsubjunctive
d.analyzesandcorrectsinappropriate
correctsinappropriateshiftsin mood.
mood.
shiftsinverbvoiceandmood.
verbvoiceandmood.
d.recognizesinappropriate
shiftsinverbvoiceandmood.

AzMERITPerformanceLevelDescriptors

D-59

AzMERIT - 2015

EnglishLanguageArts
Grade8

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
MinimallyProficient
PartiallyProficient
Detailed 8.L.2
demonstratesawarenessofthe demonstratesbasic
demonstratescommandofthe
conventionsofstandardEnglish understandingoftheconventions conventionsofstandardEnglish
capitalization,punctuation,and ofstandardEnglishcapitalization, capitalization,punctuation,and
spellingwhenwriting:
punctuation,andspellingwhen spellingwhenwriting:
writing:
a.inconsistentlyuses
a.usespunctuation(comma,ellipsis,
punctuation(comma,ellipsis, a.generallyusespunctuation
dash)toindicateapauseorbreak.
dash)toindicateapauseor
(comma,ellipsis,dash)toindicate
break.
apauseorbreak.
b.usesanellipsistoindicatean
omission.
b.inconsistentlyusesanellipsis b.usesanellipsistoindicatean
toindicateanomission.
omission.
c.spellscorrectly.
c.spellsbelowgradelevel
correctly.
Detailed

8.L.3

HighlyProficient
demonstratesstrongandpurposeful
commandoftheconventionsof
standardEnglishcapitalization,
punctuation,andspellingwhenwriting:
a.judiciouslyusespunctuation
(comma,ellipsis,dash)toindicatea
pauseorbreak.
b.usesanellipsistoindicatean
omission.
c.spellsunfamiliarandabovegrade
levelwordscorrectly.

c.spellsgradelevelwords
correctly.

demonstratesbasicknowledgeof usesknowledgeoflanguageandits
conventionswhenwriting,speaking,
languageanditsconventions
whenwriting,speaking,reading, reading,orlistening:
orlistening:
a.usesverbsintheactiveandpassive
voiceandintheconditionaland
a.inconsistentlyusesverbsin a.usesverbsintheactiveand
subjunctivemoodtoachieve
theactiveandpassivevoiceand passivevoiceandinthe
particulareffects(e.g.,emphasizing
conditionalandsubjunctive
intheconditionaland
theactororthe
moodtoachieveparticular
subjunctivemoodtoachieve
action;expressinguncertaintyor
effects(e.g.,emphasizingthe
particulareffects(e.g.,
describingastate
actororthe
emphasizingtheactororthe
action;expressinguncertainty action;expressinguncertaintyor contrarytofact).
ordescribingastatecontraryto describingastate
contrarytofact).
fact).

attemptstoapplyofthe
conventionsoflanguagewhen
writing,speaking,reading,or
listening:

AzMERITPerformanceLevelDescriptors

Standard Setting Technical Report

D-60

usescomprehensiveknowledgeof
languageanditsconventionswhen
writing,speaking,reading,orlistening:
a.strategicallyusesverbsintheactive
andpassivevoiceandintheconditional
andsubjunctivemoodtoachieve
particulareffects(e.g.,emphasizingthe
actorortheaction;expressing
uncertaintyordescribingastate
contrarytofact).

AzMERIT - 2015

EnglishLanguageArts
Grade8

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Detailed 8.L.4
inconsistentlydeterminesor
generallydeterminesorclarifies
clarifiesthemeaningof
themeaningofunknownand
unknownandmultiplemeaning multiplemeaningwordsor
wordsorphrases,usingatleast phrases,usingoneormore
onestrategy:
strategies:

Proficient
determinesorclarifiesthemeaningof
unknownand
multiplemeaningwordsorphrases,
choosingflexiblyfromarangeof
strategies:

Standard Setting Technical Report


HighlyProficient
authoritativelydeterminesorclarifies
themeaningofunknownandmultiple
meaningwordsorphrases,choosing
flexiblyfromarangeofstrategies:

a.usescontext(e.g.,theoverall
meaningofasentenceorparagraph;a
a.usescontext(e.g.,theoverall
meaningofasentenceorparagraph;a wordspositionorfunctionina
sentence)asacluetothemeaningofa
wordspositionorfunctionina
sentence)asacluetothemeaningof wordorphrase.
awordor
b.usescommon,gradeappropriate
phrase.
GreekorLatinaffixesandrootsasclues
b.usescommon,belowgrade b.usescommon,grade
b.usescommon,gradeappropriate tothemeaningofaword(e.g.,
precede,recede,secede).
GreekorLatinaffixesandroots appropriateGreekorLatinaffixes GreekorLatinaffixesandrootsas
ascluestothemeaningofa
androotsascluestothemeaning cluestothemeaningofaword
c.consultsgeneralandspecialized
word.
ofaword
(e.g.,precede,recede,secede).
referencematerials(e.g.,dictionaries,
(e.g.,precede,recede,secede).
glossaries,thesauruses),bothprintand
c.consultsgeneraland
c.consultsgeneralandspecialized
specializedreferencematerials c.consultsgeneraland
referencematerials(e.g.,dictionaries, digital,tofindthepronunciationofa
wordordetermineorclarifyitsprecise
(e.g.,dictionaries,glossaries,
specializedreferencematerials glossaries,thesauruses),bothprint
thesauruses),bothprintand
(e.g.,dictionaries,glossaries,
anddigital,tofindthepronunciation meaningoritspartofspeech.
digital,tofindthepronunciation thesauruses),bothprintand
ofawordordetermineorclarifyits
ofawordordetermineorclarify digital,tofindthepronunciation precisemeaningoritspartofspeech. d.verifiesthepreliminary
determinationofthemeaningofa
itsprecisemeaningoritspartof ofawordordetermineorclarify
wordorphrase(e.g.,bycheckingthe
speech.
itsprecisemeaningoritspartof d.verifiesthepreliminary
inferredmeaningincontextorina
speech.
determinationofthemeaningofa
d.verifiesthepreliminary
wordorphrase(e.g.,bycheckingthe dictionary).
determinationofthemeaningof d.verifiesthepreliminary
inferredmeaningincontextorina
awordorphrase(e.g.,by
determinationofthemeaningof dictionary).
h k
h f
d
d
h
(
b

a.usescontext(e.g.,theoverall
meaningofasentenceor
paragraph;awordspositionor
functioninasentence)asaclue
tothemeaningofawordor
phrase.

AzMERITPerformanceLevelDescriptors

a.usescontext(e.g.,theoverall
meaningofasentenceor
paragraph;awordspositionor
functioninasentence)asaclue
tothemeaningofawordor
phrase.

D-61

AzMERIT - 2015

EnglishLanguageArts
Grade8

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
MinimallyProficient
PartiallyProficient
demonstratesunderstandingof
demonstratesbasic
Detailed 8.L.5
demonstrateslimited
figurativelanguage,
understandingoffigurative
understandingoffigurative
language,wordrelationships, language,wordrelationships,and wordrelationships,andnuancesin
wordmeanings:
andnuancesinwordmeanings: nuancesinwordmeanings:
a.identifiesfiguresofspeech
(e.g.,verbalirony,puns)in
context.

a.interpretsfiguresofspeech
(e.g.,verbalirony,puns)in
context.

a.interpretsfiguresofspeech(e.g.,
verbalirony,puns)incontext.

Standard Setting Technical Report


HighlyProficient
demonstratesdeepunderstandingof
figurativelanguage,wordrelationships,
andnuancesinwordmeanings:
a.interpretsfiguresofspeech(e.g.,
verbalirony,puns)incontext.

b.usestherelationshipbetween
particularwordsto
b.usestherelationshipbetween
betterunderstandeachofthewords.
b.usestherelationshipbetween b.usestherelationshipbetween particularwordsto
betterunderstandeachofthewords.
particularbasicwordstobetter particularwordstobetter
c.distinguishesandevaluatesthe
understandeachofthewords. understandeachofthewords.
connotations(associations)ofwords
c.distinguishesamongthe
connotations(associations)ofwords withsimilardenotations(definitions)
c.generallydistinguishesamong c.distinguishesamongthe
withsimilardenotations(definitions) (e.g.,bullheaded,willful,firm,
theconnotations(associations) connotations(associations)of
persistent,resolute).
wordswithsimilardenotations (e.g.,bullheaded,willful,firm,
ofwordswithsimilar
persistent,resolute).
denotations(definitions)(e.g., (definitions)(e.g.,bullheaded,
willful,firm,persistent,resolute).
bullheaded,willful,firm,
persistent,resolute).

AzMERITPerformanceLevelDescriptors

D-62

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD

Detailed

Standard

EnglishLanguageArts
Grades910

Proficient
PartiallyProficient
Forgradeappropriate,lowto
Forgradeappropriate,moderateto
moderatecomplexitytexts,the
highcomplexitytexts,theProficient
PartiallyProficientstudent
student
Reading:Literature
citesstrongandthoroughtextual
9
citestextualevidencetosupport citestextualevidencetosupport
evidencetosupportanalysisofwhat
analysisofwhatthetextsays
10.RL.1 analysisofwhatthetextsays
explicitlyaswellasinferencesdrawn thetextsaysexplicitlyaswellas
explicitly.
inferencesdrawnfromthetext.
fromthetext.
MinimallyProficient
Forgradeappropriate,low
complexitytexts,theMinimally
Proficientstudent

Detailed

9
identifiesathemeorcentralidea determinesathemeorcentralidea
ofatextanddescribesindetailits
10.RL.2 ofatextanddescribesits
developmentoverthecourseofa developmentoverthecourseofa
text;providesarestatementof text;providesabasicsummaryof
thetext.
thetext.

Detailed

9
identifieshowcharacters
10.RL.3 develop,interactwithother
characters,andadvancetheplot
ordevelopthetheme.

AzMERITPerformanceLevelDescriptors

describeshowcharactersdevelop
overthecourseofthetext,interact
withothercharacters,andadvance
theplotordevelopthetheme.

D-63

determinesathemeorcentralidea
ofatextandanalyzesindetailits
developmentoverthecourseofa
text,includinghowitemergesandis
shapedandrefinedbyspecific
details;providesanobjective
summaryofthetext.

Standard Setting Technical Report

HighlyProficient
Forgradeappropriate,high
complexitytexts,theHighly
Proficientstudent
citesstrongandthoroughtextual
evidencetosupportadeepanalysis
ofwhatthetextsaysexplicitlyaswell
ascomplexinferencesdrawnfrom
thetext.
determinesandevaluatesathemeor
centralideaofatextandanalyzesin
detailitsdevelopmentoverthe
courseofatext,includinghowit
emergesandisshapedandrefinedby
specificdetails;providesa
comprehensive,objectivesummary
ofthetext.

analyzestheeffectivenessofthe
analyzeshowcomplexcharacters
author'sdevelopmentofcomplex
(e.g.,thosewithmultipleor
conflictingmotivations)developover characters(e.g.,thosewithmultiple
thecourseofthetext,interactwith orconflictingmotivations)overthe
othercharacters,andadvancethe courseofthetext,includinghowthey
interacttoadvancetheplotorshape
plotordevelopthetheme.
thetheme.

AzMERIT - 2015

EnglishLanguageArts
Grades910

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
withtextualsupport(e.g.,context
Detailed 9
withtextualsupport(e.g.,
clues,embeddeddefinition,etc.),
10.RL.4 contextclues,embedded
definition,etc.),determinesthe determinesthemeaningofwords
andphrasesastheyareusedinthe
literalmeaningofwordsand
phrasesastheyareusedinthe text,includingfigurativeand
connotativemeanings;analyzesthe
text;describestheimpactof
cumulativeimpactofspecificword
specificwordchoiceson
choicesonmeaningandtone(e.g.,
meaning.
howthelanguageevokesasenseof
timeandplace;howitsetsaformal
orinformaltone).

Standard Setting Technical Report

Proficient
determinesthemeaningofwords
andphrasesastheyareusedinthe
text,includingfigurativeand
connotativemeanings;analyzesthe
cumulativeimpactofspecificword
choicesonmeaningandtone(e.g.,
howthelanguageevokesasenseof
timeandplace;howitsetsaformal
orinformaltone).

HighlyProficient
determinesthemeaningofcomplex
wordsandphrasesastheyareused
inthetext,includingfigurativeand
connotativemeanings;analyzesand
evaluatesthecumulativeimpactof
specificwordchoicesonmeaningand
tone(e.g.,howthelanguageevokesa
senseoftimeandplace;howitsetsa
formalorinformaltone).

Detailed

9
identifiesanauthorschoices
10.RL.5 concerninghowtostructurea
text,ordereventswithinit(e.g.,
parallelplots),andmanipulate
time(e.g.,pacing,flashbacks).

describesanauthorschoices
concerninghowtostructureatext,
ordereventswithinit(e.g.,parallel
plots),andmanipulatetime(e.g.,
pacing,flashbacks).

analyzeshowanauthorschoices
concerninghowtostructureatext,
ordereventswithinit(e.g.,parallel
plots),andmanipulatetime(e.g.,
pacing,flashbacks)createsuch
effectsasmystery,tension,or
surprise.

analyzeshowanauthorschoices
concerninghowtostructureatext,
ordereventswithinit(e.g.,parallel
plots),andmanipulatetime(e.g.,
pacing,flashbacks)createsuch
effectsasmystery,tension,or
surprise,andevaluatestheirimpact
onthetextasawhole.

Detailed

9
identfiesaparticularpointof
10.RL.6 vieworculturalexperience
reflectedinaworkofliterature
fromoutsidetheUnitedStates.

describesaparticularpointofview
orculturalexperiencereflectedina
workofliteraturefromoutsidethe
UnitedStates,drawingongeneral
knowledgeofworldliterature.

analyzesaparticularpointofviewor
culturalexperiencereflectedina
workofliteraturefromoutsidethe
UnitedStates,drawingonawide
readingofworldliterature.

analyzescompetingpointsofviewor
culturalexperiencesreflectedina
workofliteraturefromoutsidethe
UnitedStates,drawingonadeep
understandingofworldliterary
traditions.

Detailed

9
identifiesthedifferencesina
10.RL.7 depictionofasubjectorakey
sceneintwodifferentartistic
media(e.g.,AudensMusedes
BeauxArtsandBreughels
LandscapewiththeFallofIcarus).

comparesandcontraststhe
analyzestheeffectofthe
analyzestherepresentationofa
differencesinadepictionofa
representationofasubjectorakey
subjectorakeysceneintwo
subjectorakeysceneintwo
sceneintwodifferentartisticmedia,
differentartisticmedia,including
differentartisticmedia,including
whatisemphasizedorabsentineach includingwhatisemphasizedor
whatisemphasizedorabsentin
treatment(e.g.,AudensMusedes absentineachtreatment(e.g.,
eachtreatment(e.g.,Audens
AudensMusedesBeauxArtsand
BeauxArtsandBreughels
MusedesBeauxArtsand
BreughelsLandscapewiththeFallof
LandscapewiththeFallofIcarus).
BreughelsLandscapewiththeFallof
Icarus).
Icarus)

AzMERITPerformanceLevelDescriptors

D-64

AzMERIT - 2015

EnglishLanguageArts
Grades910

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Detailed 9
recognizesthatanauthordraws describeshowanauthordrawson
10.RL.9 onsourcematerialinaspecific andtransformssourcematerialina
specificwork(e.g.,howShakespeare
work(e.g.,howShakespeare
treatsathemeortopicfromOvid treatsathemeortopicfromOvidor
ortheBibleorhowalaterauthor theBibleorhowalaterauthor
drawsonaplaybyShakespeare). drawsonaplaybyShakespeare).

AzMERITPerformanceLevelDescriptors

D-65

Standard Setting Technical Report

Proficient
HighlyProficient
analyzestheeffectivenessofhowan
analyzeshowanauthordrawson
andtransformssourcematerialina authordrawsonandtransforms
specificwork(e.g.,howShakespeare sourcematerialinaspecificwork
treatsathemeortopicfromOvidor (e.g.,howShakespearetreatsa
theBibleorhowalaterauthordraws themeortopicfromOvidortheBible
orhowalaterauthordrawsonaplay
onaplaybyShakespeare).
byShakespeare)inademonstration
ofdeeperunderstandingofthetext.

AzMERIT - 2015

EnglishLanguageArts
Grades910

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
MinimallyProficient
PartiallyProficient
Reading:InformationalText
citesstrongandthoroughtextual
Detailed 9
citestextualevidencetosupport citesstrongtextualevidenceto
evidencetosupportanalysisofwhat
supportanalysisofwhatthetext
10.RI.1 analysisofwhatthetextsays
thetextsaysexplicitlyaswellas
saysexplicitlyaswellassimple
explicitlyaswellassimple
inferencesdrawnfromthetext.
inferencesdrawnfromthetext. inferencesdrawnfromthetext.
Detailed

9
identifiesacentralideaofatext determinesacentralideaofatext
10.RI.2 anddescribesitsdevelopment; anddescribesitsdevelopmentover
thecourseofatext;providesa
providesarestatementofthe
summaryofthetext.
text.

Detailed

9
identifieshowtheauthorunfolds describeshowtheauthorunfoldsan analyzeshowtheauthorunfoldsan
analysisorseriesofideasorevents,
10.RI.3 ananalysisorseriesofideasor analysisoraseriesofideasor
events,includingtheorderinwhich includingtheorderinwhichthe
events,includingtheorderin
whichthepointsaremadeand thepointsaremade,howtheyare pointsaremade,howtheyare
introducedanddeveloped,andthe introducedanddeveloped,andthe
howtheyareintroducedand
connectionsthataredrawnbetween connectionsthataredrawnbetween
developed.
them.
them.

Detailed

withtextualsupport(e.g.,context
9
withtextualsupport(e.g.,
clues,embeddeddefinition,etc.),
10.RI.4 contextclues,embedded
definition,etc.),determinesthe determinesthemeaningofwords
meaningofwordsandphrasesas andphrasesastheyareusedina
theyareusedinatext;identifies text,includingfigurative,
connotative,andtechnical
theimpactofspecificword
meanings;describesthecumulative
choicesonmeaningandtone
(e.g.,howthelanguageofacourt impactofspecificwordchoiceson
meaningandtone(e.g.,howthe
opiniondiffersfromthatofa
languageofacourtopiniondiffers
newspaper).
fromthatofanewspaper).

AzMERITPerformanceLevelDescriptors

D-66

determinesacentralideaofatext
andanalyzesitsdevelopmentover
thecourseofthetext,includinghow
itemergesandisshapedandrefined
byspecificdetails;providesan
objectivesummaryofthetext.

HighlyProficient
citesstrongandthoroughtextual
evidencetosupportadeepanalysis
ofwhatthetextsaysexplicitlyaswell
ascomplexinferencesdrawnfrom
thetext.
determinesandevaluatesacentral
ideaofatextandanalyzesindetail
itsdevelopmentoverthecourseofa
text,includinghowitemergesandis
shapedandrefinedbyspecific
details;providesacomprehensive,
objectivesummaryofthetext.
evaluatestheeffectofhowthe
authorunfoldsananalysisorseriesof
ideasorevents,includingtheorderin
whichthepointsaremade,howthey
areintroducedanddeveloped,and
theconnectionsthataredrawn
betweenthem.

analyzesthemeaningofwordsand
determinesthemeaningofwords
phrasesastheyareusedinatext,
andphrasesastheyareusedina
includingfigurative,connotative,and
text,includingfigurative,
connotative,andtechnicalmeanings; technicalmeanings;evaluatesthe
cumulativerhetoricaleffectof
analyzesthecumulativeimpactof
specificwordchoicesonmeaningand
specificwordchoicesonmeaning
andtone(e.g.,howthelanguageofa tone(e.g.,howthelanguageofa
courtopiniondiffersfromthatofa courtopiniondiffersfromthatofa
newspaper).
newspaper).

AzMERIT - 2015

EnglishLanguageArts
Grades910

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Detailed 9
identifieshowanauthorsideas describeshowanauthorsideasor
claimsaredevelopedandrefinedby
10.RI.5 orclaimsaredevelopedand
refinedbyparticularsentences, particularsentences,paragraphs,or
paragraphs,orlargerportionsof largerportionsofatext(e.g.,a
atext(e.g.,asectionorchapter). sectionorchapter).

Standard Setting Technical Report

Proficient
analyzesindetailhowanauthors
ideasorclaimsaredevelopedand
refinedbyparticularsentences,
paragraphs,orlargerportionsofa
text(e.g.,asectionorchapter).

HighlyProficient
evaluateshowanauthordevelopshis
orherideasorclaimsandrefines
themwithparticularsentences,
paragraphs,orlargerportionsofa
text(e.g.,asectionorchapter).

Detailed

9
identifiesanauthorspointof
10.RI.6 vieworpurposeinatext;
identifiestheauthor'suseof
rhetorictoadvancethatpointof
vieworpurpose.

identifiesanauthorspointofview
orpurposeinatextanddescribes
howanauthorusesrhetoricto
advancethatpointofviewor
purpose.

determinesanauthorspointofview
orpurposeinatextandanalyzes
howanauthorusesrhetoricto
advancethatpointofviewor
purpose.

analyzesanauthorspointofviewor
purposeinatextandevaluatesthe
effectivenessofanauthor'suseof
rhetorictoadvancethatpointofview
orpurpose.

Detailed

9
describesvariousaccountsofa
10.RI.7 subjecttoldindifferentmedia
(e.g.,apersonslifestoryinboth
printandmultimedia).

comparesandcontrastsvarious
accountsofasubjecttoldin
differentmedia(e.g.,apersonslife
storyinbothprintandmultimedia),
identifyingwhichdetailsare
emphasizedineachaccount.

analyzesvariousaccountsofa
subjecttoldindifferentmedia(e.g.,
apersonslifestoryinbothprintand
multimedia),determiningwhich
detailsareemphasizedineach
account.

analyzesvariousaccountsofa
subjecttoldindifferentmedia(e.g.,a
personslifestoryinbothprintand
multimedia),evaluatingtheeffectof
theemphasisofdifferentdetailsin
eachaccount.

Detailed

9
delineatesandevaluatesthe
10.RI.8 argumentandclaimsinatext,
describingthereasoningand
evidenceusedtosupportthe
claim.

delineatesandevaluatesthe
delineatesandevaluatesthe
argumentandspecificclaimsina
argumentandspecificclaimsina
text,assessingwhetherthe
text,assessingwhetherthe
reasoningisvalidandtheevidenceis reasoningisvalidandtheevidenceis
relevantandsufficient;identifies
relevantandsufficient.
falsestatementsandfallacious
reasoning.

explicatesandevaluatesthe
argumentandspecificclaimsina
text,citingspecificlanguagefromthe
textinanassessmentofwhetherthe
reasoningisvalidandtheevidenceis
relevantandsufficient;identifies
subtleinstancesoffalsestatements
andfallaciousreasoning.

Detailed

9
describesspecificaspectsof
10.RI.9 seminalU.S.documentsof
historicalandliterarysignificance
(e.g.,WashingtonsFarewell
Address,theGettysburgAddress,
RooseveltsFourFreedoms
speech,KingsLetterfrom
BirminghamJail).

analyzesspecificaspectsofseminal analyzesseminalU.S.documentsof evaluatesthereasoningand


rhetoricalstrategiesemployedin
historicalandliterarysignificance
U.S.documentsofhistoricaland
(e.g.,WashingtonsFarewellAddress, seminalU.S.documentsofhistorical
literarysignificance(e.g.,
WashingtonsFarewellAddress,the theGettysburgAddress,Roosevelts andliterarysignificance(e.g.,
FourFreedomsspeech,KingsLetter WashingtonsFarewellAddress,the
GettysburgAddress,Roosevelts
GettysburgAddress,RooseveltsFour
FourFreedomsspeech,KingsLetter fromBirminghamJail),including
Freedomsspeech,KingsLetterfrom
howtheyaddressrelatedthemes
fromBirminghamJail).
BirminghamJail),includinghowthey
andconcepts.
addressrelatedthemesand
concepts.

AzMERITPerformanceLevelDescriptors

D-67

AzMERIT - 2015

EnglishLanguageArts
Grades910

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Writing
writesargumentstosupportclaims writesargumentstosupportclaims writeshighlyeffectiveargumentsto
Detailed
9
writesargumentstosupport
inananalysisofsubstantivetopicsor inananalysisofsubstantivetopicsor supportclaimsinananalysisof
10.W.1 claimsinananalysisof
substantivetopicsortexts,using
substantivetopicsortexts,using texts,usingreasoningandrelevant texts,usingvalidreasoningand
validreasoningandrelevantand
relevantandsufficientevidence.
evidence.
reasoningandevidence.
sufficientevidence.
a.introducesclaim(s)andcreates a.introducesclaim(s),distinguishes a.introducespreciseclaim(s),
a.introducesstrongandprecise
distinguishestheclaim(s)from
theclaim(s)fromalternateor
anorganization,establishing
claim(s),distinguishestheclaim(s)
alternateoropposingclaims,and
opposingclaims,andcreatesan
relationshipsamongclaim(s),
fromalternateoropposingclaims,
createsanorganizationthat
organizationthatestablishes
reasons,andevidence.
establishesclearrelationshipsamong andcreatesaneffectiveorganization
relationshipsamongclaim(s),
claim(s),counterclaims,reasons,and thatestablishesstrong,clear
counterclaims,reasons,and
b.developsclaim(s),supplying
relationshipsamongclaim(s),
evidence.
evidence.
evidenceinamannerthat
counterclaims,reasons,and
anticipatestheaudience's
evidence.
b.developsclaim(s)and
b.developsclaim(s)and
concerns.
counterclaimsfairly,supplying
counterclaims,supplyingevidence
evidenceforeachwhilepointingout b.developsstrongclaim(s)and
foreachwhilepointingoutthe
c.useswords,phrases,and
clausestolinkthemajorsections strengthsofbothinamannerthat thestrengthsandlimitationsofboth counterclaimsfairly,supplying
thoroughevidenceforeachwhile
anticipatestheaudience'sconcerns. inamannerthatanticipatesthe
ofthetextandclarifythe
pointingoutthestrengthsand
audience'sknowledgeleveland
relationshipsbetweenclaim(s)
limitationsofbothinamannerthat
c.useswords,phrases,andclauses concerns.
andreasons,andbetween
effectivelyanticipatestheaudience's
tolinkthemajorsectionsofthetext
reasonsandevidence.
andclarifytherelationshipsbetween c.useswords,phrases,andclauses knowledgelevelandconcerns.
tolinkthemajorsectionsofthetext,
claim(s)andreasons,between
d.attemptsaformalstyleand

AzMERITPerformanceLevelDescriptors

D-68

AzMERIT - 2015

EnglishLanguageArts
Grades910

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
objectivetonewhile
reasonsandevidence,andbetween
demonstratingawarenessofthe claim(s)andcounterclaims.
normsandconventionsof
standardEnglish.
d.establishesaformalstyleand
objectivetonewhiledemonstrating
e.providesaconcluding
awarenessofthenormsand
conventionsofthedisciplinein
statementorsection.
whichheorsheiswriting.
e.providesaconcludingstatement
orsectionthatsupportsthe
argumentpresented.

AzMERITPerformanceLevelDescriptors

D-69

Proficient
createcohesion,andclarifythe
relationshipsbetweenclaim(s)and
reasons,betweenreasonsand
evidence,andbetweenclaim(s)and
counterclaims.

Standard Setting Technical Report

HighlyProficient
c.usesprecisewords,phrases,and
clausestolinkthemajorsectionsof
thetext,createcohesion,andclarify
therelationshipsbetweenclaim(s)
andreasons,betweenreasonsand
evidence,andbetweenclaim(s)and
d.establishesandmaintainsaformal counterclaims.
styleandobjectivetonewhile
d.establishesandmaintainsa
attendingtothenormsand
rhetoricallyappropriateformalstyle
conventionsofthedisciplinein
andobjectivetonewhileattendingto
whichheorsheiswriting.
thenormsandconventionsofthe
e.providesaconcludingstatement disciplineinwhichheorsheis
writing.
orsectionthatfollowsfromand
supportstheargumentpresented.
e.providesaneffectiveconcluding
statementorsectionthatfollows
fromandsupportstheargument
presented.

AzMERIT - 2015

EnglishLanguageArts
Grades910

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Detailed
9
writesinformative/explanatory writesinformative/explanatorytexts writesinformative/explanatorytexts writeshighlyeffective
toexamineandconveyideas,
10.W.2 textstoexamineandconvey
toexamineandconveycomplex
informative/explanatorytextsto
ideas,concepts,andinformation concepts,andinformationaccurately ideas,concepts,andinformation
examineandconveycomplexideas,
throughtheselection,organization, clearlyandaccuratelythroughthe concepts,andinformationclearlyand
throughtheselection,
andanalysisofcontent.
organization,andanalysisof
effectiveselection,organization,and accuratelythroughtheeffective
content.
analysisofcontent.
selection,organization,andanalysis
a.statesatopic;organizesideas,
ofcontent.
concepts,andinformationtomake a.introducesatopic;organizes
a.statesatopic;attemptsan
organizationofideas,concepts, connectionsanddistinctions;
complexideas,concepts,and
a.clearlyintroducesatopic;
includesformatting(e.g.,headings) informationtomakeimportant
strategicallyorganizescomplexideas,
andinformationtomake
andgraphics(e.g.,figures,tables)to connectionsanddistinctions;
connectionsanddistinctions.
concepts,andinformationtomake
aidcomprehension.
includesformatting(e.g.,headings) importantconnectionsand
andgraphics(e.g.,figures,tables)
distinctions;includesimportant
b.developsthetopicwith
b.developsthetopicwithrelevant whenusefultoaiding
formatting(e.g.,headings)and
informationandexamples
facts,extendeddefinitions,concrete comprehension.
graphics(e.g.,figures,tables)when
appropriatetotheaudience's
details,quotations,orother
usefultoaidingcomprehension.
knowledgeofthetopic.
informationandexamples
b.developsthetopicwithwell
chosen,relevant,andsufficientfacts, b.thoroughlydevelopsthetopicwith
c.usesappropriatetransitionsto appropriatetotheaudience.
extendeddefinitions,concrete
wellchosen,relevant,andsufficient
linkthemajorsectionsofthe
c.usesappropriatetransitionstolink details,quotations,orother
facts,extendeddefinitions,concrete
texts.
themajorsectionsofthetext,create informationandexamples
details,quotations,orother
cohesion,andclarifythe
appropriatetotheaudience's
informationandexamples
d.usestopicappropriate
relationshipsamongcomplexideas knowledgeofthetopic.
appropriatetotheaudience's
languageandvocabularyto
andconcepts.
knowledgeofthetopic.
describethetopic.
c.usesappropriateandvaried
d.usestopicappropriatelanguage transitionstolinkthemajorsections c.consistentlyandeffectivelyuses
e.attemptsaformalstyleand
anddomainspecificvocabularyto ofthetext,createcohesion,and
appropriateandvariedtransitionsto
objectivetonewhile
linkthemajorsectionsofthetext,
demonstratingawarenessofthe managethecomplexityofthetopic. clarifytherelationshipsamong
complexideasandconcepts.
createscohesion,andclarifiesthe
normsandconventionsof
relationshipsamongcomplexideas
e.establishesaformalstyleand
standardEnglish.
bj ti t
hil d
t ti
d
i l
dd
i
d
t

AzMERITPerformanceLevelDescriptors

D-70

AzMERIT - 2015

EnglishLanguageArts
Grades910

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
objectivetonewhiledemonstrating d.usespreciselanguageanddomain andconcepts.
awarenessofthenormsand
specificvocabularytomanagethe
f.providesaconcluding
conventionsofthedisciplinein
complexityofthetopic.
statementorsection.
d.usespreciselanguage,domain
whichheorsheiswriting.
specificvocabulary,andfiguresof
e.establishesandmaintainsaformal speechtomanagethecomplexityof
f.providesaconcludingstatementor styleandobjectivetonewhile
thetopic.
sectionthatsupportsthe
attendingtothenormsand
informationorexplanation
conventionsofthedisciplinein
e.establishesandmaintainsa
presented.
whichheorsheiswriting.
rhetoricallyeffectiveformalstyleand
objectivetonewhileattendingtothe
f.providesaconcludingstatementor normsandconventionsofthe
sectionthatfollowsfromand
disciplineinwhichheorsheis
supportstheinformationor
writing.
explanationpresented(e.g.,
articulatingimplicationsorthe
f.providesaneffectiveconcluding
significanceofthetopic).
statementorsectionthatfollows
fromandsupportstheinformationor
explanationpresented(e.g.,
articulatingimplicationsorthe
significanceofthetopic).

AzMERITPerformanceLevelDescriptors

D-71

AzMERIT - 2015

EnglishLanguageArts
Grades910

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
producescoherentwritinginwhich
produceswritinginwhichthe
Detailed 9
10.W.4 development,organization,and thedevelopment,organization,and
styleareappropriatetothetask styleareappropriatetothetask,
6
andpurpose;strengthenswriting purpose,andaudience;strengthens
writingasneededbyplanning,
asneededbyrevisingand
revising,andediting;uses
editing;usestechnologyto
technology,includingtheInternet,
producewriting.
toproduceandpublishwriting
products,takingadvantageof
technology'scapacitytodisplay
informationflexiblyanddynamically.

Detailed

9
conductsshortresearchprojects conductsshortaswellasmore
sustainedresearchprojectsto
10.W.7 toansweragivensimple
questionorsolveagivensimple answerasimplequestion(including
aselfgeneratedquestion)orsolvea
problem;usesdiscrete
informationfromsourcesonthe simpleproblem;narrowsor
broadenstheinquirywhen
subject,demonstratinga
developingunderstandingofthe appropriate;synthesizessourceson
thesubject,demonstrating
subjectunderinvestigation.
understandingofthesubjectunder
investigation.

AzMERITPerformanceLevelDescriptors

D-72

Standard Setting Technical Report

Proficient
producesclearandcoherentwriting
inwhichthedevelopment,
organization,andstyleare
appropriatetotask,purpose,and
audience;developsandstrengthens
writingasneededbyplanning,
revising,editing,rewriting,ortrying
anewapproach,focusingon
addressingwhatismostsignificant
foraspecificpurposeandaudience;
usestechnology,includingthe
Internet,toproduce,publish,and
updateindividualorsharedwriting
products,takingadvantageof
technology'scapacitytolinktoother
informationandtodisplay
informationflexiblyanddynamically.

HighlyProficient
producesclearandcoherentwriting
inwhichthedevelopment,
organization,andstylearehighly
effectiveforthetask,purpose,and
audience;developsandstrengthens
writingbyplanning,revising,editing,
rewriting,ortryinganewapproach,
focusingonaddressingwhatismost
significantforaspecificpurposeand
audience;usestechnology,including
theInternet,toproduce,publish,and
updateindividualorsharedwriting
products,takingadvantageof
technology'scapacitytolinktoother
informationandtodisplay
informationflexiblyanddynamically.

conductsshortaswellasmore
sustainedresearchprojectsto
answeraquestion(includingaself
generatedquestion)orsolvea
problem;narrowsorbroadensthe
inquirywhenappropriate;
synthesizesmultiplesourcesonthe
subject,demonstrating
understandingofthesubjectunder
investigation.

conductsshortaswellasmore
sustainedresearchprojectstoanswer
acomplexquestion(includingaself
generatedquestion)orsolvea
complexproblem;narrowsor
broadenstheinquirywhen
appropriate;synthesizesmultiple
highqualitysourcesonthesubject,
demonstratingcomplete
understandingofthesubjectunder
investigation.

AzMERIT - 2015

EnglishLanguageArts
Grades910

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Detailed 9
gathersinformationfromprint gathersrelevantinformationfrom
multipleprintanddigitalsources,
10.W.8 anddigitalsources;integrates
usingsearcheseffectively;assesses
informationintothetext,
avoidingplagiarismandfollowing theusefulnessofeachsourcein
answeringtheresearchquestion;
astandardformatforcitation.
integratesinformationintothetext
tomaintaintheflowofideas,
avoidingplagiarismandfollowinga
standardformatforcitation.

AzMERITPerformanceLevelDescriptors

D-73

Proficient
gathersrelevantinformationfrom
multipleauthoritativeprintand
digitalsources,usingadvanced
searcheseffectively;assessesthe
usefulnessofeachsourcein
answeringtheresearchquestion;
integratesinformationintothetext
selectivelytomaintaintheflowof
ideas,avoidingplagiarismand
followingastandardformatfor
citation.

Standard Setting Technical Report


HighlyProficient
gathershighlyrelevantinformation
frommultipleauthoritativeprintand
digitalsources,usingadvanced
searcheseffectively;assessesand
analyzestheusefulnessofeach
sourceinansweringtheresearch
question;seamlesslyintegrates
informationintothetextselectively
tocreateandmaintaintheflowof
ideas,avoidingplagiarismand
followingastandardformatfor
citation.

AzMERIT - 2015

EnglishLanguageArts
Grades910

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
MinimallyProficient
PartiallyProficient
Listening
usesmultiplesourcesofinformation integratesmultiplesourcesof
Detailed 9
usesmultiplesourcesof
informationpresentedindiverse
10.SL.2 informationpresentedindiverse presentedindiversemediaor
mediaorformats(e.g.,visually, formats(e.g.,visually,quantitatively, mediaorformats(e.g.,visually,
orally),evaluatingthecredibilityand quantitatively,orally),evaluatingthe
quantitatively,orally).
credibilityandaccuracyofeach
accuracyofeachsource.
source.

HighlyProficient
effectivelyintegratesmultiple
sourcesofinformationpresentedin
diversemediaorformats(e.g.,
visually,quantitatively,orally)to
meettheneedsofaspecifictask,
audience,andpurpose,while
evaluatingthecredibilityand
accuracyofeachsource.

Detailed

9
summarizesaspeakerspointof evaluatesaspeaker'spointofview, evaluatesaspeaker'spointofview, thoroughlyevaluatesaspeaker's
reasoning,anduseofevidenceand pointofview,reasoning,anduseof
reasoning,anduseofevidence,
10.SL.3 view,reasoning,anduseof
evidenceandrhetoric,analyzingany
identifyinganyfallaciousreasoning. rhetoric,identifyinganyfallacious
evidence.
fallaciousreasoningorexaggerated
reasoningorexaggeratedor
ordistortedevidence.
distortedevidence.

Detailed

9
10.L.1

Language
demonstratesstrongcommandof
demonstratesbasicunderstandingof demonstratescommandofthe
attemptstomeetthe
theconventionsofstandardEnglish
conventionsofstandardEnglish theconventionsofstandardEnglish conventionsofstandardEnglish
grammarandusagewhenwriting grammarandusagewhenwritingor grammarandusagewhenwritingor grammarandusagewhenwritingor
speaking.
speaking.
orspeaking:usesvarioustypes speaking.
ofphrases(noun,verb,adjectival,
a.usesparallelstructure.
a.usesparallelstructure.
a.usesparallelstructure.
adverbial,participial,
prepositional,absolute)and
b.usesvarioustypesofphrases
b.usesvarioustypesofphrases
b.usesvarioustypesofphrases
clauses(independent,
(noun,verb,adjectival,adverbial,
(noun,verb,adjectival,adverbial,
(noun,verb,adjectival,adverbial,
dependent,noun,relative,
participial,prepositional,absolute) participial,prepositional,absolute) participial,prepositional,absolute)
adverbial)toaddinterestto
andclauses(independent,
andclauses(independent,
andclauses(independent,
writingorpresentations.
dependent,noun,relative,adverbial) dependent,noun,relative,adverbial) dependent,noun,relative,adverbial)
toconveymeaningsandaddinterest toconveyspecificmeaningsandadd toconveyspecificmeaningsandadd
variety,craft,style,depthof
varietyandinteresttowritingor
towritingorpresentations.
meaning,andinteresttowritingor
presentations.
presentations.

AzMERITPerformanceLevelDescriptors

D-74

AzMERIT - 2015

EnglishLanguageArts
Grades910

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
MinimallyProficient
PartiallyProficient
demonstratesbasicunderstandingof demonstratescommandofthe
Detailed 9
attemptstomeetthe
10.L.2 conventionsofstandardEnglish theconventionsofstandardEnglish conventionsofstandardEnglish
capitalization,punctuation,and
capitalization,punctuation,and capitalization,punctuation,and
spellingwhenwriting.
spellingwhenwriting.
spellingwhenwriting.
a.attemptstouseasemicolonto
linktwoormorecloselyrelated
independentclauses.
b.attemptstouseacolonto
introducealistorquotation.
c.spellscorrectly.
Detailed

9
10.L.3

usesknowledgeoflanguagefor
comprehensionwhenreadingor
listeningandmakeschoicesfor
meaningorstyle.

AzMERITPerformanceLevelDescriptors

usesknowledgeoflanguagefor
comprehensionwhenreadingor
listeningandmakeschoicesfor
meaningorstyle;writesandedits
worktoconformtoaformalor
informalstyle.

D-75

a.usesasemicolontolinktwoor
morecloselyrelatedindependent
clauses.

HighlyProficient
demonstratesstrongcommandof
theconventionsofstandardEnglish
capitalization,punctuation,and
spellingwhenwriting,usingthat
commandtoenhancestyleand
meaning.

a.usesasemicolontolinktwoor
morecloselyrelatedindependent
b.usesacolontointroducealistor clauses.
quotation.
b.usesacolontointroducealistor
quotation.
c.spellscorrectly.
appliesknowledgeoflanguageto
understandhowlanguagefunctions
indifferentcontexts,tomake
effectivechoicesformeaningor
style,andtocomprehendmorefully
whenreadingorlistening.Writes
andeditsworksothatitconformsto
theguidelinesinastylemanual(e.g.,
MLAHandbook,Turabian'sManual
forWriters)appropriateforthe
disciplineandwritingtype.

appliesknowledgeoflanguageto
demonstratehowlanguagefunctions
indifferentcontexts,tomakehighly
effectivechoicesformeaningorstyle,
andtofullycomprehendwhen
readingorlistening;writesandedits
worksothatitconformstothe
guidelinesinastylemanual(e.g.,
MLAHandbook,Turabian'sManual
forWriters)appropriateforthe
disciplineandwritingtype.

AzMERIT - 2015

EnglishLanguageArts
Grades910

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
determinesthemeaningofunknown determinesandclarifiesthemeaning determinesandclarifiesthe
Detailed 9
determinesthemeaningof
meaningsofunknownandmultiple
ofunknownormultiplemeaning
ormultiplemeaninggradelevel
10.L.4 unknownormultiplemeaning
wordsbyusingcontextclueswithin gradelevelwordsbyusingcontext meaningwords,includingabove
gradelevelwordsbyusing
clueswithinthetext;identifiesand gradelevelwords,byusingcontext
thesamesentence;identifiesand
immediatecontextcluesor
clueswithinthetext;identifiesand
correctlyusespatternsofword
attemptstousepatternsofword
attemptingtousepatternsof
correctlyusespatternsofword
changesthatindicatedifferent
changesthatindicatedifferent
wordchanges.
changesthatindicatedifferent
meaningsorpartsofspeech;
meanings;orconsultsgeneral
meaningsorpartsofspeech;consults
referencematerials,bothprintand consultsgeneralandspecialized
referencematerials,bothprintand generalandspecializedreference
digital.
materials,bothprintanddigital,to
digital,todetermineitspartof
determineitspartofspeechorits
speechoritsetymology;and/or
etymology;and/orverifiesthe
verifiesthepreliminary
determinationofthemeaningofa meaningofawordorphrase.
wordorphrase.
Detailed

9
10.L.5

recognizesfigurativelanguage
andwordrelationshipsby
identifyingfiguresofspeechand
nuancesinwordmeanings.

AzMERITPerformanceLevelDescriptors

demonstratesunderstandingof
straightforwardfigurativelanguage,
clearwordrelationships,and
nuancesinwordmeaningsby
identifyingandattemptingto
interpretfiguresofspeechintexts
andrecognizingnuancesinthe
meaningofwords.

D-76

demonstratesunderstandingof
figurativelanguage,word
relationships,andnuancesinword
meanings.

demonstratesunderstandingof
compexfigurativelanguage,complex
wordrelationships,andsubtle
nuancesinwordmeanings.

a.interpretsfiguresofspeechin
contextandanalyzestheirrolein
texts.

a.interpretsandusesfiguresof
speechincontextandanalyzestheir
roleintexts.

b.analyzesnuancesinthemeaning
ofwordswithsimilardenotations.

b.analyzesandusesnuancesinthe
meaningofwordswithsimilar
denotations.

AzMERIT - 2015

EnglishLanguageArts
Grade11

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Forgradeappropriate,lowto
Forgradeappropriate,moderateto Forgradeappropriate,high
Forgradeappropriate,low
highcomplexitytexts,theProficient complexitytexts,theHighly
complexitytexts,theMinimally moderatecomplexitytexts,the
Proficientstudent
PartiallyProficientstudent
student
Proficientstudent
Reading:Literature
citesstrongandthoroughtextual
citesstrongandthoroughtextual
Detailed 11.RL.1 citestextualevidencetosupport citesstrongtextualevidenceto
evidencetosupportanalysisofwhat evidencetosupportadeepanalysis
supportanalysisofwhatthetext
analysisofwhatthetextsays
ofwhatthetextsaysexplicitlyaswell
saysexplicitlyaswellasinferences thetextsaysexplicitlyaswellas
explicitlyaswellassimple
ascomplexinferencesdrawnfrom
inferencesdrawnfromthetext,
inferencesdrawnfromthetext. drawnfromthetext.
includingdeterminingwherethetext thetext,includingdeterminingwhere
thetextleavesmattersuncertainand
leavesmattersuncertain.
howtheycouldbeclarified.
Detailed

11.RL.2 determinestwoexplicitthemes determinestwothemesorcentral


ideasofatextandanalyzestheir
orcentralideasofatextand
describestheirdevelopmentover developmentoverthecourseofthe
thecourseofthetext;providesa text;providesasimpleobjective
summaryofthetext.
simplesummaryofthetext.

determinestwoormorethemesor determinestwoormoresubtle
centralideasofatextandanalyzes themesorcentralideasofatext;
theirdevelopmentoverthecourseof analyzesandevaluatestheir
thetext,includinghowtheyinteract developmentoverthecourseofthe
andbuildononeanothertoproduce text,includinghowtheyinteractand
buildononeanothertoproducea
acomplexaccount;providesan
complexaccount;providesa
objectivesummaryofthetext.
comprehensiveobjectivesummaryof
thetext.

Detailed

analyzestheimpactoftheauthor's
11.RL.3 describestheauthor'schoices
choicesregardinghowtodevelop
regardinghowtodevelopand
relatebasicelementsofastoryor andrelatebasicelementsofastory
drama(e.g.,setting,characters, ordrama(e.g.,setting,characters,
plot).
plot).

analyzestheimpactoftheauthor's analyzesandevaluatestheimpactof
theauthor'schoicesregardinghowto
choicesregardinghowtodevelop
developandrelateelementsofa
andrelateelementsofastoryor
drama(e.g.,whereastoryisset,how storyordrama(e.g.,whereastoryis
set,howtheactionisordered,how
theactionisordered,howthe
thecharactersareintroducedand
charactersareintroducedand
developed).
developed).

AzMERITPerformanceLevelDescriptors

D-77

AzMERIT - 2015

EnglishLanguageArts
Grade11

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
determinesthemeaningofcomplex
withtextualsupport(e.g.,context determinesthemeaningofwords
Detailed 11.RL.4 withtextualsupport(e.g.,
andphrasesastheyareusedinthe wordsandphrasesastheyareused
clues,embeddeddefinitions),
contextclues,embedded
inthetext,includingfigurativeand
text,includingfigurativeand
determinesthemeaningofwords
definitions),determinesthe
andphrasesastheyareusedinthe connotativemeanings;analyzesthe connotativemeanings;analyzesand
literalmeaningofwordsand
impactofspecificwordchoiceson evaluatestheimpactofspecificword
phrasesastheyareusedinthe text,includingfigurativeand
connotativemeanings;analyzesthe meaningandtone,includingwords choicesonmeaningandtone,
text,includingfigurativeand
impactofspecificwordchoiceson withmultiplemeaningsorlanguage includingwordswithmultiple
connotativemeanings.
meaningandtone,includingwords thatisparticularlyfresh,engaging,or meaningsorlanguagethatis
particularlyfresh,engaging,or
withmultiplemeaningsorlanguage beautiful.
beautiful.
thatisparticularlyfresh,engaging,or
beautiful.
Detailed

analyzeshowanauthorschoices
describesanauthorschoices
11.RL.5 identifiesanauthorschoices
concerninghowtostructurespecific concerninghowtostructurespecific
concerninghowtostructure
partsofatext(e.g.,thechoiceof
specificpartsofatext(e.g.,the partsofatext(e.g.,thechoiceof
choiceofwheretobeginorenda wheretobeginorendastory,the wheretobeginorendastory,the
choicetoprovideacomedicortragic choicetoprovideacomedicortragic
story,thechoicetoprovidea
resolution)contributetoitsoverall
resolution).
comedicortragicresolution).
structureandmeaningaswellasits
aestheticimpact.

analyzesandevaluatesthe
effectivenessofanauthorschoices
concerninghowtostructurespecific
partsofatext(e.g.,thechoiceof
wheretobeginorendastory,the
choicetoprovideacomedicortragic
resolution),includinghowthey
contributetoitsoverallstructureand
meaningaswellasitsaesthetic
impact.

Detailed

11.RL.6 identifiesaclearcaseinwhich
graspingpointofviewrequires
distinguishingwhatisdirectly
statedinatextfromwhatis
reallymeant(e.g.,satire,
sarcasm,irony,or
understatement).

analyzesacaseinwhichgrasping
pointofviewrequiresdistinguishing
whatisdirectlystatedinatextfrom
whatisreallymeant(e.g.,satire,
sarcasm,irony,orunderstatement).

analyzesacaseinwhichgrasping
pointofviewrequiresdistinguishing
whatisdirectlystatedinatextfrom
whatisreallymeant(e.g.,satire,
sarcasm,irony,orunderstatement),
andevaluatesitsrhetoricaleffectand
aestheticimpact.

AzMERITPerformanceLevelDescriptors

identifiesasubtlecaseinwhich
graspingpointofviewrequires
distinguishingwhatisdirectlystated
inatextfromwhatisreallymeant
(e.g.,satire,sarcasm,irony,or
understatement).

D-78

AzMERIT - 2015

EnglishLanguageArts
Grade11

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
comparesandcontrastsmultiple
Detailed 11.RL.7 describesdifferencesin
interpretationsofastory,drama, interpretationsofastory,drama,or
poem(e.g.,recordedorlive
orpoem(e.g.,recordedorlive
productionofaplayorrecorded productionofaplayorrecorded
novelorpoetry),identifyinghow novelorpoetry),describinghow
eachversioninterpretsthesource
eachversioninterpretsthe
text.
sourcetext.
Detailed

11.RL.9 demonstratesknowledgeofsome demonstratesknowledgeofacore


groupofeighteenth,nineteenth
eighteenth,nineteenthand
andearlytwentiethcentury
earlytwentiethcentury
foundationalworksofAmerican foundationalworksofAmerican
literature,includinghowtwotexts
literature,includinghowtwo
fromthesameperiodtreatsimilar
textstreatsimilartopics.
themesortopics.

AzMERITPerformanceLevelDescriptors

D-79

Standard Setting Technical Report

Proficient
HighlyProficient
analyzesmultipleinterpretationsofa analyzesmultiple,subtlydifferent
interpretationsofastory,drama,or
story,drama,orpoem(e.g.,
recordedorliveproductionofaplay poem(e.g.,recordedorlive
productionofaplayorrecorded
orrecordednovelorpoetry),
novelorpoetry),evaluatingeach
evaluatinghoweachversion
version'sinterpretationofthesource
interpretsthesourcetext.
textandhowthatinterpretation
affectstheoverallmeaning.
demonstratesknowledgeof
eighteenth,nineteenthandearly
twentiethcenturyfoundational
worksofAmericanliterature,
includinghowtwoormoretexts
fromthesameperiodtreatsimilar
themesortopics.

demonstratesthoroughknowledgeof
eighteenth,nineteenthandearly
twentiethcenturyfoundational
worksofAmericanliterature,
analyzingandevaluatinghowtwoor
moretextsfromthesameperiodin
ananalysisoftheirtreatmentof
similarthemesortopics.

AzMERIT - 2015

EnglishLanguageArts
Grade11

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Reading:InformationalText
citesstrongandthoroughtextual
citesstrongandthoroughtextual
Detailed 11.RI.1 citestextualevidencetosupport citesstrongtextualevidenceto
evidencetosupportanalysisofwhat evidencetosupportadeepanalysis
supportanalysisofwhatthetext
analysisofwhatthetextsays
ofwhatthetextsaysexplicitlyaswell
saysexplicitlyaswellasinferences thetextsaysexplicitlyaswellas
explicitlyaswellassimple
ascomplexinferencesdrawnfrom
inferencesdrawnfromthetext,
inferencesdrawnfromthetext. drawnfromthetext.
includingdeterminingwherethetext thetext,includingdeterminingwhere
thetextleavesmattersuncertainand
leavesmattersuncertain.
howtheycouldbeclarified.
Detailed

11.RI.2 determinestwoexplicitcentral determinestwocentralideasofa


ideasofatextanddescribestheir textandanalyzestheirdevelopment
developmentoverthecourseof overthecourseofthetext;provides
asimple,objectivesummaryofthe
thetext;providesasimple
text.
summaryofthetext.

Detailed

11.RI.3 describesasetofideasor
sequenceofeventsandidentifies
howspecificindividuals,ideas,or
eventsinteractanddevelopin
specificsectionsofthetext.

analyzesasetofideasorsequence
ofeventsandidentifieshowspecific
individuals,ideas,oreventsinteract
anddevelopinspecificsectionsof
thetext.

evaluatestheeffectofthe
analyzesacomplexsetofideasor
sequenceofeventsandexplainshow presentationofacomplexsetof
specificindividuals,ideas,orevents ideasorsequenceofeventsand
interactanddevelopoverthecourse explainshowspecificindividuals,
ideas,oreventsinteractanddevelop
ofthetext.
overthecourseofthetext.

Detailed

11.RI.4 withtextualsupport(e.g.,
contextclues,embedded
definitions),determinesthe
meaningofwordsandphrasesas
theyareusedinatext;identifies
howanauthorusesandrefines
themeaningofakeytermor
termsoverthecourseofatext
(e.g.,howMadisondefines
factioninFederalist No.10).

withtextualsupport(e.g.,context
clues,embeddeddefinitions),
determinesthemeaningofwords
andphrasesastheyareusedina
text,includingfigurative,
connotative,andtechnical
meanings;describeshowanauthor
usesandrefinesthemeaningofa
keytermortermsoverthecourseof
atext(e.g.,howMadisondefines
factioninFederalist No.10).

analyzesthemeaningofwordsand
determinesthemeaningofwords
phrasesastheyareusedinatext,
andphrasesastheyareusedina
includingfigurative,connotative,and
text,includingfigurative,
connotative,andtechnicalmeanings; technicalmeanings;evaluatesthe
rhetoricaleffectofhowanauthor
analyzeshowanauthorusesand
refinesthemeaningofakeytermor usesandrefinesthemeaningofakey
termsoverthecourseofatext(e.g., termortermsoverthecourseofa
text(e.g.,howMadisondefines
howMadisondefinesfactionin
factioninFederalist No.10).
Federalist No.10).

AzMERITPerformanceLevelDescriptors

D-80

determinestwoormorecentral
ideasofatextandanalyzestheir
developmentoverthecourseofthe
text,includinghowtheyinteractand
buildononeanothertoprovidea
complexanalysis;providesan
objectivesummaryofthetext.

determinestwoormoresubtle
centralideasofatext;analyzesand
evaluatestheirdevelopmentoverthe
courseofthetext,includinghowthey
interactandbuildononeanotherto
produceacomplexanalysis;provides
acomprehensive,objectivesummary
ofthetext.

AzMERIT - 2015

EnglishLanguageArts
Grade11

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Detailed 11.RI.5 analyzesthestructuretheauthor analyzesandevaluatesthe
effectivenessofthestructurean
usesinhisorherexpositionor
authorusesinhisorherexposition
argument.
orargument.

Detailed

identifiesanauthor'spointofview
11.RI.6 identifiesanauthor'spointof
vieworpurposeinatextinwhich orpurposeinatextinwhichthe
rhetoricisparticularlyeffective,
therhetoricisparticularly
describinghowstyleandcontent
effective;identifiesthe
contributetothepower,
contributionofthetext'sstyle
persuasiveness,orbeautyofthe
andcontent.
text.

Detailed

11.RI.7 usesinformationpresentedin
differentmediaorformats(e.g.,
visually,quantitatively)aswellas
inwordsinordertoaddressa
questionorsolveaproblem.

AzMERITPerformanceLevelDescriptors

integratesmultiplesourcesof
informationpresentedindifferent
mediaorformats(e.g.,visually,
quantitatively)aswellasinwordsin
ordertoaddressaquestionorsolve
aproblem.

D-81

Standard Setting Technical Report

Proficient
analyzesandevaluatesthe
effectivenessofthestructurean
authorusesinhisorherexposition
orargument,includingwhetherthe
structuremakespointsclear,
convincing,andengaging.

HighlyProficient
analyzesandevaluatesthe
effectivenessofboththestructurean
authorusesinhisorherexpositionor
argumentandalternatestructures,
includingwhetherthestructure
makespointsclear,convincing,and
engaging.

determinesanauthor'spointofview
orpurposeinatextinwhichthe
rhetoricisparticularlyeffective,
analyzinghowstyleandcontent
contributetothepower,
persuasiveness,orbeautyofthetext.

analyzesanauthor'spointofviewor
purposeinatextinwhichthe
rhetoricisparticularlyeffective;
evaluatestheeffectivenessofthe
author'sstyleandcontent,including
theircontributiontothepower,
persuasiveness,orbeautyofthetext.

integratesandevaluatesmultiple
sourcesofinformationpresentedin
differentmediaorformats(e.g.,
visually,quantitatively)aswellasin
wordsinordertoaddressaquestion
orsolveaproblem.

synthesizes,integrates,andevaluates
multiplesourcesofinformation
presentedindifferentmediaor
formats(e.g.,visually,quantitatively)
aswellasinwordsinorderto
addressaquestionorsolvea
problem;evaluatestheeffectofthe
proposedanswerorsolution.

AzMERIT - 2015

EnglishLanguageArts
Grade11

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
explicatesandevaluatesthe
delineatesandevaluatesthe
delineatesandevaluatesthe
Detailed 11.RI.8 delineatesandevaluatesthe
reasoninginseminalU.S.texts,
reasoninginseminalU.S.texts,
reasoninginseminalU.S.texts, reasoninginseminalU.S.texts,
includingtheapplicationof
includingtheapplicationof
includingtheapplicationof
describingtheapplicationof
constitutionalprinciplesanduse constitutionalprinciplesanduseof constitutionalprinciplesanduseof constitutionalprinciplesanduseof
legalreasoning(e.g.,inU.S.Supreme legalreasoning(e.g.,inU.S.Supreme legalreasoning(e.g.,inU.S.Supreme
oflegalreasoning(e.g.,inU.S.
Courtmajorityopinionsanddissents) Courtmajorityopinionsanddissents)
SupremeCourtmajorityopinions Courtmajorityopinionsand
andthepremises,purposes,and
andthepremises,purposes,and
dissents).
anddissents).
argumentsinworksofpublic
argumentsinworksofpublic
advocacy(e.g.,TheFederalist ,
advocacy(e.g.,TheFederalist,
presidentialaddresses);extrapolates
presidentialaddresses).
andevaluatestheeffectsofthese
decisionsonpubliclife.
Detailed

11.RI.9 describesthethemes,purposes, performsabasicanalysisofthe


themes,purposes,andrhetorical
andrhetoricalfeaturesof
seventeenth,eighteenth,and featuresinseventeenth,eighteenth
nineteenthcenturyfoundational ,andnineteenthcentury
U.S.documentsofhistoricaland foundationalU.S.documentsof
literarysignificance(includingthe historicalandliterarysignificance
DeclarationofIndependence,the (includingtheDeclarationof
PreambletotheConstitution,the Independence,thePreambletothe
Constitution,theBillofRights,and
BillofRights,andLincolns
LincolnsSecondInauguralAddress).
SecondInauguralAddress).

AzMERITPerformanceLevelDescriptors

D-82

analyzesseventeenth,eighteenth, referstospecifictextualevidencein
andnineteenthcenturyfoundational ananalysisofseventeenth,
eighteenth,andnineteenthcentury
U.S.documentsofhistoricaland
foundationalU.S.documentsof
literarysignificance(includingthe
historicalandliterarysignificance
DeclarationofIndependence,the
PreambletotheConstitution,theBill (includingtheDeclarationof
Independence,thePreambletothe
ofRights,andLincolnsSecond
InauguralAddress)fortheirthemes, Constitution,theBillofRights,and
LincolnsSecondInauguralAddress),
purposes,andrhetoricalfeatures.
evaluatingtheimplicationsoftheir
themes,purposes,andrhetorical
features.

AzMERIT - 2015

EnglishLanguageArts
Grade11

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Writing
Detailed 11.W.1 writesargumentstosupport
writesargumentstosupportclaims writesargumentstosupportclaims writeshighlyeffectiveargumentsto
claimsinananalysisof
inananalysisofsubstantivetopicsor inananalysisofsubstantivetopicsor supportclaimsinananalysisof
substantivetopicsortexts,using texts,usingreasoningandrelevant texts,usingvalidreasoningand
substantivetopicsortexts,using
reasoningandevidence.
evidence.
relevantandsufficientevidence.
validreasoningandrelevantand
sufficientevidence.
a.introducesclaim(s),statesthe a.introducesclaim(s),statesthe
a.introducespreciseclaim(s),
significanceoftheclaim(s),and significanceoftheclaim(s),
a.introducesstrongandprecise
establishesthesignificanceofthe
establishesrelationshipsamong distinguishestheclaim(s)from
claim(s),distinguishestheclaim(s)
claim(s),establishesthesignificance
someclaim(s),reasons,and
alternateoropposingclaims,and
fromalternateoropposingclaims, oftheclaim(s),distinguishesthe
evidence.
createsanorganizationthat
andcreatesanorganizationthat
claim(s)fromalternateoropposing
establishesrelationshipsamong
establishesclearrelationshipsamong claims,andcreatesaneffective
claim(s),counterclaims,reasons,and claim(s),counterclaims,reasons,and organizationthatestablishesstrong,
b.developsclaim(s),supplying
evidence.
evidence.
clearrelationshipsamongclaim(s),
evidenceinamannerthat
counterclaims,reasons,and
anticipatestheaudience's
evidence.
concerns.
b.developsclaim(s)and
b.developsclaim(s)and
counterclaims,supplyingevidence counterclaimsfairly,supplying
foreachwhilepointingoutthe
evidenceforeachwhilepointingout b.developsstrongclaim(s)and
c.useswords,phrases,and
clausestolinksectionsofthetext strengthsofbothinamannerthat thestrengthsandlimitationsofboth counterclaimsfairly,supplying
andclarifytherelationships
anticipatestheaudience'sconcerns. inamannerthatanticipatesthe
thoroughevidenceforeachwhile
betweenclaim(s)andreasons,
audience'sknowledgeleveland
establishingthestrengthsand
andbetweenreasonsand
limitationsofbothinamannerthat
c.useswords,phrases,andclauses concerns.
evidence.
effectivelyanticipatestheaudience's
tolinksectionsofthetextandclarify
therelationshipsbetweenclaim(s) c.useswords,phrases,andclauses knowledgelevelandconcerns.
d tt
t f
l t l
d
d
b t
d t li k th
j
ti
f th t t

AzMERITPerformanceLevelDescriptors

D-83

AzMERIT - 2015

EnglishLanguageArts
Grade11

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
andreasons,betweenreasonsand
d.attemptsaformalstyleand
objectivetonewhile
evidence,andbetweenclaim(s)and
demonstratingawarenessofthe counterclaims.
normsandconventionsof
standardEnglish.
d.establishesaformalstyleand
objectivetonewhiledemonstrating
e.providesaconcluding
awarenessofthenormsand
statementorsection.
conventionsofthedisciplinein
whichheorsheiswriting.
e.providesaconcludingstatement
orsectionthatsupportsthe
argumentpresented.

Standard Setting Technical Report

Proficient
tolinkthemajorsectionsofthetext,
createcohesion,andclarifythe
relationshipsbetweenclaim(s)and
reasons,betweenreasonsand
evidence,andbetweenclaim(s)and
counterclaims.

HighlyProficient

c.usesprecisewords,phrases,and
clausestolinkthemajorsectionsof
thetext,createcohesion,andclarify
therelationshipsbetweenclaim(s)
andreasons,betweenreasonsand
evidence,andbetweenclaim(s)and
d.establishesandmaintainsaformal counterclaims.
styleandobjectivetonewhile
d.establishesandmaintainsa
attendingtothenormsand
rhetoricallyappropriateformalstyle
conventionsofthedisciplinein
andobjectivetonewhileattendingto
whichheorsheiswriting.
thenormsandconventionsofthe
e.providesaconcludingstatement disciplineinwhichheorsheis
writing.
orsectionthatfollowsfromand
supportstheargumentpresented.
e.providesaneffectiveconcluding
statementorsectionthatfollows
fromandsupportstheargument
presented.
f.evaluatesandreflectsonthe
writingandhowwellitaddressesthe
purpose,audience,andtask.

AzMERITPerformanceLevelDescriptors

D-84

AzMERIT - 2015

EnglishLanguageArts
Grade11

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
MinimallyProficient
PartiallyProficient
Detailed 11.W.2 writesinformative/explanatory writesinformative/explanatorytexts writesinformative/explanatorytexts
textstoexamineandconvey
toexamineandconveyideas,
toexamineandconveycomplex
ideas,concepts,andinformation concepts,andinformationaccurately ideas,concepts,andinformation
throughtheselection,
throughtheeffectiveselection,
clearlyandaccuratelythroughthe
organization,andanalysisof
organization,andanalysisof
effectiveselection,organization,and
content.
content.
analysisofcontent.
a.statesatopic;organizesideas,
concepts,andinformationto
makeconnectionsand
distinctions.

HighlyProficient
writeshighlyeffective
informative/explanatorytextsto
examineandconveycomplexideas,
concepts,andinformationclearlyand
accuratelythroughtheeffective
selection,organization,andanalysis
ofcontent.

a.introducesatopic;organizes
a.introducesatopic;organizes
a.clearlyintroducesatopic;
ideas,concepts,andinformationto complexideas,concepts,and
strategicallyorganizescomplexideas,
makeconnectionsanddistinctions; informationsothateachnew
concepts,andinformationtomake
includesformatting(e.g.,headings), elementbuildsonthatwhich
graphics(e.g.,figures,tables),and precedesittocreateaunifiedwhole; importantconnectionsand
includesformatting(e.g.,headings) distinctions;includesimportant
b.developsthetopicbyselecting multimediainanattempttoaid
formatting(e.g.,headings)and
comprehension.
andgraphics(e.g.,figures,tables)
relevantfacts,extended
graphics(e.g.,figures,tables)when
whenusefultoaiding
definitions,concretedetails,
usefultoaidingcomprehension.
comprehension.
quotations,orotherinformation b.developsthetopicbyselecting
significantandrelevantfacts,
andexamples.
extendeddefinitions,concrete
b.developsthetopicthoroughlyby b.developsthetopicstrategicallyby
selectingthemostsignificantand
selectingthemostsignificantand
c.usesappropriatetransitionsto details,quotations,orother
informationandexamples
linkthemajorsectionsofthe
relevantfacts,extendeddefinitions, relevantfacts,extendeddefinitions,
text,createcohesion,andclarify appropriatetotheaudience.
concretedetails,quotations,orother concretedetails,quotations,orother
informationandexamples
therelationshipsamongcomplex
informationandexamples
appropriateandrelevanttothe
ideasandconcepts.
c.usesappropriatetransitionstolink appropriatetotheaudience's
audience'sknowledgeofthetopic.
themajorsectionsofthetext,create knowledgeofthetopic.
cohesion,andclarifythe
d.usestopicappropriate
c.consistentlyandeffectivelyuses
relationshipsamongcomplexideas c.usesappropriateandvaried
language,vocabulary,and
andconcepts.
techniquessuchasmetaphor,
transitionstolinkthemajorsections appropriateandvariedtransitionsto
linkthemajorsectionsofthetext,
simile,andanalogytodescribe
ofthetext,createcohesion,and
createscohesion,andclarifiesthe
thetopic.
d.usestopicappropriatelanguage, clarifytherelationshipsamong
relationshipsamongcomplexideas
domainspecificvocabulary,and
complexideasandconcepts.
f
l l
d
h i
h
h
i il
d

AzMERITPerformanceLevelDescriptors

D-85

AzMERIT - 2015

EnglishLanguageArts
Grade11

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
andconcepts.
techniquessuchasmetaphor,simile,
e.attemptsaformalstyleand
d.usespreciselanguage,domain
objectivetonewhile
andanalogytomanagethe
specificvocabulary,andtechniques d.effectivelyusespreciselanguage,
demonstratingawarenessofthe complexityofthetopic.
suchasmetaphor,simile,and
normsandconventionsof
domainspecificvocabulary,and
analogytomanagethecomplexityof techniquessuchasmetaphor,simile,
standardEnglish.
e.establishesaformalstyleand
objectivetonewhiledemonstrating thetopic.
andanalogytomanagethe
f.providesaconcluding
awarenessofthenormsand
complexityofthetopicandachievea
e.establishesandmaintainsaformal desiredrhetoricaleffect.
statementorsection.
conventionsofthedisciplinein
styleandobjectivetonewhile
whichheorsheiswriting.
attendingtothenormsand
e.establishesandmaintainsa
f.providesaconcludingstatementor conventionsofthedisciplinein
rhetoricallyeffectiveformalstyleand
whichheorsheiswriting.
sectionthatsupportsthe
objectivetonewhileattendingtothe
informationorexplanation
normsandconventionsofthe
f.providesaconcludingstatementor disciplineinwhichheorsheis
presented.
sectionthatfollowsfromand
writing.
supportstheinformationor
explanationpresented(e.g.,
f.providesaneffectiveconcluding
articulatingimplicationsorthe
statementorsectionthatarticulates
significanceofthetopic).
thesignificanceofthetopic,and
followsfromandsupportsthe
informationorexplanationpresented
(e.g.,articulatingimplicationsorthe
significanceofthetopic).

AzMERITPerformanceLevelDescriptors

D-86

AzMERIT - 2015

EnglishLanguageArts
Grade11

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
producescoherentwritinginwhich
Detailed 11.W.4 produceswritinginwhichthe
6
development,organization,and thedevelopment,organization,and
styleareappropriatetothetask styleareappropriatetotask,
andpurpose.Strengthenswriting purpose,andaudience.Strengthens
asneededbyrevisingandediting. writingasneededbyplanning,
Usestechnologytoproduceand revising,andediting.Uses
technology,includingtheInternet,
updatewritingproducts.
toproduce,publish,andupdate
writingproductsinresponseto
ongoingfeedback,includingnew
argumentsorinformation.

Detailed

11.W.7 conductsshortresearchprojects
toansweragivensimple
questionorsolveagivensimple
problem;usesdiscrete
informationfromsourcesonthe
subject,demonstratinga
developingunderstandingofthe
subjectunderinvestigation.

AzMERITPerformanceLevelDescriptors

conductsshortaswellasmore
sustainedresearchprojectsto
answerasimplequestion(including
aselfgeneratedquestion)orsolvea
simpleproblem;narrowsor
broadenstheinquirywhen
appropriate;synthesizessourceson
thesubject,demonstratingan
understandingofthesubjectunder
investigation.

D-87

Standard Setting Technical Report

Proficient
producesclearandcoherentwriting
inwhichthedevelopment,
organization,andstyleare
appropriatetotask,purpose,and
audience.Developsandstrengthens
writingasneededbyplanning,
revising,editing,rewriting,ortrying
anewapproach,focusingon
addressingwhatismostsignificant
foraspecificpurposeandaudience.
Usestechnology,includingthe
Internet,toproduce,publish,and
updateindividualorsharedwriting
productsinresponsetoongoing
feedback,includingnewarguments
orinformation.

HighlyProficient
producesclearandcoherentwriting
inwhichthedevelopment,
organization,andstylearehighly
effectiveforthetask,purpose,and
audience.Developsandstrengthens
writingbyplanning,revising,editing,
rewriting,ortryinganewapproach,
focusingonaddressingwhatismost
significantforaspecificpurposeand
audience.Usestechnology,including
theInternet,toproduce,publish,and
effectivelyupdateindividualor
sharedwritingproductsinresponse
toongoingfeedback,includingnew
argumentsorinformation.

conductsshortaswellasmore
sustainedresearchprojectsto
answeraquestion(includingaself
generatedquestion)orsolvea
problem;narrowsorbroadensthe
inquirywhenappropriate;
synthesizesmultiplesourcesonthe
subject,demonstrating
understandingofthesubjectunder
investigation.

conductsshortaswellasmore
sustainedresearchprojectstoanswer
acomplexquestion(includingaself
generatedquestion)orsolvea
complexproblem;narrows,
broadens,orreformulatestheinquiry
whenappropriate;synthesizes
multiplehighqualitysourcesonthe
subject,demonstratingcomplete
understandingofthesubjectunder
investigation.

AzMERIT - 2015

EnglishLanguageArts
Grade11

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
gathersrelevantinformationfrom gathersrelevantinformationfrom gathershighlyrelevantinformation
Detailed 11.W.8 gathersinformationfrom
frommultipleauthoritativeprintand
multipleauthoritativeprintand
multipleprintanddigitalsources; multipleprintanddigitalsources,
digitalsources,usingadvanced
usingsearcheseffectively;assesses digitalsources,usingadvanced
assessesthestrengthsofeach
searcheseffectively;assessesthe
thestrengthsandlimitationsofeach searcheseffectively;assessesthe
sourceintermsofthetask,
strengthsandlimitationsofeach
sourceintermsofthetask,purpose, strengthsandlimitationsofeach
purpose,andaudience;
integratesinformationintothe andaudience;integratesinformation sourceintermsofthetask,purpose, sourceintermsofthetask,purpose,
intothetexttomaintaintheflowof andaudience;integratesinformation andaudience;seamlesslyintegrates
text,avoidingplagiarismand
intothetextselectivelytomaintain informationintothetextselectively
followingastandardformatfor ideas,avoidingplagiarismand
theflowofideas,avoidingplagiarism tomaintaintheflowofideas,
followingastandardformatfor
citation.
andoverrelianceonanyonesource avoidingplagiarismandoverreliance
citation.
andfollowingastandardformatfor onanyonesourceandadheringtoa
standardformatforcitation.
citation.

AzMERITPerformanceLevelDescriptors

D-88

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
MinimallyProficient
PartiallyProficient
Listening
usesmultiplesourcesofinformation
Detailed 11.SL.2 usesmultiplesourcesof
informationpresentedindiverse presentedindiverseformatsand
formatsandmedia(e.g.,visually, media(e.g.,visually,quantitatively,
quantitatively,orally)inorderto orally)inordertomakeinformed
decisionsandsolveproblems,
makeinformeddecisionsand
evaluatingthecredibilityand
solveproblems.
accuracyofeachsourceandnoting
anydiscrepanciesamongthedata.
Detailed

11.SL.3 describesaspeaker'spointof
view,reasoning,anduseof
evidenceandrhetoric.

Detailed

11.L.1

describesaspeaker'spointofview,
reasoning,anduseofevidenceand
rhetoric,includingthestance,
premises,linksamongideas,word
choice,pointsofemphasis,andtone
used.

Language
demonstratesawarenessofthe
attemptstomeetthe
conventionsofstandardgradelevel
conventionsofstandardgrade
levelEnglishgrammarandusage Englishgrammarandusagewhen
writingorspeaking:(a)
whenwritingorspeaking:(a)
demonstratestheunderstanding demonstratestheunderstanding
thatusageisamatterofconvention,
thatusageisamatterof
convention;(b)clarifiesissuesof canchangeovertime,andis
sometimescontested;(b)resolves
usage,consultingreferences
issuesofcomplexorcontested
(e.g.,MerriamWebster's
usage,consultingreferences(e.g.,
DictionaryofEnglishUsage ,
MerriamWebster'sDictionaryof
Garner'sModernAmerican
EnglishUsage ,Garner'sModern
Usage )asneeded.
AmericanUsage )asneeded.

AzMERITPerformanceLevelDescriptors

Standard Setting Technical Report

EnglishLanguageArts
Grade11

D-89

Proficient

HighlyProficient

integratesmultiplesourcesof
informationpresentedindiverse
formatsandmedia(e.g.,visually,
quantitatively,orally)inorderto
makeinformeddecisionsandsolve
problems,evaluatingthecredibility
andaccuracyofeachsourceand
notinganydiscrepanciesamongthe
data.

effectivelyintegratesmultiple
sourcesofinformationpresentedin
diverseformatsandmedia(e.g.,
visually,quantitatively,orally)in
ordertomakeinformeddecisions
andsolveproblems,evaluatingthe
credibilityandaccuracyofeach
sourceandnotinganydiscrepancies
amongthedata.

evaluatesaspeaker'spointofview,
reasoning,anduseofevidenceand
rhetoric,assessingthestance,
premises,linksamongideas,word
choice,pointsofemphasis,andtone
used.

evaluatesandcritiquesaspeaker's
pointofview,reasoning,anduseof
evidenceandrhetoric,assessingand
analyzingthestance,premises,links
amongideas,wordchoice,pointsof
emphasis,andtoneused.

demonstratesstrongcommandof
demonstratescommandofthe
conventionsofstandardgradelevel theconventionsofstandardgrade
levelEnglishgrammarandusage
Englishgrammarandusagewhen
writingorspeaking:(a)appliesthe whenwritingorspeaking:(a)applies
understandingthatusageisamatter theunderstandingthatusageisa
ofconvention,canchangeovertime, matterofconvention,canchange
overtime,andissometimes
andissometimescontested;(b)
contested;(b)resolvesissuesof
resolvesissuesofcomplexor
complexorcontestedusage,
contestedusage,consulting
references(e.g.,MerriamWebster's consultingreferences(e.g.,Merriam
DictionaryofEnglishUsage ,Garner's Webster'sDictionaryofEnglish
ModernAmericanUsage )asneeded. Usage ,Garner'sModernAmerican
Usage )asneeded.

AzMERIT - 2015

EnglishLanguageArts
Grade11

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
MinimallyProficient
PartiallyProficient
demonstratescommandofthe
demonstratesawarenessofthe
Detailed 11.L.2 attemptstomeetthe
conventionsofstandardEnglish
conventionsofstandardEnglish conventionsofstandardEnglish
capitalization,punctuation,and
capitalization,punctuation,and capitalization,punctuation,and
spellingwhenwriting:(a)attempts spellingwhenwriting:(a)observes
spellingwhenwriting.
toobservehyphenationconventions; hyphenationconventions;(b)spells
correctly.
(b)spellscorrectly.
Detailed

11.L.3

Standard Setting Technical Report


HighlyProficient
demonstratesstrongcommandof
theconventionsofstandardEnglish
capitalization,punctuation,and
spellingwhenwriting:(a)observes
hyphenationconventions;(b)spells
correctly.

appliesdeepknowledgeoflanguage
usesknowledgeoflanguagefor usesknowledgeoflanguagetomake appliesknowledgeoflanguageto
understandhowlanguagefunctions tounderstandhowlanguage
comprehensionwhenreadingor effectivechoicesformeaningor
functionsindifferentcontexts,to
style,andtocomprehendmorefully indifferentcontexts,tomake
listening.
makehighlyeffectivechoicesfor
effectivechoicesformeaningor
whenreadingorlistening.Varies
style,andtocomprehendmorefully meaningorstyle,andtoaiddeep
syntaxforeffect,consulting
comprehensionwhenreadingor
whenreadingorlistening.Varies
references(e.g.,Tufte'sArtful
listening.Variessyntaxforeffect,
Sentences )forguidanceasneeded. syntaxforeffect,consulting
consultingreferences(e.g.,Tufte's
references(e.g.,Tufte'sArtful
Sentences )forguidanceasneeded; ArtfulSentences )forguidanceas
appliesanunderstandingofsyntax needed;appliesathorough
tothestudyofcomplextextswhen understandingofsyntaxtothestudy
ofcomplextextswhenreading.
reading.

AzMERITPerformanceLevelDescriptors

D-90

AzMERIT - 2015

EnglishLanguageArts
Grade11

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


PLD Standard
Proficient
MinimallyProficient
PartiallyProficient
determinesthemeaningofunknown determinesorclarifiesthemeaning
Detailed 11.L.4 determinesthemeaningof
ofunknownandmultiplemeaning
unknownandmultiplemeaning andmultiplemeaningwordsand
phrasesbyusingcontextclueswithin gradelevelwordsandphrasesby
wordsandphrasesbyusing
usingcontextcluesasacluetothe
thesamesentence;identifying
immediatecontextcluesand
meaningofawordorphrase;
patternsofwordchangesthat
consultinggeneralreference
materials,bothprintanddigital, indicatedifferentmeaningsorparts identifyingandcorrectlyusing
ofspeech;consultinggeneraland patternsofwordchangesthat
tofindthepronunciationofa
wordordetermineitsmeaningor specializedreferencematerials,both indicatedifferentmeaningsorparts
ofspeech;consultinggeneraland
itsstandardusage;andverifying printanddigital,tofindthe
specializedreferencematerials,both
thepreliminarydeterminationof pronunciationofawordor
printanddigital,tofindthe
themeaningofawordorphrase. determineorclarifyitsprecise
pronunciationofawordor
meaning,itspartofspeech,its
determineorclarifyitsprecise
etymology,oritsstandardusage;
meaning,itspartofspeech,its
andverifyingthepreliminary
determinationofthemeaningofa etymology,oritsstandardusage;
andverifyingthepreliminary
wordorphrase.
determinationofthemeaningofa
wordorphrase.

Detailed

11.L.5

recognizesfigurativelanguage
andwordrelationships.
Recognizesfiguresofspeechin
context.Recognizesnuancesin
themeaningofwordswith
similardenotations.

AzMERITPerformanceLevelDescriptors

demonstratesunderstandingof
straightforwardfigurativelanguage,
clearwordrelationships,and
nuancesinwordmeanings.
Interpretsfiguresofspeechin
context.Recognizesnuancesinthe
meaningofwordswithsimilar
denotations.

D-91

demonstratesunderstandingof
figurativelanguage,word
relationships,andnuancesinword
meanings.Interpretsfiguresof
speechincontextandanalyzestheir
roleinthetext.Analyzesnuancesin
themeaningofwordswithsimilar
denotations.

HighlyProficient
determinesorclarifiesthemeaning
ofunknownandmultiplemeaning
wordsandphrases,includingabove
gradelevelcontent,byusingcontext
cluesasacluetothemeaningofa
wordorphrase;identifyingand
correctlyusingpatternsofword
changesthatindicatedifferent
meaningsorpartsofspeech;
consultinggeneralandspecialized
referencematerials,bothprintand
digital,tofindthepronunciationofa
wordordetermineorclarifyits
precisemeaning,itspartofspeech,
itsetymology,oritsstandardusage;
andverifyingthepreliminary
determinationofthemeaningofa
wordorphrase.

demonstratesadeepunderstanding
offigurativelanguage,complexword
relationships,andcomplexnuances
inwordmeanings.Interprets
complexfiguresofspeechincontext
andanalyzestheirroleinthetext.
Analyzesnuancesinthemeaningof
wordswithsimilardenotations.

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD

Standard

Minimally Proficient
The Minimally Proficient student

Detailed

Detailed

3.OA.A [1 to
4]

3.OA.B [5 to
6]

Interprets whole number products


and quotients with visual support.
Multiplies and divides within 100
to solve word problems involving
equal groups and arrays when a
visual model is given. Determines
the unknown whole number in a
multiplication or division equation,
when the unknown number is the
product or quotient. Factors and
divisors are less than or equal to 5
for all problems.

Applies the properties of


operations to multiply and divide.
Solves division as unknown factor
problems by finding missing
number in the second factor
position with visual support.
Factors and divisors are less than
or equal to 5 for all problems.

AzMERIT Performance Level Descriptors

Mathematics
Grade 3
Partially Proficient

The Partially Proficient student

Standard Setting Technical Report

Proficient
The Proficient student

Operations and Algebraic Thinking


Interprets whole number products Interprets products and
and quotients with visual support. quotients of single-digit
Multiplies and divides within 100 to whole numbers using equal
solve word problems involving
groups of objects, arrays of
equal groups and arrays when a
objects and comparison.
visual model is given. Determines
Multiplies and divides within
the unknown whole number in a
100 to solve single-step word
multiplication or division equation, problems involving equal
when the unknown number is the groups, arrays, and
product or quotient. Factors and
measurement quantities.
divisors are less than or equal to 9 Determines an unknown
for all problems.
whole number, in any
position, in a multiplication
and division equation.
Applies the properties of operations
to multiply and divide. Solves
division as unknown factor
problems by finding missing
number in any position with visual
support. Factors and divisors are
less than or equal to 9 for all
problems.

D-92

Applies the properties of


operations as strategies to
multiply and divide.
Determines an appropriate
strategy for a given situation.
Understands that division
can be expressed as an
unknown factor problem by
using the relationship
between multiplication and
division.

Highly Proficient
The Highly Proficient student
Interprets products and quotients of
whole numbers within 100,
representing context using pictures,
numbers, and words. Multiplies and
divides within 100 to solve multi-step
word problems involving equal
groups, arrays, and measurement
quantities. Determines an unknown
whole number in a multiplication and
division equation. Students will use
the given context to generate an
equation or create a word problem.

Applies multiple strategies of


operations within a word problem.
Solves division as unknown factor
problems by using the relationship
between multiplication and division,
models multiplication and division in
a variety of ways.

Mathematics
Grade 3

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Multiplies and divides single-digit Fluently multiplies and divides all
numbers using a variety of
single-digit numbers using variety
strategies and supports.
strategies.
Detailed 3.OA.C [7]
PLD

Detailed

3.OA.D [8 to
9]

Solve two-step word problems


using addition and subtraction
with simple context and concrete
objects or visual representations.
Identifies additive arithmetic
patterns using visual supports,
such as an addition table.

AzMERIT Performance Level Descriptors

Solve two-step word problems


using the four operations with
simple context and visual
representations (with the unknown
in a variety of positions). Identifies
multiplicative and subtractive
arithmetic patterns using visual
supports.

D-93

Standard Setting Technical Report


Proficient
Highly Proficient
Knows from memory all
Fluently multiplies and divides within
products of two single-digit 100 using a wide range of contexts.
numbers, fluently multiplies
products within 100, fluently
divides dividends that are
less than 100.
Solve two-step word
problems using equations in
the four operations (using a
letter standing for the
unknown quantity).
Recognizes the
reasonableness of answers
using mental computation
and estimation strategies.
Identifies arithmetic patterns
and explains them using
properties of operations.

Creates two-step word problems


using multiple operations. Creates
and extends arithmetic patterns,
explains patterns using properties of
operations.

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient

Detailed

Detailed

Uses place value understanding to


round a two-digit number to the
nearest 10. Adds and subtracts
two digit numbers using visual
models or support. Skip counts by
10, 20 or 50 to multiply single-digit
3.NBT.A [1 to
whole numbers by multiples of 10
3]
in the range 10-90.

3.NF.A [1 to
2b]

Identifies the numerator and


denominator of a fraction or a
fraction on a number line where
the increments are equal to the
denominator.

AzMERIT Performance Level Descriptors

Mathematics
Grade 3

Partially Proficient
Proficient
Number and Operations in Base Ten
Uses place value understanding to Uses place value
round a three-digit number to the understanding to round
nearest 100. Adds and subtracts
whole numbers (up to 1,000)
numbers within 1,000 using visual to the nearest 10 or 100.
models or support. Uses grouping Fluently adds and subtracts
strategies (associative property) to within 1,000 using any
multiply single-digit whole numbers strategy. Multiplies singleby multiples of 10 in the range 10- digit whole
90.
numbers by multiples of 10
in the range 10-90 using any
of a variety of strategies.

Number and Operations - Fractions


Identifies the meaning of the
Understands 1/b is equal to
numerator and denominator of a
one part when the whole is
fraction. Represents a fraction on a partitioned into b equal
partitioned number line.
parts (where the
denominators are 2, 3, 4, 6
or 8). Represents a fraction
on a number line by
partitioning into equal parts.

D-94

Standard Setting Technical Report


Highly Proficient
Uses rounding strategies in realworld situations. Explains the
method used in finding the sum or
difference; recognizes and identifies
an error and shows the correct
answer. Shows product of single-digit
whole numbers by multiples of 10
using multiple strategies.

Applies understanding of unit


fractions to real world situations and
problems. Represents a set of
fractions with unlike denominators
on a number line by partitioning into
equal parts.

AzMERIT - 2015

Mathematics
Grade 3

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Understands, recognizes, and
Understands, recognizes, and
generates equivalent fractions using
generates equivalent fractions
using denominators of 2, 4 and 8 denominators of 2, 4 and 8.
given visual models. Expresses and Expresses and recognizes fractions
recognizes fractions that are
that are equivalent to whole
equivalent to 1. Compares two
numbers. Compares two fractions
fractions
with
the
same
with the same numerator and
3.NF.A [3a to
Detailed
denominator and records results records results using symbols.
3d]
using symbols.
PLD

AzMERIT Performance Level Descriptors

D-95

Standard Setting Technical Report


Proficient
Understand, recognizes, and
generates equivalent
fractions using denominators
of 2, 3, 4, 6, and 8; explains
why the fractions are
equivalent using a visual
model. Expresses whole
numbers as fractions.
Compares two fractions that
have the same numerator or
same denominator using
symbols and visual fraction
models.

Highly Proficient
Explains why two fractions are
equivalent. Identifies equivalent
fractions by creating fraction models
to compare fractions with different
denominators that pertain to the
same whole. Compares two fractions
that have the same numerator or
same denominator using symbols.

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient

Detailed

Detailed

3.MD.A [1 to
2]

3.MD.B [3 to
4]

Mathematics
Grade 3

Partially Proficient
Measurement and Data
Tells, writes, and measures time to Solves one-step word problems
the nearest minute. Using grams, involving addition or subtraction of
kilograms or liters, measures and time intervals in minutes with
estimates liquid volumes and
scaffolding. Using grams, kilograms
masses of objects using models.
or liters, solves simple one-step
measurement word problems using
either addition or subtraction.

Completes a scaled picture graph


or bar graph (with a scale factor of
1 or 5) to represent data set with
support. Generates measurement
data by measuring lengths to the
nearest half-inch. Shows the data
by making a line plot, where the
horizontal scale is marked by
whole numbers or halves with
supports.

AzMERIT Performance Level Descriptors

Completes a scaled picture graph or


bar graph to represent a data set
with support. Solves one-step "how
many more" and "how many less"
problems using information
presented in scaled bar graphs.
Generates measurement data by
measuring lengths to the nearest
quarter-inch. Shows the data by
making a line plot, where the
horizontal scale is marked by whole
numbers, halves, or quarters with
supports.

D-96

Standard Setting Technical Report


Proficient

Highly Proficient

Solves one-step word


problems involving addition
and subtraction of time
intervals in minutes. Using
grams, kilograms or liters,
estimates and solves onestep measurement word
problems involving any of
the four operations.

Solves two-step real world problems


involving addition and subtraction of
time intervals in minutes. Using
grams, kilograms or liters, estimates
and solves two-step measurement
word problems involving any of the
four operations.

Creates a scaled picture


graph or bar graph to
represent a data set. Solves
two-step "how many more"
and how many less"
problems using information
presented in scaled bar
graphs. Shows the data by
making a line plot, where the
horizontal scale is marked by
whole numbers, halves or
quarters.

Solves multi-step "how many more"


and how many less" problems using
information presented in scaled bar
graphs. Uses a line plot to answer
questions or solve problems.

Mathematics
Grade 3

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Understands what a square unit is Understands area is measured using
and that a plane figure can be
square units, finds area of a
covered without gaps or overlaps rectangle by counting the square
to find an area. Finds the area of units. Shows that the area of a
one or two rectangles by tiling.
rectangle find by tiling is the same
3.MD.C [5a to
as would be found by multiplying
Detailed
7d ]
the side lengths. Finds the area of
two rectangles by tiling and adds
the areas of the rectangles.
PLD

Detailed

3.MD.D [8]

Finds the perimeter and area of


Solves mathematical problems
polygons (given the side lengths). involving perimeters of polygons,
including finding the perimeter and
area (given the side lengths);
compares and contrasts area and
perimeter.

AzMERIT Performance Level Descriptors

D-97

Standard Setting Technical Report


Proficient
Understands area is
measured using square units,
finds area of a plane figure
by counting the square units
or multiplying the side
lengths, in the context of
solving real-world and
mathematical problems.
Represents whole number
products as rectangular
areas.

Highly Proficient
Finds the area of 2 plane figures by
counting the square units or
multiplying their side lengths and
compares their sizes. Creates a word
problem using the distributive
property to find the area of
rectangles.

Solves real-word and


mathematical problems
involving perimeters of
polygons, finding an
unknown side length, and
exhibiting rectangles with
the same perimeter and
different areas or with the
same area and different
perimeters.

Constructs rectangles that have the


same perimeter but different areas
and the reverse.

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient

Detailed

Mathematics
Grade 3

Partially Proficient
Geometry
Identifies examples of
Understands the properties of
quadrilaterals; recognizes that
quadrilaterals and the
examples of quadrilaterals have
subcategories of quadrilaterals.
shared attributes, and that the
Partitions shapes into parts with
shared attributes can define a
equal areas and expresses the area
larger category. Partitions shapes as a unit fraction of the whole
into parts with equal areas and
(limited to halves, quarters, and
expresses the area as a unit
eighths).
fraction
of
the
whole
(limited
to
3.G.A [1 to 2]
halves and quarters).

AzMERIT Performance Level Descriptors

D-98

Standard Setting Technical Report


Proficient

Highly Proficient

Recognizes and sorts


examples of quadrilaterals
that have shared attributes
and that the shared
attributes can define a larger
category; draws examples of
quadrilaterals that don't
belong to the categories of
rhombuses, rectangles, and
squares. Partitions shapes
into parts with equal areas
and expresses the area as a
unit fraction (with
denominator of 2, 3, 4, 6, or
8) of the whole.

Recognizes and sorts examples of


quadrilaterals that have shared
attributes and that the shared
attributes can define a larger
category; draws examples and nonexamples of quadrilaterals that are
not rhombuses, rectangles, or
squares. Partitions shapes in multiple
ways into parts with equal areas and
expresses the area as a unit fraction
of the whole.

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD

Standard

Minimally Proficient
The Minimally Proficient student

Detailed

4.OA.A [1 to 3.1]

Recognizes that any two factors


and their product can be read as a
comparison. Solves word problems
involving multiplicative
comparison (where the unknown
is the product or quotient), given
visual representations. Solves
multi-step word problems using
the four operations with simple
context and scaffolding, where the
final answer is the unknown.
Solves a counting problem with
two attributes using a visual
representation.

AzMERIT Performance Level Descriptors

Mathematics
Grade 4
Partially Proficient
The Partially Proficient student

Standard Setting Technical Report

Proficient
The Proficient student

Operations and Algebraic Thinking


Represents comparisons of two Represents comparisons of
two factors and their product
factors and their product as
equations using supports. Solve as equations without
word problems involving
support. Solves word
multiplicative comparison (where problems involving
the unknown is in a variety of
multiplicative comparison,
positions), given visual
where the unknown is in a
representations. Solves multi-step variety of positions. Solves
word problems (which may
multi-step word problems
include interpreting remainders) (including interpreting
using the four operations with
remainders) using the four
simple context and scaffolding,
operations. The unknown is
where the final answer is the
in a variety of positions and
unknown. Creates and uses any can be represented by a
visual representation of a
symbol or letter. Recognizes
counting problem with two
the reasonableness of
attributes.
answers using mental
computation and estimation
strategies. Creates and uses
any representation of
counting problems; analyzes
simple relationships between
counting problem
representations.

D-99

Highly Proficient
The Highly Proficient student
Recognizes that any two factors and
their product can be read as a
comparison; uses multiple strategies
and creates his or her own to
represent and describe those
comparisons. Creates own context
for multiplicative comparison. Solves
complex multi-step word problems
with multiple possible solutions and
determines which would be the most
reasonable based upon given criteria.
Analyzes relationships between any
two representations of a counting
problem and makes connections to
the multiplication principle.

Mathematics
Grade 4

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Finds factor pairs for a multiple of Finds factor pairs for any whole
10. Determines whether a whole number. Determines whether a
number in the range of 1 to 25 is whole number in the range of 1
prime or composite, given visual to 50 is prime or composite, given
representations.
visual representations.

Detailed

Detailed

4.OA.B [4]

4.OA.C [5]

Generates a number or shape


pattern that follows a given rule,
using visual models.

AzMERIT Performance Level Descriptors

Standard Setting Technical Report


Proficient
Highly Proficient
Recognizes that a whole
Applies the concepts of both factors
number is a multiple of each and prime and composite numbers in
of its factors and determines problem-solving contexts.
a given whole number in the
range of 1 to 100 is a
multiple of a given singledigit number. Determines
whether a whole number in
the range of 1 to 100 is prime
or composite.

Generates a number or shape


Generates a number or
Generates a number or shape
pattern that follows a given rule. shape pattern that follows a pattern that combines two
given rule; identifies
operations for a given rule.
apparent features that are
not explicit in the rule.

D-100

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient

Detailed

Detailed

4.NBT.A [1 to 3]

4.NBT.B [4 to 6]

With numbers within 10,000,


recognizes that a digit in one place
represents 10 times as much as it
represents in the place to its right,
reads and writes multi-digit whole
numbers in a variety of forms, and
uses place value understanding to
round multi-digit whole numbers.

Fluently adds and subtracts multidigit whole numbers using the


standard algorithm without
regrouping. Finds products of a
whole number (of up to three
digits) by a single-digit whole
number and whole number
quotients and remainders (with up
to double-digit dividends and
single-digit divisors).

AzMERIT Performance Level Descriptors

Mathematics
Grade 4

Standard Setting Technical Report

Partially Proficient
Proficient
Highly Proficient
Number and Operations in Base Ten
With numbers within 100,000,
With numbers within
Uses place value strategies,
recognizes that a digit in one
1,000,000, recognizes that a comparisons of two numbers, and
place represents 10 times as
digit in one place represents rounding in a real-world context.
much as it represents in the place 10 times as much as it
to its right, reads and writes multi- represents in the place to its
digit whole numbers in a variety right, reads and writes multidigit whole numbers in a
of forms, and uses place value
variety of forms, and uses
understanding to round multidigit whole numbers.
place value understanding to
round multi-digit whole
numbers.
Fluently adds and subtracts multidigit whole numbers using the
standard algorithm with supports.
Finds products of a whole
number (of up to four digits) by a
single-digit whole number and
whole number quotients and
remainders (with up to three-digit
dividends and single-digit
divisors).

D-101

Fluently adds and subtracts


multi-digit whole numbers
using the standard algorithm.
Finds products of a whole
number (of up to four digits)
by a single-digit whole
number or two double-digit
numbers and whole number
quotients and remainders
(with up to four-digit
dividends and single-digit
divisors) in context.
Illustrates and explains
calculations by using
equations, rectangular
arrays, and/or area models.

Recognizes and identifies an error in


an addition or subtraction and shows
the correct answer. Interprets a
multiplication or division context and
explains strategies used to solve.
Fluently adds and subtracts multidigit
whole numbers using the standard
algorithm.

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient

Detailed

Detailed

4.NF.A [1 to 2]

Uses area fraction models to


represent equivalent fractions by
partitioning unit fraction pieces
into smaller equal pieces. Uses a
visual fraction model to compare
two fractions with different
numerators and different
denominators.

4.NF.B [3]

Adds and subtracts fractions with


like denominators by joining and
separating parts referring to the
same whole with or without
context using visual or
manipulative models, with no or a
simple context. Converts mixed
numbers to equivalent fractions.

AzMERIT Performance Level Descriptors

Mathematics
Grade 4

Standard Setting Technical Report

Partially Proficient
Proficient
Number and Operations - Fractions
Uses area fraction models to
Uses area fraction models
represent equivalent fractions by and double number lines to
partitioning unit fraction pieces generate and explain why
into smaller pieces (and
fraction a /b is equivalent to
understands that this is the
a fraction (n x a)/(n x b),
same), and multiplies by 1
where n is a non-negative
represented as a fraction.
whole number. Compares
two fractions with different
numerators and different
denominators and justifies
answers using visual fraction
models.

Adds and subtracts fractions with


like denominators by joining and
separating parts referring to the
same whole using visual or
manipulative models, with no or a
simple context. Decomposes a
fraction into a sum of fractions
with the same denominator and
records the decomposition using
an equation. Converts mixed
numbers into equivalent fractions
and adds and subtracts them.

D-102

Adds and subtracts fractions


with like denominators by
joining and separating parts
referring to the same whole,
with or without context.
Decomposes a fraction into a
sum of fractions with the
same denominator in more
than one way and records
the decomposition using an
equation.

Highly Proficient
Uses a variety of strategies to
generate and explain why fraction
a/b is equivalent to a fraction (n x
a)/(n x b), where n is a non-negative
whole number. Extends
understanding to compare and order
fractions with different numerators
and different denominators.

Adds and subtracts more than 2


fractions with like denominators by
joining and separating parts referring
to the same whole, with or without
context. Decomposes a fraction into
a sum of fractions with the same
denominator in multiple ways and
records the decomposition using an
equation.

Mathematics
Grade 4

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Understands a fraction a/b as a
Understands a fraction a/b as a
multiple of 1/b by using visual
multiple of 1/b , and uses this
fraction models.
understanding to multiply a
fraction by a whole number,
using visual fraction model.
Detailed 4.NF.B [4]
PLD

Detailed

4.NF.C [5 to 7]

Expresses a fraction with


denominator 10 as an equivalent
fraction with denominator 100 by
using a model. Uses decimal
notation for fractions with a
denominator of 10, with supports.
Compares two decimals with the
same number of places (tenths or
hundredths) using supports.

AzMERIT Performance Level Descriptors

Adds two fractions with


respective denominators 10 and
100 by first finding equivalent
fractions with like denominators
by using a model. Uses decimal
notation for fractions with
denominators of 10 or 100, with
supports. Compares two decimals
to the hundredth by reasoning
about their size using models.

D-103

Standard Setting Technical Report


Proficient
Understands and solves
simple word problems by
recognizing that fraction a/b
is a multiple of 1/b , and uses
that construct to multiply a
fraction by a whole number
(in general, n x a/b is (n x
a)/b ).

Highly Proficient
Understands and solves more
complex word problems by
recognizing that fraction a/b is a
multiple of 1/b , and uses that
construct to multiply a fraction by a
whole number (in general, n x a/b is
(n x a)/b ).

Adds two fractions with


respective denominators 10
and 100 by first finding
equivalent fractions with like
denominators. Uses decimal
notation for fractions with
denominators of 10 or 100.
Compares two decimals in
the tenths and the
hundredths (using <, >, and =
) by reasoning about their
size and records the result of
the comparison using the
correct symbols.

Solves missing addend problems with


respective denominators 10 and 100
by first finding equivalent fractions
with like denominators.
Demonstrates knowledge of decimal
notation for fractions with
denominators of 10 or 100 by
converting a number with decimal
notation to a decimal fraction.
Orders decimal set composed of
tenths and hundredths by reasoning
about their size. Recognizes that the
decimals must refer to the same
whole.

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient

Detailed

Mathematics
Grade 4

Partially Proficient
Measurement and Data
Knows relative size of
Expresses measurements in a
measurement units, within one
larger unit in terms of a smaller
system of units. Uses the four
unit, within a single system, using
operations to solve word problems supports and adjacent units. Uses
(involving distance, liquid volumes, the four operations to solve word
problems (involving distance,
masses of objects, intervals of
liquid volumes, masses of objects,
time and money), including
intervals of time and money,
problems involving whole
numbers, using supports. Applies area, and perimeter), including
the area and perimeter formulas problems involving simple
fractions or decimals, using
when given all side
supports.
measurements, using supports.

4.MD.A [1 to 3]

AzMERIT Performance Level Descriptors

D-104

Standard Setting Technical Report


Proficient

Highly Proficient

Expresses measurements in a
larger unit in terms of a
variety of smaller units,
within a single system, and
records that data in a twocolumn table. Uses the four
operations to solve word
problems (involving distance,
liquid volumes, masses of
objects, intervals of time and
money), including problems
involving simple fractions or
decimals, and problems that
require expressing
measurements given in a
larger unit in terms of a
smaller unit. Represents
measurement quantities
using diagrams. Applies the
area and perimeter formulas
for rectangles in real-world
and mathematical problems,
including those where the
area/perimeter and one
factor (length or width) are
known.

Given a context, determines the


appropriate unit needed and
expresses the measurement to the
level of accuracy needed. Uses the
four operations to solve multi-step
word problems (involving distance,
liquid volumes, masses of objects,
intervals of time and money),
including problems involving
fractions or decimals, and problems
that require expressing
measurements given in a larger unit
in terms of a smaller unit. Represents
measurement quantities using
diagrams. Applies the area and
perimeter formulas for rectilinear
shapes in real-world and
mathematical problems.

Mathematics
Grade 4

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Makes a line plot to display a data Makes a line plot to display a data
set of measurements in fractions set of measurements in fractions
of a unit (with like denominators of a unit (with like denominators
of 2 or 4).
of 2 or 4), and uses addition and
subtraction of fractions to solve
problems involving information in
Detailed 4.MD.B [4]
the line plot.
PLD

Detailed

4.MD.C [5 to 7]

Measures benchmark angles.


Recognizes that angle measure is
additive. Solves addition realworld mathematical problems to
find unknown angles on a diagram
with no more than two angles,
within a 90-degree angle.

AzMERIT Performance Level Descriptors

Understands that angles are


measured in reference to a circle,
and can measure angles in whole
number degrees using a
protractor. Solves addition and
subtraction real-world
mathematical problems to find
unknown angles on a diagram
with no more than two angles,
within a 180-degree angle.

D-105

Standard Setting Technical Report


Proficient
Highly Proficient
Makes a line plot to display a Uses data in a line plot to solve a
data set of measurements in multi-step word problem.
fractions of a unit (with like
denominators limited to 2, 4
and 8), and uses addition and
subtraction of fractions to
solve problems involving
information in the line plot.

Understands that angles are


measured in reference to a
circle, and can measure
angles in whole number
degrees using a protractor.
Sketches angles of specific
measure. Solves addition and
subtraction real-world
mathematical problems to
find unknown angles on a
diagram.

Recognizes how angles are formed,


understands that angles are
measured in reference to a circle,
and can measure angles in whole
number degrees using a protractor.
Sketches angles of specific measure.
Given angle parameters, decomposes
into multiple angles and gives the
measure of each angle in relationship
to the whole.

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient

Detailed

4.G.A [1 to 3]

Mathematics
Grade 4

Partially Proficient
Geometry
Identifies points, lines, line
Identifies and draws points, lines,
segments, rays, perpendicular and line segments, rays, angles (right,
parallel lines, two-dimensional
acute, obtuse), and perpendicular
figures, including right triangles, and parallel lines. Classifies twoand line-symmetric regular figures; dimensional figures based on the
presence or absence of parallel or
classifies angles (right, acute,
perpendicular lines; identifies
obtuse).
triangles. Draws lines of
symmetry for regular twodimensional figures.

AzMERIT Performance Level Descriptors

D-106

Standard Setting Technical Report


Proficient

Highly Proficient

Draws points, lines, line


segments, rays, angles (right,
acute, obtuse), and
perpendicular and parallel
lines; identifies these in twodimensional figures.
Classifies two-dimensional
figures based on the
presence or absence of
angles of specified size.
Draws lines of symmetry for
any two-dimensional figure.

Creates a two-dimensional shape


when given specific attributes,
including the presence or absence of
parallel or perpendicular lines, the
presence or absence of angles of
specified size, and particular lines of
symmetry.

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD

Standard

Minimally Proficient
The Minimally Proficient student

Detailed

Detailed

5.OA.A [1 to
2]

5.OA.B [3]

Evaluates a simple numerical


expression using parentheses,
brackets, or braces (without
nesting). Writes a numerical
expression, using one operation,
from a written statement.

Mathematics
Grade 5
Partially Proficient

Standard Setting Technical Report

Proficient

The Partially Proficient student The Proficient student

The Highly Proficient student

Operations and Algebraic Thinking


Evaluates a numerical
Uses parentheses, brackets, or
expression using parentheses, braces in numerical expressions
brackets, or braces (without
(without nesting), and evaluates
nesting). Writes simple
expressions with these symbols.
numerical expressions and
Writes numerical expressions and
interprets numerical
interprets numerical expressions,
expressions, without evaluating without evaluating them.
them.

Inserts parentheses, brackets, or


braces (without nesting), in
numerical expressions to make a
statement true. Writes numerical
expressions using multiple
operations, involving real-world and
mathematical contexts.

Continues two numerical patterns Continues two numerical


Generates two numerical patterns
(when given a table), using two
patterns using two given rules. using two given rules. Identifies
given rules.
apparent relationships between
corresponding terms.

Number and Operations in Base Ten

AzMERIT Performance Level Descriptors

Highly Proficient

D-107

Generates two numerical patterns


using two multi-step given rules, in
mathematical contexts. Explains the
relationship between corresponding
terms.

AzMERIT - 2015

Mathematics
Grade 5

Appendix D. Performance Level Descriptors (PLDs)


Minimally Proficient
Partially Proficient
Standard
Uses visual models or calculation Uses visual models or
to demonstrate a digit in one
calculation to recognize that a
place of a whole number
digit in one place in a whole
represents 10 times as much as it number represents 10 times as
represents in the place to its right, much as it represents in the
or 1/10 of what it represents in
place to its right and 1/10 of
the place to its left. Continues a
what it represents in the place
given pattern that shows the
to its left. Recognizes patterns
number of zeroes of the product in the number of zeroes of
5.NBT.A [1 to when multiplying a number by
products when multiplying a
Detailed
2]
powers of 10.
number by powers of 10. Uses
whole number exponents
greater than zero to denote
powers of 10.
PLD

Detailed

Reads decimals to the


thousandths place. Compares two
decimals to the tenths place, using
>, =, and < symbols to record the
results of comparisons. Uses place
value understanding to round
multi-digit numbers to the tenths
5.NBT.A [3 to place.
4]

AzMERIT Performance Level Descriptors

Reads and writes decimals to


the thousandths place, using
base-ten numerals and number
names. Compares two decimals
to the hundredths place, using
>, =, and < symbols to record
the results of comparisons.
Uses place value understanding
to round multi-digit whole
numbers to the hundredths
place.

D-108

Standard Setting Technical Report


Proficient
Recognizes (in any multi-digit
number, including decimals to
thousandths) that a digit in one
place represents 10 times as much
as it represents in the place to its
right and 1/10 of what it
represents in the place to its left.
Explains patterns in the number
of zeroes of the product when
multiplying a number by powers
of 10, and explains patterns in the
placement of the decimal point
when a decimal is multiplied or
divided by a power of 10. Uses
whole number exponents to
denote powers of 10, including 10
to the power of zero.

Highly Proficient
Recognizes (in any multi-digit
number, including decimals to
thousandths) that a digit in one place
represents 10 times as much as it
represents in the place to its right
and 1/10 of what it represents in the
place to its left, in real-world or
mathematical context problems.
Interprets a multiplication problem
to identify the factor of 10 by which
one number is greater or lesser than
another.

Reads and writes decimals to the


thousandths place, using base-ten
numerals, number names, and
expanded form (e.g., 347.392 = 3
X 100 + 4 X 10 + 7 X 1 + 3 X (1/10)
+ 9 X (1/100) + 2 X (1/1000).
Compares two decimals to the
thousandths place (with varying
place values), using >, =, and <
symbols to record the results of
comparisons. Uses place value
understanding to round multidigit numbers up to any place
(within content limits).

Writes numbers in expanded form in


a variety of formats (e.g., 347.392 = 7
X 1 + 3.4 X 100 + 3 X (1/10) + 2 X
(1/1000) + (1/100) X 9). Compares
and orders decimals to the
thousandths place (with varying
place values), from least to greatest
or vice-versa. Uses rounding
strategies in real-world situations.

Mathematics
Grade 5

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


Minimally Proficient
Partially Proficient
Standard
Multiplies two two-digit numbers Multiplies three-digit by twousing a standard algorithm. Finds digit whole numbers, using a
whole-number quotients of whole standard algorithm. Finds
numbers (with up to two digit
whole-number quotients of
dividends and two-digit divisors), whole numbers (with up to
using rectangular arrays or area
three digit dividends and twomodels.
digit divisors), using strategies
5.NBT.B [5 to
based on place value and the
Detailed
6]
properties of operations.
PLD

Adds, subtracts, multiplies, and


divides decimals to the tenths
place, using concrete models,
drawings, or strategies based on
place value.
Detailed

5.NBT.B [7]

AzMERIT Performance Level Descriptors

Adds, subtracts, multiplies, and


divides decimals to the
hundredths place, using
concrete models or drawings,
strategies based on place
value, and/or the relationship
between addition and
subtraction; relates the
strategy to a written method.

D-109

Standard Setting Technical Report


Proficient
Highly Proficient
Fluently multiplies multi-digit
Fluently multiplies multi-digit whole
whole numbers using a standard numbers, in real-world and
algorithm. Finds whole-number
mathematical contexts, using a
quotients of whole numbers (with standard algorithm. Finds wholeup to four digit dividends and two- number quotients of whole numbers
(with up to four digit dividends and
digit divisors), using strategies
two-digit divisors) in context.
based on place value, the
properties of operations, and/or
the relationship between
multiplication and division.
Illustrates and explains the
calculation by using equations,
rectangular arrays, and/or area
models.
Adds, subtracts, multiplies, and
divides decimals to the
hundredths place, using concrete
models or drawings and strategies
based on place value, properties
of operations, and/or the
relationship between addition
and subtraction; relates the
strategy to a written method and
explains the reasoning used.

Adds, subtracts, multiplies, and


divides decimals to the hundredths
place, using multiple strategies, in a
real-world or mathematical context;
relates the strategy to a written
method and explains the reasoning
used.

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Minimally Proficient
Standard

Detailed

Detailed

Adds/subtracts fractions with


unlike denominators, where one
denominator is a multiple of the
other denominator, with the use
of a visual model. Solves word
problems involving addition and
subtraction of fractions with
5.NF.A [1 to 2] unlike denominators, where one
denominator is a multiple of the
other denominator, using visual
representations. Determines a
common denominator, with use of
a visual model.

5.NF.B [3]

Rewrites a fraction as a division


problem; uses manipulatives or
visual models to solve problems
involving division of whole
numbers, leading to answers in
the form of fractions or mixed
numbers.

AzMERIT Performance Level Descriptors

Mathematics
Grade 5

Standard Setting Technical Report

Partially Proficient
Proficient
Number and Operations - Fractions
Adds/subtracts fractions with Adds and subtracts fractions with
unlike denominators, where
unlike denominators (including
one denominator is a multiple mixed numbers). Solves word
of the other denominator.
problems involving addition and
Solves word problems involving subtraction of fractions with
unlike denominators (including
addition and subtraction of
mixed numbers). Assesses and
fractions with unlike
justifies reasonableness of the
denominators, where one
answer by using benchmark
denominator is a multiple of
fractions, visual models, or
the other denominator.
equations.

Solves word problems involving


division of whole numbers
leading to answers in the form
of fractions or mixed numbers.

D-110

Highly Proficient
Adds or subtracts at least 3 or more
fractions with unlike denominators
(including mixed numbers). Solves
word problems involving addition or
subtraction with at least 3 or more
fractions with unlike denominators
(including mixed numbers).

Interprets a fraction as division of Creates his or her own model to


the numerator by the
demonstrate division of fractions.
denominator (a/b = a b ); solves
word problems involving division
of whole numbers, leading to
answers in the form of fractions
or mixed numbers.

AzMERIT - 2015

Mathematics
Grade 5

Appendix D. Performance Level Descriptors (PLDs)


Minimally Proficient
Partially Proficient
Standard
Shows the product of a fraction by Shows the product of two
a whole number by repeated
fractions by using an area
addition, using visual fraction
model. Interprets
models. Interprets multiplication multiplication scaling by
scaling by comparing the size of a comparing the size of a product
product to the size of one factor to the size of one factor on the
on the basis of the size of the
basis of the size of the second
second factor, without performing factor, without performing the
the indicated multiplication
indicated multiplication (where
(where both factors are whole
one factor is a fraction less
numbers).
than one).
Detailed 5.NF.B [4 to 5]
PLD

Detailed

Solves real-world problems


involving multiplication of
fractions (limited to fractions with
single-digit numerators or
denominators) or division of
whole numbers by unit fractions
by using visual fraction models or
5.NF.B [6 to 7]
equations to represent the
problem.

AzMERIT Performance Level Descriptors

Solves real-world problems


involving multiplication of
fractions or division of unit
fractions by non-zero whole
numbers and division of whole
numbers by unit fractions
(limited to single digit whole
numbers and denominators)by
using visual fraction models or
equations to represent the
problem.

D-111

Standard Setting Technical Report


Proficient
Shows the product of two
fractions using an area model and
creates a story context for the
product. Finds the area of a
rectangle with fractional side
lengths by tiling it with squares
with unit fraction side lengths,
and shows that the area is the
same as would be found by
multiplying the side lengths.
Multiplies fractional side lengths
to find areas of rectangles, and
represents fraction products as
rectangular areas. Interprets
multiplication scaling by
comparing the size of a product to
the size of one factor on the basis
of the size of the second factor,
without performing the indicated
multiplication.

Highly Proficient
Creates a real-world context and
models representing multiplication of
fractions. Demonstrates reasoning
about fractions in both an additive
and multiplicative sense with
different wholes, and displays the
quantities with visual models.
Interprets multiplication scaling by
comparing the size of a product to
the size of one factor on the basis of
the size of the second factor by
performing the indicated
multiplication with 2 fractions.

Solves real-world problems


involving multiplication of
fractions and mixed numbers or
division of unit fractions by nonzero whole numbers and division
of whole numbers by unit
fractions, using visual fraction
models and equations to
represent the problem.

Uses several mixed numbers, often


with multi-digit numerators or
denominators, to solve real-world
problems involving multiplication of
fraction or mixed numbers. Creates
real-world problems involving
division of unit fractions by non-zero
whole numbers and division of whole
numbers by unit fractions, using
visual fraction models and equations
to represent the problem.

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Minimally Proficient
Standard
Converts among different-sized
standard measurement units
within a given measurement
system.

Detailed

Detailed

Detailed

5.MD.A [1]

5.MD.B [2]

Mathematics
Grade 5
Partially Proficient

Standard Setting Technical Report


Proficient

Measurement and Data


Converts among different-sized Converts among different-sized
standard measurement units standard measurement units
within a given measurement
within a given measurement
system; uses these conversions system; uses these conversions in
to solve single-step problems, solving multi-step, real-world
using manipulatives or visual
problems.
models.

Highly Proficient
Creates real-world multi-step
problems. Chooses the appropriate
measurement unit based on the
given context.

Plots data on a given line plot with


a data set of measurements in
fractions of a unit (1/2, 1/4, 1/8),
where the given data set is limited
to a common denominator. Solves
addition and subtraction
comparison problems using the
data.

Makes a line plot to display a


data set of measurements in
fractions of a unit (1/2, 1/4, or
1/8), where the given data set
is limited to a common
denominator. Solves problems
using all four operations.

Makes a line plot to display a data


set of measurements in fractions
of a unit (1/2, 1/4, 1/8). Uses
operations on fractions to solve
problems involving information
presented in line plots (division is
limited to a whole number divided
by a fraction or a fraction divided
by a whole number).

Uses unit cubes to find the volume


of rectangular prisms with whole
number edges (limited to single
digit dimensions). Solves volume
problems of a right rectangular
prism by using unit cubes.

Uses unit cubes (number of


unit cubes, edge length, height)
to find the volume of
rectangular prisms. Uses the
information that the number of
unit cubes is related to the
edge length; uses visual
models. Solves volume
problems by relating the
number of unit cubes in a
prism to the multiplication of
the edge lengths.

Uses unit cubes (number of unit Compares the volumes of different


cubes, edge length, height) to find prisms by using unit cubes. Creates
the volume of rectangular prisms. real-world mathematical problems
Represents the volume of a solid that would be solved by finding
figure as n cubic units. Solves real- volume.
world and mathematical problems
by applying the formulas for
volume. Finds the volume of two
non-overlapping right rectangular
prisms by adding the volumes of
the two non-overlapping parts.

5.MD.C [3 to
5]

AzMERIT Performance Level Descriptors

D-112

Makes a line plot to display a data set


of measurements in fractions of a
unit (1/2, 1/4, 1/8). Solves multi-step
word problems using the four
operations and interprets the
solution to the data.

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Minimally Proficient
Standard

Detailed

Detailed

Mathematics
Grade 5

Partially Proficient
Geometry
Identifies the key components of Interprets coordinate values of
the coordinate plane (x -axis, x points in the first quadrant
coordinate, y -axis, y -coordinate (e.g., reading line graphs), in
and origin). Locates given points in context.
5.G.A [1 to 2] the first quadrant of the
coordinate plane.

5.G.B [3 to 4]

Identifies two-dimensional figures Classifies some twobased on properties limited to


dimensional figures into
sides and angles.
categories based on their
properties (sides and angles).

AzMERIT Performance Level Descriptors

D-113

Standard Setting Technical Report


Proficient

Highly Proficient

Represents real-world and


mathematical problems by
locating and graphing points in
the first quadrant of the
coordinate plane.

Using real-world data, creates a


representation and draws
conclusions based on the data
presented.

Understands that attributes


belonging to a category of twodimensional figures also belong to
all subcategories of that category
and classifies two-dimensional
figures in the hierarchy based on
these properties.

Draws or constructs
specific two-dimensional figures
according to the definitions provided,
attributes described, or categories
given.

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD

Standard

Minimally Proficient

Mathematics
Grade 6
Partially Proficient

The Minimally Proficient student The Partially Proficient student


Identifies unit rates and
describes them using basic
language or notation.
Detailed

Detailed

Detailed

6.RP.A [1 to 2]

Uses a limited variety of


representations to solve ratio and
unit rate problems involving
whole numbers and to convert
measurement units, finds the
percent of a quantity, and
manipulates units appropriately
when multiplying or dividing
quantities.

Solves problems in contexts


involving division of whole
numbers by unit fractions using
visual fraction models and
equations.

The Number System


Solves problems in contexts
involving division of fractions by
non-zero whole numbers and vice
versa using visual fraction models
and equations.

AzMERIT Performance Level Descriptors

Proficient
The Proficient student

Ratios and Proportional Relationships


Describes the concept of ratio
Uses the concept of a ratio, ratio
using a limited variety of
language, ratio notation, and unit
representations and determines a rate associated with a ratio to
unit rate.
precisely describe a ratio
relationship between two
quantities and within context.

Identifies proportional
relationships presented in
graphical, tabular, or verbal
formats, knows the meaning of
a percent of a quantity as a rate
per hundred, and finds missing
6.RP.A.3 [a to dvalues in tables and plots values
on the coordinate plane using
whole numbers.

6.NS.A [1]

Standard Setting Technical Report

D-114

Highly Proficient
The Highly Proficient student
Uses and connects between
representation for ratio situations
and finds unit rates requiring
multiple steps.

Uses ratio and rate reasoning to


convert measurement units and
solve real-world problems, solves
unit rate problems including those
involving unit pricing and constant
speed, determines the percent of
a quantity as a rate per 100, and
solves problems involving finding
the whole given a part and a
percent.

Creates and applies ratio reasoning


to solve real-world problems
including those involving percent or
conversion of measurement units.

Solves problems in contexts


involving division of fractions by
fractions and interprets the
solution in context.

Solves problems in contexts involving


multi-step division problems
involving mixed numbers and
interprets the solution in context.

Mathematics
Grade 6

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Adds, subtracts, multiplies
Adds, subtracts, multiplies where
where decimals are limited to
dividends are limited to whole
hundredths, and finds whole
numbers, and finds whole number
number quotients and
quotients and remainders where
remainders where dividends are dividends are up to four digits and
up to four digits and divisors are divisors are up to two digits using
Detailed 6.NS.B [2 to 3]
one digit using strategies based strategies based on place value,
on place value, the properties of the properties of operations, and
operations, and the relationship the relationship between
operations.
between operations.
PLD

Detailed

6.NS.B [4]

Finds common factors of two


whole numbers less than or
equal to 50 and common
multiples of two whole numbers
less than or equal to 10 using
strategies including a visual
model.

AzMERIT Performance Level Descriptors

For two whole numbers, finds the


greatest common factor less than
or equal to 50 and the least
common multiple less than or
equal to 10.

D-115

Standard Setting Technical Report


Proficient
Fluently adds, subtracts,
multiplies and divides multi-digit
numbers including multi-digit
decimals using the standard
algorithm for each operation.

Highly Proficient
Solves real world problems by
adding, subtracting, multiplying and
dividing multi-digit numbers
including multi-digit decimals using
the standard algorithm for each
operation and assesses the
reasonableness of the result.

For two whole numbers, finds the


greatest common factor less than
or equal to 100 and the least
common multiple less than or
equal to 12 and uses the
distributive property to express a
sum of two whole numbers from
1 to 100 with a common factor as
a multiple of a sum of two whole
numbers with no common factor.

Interprets a context to construct an


equivalent expression using the
greatest common factor, least
common multiple, and the
distributive property.

AzMERIT - 2015

Mathematics
Grade 6

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Plots integer pairs on a
Plots rational pairs on a
coordinate plane and on a
coordinate plane and on a
horizontal number line,
horizontal or vertical number line,
compares two numbers on a
determines the meaning of zero in
number line, finds the absolute context, compares two numbers
value of a rational number, and including absolute values, and
determines the distance
determines the distance between
between two points on the
two points with the same first or
Detailed 6.NS.C [5 to 9] coordinate plane by counting
second coordinate. Converts
spaces.
between expressions for positive
rational numbers including
fractions and decimals.
PLD

AzMERIT Performance Level Descriptors

D-116

Standard Setting Technical Report


Proficient
Highly Proficient
Uses positive and negative
Solves real world problems involving
numbers to represent quantities the coordinate plane and absolute
in real world contexts, recognizes values.
that when two ordered pairs
differ only by sign then the
locations are related to reflections
over one or both axes, and uses
absolute value to find the
distance between two points with
the same first or second
coordinate. Converts between
expressions for positive rational
numbers including fractions,
decimals, and percents.

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient

Detailed

Detailed

Detailed

Recognizes exponential notation


as repeated multiplication,
identifies an expression
matching a written statement
where variables represent
6.EE.A [1 to 4] numbers, evaluates an
expression at a specific value for
a variable, and identifies when
two simple expressions are
equivalent.
Uses substitution to determine
whether a given value for a
variable makes an equation or
inequality true using whole
numbers and recognizes that
6.EE.B [5 to 8] inequalities of the form x<c and
x>c have infinitely many
solutions and identifies them on
a number line.

6.EE.C [9]

Given a graph or table, identifies


an algebraic equation for two
quantities that change in
relationship to one another.

AzMERIT Performance Level Descriptors

Mathematics
Grade 6
Partially Proficient

Standard Setting Technical Report


Proficient

Expressions and Equations


Evaluates a single term involving Performs arithmetic operations
whole number exponents,
including whole number
recognizes one or more parts of
exponents when no parenthesis
an expression as a single entity,
or parentheses are present and
evaluates an expression at specific applies properties of operations
values for each variable, and
to identify and generate
applies properties of operations to equivalent expressions.
identify equivalent expressions.

Highly Proficient
Evaluates multi-step problems and
generates equivalent expression
involving rational numbers and
whole number exponents in real
world contexts.

Solves an equation or inequality


with a single operation using
substitution to determine whether
a given value in a set of values for
a variable makes an equation or
inequality true, and identifies
solutions to compound
inequalities on a number line.

Solves an equation or inequality


as a process to answer a question
and determines which value(s) in
a set of values for a variable
makes an equation or inequality
true, and uses inequalities to
show constraints in a real world
context.

Creates a set of values that make an


equation or inequality true, and
creates a real world situation that
corresponds to a given expression or
constraint.

Given a graph or table, identifies


the dependent and independent
variables and creates an algebraic
equation to represent how these
quantities change in relationship
to one another.

Given a real world context,


creates an equation to express
the relationship between the
dependent and independent
variables and creates graphs and
tables relating to the equation.

Creates a real world context using


dependent and independent
variables.

D-117

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient

Detailed

Finds the area given all the


measurements for triangles or
polygons decomposed into
rectangles and triangles, finds
the volume of a right
6.G.A [1 to 3] rectangular prism with whole
number edges, and creates
polygons in the coordinate
plane given coordinates for the
vertices.
Represents three-dimensional
figures using nets comprised of
rectangles and triangles.

Detailed

6.G.A [4]

Detailed

Recognizes a statistical question


from a list of questions,
identifies a graph given a data
set or vice versa, and recognizes
mean, median, and mode as a
measure of center and range as
6.SP.A [1 to 3] a measure of variation.

AzMERIT Performance Level Descriptors

Mathematics
Grade 6

Standard Setting Technical Report

Partially Proficient

Proficient

Highly Proficient

Geometry
Finds the area given some
measures for triangles or polygons
by decomposing into rectangles
and triangles, finds the volume of
a right rectangular prism with one
fractional edge, and uses
coordinates to find the length of a
side joining points with the same
first or second coordinate.

Solves a real world context by


finding the area given some
measures for triangles or polygons
by decomposing into rectangles
and triangles, finds the volume of
a right rectangular prism with
fractional edges, and using
coordinates for vertices of a
polygon.

Solves real world multi-step


geometric problems including
decimal and fractional
measurements, finds missing side
length of a right rectangular prism
given a volume and fractional side
lengths, and finds a missing vertex of
a polygon given other vertices.

Solves real world problems by


finding surface area for threedimensional figures using nets
with whole number edges.

Solves real world problems by finding


surface area for three-dimensional
figures using nets with fractional
edges.

Finds surface area for threedimensional figures using nets.

Statistics and Probability


Changes a question from being
Recognizes that a statistical
non-statistical to statistical,
question anticipates variability,
demonstrates that a set of data
demonstrates that a set of data
collected to answer a statistical
collected to answer a statistical
question has a distribution that
question can be described by its
can be described by its measure of measure of center and spread and
overall shape, and recognizes that
center and spread, and
determines mean, median, mode, a measure of center summarizes
and range.
all the values of a data set with a
single value.

D-118

Creates a statistical question given a


context, creates a data set with a
given measure of center and/or
spread and/or overall shape, and
determines how additional data
points impact the measure of center
and/or spread and/or overall shape.

AzMERIT - 2015

Mathematics
Grade 6

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Proficient
Identifies an appropriate display Constructs an appropriate display Summarizes numerical data sets
for numerical data including dot for numerical data including dot in relation to their context.
plots, histograms, and box plots, plots, histograms, and box plots,
and summarizes data from a line and summarizes data from a line
plot by counting the number of plot by counting the number of
Detailed 6.SP.A [4 to 5] observations, determining the observations, determining the
range, and/or a measure of
range, and/or a measure of
center, and identifying outliers or
center.
other striking deviations.
PLD

AzMERIT Performance Level Descriptors

D-119

Standard Setting Technical Report


Highly Proficient
Creates a histogram or box plot given
a dot plot and creates a data set
given a display.

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD

Standard

Minimally Proficient

Mathematics
Grade 7

Partially Proficient

The Minimally Proficient student The Partially Proficient student

Detailed

Detailed

Detailed

Standard Setting Technical Report

Proficient
The Proficient student

Ratio and Proportional Relationships


Computes unit rates with ratios Computes unit rates with ratios of Computes unit rates with ratios of
of two unit fractions having like one non-unit fraction and one unit two non-unit fractions having like or
7.RP.A [1]
or different units.
fraction having like or different
different units. Ratios include side
units.
lengths.
Decides whether two quantities Decides whether two quantities are Decides whether two quantities are
are in a proportional relationship in a proportional relationship and in a proportional relationship and
identifies the constant of
identifies the constant of
and identifies the constant of
proportionality (unit rate) in any
proportionality (unit rate) in any
proportionality (unit rate) in a
simple representation, i.e. tables,
complex representation, (i.e. tables,
representation that includes
equations, diagrams, verbal
equations, diagrams, verbal
(0,0).
descriptions, graphs.
descriptions, graphs).
Identifies the equation that
Models a proportional relationship Models a proportional relationship
models a relationship from a
7.RP.A
using an equation when given a
using an equation given a complex
given representation with a
[2a to 2d]
simple table, graph, or verbal
table, graph, or verbal description.
proportional relationship.
description.
Explains what any point (x,y) on the
Explains what any point (x,y) on
Explains what any point (x,y) on the graph of a proportional relationship
the graph of a proportional
relationship means in terms of graph of a proportional relationship means in terms of the situation and
the situation, but not identify
means in terms of the situation, and identify the unit rate.
the unit rate.
can identify the unit rate when
given the point (1,r).
Uses proportional relationships Uses proportional relationships to
to solve simple ratio and percent solve simple ratio and percent
7.RP.A [3]
problems.
problems in context.

Uses proportional relationships to


solve multistep ratio and percent
problems in context.

The Number System

AzMERIT Performance Level Descriptors

D-120

Highly Proficient
The Highly Proficient student
Computes unit rates with ratios of
two mixed numbers having like or
different units. Ratios include areas.
Extends the given representation or
creates a different representation
that would represent the same
proportional relationship.
Creates a representation with a
context that would represent a given
proportional equation.
Identifies a point (x,y) on the same
graph as the point (1,r) for a
proportional relationship and
interprets the meaning of (x,y) in
terms of the situation.

Creates equivalent proportional


equations that could be used to
solve the same ratio/percent
problem in context.

AzMERIT - 2015
PLD

Mathematics
Grade 7

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Adds, subtracts, multiplies and Adds, subtracts, multiplies and
divides rational numbers using a divides simple rational numbers.
Recognizes that the sum of a
number line or other
number and its opposite equals
manipulatives.
zero.

Detailed

7.NS.A
[1a to 2d]

Detailed

Solves simple real-world and


mathematical problems
7.NS.A [3] involving the four operations
with rational numbers using the
number line or other
manipulatives

AzMERIT Performance Level Descriptors

Solves simple real-world and


mathematical problems involving
the four operations with rational
numbers.

D-121

Standard Setting Technical Report


Proficient
Adds, subtracts, multiplies, and
divides rational numbers and
determines the reasonableness of
the solution.
Understands p + q as the number
located a distance |q| from p in a
positive or negative direction, and
understand subtraction as adding
the additive inverse.
Understands that -(q/p)
= (-p)/q= p/(-q). Converts a rational
number to a decimal using long
division and knows that the rational
number terminates in 0 or
eventually repeats. Knows that
division by zero is undefined.

Highly Proficient
Interprets the sums of rational
numbers in real-world contexts.
Justifies the steps taken to add or
subtract rational numbers. Interprets
products and quotients of rational
numbers in a real-world context.

Solves real-world and multistep


mathematical problems involving
the four operations with rational
numbers.

Creates a story problem to model a


given number sentence based on a
real-world context and uses this to
solve problems.

AzMERIT - 2015

Mathematics
Grade 7

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Proficient
Expressions and Equations
Applies properties of operations Applies properties of operations as Applies properties of operations as
as strategies to add, subtract,
strategies to add, subtract, factor, strategies to add, subtract, factor,
factor, and expand linear
and expand linear expressions
and expand linear expressions (with
expressions (with whole number (with integer coefficients).
simple rational coefficients).
7.EE.A [1 coefficients).
Recognizes
and
explains
the
Understands that rewriting an
Detailed
to 2]
Recognizes and explains the
meaning of an expression in
expression in different forms in a
meaning of an expression in
context (with rational coefficients). problem context can shed light on
context (with integer
the problem and how the quantities
coefficients).
in it are related.
PLD

Detailed

Detailed

Detailed

Detailed

Solves equations of the form px Solves real-world or mathematical


+ q = r and p(x + q) = r with
problems of the form px + q = r,
7.EE.B [3 (rational coefficients).
p(x + q) = r, px + q > r, and px + q <
to 4b]
r with rational coefficients.

Finds actual lengths given a


geometric figure and a scale
7.G.A [1] factor.

Geometry
Finds actual lengths given two
geometric figures with some
unknown side measure when given
the scale factor that relates the two
figures.

Identifies geometric shapes


Constructs geometric shapes given
given conditions on the sides or a combination of angle and side
7.G.A [2] angles.
conditions and determines
whether it makes a particular
shape.
Identifies the 2-dimensional
Identifies the 2-dimensional figure
figure that results from a vertical that results from a vertical or
7.G.A [3] or horizontal cut of a right
horizontal cut of right rectangular
rectangular prism.
pyramids.

AzMERIT Performance Level Descriptors

D-122

Highly Proficient
Applies properties of operations as
strategies to add, subtract, factor,
and expand linear expressions (with
complex rational coefficients).
Creates equivalent expressions given
a problem context and explains key
terms and factors of the problem for
each expression.

Creates a model and solves realworld or mathematical problems of


the form px + q = r, p(x + q) = r, px +
q > r, and px + q < r with rational
coefficients.

Creates a model and solves realworld or mathematical problems


using equations and inequalities
with rational coefficients and
explains what the solution means.

Computes actual lengths and areas


from a scale drawing, creates a scale
drawing based on a context, and
reproduces a scale drawing using a
different scale.

Explains the relationship between


scale factors of length and scale
factors of areas for geometric figures
and reproduce a scale drawing using
a different scale.

Notices when conditions determine Justifies the conditions for a unique


a unique triangle, more than one
triangle, more than one triangle or
triangle, or no triangle.
no triangle.
Describes the 2-dimensional figure
that results from a vertical,
horizontal, or angled slice of a right
rectangular prism.

Draws the 2-dimensional figure that


results from a vertical, horizontal or
angled slice of a right prism or
pyramid.

AzMERIT - 2015

Mathematics
Grade 7

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Recognizes the formulas for
Calculates area and circumference
area and circumference of a
given radius or diameter. Calculates
circle.
radius or diameter given the
circumference.
Detailed 7.G.B [4]
PLD

Detailed

Detailed

Standard Setting Technical Report


Proficient
Determines the area given the
circumference or vice versa. Solves
real-world problems involving area
and circumference. Gives an
informal derivation of the
relationship between circumference
and area of a circle.

Highly Proficient
Understands how and why the
formulas for area and circumference
of a circle work. Explains the
relationship between area of a circle
and area of a parallelogram.

Creates and solves multi-step


equations to find unknown angle
measures given a figure with
intersecting lines.

Identifies supplementary,
complementary, vertical and
7.G.B [5] adjacent angles.

Finds the unknown angle given


another angle and their
relationship.

Finds any of the unknown angles


formed by two intersecting lines
when measures are given algebraic
expressions.

Finds the area of triangles,


quadrilaterals and regular
7.G.B [6] polygons. Finds the volume of
cubes and right prisms.

Solves real-world problems


involving surface area of 2dimensional figures. Solve realworld volume problems for cubes
and right prisms.

Solves real-world problems


Uses relationships between volume
involving surface area of composite and surface area of 3 dimensional
2- dimensional figures. Solves real- shapes to solve real-world problems.
word problems involving volume of
3- dimensional objects.

AzMERIT Performance Level Descriptors

D-123

Mathematics
Grade 7

AzMERIT - 2015

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Proficient
Statistics and Probability
Identifies and recognizes sample Recognizes that a random sample Makes inferences about a
populations given a scenario
produces the most valid
population based on representative
describing
the
entire
representation
of
the
entire
samples. Uses multiple samples to
7.SP.A [1
Detailed
population.
population.
gauge variations in estimates or
to 2]
predictions.
PLD

Detailed

Detailed

Detailed

Identify basic measures of


central tendency to compare
7.SP.B [3 two different populations.
to 4]

Uses measures of central tendency Uses measures of central tendency


and variability to make comparative
to draw comparisons about two
inferences about two populations in
different populations.
any context.

Understands that the probability Understands that if the probability


of a chance event is a number
of a chance event is closer to 1, it is
between 0 and 1.
likely to happen and if it is closer to
7.SP.C [5]
0, it is not likely to happen.

7.SP.C [6]

Makes approximations of
probability for a chance event.

Determines the theoretical


probability of a simple event.
Detailed

7.SP.C [7a
to 7b]

Detailed

Determines the sample space


7.SP.C [8a
for compound events.
to 8c]

AzMERIT Performance Level Descriptors

Identifies the probability of a


chance event as impossible (0),
unlikely, equally likely or unlikely
(.5), more likely, or certain (1).
Represents the probability as a
fraction, decimal, or percent.

Highly Proficient
Identifies and justifies the most
representative sampling method for
a situation. Chooses or creates a
method of generating multiple
samples to gauge variations in
estimates or predictions.
Compares two visual representations
of data to make comparative
inferences about the central
tendency and variability of two
populations in context.
Compares probabilities of two or
more events and justify the
likelihood of each event.

Uses the results of an experiment


to make approximations of
probability for an event.

Compares the relative frequency of Recognizes and justifies why the


an event to the theoretical
experimental probability approaches
probability of the event.
the theoretical probability as the
relative frequency of an event
increases.
Determines the theoretical
Determines the theoretical
Compares and justifies the
probability of a simple event and
probability of an event and uses
experimental and theoretical
uses observed frequencies to
observed frequencies to create a
probability in a given situation.
create a uniform probability model. probability model for the data from
a chance process (where outcomes
are uniform or not uniform).
Determines the theoretical
probability of a compound event.

D-124

Designs a simulation to generate


frequencies for compound events.

Compares different simulations to


see which best predicts the
probability.

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD

Standard

Minimally Proficient
The Minimally Proficient student

Detailed

Identifies square roots of


nonsquare numbers and pi as
irrational numbers. Understands
that every number has a decimal
8.NS.A [1 to 2] expansion. Identifies rational or
irrational numbers and converts
familiar rational numbers with one
repeating digit to fraction form.

Knows the properties of natural


number exponents. Evaluates
square roots of small perfect
squares.
Detailed

Detailed

8.EE.A [1 to 2]

8.EE.A [3 to 4]

Uses numbers expressed in the


form of a single digit times an
integer power of 10. Represents
very large and very small
quantities in scientific notation.

AzMERIT Performance Level Descriptors

Mathematics
Grade 8
Partially Proficient

Standard Setting Technical Report

Proficient

Highly Proficient

The Partially Proficient student

The Proficient student

The Highly Proficient student

The Number System


Compares and orders rational and
irrational numbers. Identifies
irrational decimal expansions as
approximations. Identifies rational
and irrational numbers and
converts less familiar rational
numbers to fraction form.

Places irrational numbers on a


number line. Uses
approximations of irrational
numbers to estimate the value
of an expression. Converts
decimals into rational numbers.

Explains how to get more precise


approximations of square roots.
Notices and explains the patterns
that exist when writing rational
numbers as fractions.

Expressions and Equations


Applies the properties of natural Knows and applies the
number exponents to generate
properties of integer exponents
equivalent numerical expressions. to generate equivalent
Solves mathematical equations
numerical expressions. Uses
without context of the form
square root and cube root
x ^2=p and x ^3=p , where p is a symbols to represent solutions
positive rational number.
to equations of the form x ^2=p
and x ^3=p , where p is a
positive rational number.

Uses numbers expressed in the


form of a single digit times an
integer power of 10 to estimate
very large or very small quantities.
Multiplies and divides numbers in
scientific notation.

D-125

Expresses how many times as


much a number written as an
integer power of 10 is than
another number. Performs
operations with numbers
expressed in scientific notation,
including problems where both
decimal and scientific notation
are used.

Uses properties of integer


exponents to order or evaluate
multiple numerical expressions with
integer exponents. Explains how
square roots and cube roots relate
to each other and to their
radicands.

Converts between decimal notation


and scientific notation and
compares numbers written in
different forms. Calculates and
interprets values written in
scientific notation within a context.

Mathematics
Grade 8

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Graphs proportional relationships, Compares two different
interpreting the unit rate as the
proportional relationships using
slope. Determines the slope of a the same representation. Derives
line given a graph.
the equation y=mx for a line
through the origin.
Detailed 8.EE.B [5 to 6]
PLD

Solves simple linear equations


with integer coefficients. Identifies
systems of equations that have
one, infinite, or no solutions from
graph. Estimates the solution of a
system given a graph.
Detailed

8.EE.C [7 to
8c]

AzMERIT Performance Level Descriptors

Solves multistep linear equations


with rational coefficients and
identifies equations that have one
solution, infinitely many solutions,
or no solutions. Solves a system of
linear equations using any
method.

D-126

Standard Setting Technical Report


Proficient
Compares two different
proportional relationships
represented in different ways.
Recognizes and explains why the
slope m is the same between
any two distinct points on a nonvertical line. Derives the
equation y=mx+b for a line that
does not pass through the origin.

Highly Proficient
Generates a representation of a
proportional relationship with
specific qualities. Compares and
contrasts situations in which similar
triangles would and would not yield
lines with the same slope.

Solves multistep linear equations


with rational coefficients and
variables on both sides and
provides examples of equations
that have one solution, infinitely
many solutions, or no solutions.
Provides examples of systems of
equations that have a specified
number of solutions. Creates
and utilizes a system of linear
equations to solve a real-world
problem.

Justifies why an equation has one


solution, infinitely many solutions,
or no solution. Solves real-world
and mathematical problems leading
to two linear equations in two
variables.

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient

Detailed

Detailed

8.F.A [1 to 3]

8.F.B [4 to 5]

Mathematics
Grade 8

Partially Proficient
Functions
Identifies whether a relation is a Identifies whether a relation is a
function from a graph or a
function from any representation.
mapping. Creates a graph from a Given a representation of a
function expressed as an equation. function, creates another
Determines whether a function is representation of that function.
linear or nonlinear from a graph. Determines whether a function is
linear or nonlinear from an
equation.

Determines the rate of change of


the function from a graphical
description of the linear function.
Describes qualitatively the
functional relationship between
two quantities by analyzing some
features of a graph (e.g., linear
and nonlinear).

Determines the rate of change and


initial value of the function from
two (x,y) values. Creates a graph
of identified information.
Describes qualitatively the
functional relationship between
two quantities by analyzing a
graph (e.g., where the function is
increasing or decreasing).

Geometry

AzMERIT Performance Level Descriptors

D-127

Standard Setting Technical Report


Proficient

Highly Proficient

Explains that a function is a rule


that assigns to each input
exactly one output and that the
graph of a function is the set of
ordered pairs consisting of an
input and the corresponding
output. Compares properties of
two functions each represented
in a different way. Determines
whether or not a function is
linear or nonlinear from any
representation. Gives examples
of functions that are not linear.

Creates any representation of a


relation and explain why it is a
function or not a function. Justifies
whether two functions represented
in different ways are equivalent or
not by comparing their properties.
Explains why the function is linear
or nonlinear.

Interprets the rate of change


and initial value of a linear
function in terms of the situation
it models or its graph/table of
values. Constructs a function to
model a linear relationship
between two quantities.
Sketches a graph that exhibits
given qualitative features of a
function.

Identifies what prevents a set of


values in either a table or graph
from being linear and adjusts the
values to make them linear.
Interprets qualitative features of a
function in a context.

Mathematics
Grade 8

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Identifies visual representations
Identifies the angles that
and congruent figures that result correspond after a transformation.
after one transformation.
Identifies a transformation
Recognizes that it takes a
between two congruent figures.
combination of transformations
Describes the effect of reflections
and dilations to produce a similar and translations on twoDetailed 8.G.A [1a to 4]
figure.
dimensional figures using
coordinates and coordinate
notation. Identifies dilations of
figures by a given scale factor and
transformations.
PLD

Detailed

Detailed

8.G.A [5]

Knows that the sum of angles of a


triangle equals 180 degrees, and
identifies angle pairs when parallel
lines are cut by a transversal.

Knows the Pythagorean Theorem


and that it applies to right
triangles. Calculates unknown
hypotenuse side length given the
Pythagorean Theorem. Applies the
Pythagorean Theorem to find the
distance between two points in a
8.G.B [6 to 8] coordinate system with the right
triangle drawn where the
Pythagorean Theorem is given.

AzMERIT Performance Level Descriptors

Standard Setting Technical Report


Proficient
Verifies experimentally the
properties of rotations,
reflections and translations.
Describes the effect of
transformations on twodimensional figures using
coordinates and coordinate
notation, including whether the
transformations lead to similar
or congruent figures.

Highly Proficient
Recognizes and explains the
properties of transformations in
real-world graphic illustrations and
visual representations, including
whether the transformations lead
to similar or congruent figures.

Finds unknown angle measures in


a triangle, and unknown angle
measures for angle pairs when
parallel lines are cut by a
transversal.

Gives an informal argument for


the sum of angles of a triangle,
the measure of an exterior angle
of a triangle, and congruent
angle relationships when parallel
lines are cut by a transversal.

Gives an informal argument that a


triangle can only have one 90
degree angle. Gives an informal
argument for the pairs of angles
that are supplementary when
parallel lines are cut by a
transversal.

Understands the proof of the


Pythagorean
Theorem and its converse.
Calculates unknown side lengths
using the Pythagorean Theorem
given at least two different side
lengths of a right triangle. Applies
the Pythagorean Theorem to find
the distance between two points
in a coordinate system with the
right triangle drawn where the
Pythagorean Theorem is not given.

Understands and explains the


proof of the Pythagorean
Theorem and its converse.
Applies the Pythagorean
Theorem to a real-world
situations in two and three
dimensions to determine
unknown side lengths. Applies
the Pythagorean Theorem to
find the distance between two
points in a coordinate system.

Models a proof of the Pythagorean


Theorem and its converse using a
pictorial representation. Recognizes
situations and applies the
Pythagorean Theorem in multi- step
problems. Finds the coordinates of
a point which is a given distance
(non-vertical and non- horizontal)
from another point.

D-128

AzMERIT - 2015
PLD

Mathematics
Grade 8

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Finds the volume of a cylinder.
Finds the volume of a cone,
cylinder or sphere.

Detailed

8.G.C [9]

AzMERIT Performance Level Descriptors

D-129

Standard Setting Technical Report


Proficient
Knows the formulas for the
volumes of cones, cylinders, and
spheres and use them to solve
real-world mathematical
problems.

Highly Proficient
Describes the relationship between
the formulas for volumes of cones,
cylinders, or spheres. Explains the
derivation of the formulas for
cones, cylinders, and spheres.

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient

Detailed

Mathematics
Grade 8

Partially Proficient
Statistics and Probability
Constructs a scatter plot.
Constructs a scatter plot and
Recognizes a straight line can be describes the pattern as positive,
used to describe a linear
negative or no relationship. Draws
association on a scatter plot.
a straight line on a scatter plot
Identifies the slope and ythat closely fits the data points.
intercept of a linear model on a
Identifies possible data points
scatter plot. Completes a partially given a linear model. Constructs a
8.SP.A [1 to 4]
filled-in two-way table and
two-way table of categorical data.
interpret the table by row or
column.

AzMERIT Performance Level Descriptors

D-130

Standard Setting Technical Report


Proficient

Highly Proficient

Describes patterns in a scatter


plot. Judges how well the trend
line fits the data by looking at
the closeness of the data points.
Interprets the meaning of the
slope and y-intercept in context.
Interprets and describes relative
frequencies for possible
associations from a two-way
table.

Constructs and interprets scatter


plots to investigate patterns of
association between two quantities.
Compares more than one trend line
for the same scatter plot and
justifies which one best fits the
data. Creates and uses a linear
model based on a set of bivariate
data to solve a real-world problem.
Interprets and compares relative
frequencies to identify patterns of
association.

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD

Standard

Minimally Proficient

Mathematics
Algebra I
Partially Proficient

The Minimally Proficient student The Partially Proficient student

Detailed

Detailed

Detailed

Detailed

Detailed

N-RN.B [3]

The Proficient student

Highly Proficient
The Highly Proficient student

Interprets complicated expressions Explains the context of different


by viewing one or more of their
parts of a formula presented as a
parts as a single entity.
complicated expression.

Can identify different forms for


the same expression.

Justifies the different forms based Recognizes equivalent forms of


on mathematical properties.
numerical and polynomial
expressions in one variable and
uses the structure of the expression
to identify ways to rewrite it.

Rewrites numerical and


polynomial expressions to
equivalent forms, using the
structure of the expression.
Interprets different symbolic
notation. Makes generalizations
by rewriting expressions in
context, using their structure.

Identifies the zeroes of a


quadratic expression written in
factored form.

Factors a quadratic expression


without a leading coefficient.

Explains conditions for two, one,


and no real roots.

A-SSE.A [2]

A-SSE.B [3b]

Proficient

Number and Quantity


Explains why adding and
Explains why adding a rational
Explains why multiplying a nonzero Generalizes and develops rules for
multiplying two rational numbers number to an irrational number
number to an irrational number
sum and product properties of
results in an irrational number
results in an irrational number.
rational and irrational numbers.
results in a rational number

Algebra
Identifies some of the basic
Identifies all of the basic terms
terms (base, exponent,
(base, exponent, coefficient, and
coefficient, and factor) of a linear factor) of linear and exponential
A-SSE.A [1a to
or exponential expression.
expressions.
1b]

A-SSE.B [3a]

Standard Setting Technical Report

Factors a quadratic expression to


reveal the zeroes of the function it
defines.

Identifies the maximum or


Identifies the maximum or
Completes the square in a
Completes the square in a
minimum of a function, using the minimum of a function when given quadratic expression to reveal the quadratic expression (where b is
in vertex form.
maximum or minimum value of the not divisible by two).
graph.
function it defines.

AzMERIT Performance Level Descriptors

D-131

Mathematics
Algebra I

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Knows the properties of
Applies the properties of
exponents
exponents.

Detailed

A-SSE.B [3c]

Identifies polynomial
expressions.
Detailed

Detailed

Detailed

Detailed

Adds, subtracts, and multiplies


polynomials.

A-APR.A [1]

A-APR.B [3]

Standard Setting Technical Report


Proficient
Uses the properties of exponents to
transform expressions for
exponential functions with integer
exponents modeling a real-world
context.

Highly Proficient
Interprets properties of
exponential functions by
transforming them into equivalent
expressions that reveal properties
within a context.

Understands that polynomials are


closed under the operations of
addition, subtraction, and
multiplication.

Creates equivalent polynomial


expressions using the fact that
polynomials are closed under the
four operations.

Identifies the zeros of a quadratic Use zeros to sketch the graph of a


function from a graph.
quadratic function given in
factored form.

Factor a quadratic function and use Identify zeros from the graph and
zeroes to sketch a graph of the
use zeroes to construct the
function.
quadratic function.

Distinguishes between linear


Solves linear equations and
equations, inequalities, and non- inequalities in one variable with
linear equations.
constant coefficients.

Creates and solves linear equations


and inequalities in one variable,
including equations with
coefficients represented by letters
to solve problems with a real world
context. Rearranges formulas to
highlight a quantity of interest,
using the same reasoning as in
solving equations.

Creates, rearranges, and solves


exponential equations with
integer exponents or quadratic
equations.

Constructs equations and graphs


that model linear and exponential
relationships (with context). Graphs
solutions of the system of
inequalities and identifies the
solution set as a region of the
coordinate plane that satisfies both
inequalities.

Compares and contrasts equations


and graphs that model linear and
exponential relationships. Writes
or creates a system of linear
inequalities given a context or
graph and identifies the solution
set as a region of the coordinate
plane that satisfies all inequalities.

A-CED.A [1
and 4];
AREI.B [3]

Writes and graphs an equation to


represent a linear relationship.
Identifies a solution region
when the graph of a linear
A-CED.A [2]; inequality is given.
A-REI.D [12]

AzMERIT Performance Level Descriptors

Writes and graphs an equation to


represent an exponential
relationship. Graphs the solutions
to a linear inequality in two
variables as a half-plane.

D-132

Mathematics
Algebra I

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Determines whether a point is a Interprets solutions as viable or
solution to a system of equations non-viable options in a modeling
and/or inequalities given a graph context where constraints are
Detailed A-CED.A [3]
presented verbally.
or equations.
PLD

Detailed

Detailed

A-REI.A [1]

Standard Setting Technical Report


Proficient
Represents constraints by
equations or inequalities, and by
systems of equations and/or
inequalities.

Highly Proficient
Defends and justifies solutions
or non-solutions in a modeling
context.

Solves a quadratic equation with Describes the steps in solving


multiple steps, without justifying quadratic equations.
the steps involved in solving.

Explains and justifies the steps


in solving linear equations by
applying the properties of equality,
inverse, and identity.

Explains and justifies the steps


in solving linear and quadratic
equations by applying and naming
the properties of equality, inverse,
and identity.

Solves quadratic equations with


real solutions by simple
inspection.

Solves quadratic equations with


real solutions by inspection (e.g.,
for x^2 = 49 )-- taking square
roots, completing the square, the
quadratic formula, and factoring-as appropriate to the initial form of
the equation.

Determines the most efficient


method for solving a quadratic
equation and justifies the choice
selected. Recognize cases in which
a quadratic equation has no real
solutions.

Solves quadratic equations by


factoring.

A-REI.B. [4a
to 4b]

AzMERIT Performance Level Descriptors

D-133

AzMERIT - 2015
PLD

Mathematics
Algebra I

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Explains the use of the
Explains why the sum of two
multiplication property of
equations is justifiable in the
equality to solve a system of
solving of a system of equations.
equations. Solves a system of
Tests a solution to the system in
linear equations approximately both original equations (both
when given a graph of the
graphically and algebraically).
system.

Detailed

Detailed

A-REI.C [5 to
6]

Identifies solutions and nonsolutions of linear equations in


two variables. Finds the point
where two lines or exponential
curves intersect on a graph or
approximates solutions using
A-REI.D [10 to
other methods such as a table or
11]
technology.

AzMERIT Performance Level Descriptors

Identifies solutions and nonsolutions of exponential equations


in two variables. Finds and explains
why the solution to a system
linear, polynomial, rational, or
absolute value equations is the
point where the two intersect.

D-134

Standard Setting Technical Report


Proficient
Relates the process of linear
combinations with the process of
substitution for solving a system of
linear equations. Solves a system of
linear equations exactly and
approximately by choosing the best
method depending on the
representation of the equations

Highly Proficient
Proves that, given a system of two
equations in two variables,
replacing one equation by the sum
of that equation and a multiple of
the other produces a system with
the same solutions. Analyzes the
system of
equations and is able to solve
exactly and approximately given a
context or real-world situation.
Solves a system of equations and
manipulates one of the equations
to provide additional information
or an additional given solution.

Graphs points that satisfy linear and


exponential equations. Models the
solutions of a system of linear
equations and/or exponential
equations showing the solutions
using technology, tables, graphs,
approximations. Finding the
solutions approximately is limited
to cases where f(x) and g(x) are
polynomial functions.

Describes viable solutions using


the knowledge that continuous
lines and curves contain an infinite
number of solutions. Explains why
there are infinitely many solutions
when
f(x) = g(x) .

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient

Detailed

Detailed

Detailed

Identifies functions and their


domains

Partially Proficient
Functions
Evaluates a function for inputs in
the domain, and writes functions
using function notation (without
context).

Identifies the key features (as


listed in the Standard) when
given a linear, quadratic, square
root, cube root, piecewisedefined functions (including step
functions and absolute value
functions), and exponential
functions (with domains in the
integers).

Interprets the key features (as


listed in the Standard) when given
a graph of a linear, quadratic,
square root, cube root, piecewisedefined functions (including step
functions and absolute value
functions), and exponential
functions (with domains in the
integers).

F-IF.A [1 to 2];
F-IF.B [5]

F-IF.B [4]

F-IF.B [6]

Standard Setting Technical Report


Proficient

Highly Proficient

Uses function notation and


evaluates functions for inputs in
their domain, and interprets
statements that use function
notation in terms of context.

Applies and extends knowledge of


domain and range to real world
situations and contexts; creates a
function for a given context where
the domain meets given
parameters.

Identifies and interprets the key


features (as listed in the Standard)
when given a table of values.
Sketches graphs of linear,
quadratic, square root, cube root,
piecewise-defined functions
(including step functions and
absolute value functions), and
exponential functions (with
domains in the integers) showing
key features, when given a verbal
description of the relationship.

Accurately creates a story or


context that models the given key
features of linear, quadratic,
square root, cube root, piecewisedefined functions (including step
functions and absolute value
functions), and exponential
functions (with domains in the
integers).

Determines the rate of change of Determines the rate of change of Calculates and interprets the
Describes the different rates of
a linear function presented
an exponential function presented average rate of change of a
change over given intervals of the
algebraically.
algebraically, over a given interval. function presented symbolically or graph .
as a table over a specified interval.
Evaluates linear, quadratic,
piecewise, step, and absolute
value functions

Detailed

Mathematics
Algebra I

Identifies key features of linear,


quadratic, piecewise, step, and
absolute value functions when the
graph is given.

F-IF.C [7a to
7b, and 8a]

AzMERIT Performance Level Descriptors

D-135

Graphs linear, quadratic, piecewise,


step, and absolute value functions,
showing intercepts, maxima, and
minima. Can graph functions
expressed symbolically and can
show key features of the graph (by
hand in simple cases, and using
technology for more complicated
cases).

Graphs and compares linear,


quadratic, piecewise, step, and
absolute value functions in various
forms.

Mathematics
Algebra I

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Compares slopes and y Compares growth rates and
intercepts of two linear functions intercepts of two functions where
where one is presented
one is presented graphically and
graphically and the other is
the other is presented in function
presented in slope-intercept
notation.
form.
Detailed F-IF.C [9]
PLD

Detailed

Detailed

F-BF.A [1];
F-IF.A [3];
F-LE.A [2]

F-BF.B [3]

Standard Setting Technical Report


Proficient
Uses tables, graphs, algebra, and
verbal descriptions to compare
properties of two functions (linear,
quadratic, square root, cube root,
piecewise-defined functions
(including step functions and
absolute value functions), and
exponential functions with domains
in the integers), when each is
presented a different way.

Highly Proficient
Constructs a linear, quadratic,
square root, cube root, piecewisedefined functions (including step
functions and absolute value
functions), and exponential
functions with domains in the
integers that has a characteristic
(i.e. slope, intercept, maximum)
that is greater than or lesser than
a given function.

Identifies the parts of a recursive Defines and expresses a recursive


function or sequence.
sequence as a function, constructs
a linear function (not multi-step)
given a graph, a description of a
relationship, or two input-output
pairs.

Recognizes that sequences are


Applies sequences, sometimes
functions with a domain that is a
expressed as recursive functions,
subset of the integers, can generate to real world contexts.
a recursive function to express a
sequence and generate a sequence
given a recursive function,
constructs an exponential function
(not multi-step) given a graph, a
description of a relationship, or two
input-output pairs.

Relates the vertical translation of Performs vertical translations on


linear, quadratic, square root, cube
a linear function to its yintercept.
root, piecewise-defined functions
(including step functions and
absolute value functions), and
exponential functions with
domains in the integers.

Performs vertical translations on


graphs. Describes what will happen
to a function when f(x) is replaced
by f(x) + k, k f(x), f(kx), and
f(x+k) for different values of k.

AzMERIT Performance Level Descriptors

D-136

Finds the value of k given f(x)


replaced by f(x) + k, k f(x), f(kx),
and f(x+k) on a graph of linear,
quadratic, square root, cube root,
piecewise-defined functions
(including step functions and
absolute value functions), and
exponential functions with
domains in the integers.

Mathematics
Algebra I

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Recognizes situations in which
Recognizes relationships in tables
one quantity changes at a
and graphs that can be modeled
constant rate per unit interval
with linear functions (constant rate
relative to another.
of change) and with exponential
F-LE.A [1a to
functions (multiplicative rate of
Detailed
1c]
change)
PLD

Detailed

Detailed

F-LE.A [3]

Standard Setting Technical Report


Proficient
Justifies that linear functions grow
by equal differences over equal
intervals; exponential functions
grow by equal factors over equal
intervals. (ex- percent change)

Highly Proficient
Describes the rate of change per
unit as constant or the growth
factor as a constant percentage.
Proves that linear functions grow
by equal differences over equal
intervals; exponential functions
grow by equal factors over equal
intervals.

Compares the values of linear


and exponential functions at
specific points.

Compares the values of linear and Observes, using graphs and tables,
exponential functions over various that a quantity increasing
intervals.
exponentially eventually exceeds a
quantity that is increasing linearly
or quadratically.

Observes, explores, predicts,


models, and evaluates different
situations in which linear and
exponential functions are
compared.

Identifies which values are


constant from a given context.

Interprets the slope and x-and yintercepts in a linear function in


terms of a context.

Interprets the base value and


initial value in an exponential
function of the form f(x) = a*b^x,
where b is an integer, and a can be
any positive integer including 1, in
terms of context.

F-LE.B [5]

AzMERIT Performance Level Descriptors

D-137

Interprets the base value and


vertical shifts in an exponential
function of the form f(x) = b^x+ k ,
where b is an integer and k can
equal zero, in terms of context.

AzMERIT - 2015
PLD

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Identifies dot plots, histograms,
and box plots for a given set of
data.

Detailed

Detailed

Detailed

Mathematics
Algebra I

Partially Proficient
Statistics
Graphs numerical data on a real
number line using dot plots,
histograms, and box plots.

S-ID.A [1]

Proficient

Highly Proficient

Describes and gives a simple


interpretation of a graphical
representation of data on dot plots,
histograms, and box plots.

Determines and justifies which


type of data plot on a real number
line would be most appropriate
for a set of data. Identify
advantages and disadvantages of
different types of data plots.

Explains and interprets similarities


and differences using specific
measures of center and spread,
given two sets of data or two
graphs with possible effects from
existing outliers.

Plots data based on situations


with multiple data sets, and then
compares and discusses using
measures of center and spread
and explores the manipulation of
additional data points.. Justifies
which measure(s) are most
appropriate for comparison.
Identifies advantages and
disadvantages of using each
measure of center and spread.

Describes informally the center


and spread of a single set of data
or graph. Identifies shape,
center, and spread of a data set.

Compares informally the


similarities or differences in shape,
center, or spread between two
graphs. Identifies and states the
effects of existing outliers.

Explains data in a two-way


frequency table.

Creates a two-way frequency


Finds and interprets joint,
table showing the relationship
marginal and conditional relative
between two categorical variables. frequencies. Recognizes possible
associations and trends in the data.

S-ID.A [2 to 3]

S-ID.B [5]

Creates a scatter plot of bivariate Determines if a plotted data set


data.
is approximately linear.
Detailed

Standard Setting Technical Report

S-ID.B [6a to
6c]

AzMERIT Performance Level Descriptors

D-138

Creates a scatter plot of


bivariate data and estimates a
linear function that fits the
data. Uses this function to solve
problems in the context of the data.

Given a context, interprets,


identifies, and describes
associations and trends using a
two-way frequency table.
Compares the fit of different
functions, including exponential
functions with domains in the
integers, to data and determines
which function has the best fit.

Mathematics
Algebra I

AzMERIT - 2015

Standard Setting Technical Report

Appendix D. Performance Level Descriptors (PLDs)


Standard
Minimally Proficient
Partially Proficient
Proficient
Highly Proficient
Identifies a linear model of
Graphs data in a scatter plot,
Using a line fitted to data,
Using a function that best fits the
bivariate data.
identify the slope and y- intercept interprets the slope (rate of
data, interpolates and
of a linear model.
change) and the intercept (constant extrapolates trends in the data.
Detailed S-ID.C [7]
term) of a linear model in the
context of the situation and data.
PLD

Uses a table or graph of a set of


data to informally describe a
correlation. Defines causation
and correlation.
Detailed

S-ID.C [8 to 9]

AzMERIT Performance Level Descriptors

Identifies the existence of or nonexistence of causation in the


context of a correlated problem.
Computes the correlation
coefficient of a set of linearlyrelated data using technology.

D-139

Interprets the correlation


coefficient of a linear fit in the
context of a situation using
technology. Determines whether
the correlation shows a weak
positive, strong positive, weak
negative, strong negative, or no
correlation. Distinguishes between
causation and correlation in the
context of a situation with data.

Supports or refutes a
hypothesized correlation between
two sets of data. Supports or
refutes claims of
causation with the understanding
that a strong correlation does not
imply causation.

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD

Standard

Minimally Proficient

Mathematics
Geometry
Partially Proficient

The Minimally Proficient student The Partially Proficient student

Detailed

Detailed

Detailed

G-CO.A [1]

G-CO.A [2
and 4]

G-CO.A [3]

Identifies an angle, circle,


perpendicular line, parallel line,
and line segment using proper
notation.

Congruence
Informally defines an angle,
circle, perpendicular line, parallel
line, and line segment using
examples and non- examples.

Standard Setting Technical Report

Proficient

Highly Proficient

The Proficient student

The Highly Proficient student

Can explain definitions of an


angle, circle, perpendicular line,
parallel line, and line segment
based on the notions of point,
line, distance along a line, and
distance around a circular arc.

Identifies real-life examples of an


angle, circle, perpendicular line,
parallel line, and line segment using
precise definitions.

Describes reflections, rotations,


and translations. Identifies
rotations, reflections, and
translations given an image and
its transformation.

Describes dilations. Informally


describes rotations, reflections,
and translations using examples
and non- examples.

Compares transformations in the


plane and understands them as
functions that take points in the
plane as inputs and give other
points as outputs. Develops
definitions of rotations,
reflections, and translations using
the terms angles, circles,
perpendicular lines, parallel lines,
and line segments.

Represents functions to describe


transformations using a variety of
media. Justifies statements about
rotations, reflections, and
translations on the coordinate plane.

Distinguishes between rotations


and reflections given a rectangle,
parallelogram, trapezoid, or
regular polygon and its
transformation.

Identifies lines and points of


symmetry given a rectangle,
parallelogram, trapezoid, or
regular polygon and its reflection
or rotation.

Describes the rotations and


reflections that carry a given
rectangle, parallelogram,
trapezoid, or regular polygon
onto itself.

Identifies a rectangle, parallelogram,


trapezoid, or regular polygon that
satisfies a description of rotational
symmetry or lines of symmetry.

AzMERIT Performance Level Descriptors

D-140

Mathematics
Geometry

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


PLD
Standard
Minimally Proficient
Partially Proficient

Detailed

Detailed

Detailed

Detailed

Detailed

G-CO.B [7]

G-CO.B [8]

G-CO.C [9]

Proficient

Highly Proficient

Performs rotations, reflections,


and translations on a given
figure.

Identifies a sequence of
Performs rotations, reflections,
Explains how the order of a sequence
transformations that will carry a and translations using a variety of of transformations is performed may
given figure onto another.
methods and specifies the
result in different outcomes.
sequence of transformations that
will carry a given figure onto
another.

Explains transformations of a
given figure based on
descriptions of rigid motion.

Predicts the effect of a


transformation of a given figure
based on descriptions of rigid
motion.

Creates congruent figures using


transformations of rigid motion.

Justifies the congruence of two


complex figures using properties of
rigid motion.

Identifies corresponding pairs of


angles or corresponding pairs of
sides of two triangles that are
congruent.

Identifies corresponding pairs of


angles and corresponding pairs of
sides of two triangles that are
congruent.

Shows that two triangles are


congruent if and only if
corresponding pairs of sides and
corresponding pairs of angles are
congruent (CPCTC) using the
definition of congruence in terms
of rigid motions.

Justifies that two triangles are


congruent if and only if
corresponding pairs of sides and
corresponding pairs of angles are
congruent in a context.

G-CO.A [5]

G-CO.B [6]

Standard Setting Technical Report

Identifies corresponding parts of Identifies the minimum


Demonstrates how the criteria for Understands and explains why SSA
two congruent triangles.
conditions necessary for triangle triangle congruence (ASA, SAS,
and AAA do not provide enough
congruence (ASA, SAS, SSS).
SSS) follow from the definition of evidence for triangle congruence.
congruence in terms of rigid
motions.
Describes examples of theorems Determines the validity of
about lines and angles.
statements within a given proof
of a theorem about lines and
angles.

AzMERIT Performance Level Descriptors

D-141

Proves theorems about lines and Applies theorems about lines and
angles.
angles to a real-life context.

Mathematics
Geometry

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


PLD
Standard
Minimally Proficient
Partially Proficient

Detailed

Detailed

Detailed

G-CO.D [12
to 13]

Proficient

Highly Proficient

Describes examples of theorems Determines the validity of


about triangles.
statements within a given proof
of a theorem about triangles.

Proves theorems about triangles. Applies theorems about triangles to a


(Theorems include: measures of real-life context.
interior angles of a triangle sum
to 180; base angles of isosceles
triangles are congruent; the
segment joining midpoints of two
sides of a triangle is parallel to the
third side and half the length; the
medians of a triangle meet at a
point.)

Defines theorems about


parallelograms.

Determines the validity of


statements within a given proof
of a theorem about
parallelograms.

Proves theorems about


parallelograms.

Applies theorems about


parallelograms to a real-life context.

Copies a line segment and an


angle. Constructs congruent
segments and perpendicular
lines.

Bisects a line segment and an


angle. Constructs an equilateral
triangle, a square, and a regular
hexagon.

Constructs perpendicular lines, a


perpendicular bisector of a line
segment, and a line parallel to a
given line through a point not on
the line. Constructs an equilateral
triangle, a square, and a regular
hexagon inscribed in a circle.

Creates a polygon given certain


attributes using geometric
constructions. Explores the
construction of other regular
polygons inscribed in a circle.

G-CO.C [10]

G-CO.C [11]

Standard Setting Technical Report

AzMERIT Performance Level Descriptors

D-142

Mathematics
Geometry

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


PLD
Standard
Minimally Proficient
Partially Proficient
Identifies dilations.

Detailed

Detailed

Detailed

Detailed

Detailed

G-SRT.A [1a
to 1b]

G-SRT.A [2]

Proficient

Highly Proficient

Similarity, Right Triangles and Trigonometry


Identifies the scale factors of
Verifies the properties of dilations Locates the center of dilation and
dilations.
given by a center and a scale
scale factor, given a pair of similar
factor, by understanding that a
figures on a coordinate plane.
dilation creates parallel lines and
line segments in ratios of the
scale factor.

Identifies corresponding parts of Determines if two given figures


two similar figures.
are similar.

Explains that two given figures are Proves or disproves that two given
similar in terms of similarity
figures are similar, using
transformations.
transformations and the definitions
of similarity.
Establishes the AA criterion for
Proves that two triangles are similar
two triangles to be similar by
if two angles of one triangle are
using the properties of similarity congruent to two angles of the other
transformations.
triangle, using the properties of
similarity transformations.

Identifies similarity
transformations.

Identifies triangle similarity by


the use of the AA
criterion.

Defines theorems about


triangles.

Determines the validity of


statements within a given proof
of a theorem about triangles.

Proves theorems about triangles. Applies theorems about triangles to a


(Theorems include: a line parallel real-life context.
to one side of a triangle divides
the other two proportionally, and
conversely; the Pythagorean
Theorem proved using triangle
similarity.)

Finds measures of sides and


angles of congruent and similar
triangles.

Solves problems involving


triangles, using congruence and
similarity criteria.

Solves problems and proves


relationships in geometric figures
by using congruence and
similarity criteria for triangles.
Includes problems from context.

G-SRT.A [3]

G-SRT.B [4]

G-SRT.B [5]

Standard Setting Technical Report

AzMERIT Performance Level Descriptors

D-143

Proves conjectures about congruence


or similarity in geometric figures,
using congruence and similarity
criteria for triangles. Includes
problems from context.

Mathematics
Geometry

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


PLD
Standard
Minimally Proficient
Partially Proficient

Detailed

Detailed

Detailed

G-SRT.C [6]

G.SRT.C [7]

Standard Setting Technical Report


Proficient

Highly Proficient
Determines the similarity of right
triangles by comparing the
trigonometric ratios of the
corresponding sides. Solves for
missing angles of right triangles using
sine and cosine.

Understands that, in similar


triangles, corresponding angles
are congruent and ratios of
corresponding sides are equal.
Understands that the acute
angles of a right triangle are
complementary.

Defines sine, cosine, and tangent


as the ratio of sides of a right
triangle. Identifies the
relationship between the sine
and cosine of the acute angles of
a right triangle.

Understands that the ratio of two


sides in one triangle is equal to
the ratio of the corresponding
two sides of all other similar
triangles, leading to definitions of
trigonometric ratios for acute
angles. Explains the relationship
between the sine and cosine of
complementary angles.

Understands that the acute


angles of a right triangle are
complementary.

Identifies the relationship


between the sine and cosine of
the acute angles of a right
triangle.

Explains the relationship between Solves for missing side lengths of


the sine and cosine of
right triangles when given a fraction
complementary angles.
that is equivalent to the sine or
cosine of one of the angles.

Solves right triangles using the


Pythagorean
Theorem.

Applies the Pythagorean


Theorem in real-life and
mathematical contexts.

Solves right triangles using


trigonometric ratios and the
Pythagorean Theorem in
applied/contextual problems.

G-SRT.C [8]

AzMERIT Performance Level Descriptors

D-144

Models solutions to situations, using


trigonometric ratios and the
Pythagorean Theorem, by
constructing equations that can be
used to solve the problem. Including
problems from context.

Mathematics
Geometry

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


PLD
Standard
Minimally Proficient
Partially Proficient

Detailed

Detailed

Detailed

Detailed

G-C.A [1]

G-C.A [2]

G-C.A [3]

G-C.B [5]

Knows that the definition of a


circle as points equidistant to a
given point.

Circles
Recognizes that all circles are
similar.

Standard Setting Technical Report


Proficient
Proves that all circles are similar.

Highly Proficient
Solves applied math problems, using
the fact that all circles are similar.

Identifies inscribed angles, radii, Recognizes relationships among


and chords in circles.
inscribed angles, radii, and
chords in circles.

Describes relationships among


Solves problems using relationships
inscribed angles, radii, and chords among inscribed angles, radii, and
chords in circles.
in circles.

Identifies inscribed and


circumscribed circles of a
polygon.

Constructs the inscribed and


circumscribed circles of a
triangle.

Proves properties of angles for a Proves the unique relationships


quadrilateral inscribed in a circle. between the angles of a triangle or
quadrilateral inscribed in a circle.

Defines a sector area of a circle


as a proportion of the entire
circle.

Develops the definition of


radians as a unit of measure by
relating to arc length.

Derives the formula for the area


of a sector, and derives, using
similarity, the fact that the length
of the arc intercepted by an angle
is proportional to the radius.

AzMERIT Performance Level Descriptors

D-145

Proves that the length of the arc


intercepted by an angle is
proportional to the radius, with the
radian measure of the angle being
the constant of proportionality.

Mathematics
Geometry

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


PLD
Standard
Minimally Proficient
Partially Proficient

Detailed

Detailed

Detailed

Detailed

Detailed

G-GPE.A [1]

G-GPE.B [4]

G-GPE.B [5]

G-GPE.B [6]

Standard Setting Technical Report


Proficient

Highly Proficient

Expressing Geometric Properties with Equations


Identifies the center and radius Creates the equation for a circle, Completes the square to find the Determines the equation of a circle,
of a circle, given an equation
when given the center and
center and radius of a circle given given points of tangency.
written in (x - h)^2 + (y - k)^2 = radius.
by its equation.
r^2 form.
Solves problems algebraically,
using geometric theorems
involving a circle on the
coordinate plane. Locates
segments on a coordinate plane
that are parallel or
perpendicular by calculating
slope.

Proves simple geometric


theorems algebraically using
coordinates, such as proving a
point lies on a given circle.

Constructs visual representations on


the coordinate plane that meet given
conditions for coordinates. Justifies
statements about
geometric figures using coordinates.

Can explain why the slopes of


Creates the equation of a line
parallel lines are equal and the that passes through a specific
slopes of perpendicular lines are point given its slope.
negative reciprocals or one that
is 0 and the other that is
undefined.

Creates the equation of a line


parallel or perpendicular to a
given line that passes through a
given point.

Creates the equation of a line parallel


or perpendicular to a given line that
passes through a given point in a
context.

Finds the point on a line


segment that partitions the
segment in a given ratio, given a
visual representation of the line
segment.

Finds the point on a line segment


that partitions the segment in a
given ratio, given coordinates for
the line segment.

Finds the point on a directed line Constructs a line segment that is


segment (between two given
partitioned in a given ratio.
points) that partitions the
segment in a given ratio.

Calculates the perimeter of a


polygon.

Calculates areas of a rectangle


and right triangle given their
coordinates.

Calculates areas of any triangle


given its coordinates.

G-GPE.B [7]

AzMERIT Performance Level Descriptors

Proves simple geometric


theorems using coordinates,
when given a visual
representation on the coordinate
plane.

D-146

Calculates perimeters of polygons


and areas of triangles and rectangles
using their coordinates from a
contextual problem.

Mathematics
Geometry

AzMERIT - 2015

Appendix D. Performance Level Descriptors (PLDs)


PLD
Standard
Minimally Proficient
Partially Proficient

Detailed

Detailed

Detailed

Detailed

Detailed

Detailed

Informally describes the


formulas for the circumference
G-GMD.A [1] and area of a circle.

Standard Setting Technical Report


Proficient

Geometric Measurement and Dimension


Informally describes the formulas Explains the formulas for the
for the volume of a cylinder,
circumference of a circle, area of
pyramid, and cone by the use of a circle, volume of a cylinder,
dissection arguments.
pyramid, and cone.

Highly Proficient
Justifies the formulas for the
circumference of a circle, area of a
circle, volume of a cylinder, pyramid,
and cone.

Substitutes given dimensions


Computes the volume of
into the formulas for the volume cylinders, pyramids, cones, and
spheres, given a graphic.
G-GMD.A [3] of cylinders, pyramids, cones,
and spheres.

Solves problems using the volume Finds the volume of cylinders,


formulas for cylinders, pyramids, pyramids, cones, and spheres in a
cones, and spheres.
real-life context.

Identifies the shapes of twodimensional cross- sections


formed by a vertical or
G-GMD.B [4] horizontal plane.

Identifies the shapes of twodimensional cross-sections of


three- dimensional objects.
Identifies a three-dimensional
object generated by rotations of
two-dimensional objects.

G-MG.A [1]

G-MG.A [2]

G-MG.A [3]

Identifies a three-dimensional
object generated by rotations of
a simple two-dimensional object
about a line of symmetry of the
object.

Modeling with Geometry


Identifies geometric shapes that Uses a geometric shape modeled Uses geometric shapes, measures,
model a real-world object.
in a simple real-world object to and properties to model and
determine the appropriate
describe objects.
measures.
Calculates density based on
Calculates density based on
Uses properties of density based
area, when a formula is given.
volume (when a formula is
on area and volume to model a
given), and identifies appropriate situation in context.
unit rates.
Identifies relevant geometric
models for use in solving a
design problem.

AzMERIT Performance Level Descriptors

Compares quantitatively
Designs a structure to meet
different proposed solutions to a constraints and optimization
design problem, using geometric requirements.
properties of the solution.

D-147

Sketches the shape of a particular


two-dimensional cross-section of a
three- dimensional shape. Sketches
the three-dimensional object that
results from the rotation of a given
two-dimensional object.

Uses composite geometric shapes,


measures, and properties to model
and describe objects.
Compares and contrasts density rates
in a modeling context.

Designs a composite structure to


meet constraints and optimization
requirements.

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD

Standard

Minimally Proficient
The Minimally Proficient
student

Detailed

Detailed

N-RN.A [1 to
2]

N-CN.A [1 to
2]

Mathematics
Algebra II
Partially Proficient

The Partially Proficient student

Standard Setting Technical Report

Proficient
The Proficient student

Number and Quantity


Uses proper notation and uses Uses proper notation for radicals Explains and uses the meaning of
structure for integer exponents in terms of rational exponents, but rational exponents in terms of
only. Converts radical notation is unable to explain the meaning. properties of integer exponents,
to rational exponent notation. Identifies equivalent forms of
and uses proper notation for
radicals in terms of rational
expressions involving rational
exponents (but is not able to re- exponents. Rewrites expressions
involving radicals and rational
write or find the product of
multiple radical expressions).
exponents, using the properties of
exponents; identifies equivalent
forms of expressions involving
rational exponents; and converts
radical notation to rational
exponent notation.

Recognizes that the square root


of a negative number is not a
real number. Adds, subtracts,
and multiplies using single
operations with complex
numbers (e.g.: 4i + 5i =9i).

AzMERIT Performance Level Descriptors

Converts simple "perfect" squares


to complex number form (bi) ,
such as the square root of -25 is
5i . Uses the Commutative,
Associative, and Distributive
properties to identify products
and sums of complex numbers.

D-148

Knows that there is a complex


number i such that i^2=-1 , and
identifies the proper a+bi form
(with a and b real). Calculates
sums and products of complex
numbers for multi-step problems.

Highly Proficient
The Highly Proficient student
Proves, uses, and explains the
properties of rational exponents
(which are an extension of the
properties of integer exponents),
and extends to real world context.
Compares contexts where radical
form is preferable to rational
exponents, and vice versa.

Generalizes or develops a rule that


explains complex numbers and
their properties. Generalizes or
develops rules for abstract
problems, such as explaining what
type of expression results, when
given (a + bi)(c + di).

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD
Standard
Minimally Proficient

Mathematics
Algebra II
Partially Proficient

Algebra
Identifies structure used to
Identifies structure used to rewrite
rewrite polynomial expressions. rational, polynomial, and
exponential expressions with
rational or real exponents.
Detailed

Detailed

Detailed

Detailed

Detailed

A-SSE.A [2];
A-SSE.B [3c]

A-SSE.B [4];
F-BF.A [2]

Recognizes if a sequence is
arithmetic, geometric, or
neither.

Writes arithmetic and/or


geometric sequences with an
explicit formula.

Standard Setting Technical Report


Proficient

Highly Proficient

Recognizes equivalent forms of


complicated expressions,
particularly those involving
rational, polynomial, or
exponential functions with
rational or real exponents, and
uses the structure of the
expression to identify ways to
rewrite it.

Rewrites complicated expressions


(including those involving rational,
polynomial, or exponential
functions with rational or real
exponents) to equivalent forms
using the structure of the
expression. Makes generalizations
by rewriting expressions in context
using their structure.

Writes arithmetic and geometric


sequences both recursively and
with an explicit formula.

Models contextual situations with


arithmetic and geometric
sequences (as appropriate).

Using the Remainder Theorem,


decides whether (x - a) is factor
of a given polynomial.

Explains why (x-a) is a factor of


p(x)=0 when p(a)=0 .

A-APR.B [2]

Given a polynomial in factored Divides a polynomial by a factor


form, identifies the zeroes of
(x - a) .
the polynomial.

A-APR.B [3]

Identifies the zeroes of a


function from a graph.

Uses zeroes to sketch the graph of Factors a polynomial and uses


a function given in factored form. zeroes to sketch a graph of the
function.

Identifies zeroes from a graph and


uses zeroes to construct the
function.

Identifies a polynomial
identity.

Justifies a polynomial identity by


testing with specific numbers.

Algebraically justifies the validity


of polynomial identities. Uses the
identity to describe numerical
relationships in a given context.

A-APR.C [4]

AzMERIT Performance Level Descriptors

D-149

Proves polynomial identities and


uses them to describe numerical
relationships.

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD
Standard
Minimally Proficient

Detailed

Detailed

Detailed

Detailed

A-APR.D [6]

A-CED.A [1]

A-REI.A [1]

A-REI.A [2]

Rewrites simple rational


expressions in different forms,
such as rewriting a(x)/x in the
form q(x) + 0 , where a(x) and
q(x) are polynomials.

Mathematics
Algebra II
Partially Proficient

Proficient

Highly Proficient

Rewrites simple rational


expressions in different forms,
such as rewriting a(x)/b(x) in the
form q(x) + r(x)/b(x) , where a(x) ,
b(x) , q(x) and r(x) are
polynomials, with the degree of
r(x) less than the degree of b(x) .

Rewrites simple rational


expressions in different forms such
as rewriting a(x)/b(x) in the form
q(x) + r(x)/b(x) where a(x) , b(x) ,
q(x) and r(x) are polynomials, with
the degree of r(x) less than the
degree of b(x) , and b(x) with
degree 2 or above.

Identifies exponential equation Identifies exponential equation


with integer exponents that
with rational or real exponents
models a given situation.
and rational functions that
models a given situation.

Creates a rational or exponential


equation with rational or real
exponents and uses it to solve
problems.

Explains the meaning of solutions


(including extraneous), in
reference to context.

Solves simple rational or radical Describes the steps in solving


equations with multiple steps, simple rational or radical
without justifying the steps
equations.
involved in solving.

Explains and justifies the steps in


solving simple rational or radical
equations by applying the
properties of equality, inverse, and
identity.

Explains and justifies the steps in


solving simple rational and radical
equations by applying naming
properties.

Identifies simple rational and


radical equations.

AzMERIT Performance Level Descriptors

Rewrites simple rational


expressions in different forms,
such as rewriting a(x)/x in the
form q(x) + r/x , where a(x) and
q(x) are polynomials and r is an
integer.

Standard Setting Technical Report

Identifies the number of solutions Solves simple rational and radical


and extraneous solutions, given a equations and identifies
simple rational or radical
extraneous solutions.
equation.

D-150

Solves complicated rational and


radical equations and justifies
extraneous solutions.

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD
Standard
Minimally Proficient

Detailed

A-REI.B [4b];
N-CN.C [7]

Detailed

A-REI.C [6 to
7]

Mathematics
Algebra II
Partially Proficient

Proficient

Highly Proficient

Solves quadratic equations by


simple inspection. Understands
the meaning of a complex
number.

Solves quadratic equations by


factoring. Understands the
meaning of a complex number and
identifies when quadratic
equations will have non-real
solutions (but is unable to identify
the complex solution).

Solves quadratic equations by


inspection (e.g., for x^2 = 49 )-taking square roots, completing
the square, the quadratic formula,
and factoring-- as appropriate to
the initial form of the equation. In
the case of equations that have
roots with nonzero imaginary
parts, writes the solutions as a bi
for real numbers a and b.

Determines the most efficient


method for solving a quadratic
equation and justifies the choice
selected. Creates a quadratic
function without x -intercepts, and
verifies that the solutions are
complex.

Identifies by inspection the


number of solutions for a
system of equations.

Finds approximate solutions of a Solves a simple system of


system of equations from a graph. equations algebraically and
graphically.

Finds the solution to f(x)=g(x) , Finds the solution to f(x)=g(x) ,


where f(x) and g(x) are linear, where f(x) and g(x) are absolute
and the solution to quadratic
value and exponential functions.
functions presented in a graph.
Detailed

Standard Setting Technical Report

A-REI.D [11]

AzMERIT Performance Level Descriptors

D-151

Finds the solution to f(x)=g(x) ,


where f(x) and g(x) are
polynomial, rational, radical,
absolute value, exponential, or
logarithmic functions presented in
different forms. Justifies why the
x -coordinates of the points of
intersection are solutions to the
equation f(x)=g(x) .

Generalizes the number of


solutions to a system of equations.

Interprets solutions to f(x)=g(x) ,


where f(x) and g(x) are
polynomial, rational, radical,
absolute value, exponential, or
logarithmic functions presented in
different forms, in reference to
context.

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD
Standard
Minimally Proficient

Detailed

Detailed

Detailed

Detailed

Interprets key features of


graphs and tables that model a
linear function. Sketches graphs
F-IF.B [4 to 5]; showing key features, given a
verbal description of a linear
F-IF.C [9]
relationship.

Partially Proficient

Functions
Interprets key features of graphs
and tables that model a quadratic
function. Sketches graphs showing
key features, given a verbal
description of a quadratic
relationship.

Standard Setting Technical Report


Proficient

Highly Proficient

Interprets key features of graphs Interprets complex features of a


and tables that model a function function modeling a real-world
that is neither linear nor quadratic. context, given a verbal description.
Sketches graphs showing key
features, given a verbal description
of a relationship that is not linear
or quadratic.

Calculates and interprets the


average rate of change of a
polynomial or radical function
over a specified interval. Estimates
the rate of change from a graph of
a function.

Calculates and interprets the


average rate of change of a
logarithmic or trigonometric
function over a specified interval.
Estimates the rate of change from
a graph.

Compares the average rate of


change of two non-linear and nonquadratic functions over a specified
interval.

Graphs quadratic functions and


identifies zeroes and describes
end behavior. Graphs simple
exponential functions and
F-IF.C [7c and identifies intercepts and end
behavior.
7e]; F-IF.C
[8b]

Chooses the graph of a polynomial


function (degree 3 or higher) that
matches given key features.
Graphs complex exponential
functions and simple logarithmic
and trigonometric functions and
describes key features.

Graphs a polynomial function


(degree 3 or higher); correctly
identifies zeroes and describes end
behavior. Graphs any exponential
or logarithmic function and
describes key features. Graphs
trigonometric functions with at
most 2 transformations.

Identifies additional features (such


as multiplicity of zeroes, locations
of minimums and maximums,
domain and range appropriate to a
context, or intervals where the
function is increasing or
decreasing) for a polynomial
function of degree 3 or higher.
Graphs trigonometric functions
with 3 or more transformations.

Adds a constant to a function or


multiplies a function by a
F-BF.A [1a to constant to model a real-world
1b]
context.

Applies arithmetic operations to


multiple linear or exponential
functions to build a new function
to model a real- world context.

Combines standard functions using Determines whether combining


arithmetic operations.
two functions is appropriate to a
context, and performs the correct
operations.

F-IF.B [6]

Calculates and interprets the


average rate of change of a
simple rational function over a
specified interval from a graph
of the function.

Mathematics
Algebra II

AzMERIT Performance Level Descriptors

D-152

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD
Standard
Minimally Proficient

Detailed

Detailed

Detailed

F-BF.B [3]

F-BF.B [4a]

F-IF.A [3];
F-LE.A [2]

Partially Proficient

Standard Setting Technical Report


Proficient

Highly Proficient

For a linear and exponential


function, f(x) , identifies the
effect on the graph of replacing
f(x) with f(x) + k , k(f(x)) , f(kx) ,
and f(x + k) for specific values
of k (both positive and
negative). Estimates the value
of k given the graphs.
Compares two functions of the
same kind that differ by a
transformation, and identifies
the transformation.

For quadratic and logarithmic


functions, f(x) , identifies the effect
on the graph of replacing f(x) with
f(x) + k , k(f(x)) , f(kx) , and f(x + k)
for specific values of k (both
positive and negative). Estimates
the value of k given the graphs.
Compares two functions of the
same kind that differ by a
transformation, and identifies the
transformation.

For any function, f(x) , identifies


Recognizes even and odd functions
the effect on the graph of
from their graphs and algebraic
replacing f(x) with f(x) + k , k(f(x)) , expressions.
f(kx , and
f(x + k) for specific
values of k (both positive and
negative). Estimates the value of k
given the graphs. Compares two
functions of the same kind that
differ by a transformation, and
identifies the transformation.

Finds inverse functions for


linear functions. Identifies
whether a function has an
inverse from its graph.

Identifies whether a function has


an inverse from any
representation.

Finds the inverse function for a


simple non-linear function, if it
exists.

Identifies the parts of a


Defines and expresses a recursive
recursive function or sequence. sequence as a function,
constructs a linear function (multistep) given a graph, a description
of a relationship, or two inputoutput pairs.

Evaluates a logarithm using


technology.
Detailed

Mathematics
Algebra II

F-LE.A [4]

AzMERIT Performance Level Descriptors

Recognizes that sequences are


Applies the ideas of sequences
functions. Recognizes that a
being functions to real world
sequence has a domain, which is contexts.
the subset of integers, and can
generate a sequence given a
recursive function, constructs a
linear function (multi-step) given a
graph, a description of a
relationship, or two input-output
pairs.

Expresses a logarithmic expression Expresses the solution to


(with no variables) in equivalent ab^(ct)=d as a logarithm (where
exponential form.
b is 2, 10, or e ). Evaluates a
logarithm using technology.

D-153

Restricts the domain of a function


in order to find its inverse.

Applies logarithms to solve for


variables in exponents for
contextual problems (such as
continuous interest or uninhibited
growth/decay).

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD
Standard
Minimally Proficient
Knows that a full rotation of a
circle is 2 radians.
Detailed

Detailed

Detailed

Detailed

Mathematics
Algebra II
Partially Proficient

Standard Setting Technical Report


Proficient

Locates a radian measure between Locates any radian measure on a


0 and 2 on a unit circle.
unit circle.

F-TF.A [1]

F-TF.A [2]

Explains that the radian measure of


an angle is equivalent to the length
of the arc on the unit circle
subtended by the angle.

Identifies the sine and cosine of Identifies the sine and cosine of
angles in the first quadrant of a angles on the unit circle.
unit circle. Recognizes that the
coordinates of any point on the
unit circle may be defined as
(cos , sin ) .

Explains that one can travel


around the unit circle any real
number of units and arrive at a set
of coordinates that defines
trigonometric functions for all real
numbers.

Explains that one can travel around


any circle any real number of units
and arrive at a set of coordinates
that defines trigonometric
functions for all real numbers.

Identifies the amplitude,


frequency, and midline of a
given trigonometric function.

Writes an appropriate
trigonometric function to model a
real-world context (where the
information about amplitude,
frequency, and midline are given
clearly).

Analyzes a real-world context to


determine which information can
be used to write a trigonometric
function. Uses this analysis to
model the context with a
trigonometric function.

Proves the Pythagorean Identity


sin^2x+cos^2x= , and uses it to
find basic trig values, given one
trig value and the quadrant.

Extends the Pythagorean Identity


to prove that trig ratios are
constant for similar triangles.

Writes a trigonometric function


(given a specific amplitude,
frequency, and midline).

F-TF.B [5]

F-TF.C [8]

Highly Proficient

Shows that the Pythagorean


Finds an unknown trigonometric
Identity is valid, given numerical value by using the Pythagorean
Identity.
values for the identity.

AzMERIT Performance Level Descriptors

D-154

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD
Standard
Minimally Proficient

Detailed

Detailed

Detailed

Detailed

Detailed

G-GPE.A [2]

S-ID.A [4]

Partially Proficient

Standard Setting Technical Report


Proficient

Geometry
Identifies the directrix and
Identifies the directrix and focus of Derives the equation of a
focus of a parabola when given a parabola when given the
parabola, given a focus and
its graph.
equation.
directrix.
Statistics
Labels a blank normal
Uses the Empirical Rule to label a
distribution curve with the
blank normal distribution curve
appropriate mean and standard with the appropriate percentages
(68%-95%-99.7%).
deviations.

Uses the mean and standard


deviation of a data set to fit it to a
normal distribution and to
estimate population percentages
using the Empirical Rule.

Highly Proficient
Justifies conditions for when a
point is or is not part of a parabola,
given information about the focus
and directrix.
Additionally, recognizes that there
are data sets for which such a
procedure is not appropriate. Uses
technology or tables to estimate
areas under the normal curve.

Creates a scatter plot of


bivariate data.

Determines if a plotted data set is Creates a scatter plot of bivariate Compares the fit of different
data and estimates an exponential functions to data and determines
approximately linear.
(with domains not in the integers) which function has the best fit.
or trigonometric function that fits
the data. Uses this function to
solve problems in the context of
the data.

Describes why a particular


sample is not representative.

Describes why a particular sample


is not random. Determines what
inferences can be made about a
population from a given
representative random sample.

S-ID.B [6a]

S-IC.A [1]

S-IC.A [2]

Mathematics
Algebra II

Given two results, decides


Explains why a specific model is
which is more consistent with a not consistent with given dataspecific data-generating
generated results.
process.

AzMERIT Performance Level Descriptors

D-155

Explains why a representative


random sample is appropriate to
make inferences about a
population. Explains how a
sample may be random, but not
representative of the underlying
population, or how a sample may
be representative, but not
random.

Explains how to select a


representative random sample
from a particular population.

Decides if a specified model is


consistent with results from a
given data-generating process,
such as a simulation.

Designs a data-generating process


(e.g., simulation) to evaluate
whether a specified model is
consistent with given results.

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD
Standard
Minimally Proficient

Detailed

Detailed

Detailed

Detailed

S-IC.B [3]

S-IC.B [4]

S-IC.B [5]

S-IC.B [6]

Mathematics
Algebra II
Partially Proficient

Proficient

Highly Proficient

Identifies whether random


sampling was used in a
particular study.

Matches a given study to its


category: survey, observational
study, or experiment.

Explains the differences among


sample surveys, experiments, and
observational studies. Explains
how randomization relates to each
type of study.

Chooses an interval that


represents possible population
proportions or means, for a
particular sample proportion or
mean.

Interprets whether a particular


proportion is possible, given a
sample proportion or mean in
context and a margin of error.

Uses +/-2 standard deviations from Develops a margin of error for a


a sample proportion or mean to
given survey through use of a
create an interval that can be used simulation model.
to estimate possible population
proportion or mean.

Determines if the differences


between two treatments are
typically positive, negative, or
centered about zero, given
results of a randomized
experiment comparing the
treatments.

Calculates statistics related to a


Compares the results of a
Designs and runs a simulation to
randomized experiment using two randomized experiment using two build a distribution for possible
treatments.
treatments to simulations in order differences, for a given experiment.
to determine if differences in the
treatments are significant.

Determines the question being


investigated and the groups
that were considered, given a
report based on data.

Determines the way


randomization was used in the
design and the results, given a
report based on data.

Identifies an event as a subset Identifies or shows relationships


of a set of outcomes (a sample between sets of events, using
space).
Venn diagrams.
Detailed

Standard Setting Technical Report

S-CP.A [1]

AzMERIT Performance Level Descriptors

D-156

Explains the purposes and


limitations of sample surveys,
experiments, and observational
studies. Designs an appropriate
study for a given situation.

Evaluates the reasonableness of a Interprets the consequences of the


report based on data.
results, given a report based on
data, and discusses the statistical
validity of the findings.
Describes events as subsets of
sample space using characteristics
of the outcomes, or using
appropriate set language and
appropriate set representations
(unions, intersections, or
complements).

Using complex representations,


makes sense of outcomes in
context. (For example: unions of all
subsets would equal the sample
space).

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD
Standard
Minimally Proficient

Detailed

Detailed

Detailed

Detailed

Detailed

Mathematics
Algebra II
Partially Proficient

S-CP.A [4]

S-CP.A [5]

Proficient

Highly Proficient

Calculates probabilities for


events (including joint
probabilities).

Identifies whether events are


independent or dependent.

Understands that two events, A


and B , are independent, if the
probability of A and B occurring
together is the product of their
probabilities, and uses this
characterization to determine if
they are independent.

Contrasts several events in a


sample space and determines if
they are independent by
calculating the event probabilities.

Understands conditional
probability and how it applies
to real life events.

Calculates conditional
probabilities.

Determines the independence of


A and B using conditional
probabilities.

Constructs two-way frequency


tables of data.

Approximates conditional
probabilities using two-way
frequency tables.

Interprets two-way frequency


tables of data and uses them to
decide if events are independent.

Identifies and interprets


independence of events in
contextual problems, using
conditional probabilities.
Constructs, interprets, and finds
missing values of a two-way
frequency table.

S-CP.A [2]

S-CP.A [3]

Standard Setting Technical Report

Expresses conditional
Interprets conditional probabilities Recognizes and explains the
probabilities and independence and independence in context.
concepts of conditional probability
using probability notation.
and independence, in everyday
language and everyday situations.

Using concepts of conditional


probability and independence,
extrapolates the meaning behind
probabilities that were calculated
from real-world context.

Distinguishes between
compound and conditional
probability scenarios.

Compares and contrasts


conditional probabilities and
compound probabilities. (For
example: from a table, determines
the probability of getting the flu,
and then compares that to the
probability of getting the flu given
the individual never washes their
hands).

S-CP.B [6]

AzMERIT Performance Level Descriptors

Finds the conditional probability Interprets conditional probability


of A , given B as the fraction of B 's in terms of a uniform probability
model.
outcomes that also belong to
A , using a two-way table, Venn
diagram, or tree diagram.

D-157

AzMERIT - 2015
Appendix D. Performance Level Descriptors (PLDs)
PLD
Standard
Minimally Proficient
Recalls the Addition Rule.

Detailed

S-CP.B [7]

AzMERIT Performance Level Descriptors

Mathematics
Algebra II
Partially Proficient

Applies the Addition Rule, P(A or


B) = P(A) + P(B) - P(A and B) to
calculate a probability, in a given
context.

D-158

Standard Setting Technical Report


Proficient

Highly Proficient

Applies the general Addition Rule


to a uniform probability model,
and interprets the answer in terms
of the model.

Applies the Addition Rule to


different representations of
probability models (Venn diagram,
tree diagram, and two-way tables),
and interprets the answer in an
abstract or real-world context.

AzMERIT

Appendix E Test Blueprints

Appendix

StandardSettingTechnicalReport

AzMERIT

Standard Setting Technical Report

Appendix E. Test Blueprints

AzMERIT
Grade 3
Strands
Reading Standards for
Literature
Reading Standards for
Informational Text
Listening Comprehension
(Informational)
Language
Writing

Min

Max

26%

35%

26%

35%

0%

13%

13%
17%

19%
19%

Arizonas Statewide Achievement Assessment


for English Language Arts and Mathematics

Grade 4
Strands
Reading Standards for
Literature
Reading Standards for
Informational Text
Listening Comprehension
(Informational)
Language
Writing

Grade 6
Strands
Reading Standards for
Literature
Reading Standards for
Informational Text
Listening Comprehension
(Informational)
Language
Writing
Grade 9
Strands
Reading Standards for
Literature
Reading Standards for
Informational Text
Listening Comprehension
(Informational)
Language
Writing

Min

Max

26%

35%

26%

35%

0%

13%

13%
17%

19%
19%

English Language Arts


Assessment Blueprint
Grade 5
Strands
Reading Standards for Literature
Reading Standards for
Informational Text
Listening Comprehension
(Informational)
Language
Writing

Grade 7
Min

Max

24%

31%

30%

38%

0%

13%

13%
17%

19%
19%

Min

Max

23%

30%

31%

40%

0%

13%

13%
16%

18%
18%

Listening Standards will only be assessed on the


computer-based assessment.
In Grades 3-5 some items in the Reading and
Language Strands will also be aligned to the
standards for Reading: Foundational Skills.

Strands
Reading Standards for
Literature
Reading Standards for
Informational Text
Listening Comprehension
(Informational)
Language
Writing
Grade 10
Strands
Reading Standards for
Literature
Reading Standards for
Informational Text
Listening Comprehension
(Informational)
Language
Writing

Grade

Min

Max

24%

31%

30%

38%

0%

13%

13%
17%

19%
19%

Min

Max

23%

30%

31%

40%

0%

13%

13%
16%

18%
18%

10%-20%

E-1

Max

26%

35%

26%

35%

0%

13%

13%
17%

19%
19%

Min

Max

24%

31%

30%

38%

0%

13%

13%
17%

19%
19%

Min

Max

23%

30%

31%

40%

0%

13%

13%
16%

18%
18%

Grade 8
Strands
Reading Standards for
Literature
Reading Standards for
Informational Text
Listening Comprehension
(Informational)
Language
Writing
Grade 11
Strands
Reading Standards for
Literature
Reading Standards for
Informational Text
Listening Comprehension
(Informational)
Language
Writing

Percentage of Points by Depth of Knowledge Level


DOK Level 1
DOK Level 2
DOK Level 3

3-11

Min

50%-60%

15%-25%

DOK Level 4
16%-19% (Writing)

For more information go to www.azed.gov/AzMERIT

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Appendix E. Test Blueprints

AzMERIT

Arizonas Statewide Achievement Assessment


for English Language Arts and Mathematics

Grade 4

Grade 3
Domain

Min.

Max.

Operations and Algebraic


Thinking & Numbers in Base Ten

49%

53%

Number and OperationsFractions

18%

22%

Measurement and Data &


Geometry

26%

30%

Domain
Operations and Algebraic
Thinking
Number and Operations in Base
Ten

Grade 5
Min.

Max.

22%

26%

24%

28%

Number and OperationsFractions

29%

33%

Measurement and Data &


Geometry

15%

19%

Grade 6
Domain

Mathematics
Assessment Blueprint
Domain

Min.

Max.

Numbers and Operations Fractions

31%

35%

Number and Operations in Base


Ten & Algebraic Thinking

38%

42%

Measurement and Data &


Geometry

24%

28%

Min.

Max.

Grade 7
Min.

Max.

Domain

Grade 8
Min.

Max.

Domain

19%

23%

Functions

21%

25%

Ratio and Proportional


Relationships

19%

23%

Ratio and Proportional


Relationships

Expressions and Equations

29%

33%

The Number System

19%

23%

Expressions and Equations

32%

36%

Geometry &
Statistics and Probability

Expressions and Equations


17%

21%

23%

27%

Geometry

23%

27%

Geometry

12%

16%

The Number System

25%

29%

Statistics and Probability

15%

19%

Statistics and Probability &


The Number System

15%

19%

Min.

Max.

Algebra

34%

38%

Functions

32%

36%

Statistics

27%

31%

Algebra I
Conceptual Categories
Algebra
Functions
Statistics

Geometry
Min.

Max.

40%

44%

36%
17%

40%
21%

Percentage of Points by Depth of Knowledge Level


Grade
DOK Level 1 DOK Level 2 DOK Level 3
3-11
10%-20%
60%-70%
12%-30%

Domain
Congruence
Similarity, Right Triangles, and
Trigonometry
Circles & Geometric
Measurement & Geometric
Properties with Equations
Modeling with Geometry

Algebra II
Min.

Max.

23%

27%

27%

31%

23%

27%

17%

21%

Conceptual Categories

Within a test, approximately 70% of the assessment will be on major content within that grade or course.

E-2

For more information go to www.azed.gov/AzMERIT

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Appendix F Summary of Ordered Item Booklets

Appendix

AzMERIT

Standard Setting Technical Report

TableF.SummaryofOrderedItemBookletsbyTest

Test
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

Overall
Percent of
Students At
or Above
Standard

-1.79

0.02

98

-1.37

0.04

93

-1.19

0.01

90

-1.00

0.03

85

-0.92

0.01

82

-0.86

0.01

81

-0.77

0.01

78

-0.75

0.01

78

-0.73

0.01

77

10

-0.65

0.01

74

11

-0.48

0.01

70

12

-0.46

0.01

68

13

-0.40

0.01

67

14

-0.30

0.04

62

15

-0.19

0.01

59

16

-0.19

0.01

59

17

-0.17

0.01

59

18

-0.09

0.01

56

19

-0.01

0.01

53

20

0.05

0.01

49

21

0.12

0.01

48

22

0.15

0.01

46

23

0.22

0.01

43

24

0.26

0.01

43

25

0.29

0.01

41

26

0.33

0.01

40

27

0.36

0.01

38

28

0.42

0.01

36

29

0.46

0.03

35

30

0.51

0.01

33

31

0.57

0.01

30

32

0.60

0.01

30

33

0.60

0.01

30

34

0.61

0.03

30

35

0.65

0.05

29

F-1

Locationof
External
Benchmarks
AIMSApproaches;

AIMSMeets

SBACLevel2;

SBACLevel3;

American Institutes for Research

AzMERIT

Test

Standard Setting Technical Report

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

Overall
Percent of
Students At
or Above
Standard

Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA
Grade3ELA

36

0.76

0.01

24

37

0.79

0.01

24

38

0.88

0.01

21

39

0.92

0.01

20

40

0.93

0.01

20

41

0.98

0.01

18

42

0.99

0.01

18

43

1.10

0.01

15

44

1.14

0.01

15

45

1.21

0.01

12

46

1.27

0.01

12

47

1.31

0.01

11

48

1.35

0.01

10

49

1.36

0.01

10

50

1.43

0.05

51

1.50

0.01

52

1.51

0.01

53

1.58

0.05

54

1.70

0.01

55

1.72

0.01

56

1.75

0.01

57

1.76

0.01

58

1.77

0.01

59

1.90

0.01

60

1.92

0.01

61

2.07

0.06

62

2.39

0.06

63

2.91

0.07

64

3.16

0.02

65

3.60

0.03

66

3.62

0.03

67

5.54

0.05

Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA

-1.59

0.05

98

-1.47

0.02

97

-1.22

0.01

95

-1.11

0.01

93

F-2

Locationof
External
Benchmarks

SBACLevel4;

AIMSExceeds;

AIMSApproaches;

American Institutes for Research

AzMERIT

Test
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA

Standard Setting Technical Report

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

Overall
Percent of
Students At
or Above
Standard

Locationof
External
Benchmarks

-0.85

0.01

89

-0.62

0.01

83

-0.61

0.01

82

-0.56

0.01

82

-0.54

0.01

81

10

-0.45

0.03

79

11

-0.30

0.06

75

12

-0.20

0.01

71

13

-0.09

0.01

67

14

-0.06

0.01

67

15

-0.03

0.01

16

0.07

0.01

SBACLevel2
61 NAEPBasic;

17

0.09

0.01

61

18

0.10

0.05

61

19

0.14

0.01

57

20

0.16

0.03

57

21

0.20

0.01

56

22

0.31

0.01

51

23

0.33

0.01

51

24

0.33

0.01

51

25

0.43

0.01

47

26

0.45

0.01

46

27

0.47

0.01

44

28

0.50

0.01

44

29

0.52

0.01

42

30

0.54

0.01

42

31

0.55

0.01

41

32

0.60

0.01

39

33

0.66

0.01

37

34

0.67

0.01

37

35

0.74

0.01

34

36

0.80

0.01

32

37

0.83

0.01

30

38

0.88

0.01

28

39

0.89

0.01

27

40

0.90

0.01

27

F-3

AIMSMeets;

64

SBACLevel3

NAEPProficient;

American Institutes for Research

AzMERIT

Test
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade4ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA

Standard Setting Technical Report

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

Overall
Percent of
Students At
or Above
Standard

41

0.96

0.01

25

42

1.09

0.01

20

43

1.09

0.01

20

44

1.13

0.03

18

45

1.20

0.01

17

46

1.21

0.01

17

47

1.27

0.01

16

48

1.30

0.01

14

49

1.32

0.01

13

50

1.32

0.01

13

51

1.39

0.01

13

52

1.45

0.05

10

53

1.56

0.01

54

1.68

0.01

55

1.77

0.03

56

1.79

0.01

57

1.80

0.01

58

1.89

0.05

59

2.21

0.06

60

2.39

0.01

61

2.48

0.01

62

2.73

0.02

63

2.80

0.04

64

3.06

0.02

65

3.82

0.02

66

4.94

0.06

67

5.16

0.06

-3.14

0.04

100

-1.36

0.01

96

-1.20

0.01

93

-1.06

0.01

90

-0.90

0.01

87

-0.78

0.04

84

-0.73

0.03

82

-0.70

0.01

82

-0.49

0.01

75

F-4

Locationof
External
Benchmarks

SBACLevel4

AIMSExceeds;

NAEPAdvanced

AIMSApproaches;

AIMSMeets;

American Institutes for Research

AzMERIT

Test
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA

Standard Setting Technical Report

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

Overall
Percent of
Students At
or Above
Standard

10

-0.37

0.01

70

11

-0.37

0.06

70

12

-0.29

0.01

69

13

-0.25

0.01

66

14

-0.15

0.01

63

15

-0.13

0.01

63

16

-0.10

0.01

59

17

-0.08

0.01

59

18

-0.03

0.04

59

19

0.10

0.05

52

20

0.14

0.01

51

21

0.30

0.01

44

22

0.33

0.01

44

23

0.35

0.01

42

24

0.35

0.01

42

25

0.40

0.01

40

26

0.40

0.01

40

27

0.45

0.01

37

28

0.46

0.01

37

29

0.51

0.01

37

30

0.51

0.01

37

31

0.63

0.01

30

32

0.63

0.01

30

33

0.63

0.01

30

34

0.69

0.01

28

35

0.69

0.01

27

36

0.76

0.05

25

37

0.82

0.01

24

38

0.83

0.01

24

39

0.85

0.01

22

40

0.91

0.01

20

41

0.94

0.01

19

42

0.95

0.01

19

43

1.00

0.01

18

44

1.06

0.01

16

45

1.15

0.01

14

F-5

Locationof
External
Benchmarks

SBACLevel2

SBACLevel3

American Institutes for Research

AzMERIT

Test
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade5ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA

Standard Setting Technical Report

RP67/
RP50
(EOC
Math)

Overall
Percent of
Students At
or Above
Standard

46

1.15

0.01

Locationof
External
Benchmarks
14 SBACLevel4

47

1.18

0.03

12

48

1.33

0.01

49

1.45

0.05

50

1.61

0.05

51

1.64

0.01

52

1.70

0.01

53

1.80

0.01

54

1.85

0.01

55

1.86

0.01

56

1.86

0.01

57

1.96

0.01

58

1.97

0.01

59

2.01

0.01

60

2.04

0.01

61

2.12

0.06

62

2.38

0.02

63

2.46

0.02

64

2.76

0.02

65

2.87

0.02

66

3.81

0.10

67

4.08

0.04

68

4.13

0.04

-1.81

0.02

99

-1.67

0.06

99

-1.28

0.05

95

-1.11

0.01

91

-1.04

0.04

89

-0.92

0.01

87

-0.89

0.01

87

-0.86

0.01

85

-0.69

0.01

80

10

-0.64

0.04

78

11

-0.55

0.01

76

12

-0.39

0.05

70

13

-0.29

0.01

Item
Map
Order

Item
Score
Category

F-6

SE

AIMSExceeds;

AIMSApproaches;

AIMSMeets;

SBACLevel2
67 NAEPBasic;

American Institutes for Research

AzMERIT

Test
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA

Standard Setting Technical Report

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

Overall
Percent of
Students At
or Above
Standard

14

-0.21

0.01

64

15

-0.17

0.01

62

16

-0.12

0.01

61

17

-0.11

0.01

61

18

-0.07

0.01

59

19

0.04

0.01

54

20

0.06

0.01

53

21

0.07

0.01

53

22

0.12

0.01

51

23

0.17

0.01

50

24

0.27

0.01

47

25

0.32

0.01

43

26

0.33

0.01

43

27

0.36

0.01

43

28

0.49

0.01

37

29

0.56

0.01

35

30

0.58

0.01

34

31

0.63

0.01

34

32

0.66

0.01

31

33

0.68

0.01

31

34

0.69

0.01

31

35

0.78

0.01

28

36

0.80

0.01

28

37

0.81

0.01

28

38

0.93

0.01

23

39

0.96

0.01

22

40

0.97

0.04

22

41

0.99

0.01

22

42

1.00

0.01

22

43

1.00

0.04

22

44

1.01

0.01

21

45

1.02

0.01

21

46

1.02

0.01

21

47

1.05

0.01

19

48

1.16

0.01

17

49

1.19

0.01

17

F-7

Locationof
External
Benchmarks

SBACLevel3

NAEPProficient;

American Institutes for Research

AzMERIT

Test
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade6ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA

Standard Setting Technical Report

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

Overall
Percent of
Students At
or Above
Standard

50

1.23

0.04

15

51

1.26

0.01

14

52

1.32

0.01

14

53

1.43

0.05

54

1.59

0.01

55

1.74

0.01

56

1.76

0.01

57

1.88

0.04

58

2.03

0.01

59

2.11

0.01

60

2.20

0.01

61

2.22

0.01

62

2.51

0.02

63

2.69

0.02

64

2.72

0.07

65

3.85

0.03

66

3.87

0.03

67

5.40

0.05

-1.58

0.02

97

-1.42

0.01

96

-1.14

0.04

91

-1.11

0.01

91

-0.83

0.04

85

-0.77

0.01

83

-0.60

0.01

79

-0.59

0.01

78

-0.38

0.01

71

10

-0.37

0.01

71

11

-0.27

0.01

68

12

-0.18

0.06

65

13

-0.17

0.06

65

14

-0.11

0.01

62

15

-0.07

0.01

59

16

-0.04

0.01

59

17

-0.03

0.01

59

18

-0.02

0.01

59

F-8

Locationof
External
Benchmarks

AIMSExceeds
12 SBACLevel4

NAEPAdvanced

AIMSApproaches

AIMSMeets;

SBACLevel2

American Institutes for Research

AzMERIT

Test
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA

Standard Setting Technical Report

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

Overall
Percent of
Students At
or Above
Standard

19

0.00

0.01

56

20

0.05

0.01

56

21

0.11

0.01

52

22

0.12

0.01

52

23

0.17

0.01

49

24

0.18

0.01

49

25

0.24

0.01

47

26

0.27

0.01

46

27

0.32

0.01

44

28

0.38

0.03

42

29

0.45

0.01

39

30

0.48

0.01

37

31

0.51

0.01

36

32

0.53

0.01

36

33

0.57

0.05

34

34

0.58

0.01

34

35

0.58

0.01

34

36

0.61

0.01

33

37

0.61

0.03

33

38

0.72

0.05

30

39

0.74

0.01

28

40

0.77

0.01

27

41

0.79

0.01

27

42

0.80

0.05

27

43

0.86

0.01

25

44

0.87

0.05

24

45

0.95

0.01

22

46

1.04

0.01

19

47

1.07

0.01

18

48

1.08

0.01

18

49

1.09

0.03

18

50

1.21

0.01

15

51

1.21

0.01

14

52

1.23

0.01

14

53

1.35

0.05

12

54

1.45

0.01

10

F-9

Locationof
External
Benchmarks

SBACLevel3

AIMSExceeds;

American Institutes for Research

AzMERIT

Test
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade7ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA

Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA

Standard Setting Technical Report

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

Overall
Percent of
Students At
or Above
Standard

55

1.56

0.01

56

1.59

0.01

57

1.66

0.01

58

1.71

0.01

59

1.82

0.01

60

1.84

0.01

61

1.90

0.01

62

1.91

0.01

63

1.93

0.01

64

2.09

0.01

65

2.19

0.01

66

2.49

0.01

67

3.86

0.03

68

3.88

0.03

-1.66

0.02

98

-1.64

0.02

98

-1.58

0.02

97

-1.52

0.07

97

-1.47

0.02

97

-1.24

0.01

94

-1.16

0.04

93

-1.04

0.04

90

-0.92

0.01

88

10

-0.89

0.01

87

11

-0.82

0.01

85

12

-0.73

0.04

82

13

-0.62

0.06

80

14

-0.56

0.06

77

15

-0.42

0.01

73

Locationof
External
Benchmarks
SBACLevel4

AIMSApproaches;

16

-0.36

0.05

NAEPBasic;AIMS
Meets;SBACLevel
72 2;

17

-0.16

0.01

63

18

-0.10

0.01

61

19

-0.06

0.01

60

20

-0.05

0.01

60

F-10

American Institutes for Research

AzMERIT

Test
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA

Standard Setting Technical Report

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

Overall
Percent of
Students At
or Above
Standard

21

-0.04

0.01

60

22

-0.02

0.01

58

23

0.02

0.01

57

24

0.17

0.01

50

25

0.18

0.01

50

26

0.19

0.01

50

27

0.20

0.01

49

28

0.27

0.01

47

29

0.33

0.01

44

30

0.34

0.01

44

31

0.36

0.05

42

32

0.43

0.03

41

33

0.46

0.05

39

34

0.50

0.05

37

35

0.57

0.01

36

36

0.60

0.01

34

37

0.64

0.01

33

38

0.64

0.01

33

39

0.67

0.01

31

40

0.69

0.01

29

41

0.73

0.01

29

42

0.74

0.01

29

43

0.77

0.01

28

44

0.77

0.01

28

45

0.82

0.01

27

46

0.83

0.01

27

47

0.89

0.01

24

48

0.94

0.01

24

49

1.02

0.01

21

50

1.05

0.03

19

51

1.06

0.01

18

52

1.08

0.01

18

53

1.12

0.01

18

54

1.16

0.01

15

55

1.25

0.01

15

56

1.35

0.01

13

F-11

Locationof
External
Benchmarks

SBACLevel3;

NAEPProficient;

American Institutes for Research

AzMERIT

Test
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade8ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA

Standard Setting Technical Report

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

Overall
Percent of
Students At
or Above
Standard

57

1.37

0.01

13

58

1.46

0.01

10

59

1.49

0.01

10

60

1.59

0.01

61

1.69

0.01

62

1.72

0.01

63

1.75

0.01

64

1.87

0.01

65

2.11

0.01

66

2.45

0.06

67

3.27

0.02

68

4.02

0.02

69

4.05

0.03

70

4.41

0.03

-1.53

0.02

95

-1.53

0.02

95

-1.14

0.02

87

-0.81

0.04

78

-0.51

0.01

68

-0.50

0.03

67

-0.48

0.01

67

-0.34

0.01

61

-0.33

0.01

60

10

-0.32

0.08

60

11

-0.28

0.01

58

12

-0.26

0.01

58

13

-0.22

0.01

57

14

-0.17

0.03

55

15

-0.17

0.01

55

16

-0.15

0.01

53

17

-0.12

0.01

53

18

-0.04

0.01

50

19

0.05

0.03

47

20

0.14

0.01

43

21

0.16

0.01

41

22

0.24

0.01

40

F-12

Locationof
External
Benchmarks

SBACLevel4
AIMSExceeds;

NAEPAdvacned

American Institutes for Research

AzMERIT

Test
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA

Standard Setting Technical Report

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

Overall
Percent of
Students At
or Above
Standard

23

0.26

0.01

38

24

0.31

0.01

37

25

0.38

0.01

33

26

0.43

0.01

33

27

0.49

0.01

30

28

0.50

0.01

30

29

0.53

0.01

28

30

0.54

0.01

27

31

0.56

0.01

27

32

0.59

0.01

27

33

0.63

0.01

24

34

0.63

0.01

24

35

0.72

0.01

21

36

0.77

0.01

21

37

0.84

0.01

18

38

0.87

0.01

18

39

0.89

0.01

18

40

0.99

0.01

16

41

1.06

0.01

14

42

1.08

0.01

14

43

1.09

0.01

14

44

1.16

0.01

11

45

1.24

0.07

10

46

1.26

0.07

47

1.34

0.01

48

1.36

0.01

49

1.37

0.01

50

1.39

0.01

51

1.46

0.01

52

1.49

0.01

53

1.49

0.01

54

1.51

0.01

55

1.54

0.01

56

1.57

0.01

57

1.62

0.01

58

1.69

0.04

F-13

Locationof
External
Benchmarks

American Institutes for Research

AzMERIT

Test
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade9ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA

Standard Setting Technical Report

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

Overall
Percent of
Students At
or Above
Standard

59

1.71

0.08

60

1.75

0.08

61

2.01

0.04

62

2.11

0.04

63

2.21

0.02

64

2.27

0.02

65

2.44

0.09

66

2.65

0.02

67

3.05

0.02

68

4.29

0.07

69

4.41

0.06

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1

-1.60
-1.50
-1.27
-1.14
-0.98
-0.77
-0.57
-0.49
-0.39
-0.31
-0.26
-0.24
0.11
0.15
0.17
0.18
0.20
0.26
0.29
0.32
0.38
0.39
0.39
0.39
0.44
0.45
0.47
0.50

F-14

0.02
0.02
0.02
0.02
0.02
0.07
0.07
0.08
0.07
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.06
0.01
0.01
0.01
0.07
0.07
0.01
0.01
0.01
0.07
0.01

97
96
94
92
88
83
77
74
71
66
66
65
51
48
48
48
46
45
42
42
39
39
39
39
36
36
36
33

Locationof
External
Benchmarks

PISALevel1;AIMS
Approaches;

PISALevel2;

AIMSMeets;

PISALevel3;

American Institutes for Research

AzMERIT

Standard Setting Technical Report

Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA
Grade10ELA

Item
Map
Order
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

Item
Score
Category
1
1
1
2
1
1
1
1
1
1
1
1
1
1
2
1
1
1
1
2
1
2
1
1
1
2
1
1
2
1
1
2
2
1
2
1
2
2
3
3

RP67/
RP50
(EOC
Math)
0.53
0.54
0.57
0.58
0.59
0.61
0.64
0.65
0.72
0.73
0.74
0.76
0.78
0.79
0.79
0.82
0.87
0.88
0.90
0.92
0.94
0.98
1.00
1.02
1.18
1.23
1.24
1.31
1.33
1.38
1.42
1.46
1.56
1.60
1.62
1.87
2.49
2.67
3.17
3.20

Grade10ELA

-1.60

Test

F-15

SE
0.01
0.01
0.01
0.01
0.01
0.01
0.07
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.07
0.01
0.01
0.02
0.02
0.02
0.03
0.03
0.02

Overall
Percent of
Students At
or Above
Standard
33
33
31
30
30
30
29
29
26
25
25
25
23
23
23
22
20
20
20
20
18
17
17
16
13
12
12
10
10
9
8
7
6
6
5
3
1
0
0

Locationof
External
Benchmarks

PISALevel4;
AIMSExceeds;

PISALevel5;

0
PISALevel1;AIMS
97 Approaches;

American Institutes for Research

AzMERIT

Test
Grade10ELA
Grade10ELA
Grade10ELA

11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA

Standard Setting Technical Report

SE
0.02
0.02
0.02

Overall
Percent of
Students At
or Above
Standard
96
94
92

Item
Map
Order
2
3
4

Item
Score
Category
1
1
1

RP67/
RP50
(EOC
Math)
-1.50
-1.27
-1.14

-2.08

0.03

96

-1.50

0.06

93

-1.15

0.02

85

-0.96

0.02

81

-0.88

0.07

78

-0.71

0.02

72

-0.61

0.02

69

-0.46

0.02

62

-0.46

0.01

62

10

-0.37

0.01

59

11

-0.25

0.02

54

12

-0.11

0.02

49

13

-0.02

0.01

46

14

-0.02

0.02

46

15

0.00

0.02

44

16

0.01

0.06

44

17

0.10

0.01

41

18

0.13

0.01

40

19

0.21

0.04

37

20

0.30

0.01

34

21

0.32

0.01

34

22

0.34

0.04

32

23

0.36

0.01

32

24

0.36

0.01

32

25

0.40

0.01

31

26

0.42

0.01

31

27

0.43

0.01

29

28

0.50

0.01

28

29

0.52

0.01

26

30

0.61

0.01

23

31

0.62

0.01

23

32

0.63

0.01

23

33

0.64

0.01

22

34

0.67

0.01

22

35

0.69

0.01

22

36

0.69

0.01

22

F-16

Locationof
External
Benchmarks

PISALevel2;

SBACLevel2

ACTCollegeReady;

SBACLevel3

American Institutes for Research

AzMERIT

Test
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
11ELA
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math

Standard Setting Technical Report

37

0.71

0.01

Overall
Percent of
Students At
or Above
Standard
20

38

0.71

0.01

20

39

0.82

0.01

17

40

0.83

0.01

17

41

0.84

0.01

17

42

0.92

0.06

15

43

0.93

0.06

15

44

1.02

0.06

12

45

1.04

0.06

12

46

1.08

0.01

12

47

1.08

0.01

12

48

1.19

0.02

10

49

1.22

0.01

50

1.23

0.01

51

1.23

0.01

52

1.27

0.02

53

1.28

0.02

54

1.41

0.02

55

1.53

0.02

56

1.61

0.02

57

1.75

0.07

58

1.84

0.02

59

1.87

0.02

60

1.92

0.02

61

2.13

0.02

62

2.26

0.02

63

2.52

0.06

64

2.72

0.02

65

2.73

0.02

66

3.75

0.05

67

3.89

0.05

-1.31

0.02

96

-0.95

0.01

90

-0.75

0.01

86

-0.62

0.01

83

-0.38

0.01

78

-0.30

0.01

76

-0.30

0.01

76

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

F-17

SE

Locationof
External
Benchmarks

SBACLevel4

AIMSApproaches

American Institutes for Research

AzMERIT

Test
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math

Standard Setting Technical Report

-0.24

0.01

Overall
Percent of
Students At
or Above
Standard
75

-0.20

0.01

74

10

-0.16

0.06

73

11

-0.12

0.01

73

12

-0.02

0.01

70

13

-0.02

0.01

70

14

0.02

0.01

69

15

0.06

0.01

67

16

0.13

0.01

66

17

0.15

0.01

66

18

0.34

0.01

60

19

0.39

0.01

59

20

0.45

0.01

57

21

0.50

0.01

57

22

0.56

0.05

54

23

0.59

0.05

54

24

0.65

0.05

51

25

0.78

0.01

48

26

0.79

0.01

48

27

0.83

0.01

48

28

0.88

0.01

47

29

0.90

0.01

45

30

0.91

0.01

45

31

0.94

0.01

45

32

0.95

0.01

45

33

1.04

0.01

42

34

1.09

0.01

42

35

1.11

0.01

42

36

1.14

0.01

38

37

1.18

0.01

38

38

1.25

0.01

37

39

1.31

0.01

35

40

1.34

0.01

35

41

1.47

0.01

32

42

1.52

0.05

32

43

1.63

0.01

28

44

1.72

0.05

25

45

1.83

0.01

25

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

F-18

SE

Locationof
External
Benchmarks

SBACLevel2

AIMSMeets

SBACLevel3

AIMSExceeds;

American Institutes for Research

AzMERIT

Test
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade3Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math

Standard Setting Technical Report

46

1.93

0.01

Overall
Percent of
Students At
or Above
Standard
21

47

1.96

0.05

21

48

1.99

0.05

21

49

2.09

0.01

18

50

2.14

0.01

18

51

2.37

0.01

15

52

2.43

0.05

15

53

2.55

0.01

11

54

2.56

0.01

11

55

2.68

0.05

11

56

2.79

0.06

10

57

2.91

0.06

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

58

3.23

0.06

-1.50

0.02

97

-1.22

0.06

93

Locationof
External
Benchmarks

SBACLevel4

-0.73

0.05

NAEPBasic;AIMS
82 Approaches;

-0.58

0.01

77

-0.57

0.01

77

-0.49

0.01

77

-0.47

0.01

74

-0.44

0.01

74

-0.34

0.01

71

10

-0.31

0.01

71

11

-0.30

0.01

71

12

-0.28

0.01

71

13

-0.24

0.01

68

14

-0.22

0.05

68

15

-0.14

0.01

65

16

-0.03

0.01

62

17

0.02

0.01

62

18

0.14

0.01

58

19

0.17

0.01

58

20

0.18

0.01

57

21

0.24

0.05

55

22

0.27

0.01

55

23

0.30

0.01

52

24

0.39

0.01

52

F-19

SBACLevel2

American Institutes for Research

AzMERIT

Test
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade4Math

Standard Setting Technical Report

25

0.40

0.01

Overall
Percent of
Students At
or Above
Standard
52

26

0.42

0.01

49

27

0.50

0.01

49

28

0.54

0.01

45

29

0.54

0.01

45

30

0.60

0.05

45

31

0.60

0.05

45

32

0.65

0.05

43

33

0.72

0.05

42

34

0.74

0.05

42

35

0.76

0.05

42

36

0.78

0.05

38

37

0.89

0.01

36

38

0.89

0.01

36

39

0.91

0.05

35

40

0.95

0.01

35

41

0.97

0.01

35

42

1.01

0.01

35

43

1.02

0.01

33

44

1.15

0.05

31

45

1.24

0.01

28

46

1.35

0.01

25

47

1.38

0.05

25

48

1.44

0.01

25

49

1.45

0.01

22

50

1.45

0.01

22

51

1.47

0.01

22

52

1.55

0.05

22

53

1.66

0.01

19

54

1.73

0.05

19

55

1.85

0.05

16

56

2.07

0.05

AIMSExceeds;
13 SBACLevel4

57

2.11

0.05

13

58

2.20

0.01

10

59

2.21

0.05

10

60

2.23

0.01

10

61

2.33

0.01

10

62

2.39

0.01

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

F-20

SE

Locationof
External
Benchmarks
AIMSMeets;

NAEPProficient

SBACLevel3

American Institutes for Research

AzMERIT

Test
Grade4Math
Grade4Math
Grade4Math
Grade4Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math

Standard Setting Technical Report

63

2.41

0.01

Overall
Percent of
Students At
or Above
Standard
7

64

2.66

0.01

65

3.34

0.07

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

66

3.68

0.02

-1.83

0.02

97

-1.52

0.07

92

-1.13

0.01

85

-0.65

0.05

71

-0.53

0.01

66

-0.40

0.01

61

-0.38

0.05

61

-0.35

0.01

61

-0.29

0.01

58

10

-0.20

0.01

56

11

-0.18

0.01

55

12

-0.15

0.01

55

13

-0.11

0.01

55

14

-0.06

0.01

52

15

-0.02

0.01

52

16

-0.01

0.01

52

17

-0.01

0.01

52

18

0.05

0.05

48

19

0.07

0.05

48

20

0.09

0.04

48

21

0.27

0.05

43

22

0.28

0.05

43

23

0.31

0.01

43

24

0.35

0.01

41

25

0.37

0.05

40

26

0.38

0.01

40

27

0.41

0.01

40

28

0.48

0.01

37

29

0.57

0.01

35

30

0.63

0.05

34

31

0.68

0.01

33

32

0.72

0.01

31

33

0.74

0.01

31

34

0.75

0.01

31

F-21

Locationof
External
Benchmarks
NAEPAdvanced

AIMSApproaches;
SBACLevel2;

AIMSMeets;

SBACLevel3;

American Institutes for Research

AzMERIT

Test
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade5Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math

Standard Setting Technical Report

35

0.87

0.01

Overall
Percent of
Students At
or Above
Standard
29

36

0.90

0.01

27

37

0.93

0.01

26

38

0.98

0.01

26

39

0.99

0.01

26

40

1.02

0.01

25

41

1.08

0.01

24

42

1.16

0.05

22

43

1.26

0.01

21

44

1.26

0.01

20

45

1.28

0.05

20

46

1.34

0.01

19

47

1.37

0.05

19

48

1.46

0.01

16

49

1.51

0.01

16

50

1.62

0.01

14

51

1.71

0.05

13

52

1.74

0.01

13

53

1.81

0.05

12

54

1.81

0.05

12

55

1.82

0.01

12

56

1.87

0.05

12

57

1.95

0.01

10

58

2.01

0.01

10

59

2.16

0.01

60

2.16

0.01

61

2.29

0.01

62

2.41

0.01

63

2.52

0.06

64

2.67

0.06

65

3.01

0.07

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

Locationof
External
Benchmarks

AIMSExceeds;

SBACLevel4;

66

3.24

0.07

-1.42

0.01

91

-1.02

0.06

80

-0.86

0.01

AIMSApproaches;
73 NAEPBasic;

-0.73

0.01

69

-0.65

0.01

66

-0.64

0.01

66

F-22

American Institutes for Research

AzMERIT

Test
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math

Standard Setting Technical Report

-0.58

0.05

Overall
Percent of
Students At
or Above
Standard
66

-0.52

0.05

62

-0.48

0.01

62

10

-0.34

0.01

55

11

-0.31

0.05

55

12

-0.28

0.01

55

13

-0.26

0.01

55

14

-0.23

0.01

51

15

-0.12

0.01

48

16

-0.03

0.01

45

17

0.00

0.01

45

18

0.02

0.05

45

19

0.02

0.01

45

20

0.03

0.01

45

21

0.19

0.05

38

22

0.28

0.01

35

23

0.30

0.01

35

24

0.33

0.01

35

25

0.35

0.01

35

26

0.41

0.01

32

27

0.47

0.01

31

28

0.52

0.01

29

29

0.53

0.01

29

30

0.60

0.01

27

31

0.67

0.01

27

32

0.74

0.01

24

33

0.78

0.01

24

34

0.79

0.01

24

35

0.81

0.01

23

36

0.90

0.01

22

37

0.97

0.01

19

38

0.99

0.01

19

39

1.00

0.01

19

40

1.02

0.01

19

41

1.15

0.01

17

42

1.18

0.05

16

43

1.35

0.01

13

44

1.36

0.01

13

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

F-23

SE

Locationof
External
Benchmarks
SBACLevel2;

AIMSMeets;

NAEPProficient;

SBACLevel3;

AIMSExceeds;

American Institutes for Research

AzMERIT

Test
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade6Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math

Standard Setting Technical Report

45

1.43

0.01

Overall
Percent of
Students At
or Above
Standard
13

46

1.55

0.05

11

47

1.66

0.01

48

1.69

0.01

49

1.85

0.06

50

1.85

0.01

51

1.98

0.01

52

2.10

0.01

53

2.14

0.01

54

2.39

0.01

55

2.56

0.07

56

3.64

0.02

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

57

4.04

0.02

-1.37

0.01

86

-0.98

0.01

76

-0.94

0.01

76

-0.73

0.01

69

-0.60

0.01

66

-0.57

0.01

62

-0.53

0.01

62

-0.52

0.05

62

-0.47

0.01

59

10

-0.41

0.01

59

11

-0.19

0.01

52

12

-0.07

0.05

49

13

-0.06

0.05

49

14

-0.06

0.01

47

15

-0.03

0.01

46

16

0.00

0.05

46

17

0.04

0.01

46

18

0.11

0.01

43

19

0.12

0.01

43

20

0.14

0.01

43

21

0.17

0.01

41

22

0.18

0.01

41

23

0.25

0.01

39

24

0.30

0.01

38

25

0.30

0.01

36

F-24

Locationof
External
Benchmarks
SBACLevel4;

NAEPAdvanced

AIMSApproaches;

SBACLevel2;
AIMSMeets

American Institutes for Research

AzMERIT

Test
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade7Math
Grade8Math
Grade8Math

Standard Setting Technical Report

26

0.32

0.01

Overall
Percent of
Students At
or Above
Standard
36

27

0.41

0.01

33

28

0.44

0.01

33

29

0.45

0.01

33

30

0.59

0.05

30

31

0.71

0.01

27

32

0.72

0.05

27

33

0.73

0.01

27

34

0.81

0.01

25

35

0.82

0.01

25

36

0.83

0.01

25

37

0.84

0.01

25

38

0.87

0.05

23

39

0.87

0.01

23

40

0.89

0.01

23

41

0.98

0.05

22

42

1.06

0.06

20

43

1.25

0.01

17

44

1.29

0.01

16

45

1.42

0.05

14

46

1.51

0.06

13

47

1.53

0.01

13

48

1.54

0.01

13

49

1.64

0.01

11

50

1.71

0.01

11

51

1.89

0.06

52

1.95

0.01

53

1.99

0.01

54

2.20

0.06

55

2.23

0.01

56

2.35

0.01

57

2.63

0.07

58

2.87

0.02

59

3.24

0.02

60

3.40

0.02

61

3.43

0.02

-2.04

0.02

93

-1.93

0.02

93

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

F-25

SE

Locationof
External
Benchmarks

SBACLevel3;

AIMSExceeds

SBACLevel4

American Institutes for Research

AzMERIT

Test
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math

Standard Setting Technical Report

-1.85

0.06

Overall
Percent of
Students At
or Above
Standard
91

-1.56

0.01

84

-1.54

0.01

84

-1.29

0.07

77

-1.24

0.01

77

-1.21

0.05

73

-1.15

0.05

73

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

SE

Locationof
External
Benchmarks

10

-1.03

0.01

NAEPBasic;AIMS
69 Approaches;

11

-0.89

0.01

65

12

-0.89

0.01

65

13

-0.88

0.05

65

14

-0.74

0.01

61

15

-0.69

0.01

57

16

-0.60

0.01

53

17

-0.53

0.01

53

18

-0.40

0.01

49

19

-0.36

0.01

45

20

-0.31

0.05

45

21

-0.30

0.05

45

22

-0.27

0.06

42

23

-0.19

0.01

42

24

-0.16

0.01

38

25

-0.08

0.05

38

26

-0.01

0.01

35

27

0.03

0.01

35

28

0.04

0.01

35

29

0.09

0.01

32

30

0.09

0.01

NAEPProficient;
32 SBACLevel3;

31

0.20

0.01

30

32

0.23

0.04

30

33

0.34

0.06

27

34

0.36

0.01

27

35

0.38

0.05

24

36

0.43

0.05

24

37

0.46

0.06

24

38

0.47

0.05

24

39

0.59

0.01

22

F-26

SBACLevel2;

AIMSMeets;

American Institutes for Research

AzMERIT

Test
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
Grade8Math
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI

Standard Setting Technical Report

40

0.68

0.01

Overall
Percent of
Students At
or Above
Standard
20

41

0.70

0.01

20

42

0.77

0.05

18

43

0.79

0.01

18

44

0.81

0.05

18

45

0.94

0.01

16

46

1.11

0.06

13

47

1.15

0.05

13

48

1.26

0.01

11

49

1.29

0.01

11

50

1.37

0.01

10

51

1.37

0.01

10

52

1.41

0.01

10

53

1.41

0.01

10

54

1.42

0.01

55

1.53

0.01

56

1.58

0.01

57

1.67

0.01

58

1.67

0.01

59

1.85

0.01

60

1.93

0.01

61

2.13

0.01

62

2.32

0.02

63

2.39

0.02

64

2.51

0.07

65

2.52

0.02

66

2.60

0.02

67

2.70

0.02

68

2.87

0.02

69

3.37

0.02

70

3.91

0.02

-1.57

0.01

88

-1.42

0.06

84

-1.24

0.06

79

-1.18

0.01

74

-1.18

0.01

74

-1.14

0.01

74

-1.12

0.06

74

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

F-27

SE

Locationof
External
Benchmarks

AIMSExceeds

SBACLevel4

NAEPAdvanced

American Institutes for Research

AzMERIT

Test
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI

Standard Setting Technical Report

-1.01

0.01

Overall
Percent of
Students At
or Above
Standard
69

-0.99

0.01

67

10

-0.98

0.01

64

11

-0.87

0.01

59

12

-0.80

0.01

59

13

-0.78

0.01

59

14

-0.77

0.06

55

15

-0.70

0.06

55

16

-0.69

0.01

55

17

-0.69

0.01

55

18

-0.64

0.01

51

19

-0.57

0.01

51

20

-0.57

0.01

51

21

-0.57

0.01

50

22

-0.53

0.01

48

23

-0.53

0.01

48

24

-0.50

0.01

48

25

-0.33

0.01

41

26

-0.32

0.01

41

27

-0.29

0.01

41

28

-0.29

0.01

41

29

-0.28

0.01

41

30

-0.19

0.01

38

31

-0.12

0.06

35

32

-0.03

0.01

32

33

-0.03

0.01

32

34

0.00

0.01

32

35

0.06

0.01

30

36

0.09

0.01

30

37

0.10

0.01

30

38

0.18

0.01

27

39

0.23

0.06

25

40

0.27

0.01

25

41

0.28

0.01

25

42

0.33

0.06

23

43

0.41

0.01

23

44

0.41

0.01

23

45

0.42

0.01

22

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

F-28

SE

Locationof
External
Benchmarks

American Institutes for Research

AzMERIT

Test
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
AlgebraI
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry

Standard Setting Technical Report

46

0.47

0.01

Overall
Percent of
Students At
or Above
Standard
21

47

0.53

0.01

20

48

0.62

0.07

18

49

0.82

0.07

16

50

0.94

0.01

14

51

0.95

0.01

14

52

0.99

0.02

12

53

1.03

0.02

12

54

1.04

0.07

12

55

1.16

0.02

11

56

1.27

0.08

57

1.45

0.08

58

1.51

0.08

59

1.60

0.08

60

1.70

0.09

61

1.81

0.09

62

1.97

0.09

63

2.06

0.09

64

2.08

0.09

65

2.15

0.02

66

2.20

0.02

67

2.30

0.10

68

2.33

0.10

69

2.77

0.02

70

2.81

0.02

-2.35

0.02

89

-2.17

0.06

84

-2.17

0.02

84

-1.99

0.06

79

-1.79

0.06

PISALevel2
69 AIMSApproaches

-1.68

0.01

67

-1.64

0.01

64

-1.64

0.06

64

-1.64

0.01

64

10

-1.61

0.06

64

11

-1.59

0.06

62

12

-1.54

0.01

62

13

-1.44

0.06

57

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

F-29

SE

Locationof
External
Benchmarks

PISALevel1;

American Institutes for Research

AzMERIT

Test
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry

Standard Setting Technical Report

14

-1.39

0.06

Overall
Percent of
Students At
or Above
Standard
57

15

-1.37

0.01

53

16

-1.37

0.01

53

17

-1.34

0.01

53

18

-1.30

0.01

52

19

-1.26

0.01

52

20

-1.16

0.01

47

21

-1.15

0.01

47

22

-1.12

0.06

44

23

-1.02

0.01

43

24

-1.02

0.01

43

25

-0.95

0.01

39

26

-0.86

0.01

37

27

-0.83

0.01

36

28

-0.76

0.01

34

29

-0.75

0.06

34

30

-0.58

0.02

30

31

-0.50

0.02

28

32

-0.40

0.02

25

33

-0.33

0.02

24

34

-0.30

0.06

23

35

-0.27

0.02

23

36

-0.27

0.07

23

37

-0.15

0.02

20

38

-0.14

0.06

20

39

-0.12

0.02

20

40

-0.10

0.02

20

41

0.10

0.06

16

42

0.16

0.02

14

43

0.24

0.02

14

44

0.27

0.02

13

45

0.27

0.02

13

46

0.33

0.02

12

47

0.48

0.07

11

48

0.61

0.02

49

0.78

0.02

50

0.85

0.08

51

0.87

0.02

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

F-30

SE

Locationof
External
Benchmarks
AIMSMeets

PISALevel3

AIMSExceeds

PISALevel4

PISALevel5;

American Institutes for Research

AzMERIT

Test
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
Geometry
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII

Standard Setting Technical Report

52

0.96

0.02

Overall
Percent of
Students At
or Above
Standard
6

53

1.12

0.02

54

1.14

0.02

55

1.15

0.02

56

1.44

0.02

57

1.50

0.02

58

1.65

0.02

59

1.80

0.03

60

1.81

0.03

61

2.00

0.03

62

2.28

0.03

63

2.38

0.03

64

2.66

0.04

65

2.95

0.05

66

3.04

0.06

-2.47

0.02

86

-2.36

0.06

82

-2.32

0.02

82

-2.31

0.06

82

-2.29

0.02

82

-2.25

0.06

76

-2.02

0.02

70

-1.99

0.02

70

-1.98

0.06

70

10

-1.98

0.02

70

11

-1.96

0.06

70

12

-1.71

0.02

58

13

-1.53

0.06

53

14

-1.50

0.02

53

15

-1.49

0.06

53

16

-1.43

0.02

48

17

-1.36

0.06

48

18

-1.31

0.02

47

19

-1.25

0.06

44

20

-1.20

0.06

42

21

-1.18

0.01

42

22

-1.17

0.02

42

23

-1.15

0.02

40

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

F-31

SE

Locationof
External
Benchmarks

PISALevel6

AIMSApproaches

AIMSMeets

SBACLevel2

American Institutes for Research

AzMERIT

Test
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII

Standard Setting Technical Report

24

-1.02

0.02

Overall
Percent of
Students At
or Above
Standard
36

25

-0.98

0.06

36

26

-0.94

0.02

35

27

-0.90

0.02

32

28

-0.85

0.02

SBACLevel3
32 AIMSExceeds

29

-0.78

0.07

29

30

-0.77

0.02

29

31

-0.70

0.06

28

32

-0.68

0.02

28

33

-0.60

0.02

26

34

-0.57

0.02

25

35

-0.55

0.02

25

36

-0.52

0.06

23

37

-0.46

0.07

23

38

-0.40

0.07

21

39

-0.30

0.02

18

40

-0.19

0.02

18

41

-0.18

0.02

16

42

-0.14

0.07

16

43

0.06

0.02

12

44

0.09

0.02

12

45

0.22

0.02

11

46

0.24

0.08

11

47

0.34

0.02

48

0.45

0.08

49

0.57

0.02

50

0.70

0.09

51

0.80

0.02

52

0.84

0.02

53

0.97

0.02

54

0.98

0.02

55

1.13

0.03

56

1.17

0.03

57

1.17

0.03

58

1.18

0.03

59

1.41

0.03

60

1.67

0.03

61

1.72

0.03

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

F-32

SE

Locationof
External
Benchmarks
ACTCollegeReady

SBACLevel4

American Institutes for Research

AzMERIT

Test
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII
AlgebraII

Standard Setting Technical Report

62

1.83

0.03

Overall
Percent of
Students At
or Above
Standard
1

63

1.96

0.03

64

2.03

0.03

65

2.07

0.04

66

2.07

0.04

67

2.22

0.04

68

3.01

0.05

Item
Map
Order

Item
Score
Category

RP67/
RP50
(EOC
Math)

F-33

SE

Locationof
External
Benchmarks

American Institutes for Research

AzMERIT

Appendix G OIB Item Data Plots

Appendix

StandardSettingTechnicalReport

AzMERIT

StandardSettingTechnicalReport

FigureG1.OIBItemDataPlotGrade3ELA

G-1

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureG2.OIBItemDataPlotGrade4ELA

G-2

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureG3.OIBItemDataPlotGrade5ELA

G-3

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureG4.OIBItemDataPlotGrade6ELA

G-4

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureG5.OIBItemDataPlotGrade7ELA

G-5

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureG6.OIBItemDataPlotGrade8ELA

G-6

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureG7.OIBItemDataPlotGrade9ELA

G-7

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureG8.OIBItemDataPlotGrade10ELA

G-8

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureG9.OIBItemDataPlotGrade11ELA

G-9

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureG10.OIBItemDataPlotGrade3Math

G-10

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureG11.OIBItemDataPlotGrade4Math

G-11

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureG12.OIBItemDataPlotGrade5Math

G-12

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureG13.OIBItemDataPlotGrade6Math

G-13

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureG14.OIBItemDataPlotGrade7Math

G-14

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureG15.OIBItemDataPlotGrade8Math

G-15

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureG16.OIBItemDataPlotAlgebraI

G-16

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureG17.OIBItemDataPlotGeometry

G-17

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureG18.OIBItemDataPlotAlgebraII

G-18

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Appendix H Bookmark Placement Readiness Forms

Appendix

AzMERIT

StandardSettingTechnicalReport

Document H. AzMERIT Bookmark Placement Readiness Form

Subject: ____________________________
Panelist ID number _____________

Preparation for Round 1 Proficient, Partially Proficient, and Highly Proficient

Yes

No

a.

The workshop training has prepared me to review the Performance Level


Descriptors

b.

The training fully explained the concept of a student who just barely
meets the criteria described in the Arizona Performance Level
Descriptors.

c.

The workshop training has prepared me to review the Ordered Item Book
(OIB).

d.

The workshop training has prepared me to fill out the bookmark


placement sheet.

I have answered, Yes to the above questions and I understand what I need to do to place my
Bookmarks.
Yes _____

No _____

Initials __________

If I answered No to any of the above questions, I received additional training.


Yes _____

No _____

Initials __________

Following the additional training, I now feel sufficiently trained on what I need to do to place my
Bookmarks.
Yes _____

No _____

Initials __________

H-1

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Test ____________________________
Panelist ID number _____________

Preparation for Round 2 Proficient, Partially Proficient, and Highly Proficient

Yes

No

a.

The workshop training has prepared me to review the Performance Level


Descriptors

b.

The training fully explained the concept of a student who just barely
meets the criteria described in the Arizona Performance Level
Descriptors.

c.

The workshop training has prepared me to review the Ordered Item Book
(OIB).

d.

The workshop training has prepared me to fill out the bookmark


placement sheet.

e.

The training fully explained the panel feedback data that was presented.

f.

The training fully explained the student impact data that was presented.

I have answered, Yes to the above questions and I understand what I need to do to place my
Bookmarks.
Yes _____

No _____

Initials __________

If I answered No to any of the above questions, I received additional training.


Yes _____

No _____

Initials __________

Following the additional training, I now feel sufficiently trained on what I need to do to place my
Bookmarks.
Yes _____

No _____

Initials __________

H-2

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Appendix I Investigation of Equating Student Scores


Across AzMERIT Test Administration Modes

Appendix

AzMERIT

StandardSettingTechnicalReport

Equating Student Scores across AzMERIT


Test Administration Modes
Introduction
Priortoconductingthestandardsettingworkshopsandreportingtestscoresforthespring 2015
assessments, a mode comparability study was performed to evaluatedifferencesintestperformance
attributabletothemodeoftestadministration,andtoidentifythelinkingconstants necessaryto place
item parameter estimates across modes on a common scalefortestscoringandreporting.Asingle,
fixedoperationaltestformwasusedtoadministerAzMERITonline.Inaddition,anearlyequivalent
testformwasconstructedforpaperbasedtestadministrations.Althoughthepaperformwasdesigned
tobeassimilaraspossibletotheonlineform,someonlineitemsthatcouldnotberenderedonpaper
weremodifiedorreplaced.Thecommonitemsbetweentheonlineandpaperformsprovidedthebasis
foramodecomparabilitystudytocomparetheperformanceofitemsbetweentheonlineandpaper
modesoftestadministration.Inadditiontotheoperationaltestitems,boththeonlineandpapertests
includedembeddedfieldtestblocks.Onpaper,theseitemslotswereusedtofieldtestonlineitems
renderedforpaperadministrationaswellastoestablishalinktothepreviousAIMSscale.Only
operationalitemsthatwerecommontoboththeonlineandpaperformswereusedasthebasisforthe
modecomparabilitystudy.
Amatchedsamplesdesign(Way,Davis,andFitzpatrick,2006)wasusedtoinvestigatemode
comparability.Acovariateregressionapproachwasimplementedtoconstructequivalentgroupsof
studentstakingtheAzMERITassessmentsforbothmodesoftestadministration.Theregressionanalysis
identifiedforeachstudentapredictedscoreonthepaperAzMERITassessmentfrompreviousyear
achievement,covaryingdemographicvariablesthatincludedgender,ethnicity,incomelevelstatus,
Englishlanguagelearner(ELL)status,andIndividualizedEducationProgram(IEP)inthedevelopmentof
thepredictionequation.Anearestneighborsearchprocedurewasthenappliedtothepredicted
AzMERITscorestoselecttheequivalentgroupsofstudents.Thisprocedureresultedintheidentification
oftwomatchedsamplesforeachassessmenttoconductthemodecomparabilitystudy.

Common Items and Test Form Equivalence


TheonlineandpaperversionsoftheAzMERITtestformsweredesignedtobeasequivalentaspossible.
BecauseAzMERITisdesignedasanonlineassessment,therewereinevitablysomeitemsthatcouldnot
berenderedforpaperadministration.Intheseinstances,differentitemswereusedtomeasurethe
samecontentstandardsbetweenthepaperandonlinetestforms.
Table1showsthetotalnumberofoperationaltestitemsperassessmentandthenumberof
commonitemsusedforthemodecomparabilitystudy.AsTable1indicates,mostoperationalitems
werecommonacrosstheonlineandpaperbasedtestadministrationmodes,indicatingthatADEwas
successfulinproducingpaperequivalentsforalmostallAzMERITitems.Nevertheless,therewere
someitemsthatcouldnotberenderedforpaperadministration,andinthoseinstancesdifferent

I-1

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

itemswereusedtoassessthesamestandardsbetweentheonlineandpapertestforms.Thetest
characteristiccurvesinAppendixI.1showthatthedistributionoftestinformationacrosstheonline
andpapertestformswasnearlyidentical,indicatingthatalthoughtheonlineandpapertestforms
werenotidentical,theformsmeasuredstudentachievementequivalentlyacrosstheability
distribution.Themodecomparabilityanalyseswerebasedonlyontheitemscommontobothforms.

Table 1: Number of Common Items Between Online and Paper Test Forms
Grade

Subject

3
4
5
6
7
8
9
10
11

ELA
ELA
ELA
ELA
ELA
ELA
ELA
ELA
ELA

3
4
5
6
7
8

Math
Math
Math
Math
Math
Math
AlgebraI
Geometry
AlgebraII

NumberOfItems
TotalOperational
CommonItems
ELA
42
41
42
37
42
41
42
42
42
40
42
40
44
43
44
44
44
42
Math
45
43
45
45
45
45
47
46
47
47
47
46
47
46
47
46
47
46

Matched Samples
Thefollowingprocedureswereusedtodefinethematchedsamplesbetweentheonlineandpapertest
administrationmodes.
1.Forstudentsparticipatinginthepapertestadministration,2015AzMERITrawscoreswereregressed
onpreviousspringachievement,individualleveldemographicvariablesandschoollevelvariables.The
previousachievementindicatorwasthe2014AIMSscore,whereavailable,and/orthepreviousyear
Stanford10scores.Theindividualdemographicvariablesincludedethnicity,gender,freeandreduced
lunch(FRL)eligibility,andEnglishlanguagelearner(ELL)andspecialeducation(SPED)status.School
levelvariablesincludedratioofAfricanAmericanstudents,ratioofHispanicstudents,ratioofmulti
ethnicstudents,ratioofFRL,ratioofELL,ratioofSPEDandaverageachievementasindexedbythe2014
AIMSscores.Theratioofgroupenrollmentinschoolsforthedemographicvariableswascategorizedas
loworhighbymediansplit,whileschoollevelachievementwasclassifiedbyquintile.Variableswere

I-2

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

enteredintotheequationinastepwisefashionsothatonlyvariablesaccountingforsignificantvariation
inthepredictionof2015AzMERITtestperformancewereincludedinthefinalregressionequation:

whereY isthepredicted2015AzMERITrawscore,nreferstotheestimatedregressionweightfor
covariateXn.
2.Withtheobtainedregressionweights,thepredictionequationwasappliedtoallstudents
participatinginAzMERITacrosstestadministrationmodes,yieldingapredicted2015AzMERITrawscore
foreachstudent.
3.Usingthepredicted2015rawscoredistribution,thesamplewiththesmallernumberofstudentswas
dividedin20equalsizedgroups.Thepredictedrawscoredistributioncutpointsdeterminedbythe
equalsizedgroupswasusedtodividestudentsinthelargersampleintoeachofthe20abilitylevel
groups.Withineachofthe20abilitygroupsinthelargersample,arandomsampleofstudentswas
drawn,equalinsizetothenumberofonlinestudentsineachofthepredictedabilitylevelgroups.Table
2showsthesizeofthematchedsamplesforeachoftheAzMERITassessments.

Table 2: Number of Students Selected for Each of the Matched Samples


Grade

Subject

3
4
5
6
7
8
9
10
11

ELA
ELA
ELA
ELA
ELA
ELA
ELA
ELA
ELA

3
4
5
6
7
8

Math
Math
Math
Math
Math
Math
AlgebraI
Geometry
AlgebraII

SizeofMatchedSamples
ELA
35220
33380
32880
34420
33080
34360
20420
20400
15680
Math
28400
30700
31140
37420
31780
32140
25760
19520
16360


I-3

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Comparing the Matched Samples


ThetablesinAppendixI.2provideacomparisonofthedemographicandachievementcharacteristics
betweenthematchedonlineandpapersamplesdrawnforthemodecomparabilitystudy.Foreach
sample,thetablepresentstheproportionofstudentsclassifiedineachdemographiccategory,the
meanandstandarddeviationoftestscoreonthespring2014AIMSassessment,aswellastheaverage
predictedrawscoreonspring2015AzMERITassessment.
Notethattherawscoresummaryisbasedonthecompletesetofoperationalitemsbetweenthepaper
andonlineversionoftheAzMERIT,andnotonlytheitemscommontobothmodes.Resultsindicatethat
thedemographiccompositionandpriorachievementofthematchedsamplesisquitesimilarandthat
thematchingprocedurewaseffective.

Results
IRTparameterestimateswerecalibratedindependentlyforthematchedonlineandpapertest
administrationmodesamples.Thelinkingconstantnecessarytobringthematchedsamplepaperitem
parametersontothematchedsampleonlinescalewasthencomputed.Thelinkageswerecomputedin
twoways.Meanlinkingwastakenasthedifferencebetweentheaverageitemdifficultyestimatesfrom
thematchedsamplepapercalibrationandtheaverageitemdifficultyestimatesfromthematched
sampleonlineitemparameterestimates.Meansigmalinkingequatingwasalsousedtoplacethepaper
itemparametersontheonlinescale.
Table2showsthemeandifficultyoftestitemsresultingfromindependentcalibrationsbasedonthe
matchedsamplesfromtheonlineandpapertestadministrations,aswellasthelinkingconstants
necessarytobringthepaperitemparametersontotheonlinescale.Linkingconstantswerebasedon
thedifferencebetweenthemeanitemdifficultiesbetweentheonlineandpaperformsusingall
commonitemsaslinkingitems.Meansigmaequatingconstantsarealsoprovided.Asthelinking
constantsindicate,parameterestimatesresultingfromtheindependentcalibrationsofthepaperand
onlineassessmentsarequitecomparable.Thelargestidentifiedmodeeffectwasforitemsonthegrade
3ELAassessmentwhichwere,onaverage,slightlymoredifficultforstudentswhowereadministered
theassessmentonline.Examinationofthelinkeditemparameterestimatesindicatedthatitemswith
thegreatestdiscrepancybetweenonlineandpaperwerenotisolatedwithinaparticularcontent
standardoritemtype.Conversely,itemsontheAlgebraIIassessmentwereveryslightlyeasierfor
studentswhenadministeredonlinethanonpaper.Otherwise,however,meandifficultyofitem
parameterestimatesacrossmodesarecomparable,aspresentedinTable3.

I-4

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Table 3: Linking Constants Resulting from the Matched Samples Equating

Grade

Subject

MeanItemDifficulties
Online
Paper

3
4
5
6
7
8
9
10
11

ELA
ELA
ELA
ELA
ELA
ELA
ELA
ELA
ELA

0.10
0.14
0.09
0.17
0.01
0.05
0.11
0.06
0.01

0.03
0.18
0.06
0.13
0.01
0.03
0.07
0.07
0.01

3
4
5
6
7
8

Math
Math
Math
Math
Math
Math
AlgebraI
Geometry
AlgebraII

0.01
0.00
0.00
0.03
0.00
0.02
0.01
0.00
0.07

0.04
0.00
0.00
0.03
0.00
0.03
0.03
0.01
0.01

MeanLinking
Constant
ELA
0.13
0.04
0.04
0.04
0.01
0.02
0.03
0.00
0.00
Math
0.03
0.00
0.00
0.00
0.00
0.01
0.01
0.00
0.06

MeanSigmaLinkingConstants
Slope
Intercept
1.02
0.96
0.93
0.94
0.89
0.94
0.91
0.89
0.83

0.13
0.03
0.04
0.05
0.01
0.02
0.04
0.00
0.00

0.96
0.92
0.93
0.95
0.99
1.01
0.99
0.97
1.00

0.03
0.00
0.00
0.00
0.00
0.01
0.01
0.00
0.06

Tohelpevaluatethemagnitudeoftheseresults,Table4presents,foreachtest,theexpectedscale
scoredifferencethatwouldbeobservedifthemodecorrectionconstantwasappliedtothescoringof
paperbasedassessmentsrelativetoscoringpapertestsusingtheonlinebankparameters.Forexample,
becausetheitemsinthegrade3ELAassessmentareslightlyeasierwhenadministeredonpaperthan
online,theabilityofstudentstakingpaperassessmentsisslightlyoverestimatedwhenscoredusingthe
onlinebankitemparameters.Inthiscase,applyingthemodecorrectionwouldeffectivelylowerthe
observedscalescoreforpapertestersbyaboutfourpointsontheAzMERITscale(aboutonerawscore
point),andwouldresultinapproximately1.6%fewerstudents,statewide,notmeetingtheproficient
performancestandard.Conversely,becausetheAlgebraIIitemsare,onaverage,slightlymoredifficult
whenadministeredonpaperthanonline,themodecorrectionwouldeffectivelyaddtwoscalescore
pointstopapertesttakers,butwouldonlyincreasethepercentageofstudentsmeetingthepartially
proficientperformancestandard.

I-5

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Table 4: Anticipated Impact of Applying the Mode Correction Constant versus Scoring All
Assessments on the AzMERIT Reference Scale

Grade

Subject

3
4
5
6
7
8
9
10
11
3
4
5

ELA
ELA
ELA
ELA
ELA
ELA
ELA
ELA
ELA
Math
Math
Math

6
7
8

Math
Math
Math
AlgebraI
Geometry
AlgebraII

Correction
Magnitudein
AzMERIT
ScaleScore
4
+1
1
1
0
1
1
0
0
1
0
0
0
0
0
0
0
+2

ApproximateChangeinPercentageofStudentsReaching
EachPerformanceStandard
PartiallyProficient
1.6%
1.6%
1.7%
0.0%
0.0%
1.6%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%

Proficient
1.6%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%

HighlyProficient
2.0%
0.0%
0.0%
0.7%
0.0%
1.1%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%

0.0%
0.0%
0.0%
0.0%
0.0%
2.3%

0.0%
0.0%
0.0%
0.0%
0.0%
0.0%

0.0%
0.0%
0.0%
0.0%
0.0%
0.0%

WenotethatADEindependentlyinvestigatedmodecomparabilityusingastrategybasedonthe
operationaltestadministrationstatewide(Scott,2015)andthisstudyisincludedinAppendixI.3.Inher
study,Scott(2015)firstidentifiedwhichArizonaschoolselectedtoadministerAzMERITonlineand
whichonpaper,andthenexaminedthetwosamplesofschoolsforanydifferencesinperformanceon
thespring2014administrationofAIMS.Havingfoundnodifferenceinmean2014performance
betweenthetwogroups,therewouldbenoexpectationforperformancedifferencesonAzMERIT
exceptasafunctionoftestadministrationmode.Followingthespring2015administrationofAzMERIT,
ADEexaminedtheperformanceofschoolsparticipatingonlineandonpaper,andfoundperformanceon
theAzMERITtobecomparablebetweenthetwosetsofschools,asexpectedbasedontheirspring2014
AIMSresults.

I-6

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Conclusion
Themodecomparabilitystudydescribedinthisdocumentexaminedthecomparabilityofitem
parametersandresultingtestscoresfromtheonlineandpaperadministrationsofthespring2015
AzMERITassessmentsinELAandmath.Thematchedsamplesanalysesrevealedgenerallythatitem
difficultyestimatesandresultingstudentabilityestimateswerecomparableacrosstestadministration
modes.SmallmodeeffectswereidentifiedforsomegradesintheELAassessments,withitemsinthe
grade3assessmentprovingslightlymoredifficultwhenadministeredonline.Bycontrast,itemsonthe
AlgebraIIassessmentwereslightlyeasierwhenadministeredonline.Evenforthelargesteffectingrade
3ELA,themagnitudeofthemodedifferencewasquitesmall,amountingtojustunderonerawscore
point(approximatelyfourpointontheAzMERITscale),impactingtheproficientratebyabout1.6%.
Giventhegenerallystrongcomparabilityofitemdifficultyacrossmode,ADEmayconsideradoptinga
singlesetofbankparametersforscoringstudentresponsesontheAzMERITacrosstheonlineandpaper
testadministrationmodes.

Reference
Scott,L.(2015).AnalysisofModeComparabilityofAzMERITsOnlineandPaperAdministrationsfor
Spring2015.Unpublishedmanuscript,ArizonaDepartmentofEducation.
Way,W.D.,Davis,L . L.,&Fitzpatrick,S.(2006,April).Scorecomparabilityofonlineandpaper
administrationsoftheTexasAssessmentofKnowledgeandSkills.Paperpresentedatthe
annualmeetingoftheNationalCouncilonMeasurementinEducation,SanFrancisco,CA.

I-7

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Appendix I.1 Test Characteristic Curves of Online and Paper Test


Forms
ThefiguresinAppendixI.1presentthetestcharacteristiccurvesoftheonlineandpapertest
forms,whichrepresentthedistributionoftestinformationacrosseachform.Thexaxis
representsoverallstudentabilityestimateinlogitmeasure,andforeachabilityestimate,they
axisrepresentsthetotalrawscorepointsthatastudentcantheoreticallyachieve.
FigureI.1.1.TestCharacteristicCurvesforOnlineandPaperFormsGrade3ELA

I-8

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureI.1.2.TestCharacteristicCurvesforOnlineandPaperFormsGrade4ELA

FigureI.1.3.TestCharacteristicCurvesforOnlineandPaperFormsGrade5ELA

I-9

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureI.1.4.TestCharacteristicCurvesforOnlineandPaperFormsGrade6ELA

FigureI.1.5.TestCharacteristicCurvesforOnlineandPaperFormsGrade7ELA

I-10

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureI.1.6.TestCharacteristicCurvesforOnlineandPaperFormsGrade8ELA

FigureI.1.7.TestCharacteristicCurvesforOnlineandPaperFormsGrade9ELA

I-11

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureI.1.8.TestCharacteristicCurvesforOnlineandPaperFormsGrade10ELA

FigureI.1.9.TestCharacteristicCurvesforOnlineandPaperFormsGrade11ELA

I-12

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureI.1.10.TestCharacteristicCurvesforOnlineandPaperFormsGrade3Math

FigureI.1.11.TestCharacteristicCurvesforOnlineandPaperFormsGrade4Math

I-13

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureI.1.12.TestCharacteristicCurvesforOnlineandPaperFormsGrade5Math

FigureI.1.13.TestCharacteristicCurvesforOnlineandPaperFormsGrade6Math

I-14

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureI.1.14.TestCharacteristicCurvesforOnlineandPaperFormsGrade7Math

FigureI.1.15.TestCharacteristicCurvesforOnlineandPaperFormsGrade8Math

I-15

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureI.1.16.TestCharacteristicCurvesforOnlineandPaperFormsAlgebraI

FigureI.1.17.TestCharacteristicCurvesforOnlineandPaperFormsGeometry

I-16

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureI.1.18.TestCharacteristicCurvesforOnlineandPaperFormsAlgebraII

I-17

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Appendix I.2 Comparison of Matched Samples


TableI.2.1.ComparisonofMatchedSamplesGrade3ELA
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014ReadingAchievementMean
Spring2014ReadingAchievementStandardDeviation

OnlineSample
0.51
0.49
0.83
0.08
0.04
0.09
0.47
0.01
0.63
0.16
0.11
0.00
27.36
7.99
0.13
1.90
52.69
27.68

PaperSample
0.51
0.49
0.85
0.08
0.04
0.07
0.46
0.01
0.53
0.16
0.12
0.00
27.36
7.99
0.13
1.89
52.58
27.65

I-18

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

TableI.2.2.ComparisonofMatchedSamplesGrade3Math
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014MathAchievementMean
Spring2014MathAchievementStandardDeviation

OnlineSample
0.51
0.49
0.83
0.07
0.04
0.09
0.46
0.01
0.63
0.16
0.11
0.00
27.83
6.92
0.15
1.97
53.26
29.53

PaperSample
0.50
0.50
0.85
0.07
0.04
0.06
0.47
0.01
0.54
0.16
0.12
0.00
27.81
6.93
0.13
1.95
52.69
29.59

TableI.2.3.ComparisonofMatchedSamplesGrade4ELA
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014ReadingAchievementMean
Spring2014ReadingAchievementStandardDeviation

I-19

OnlineSample
0.51
0.49
0.83
0.07
0.03
0.09
0.46
0.01
0.62
0.11
0.12
0.00
30.06
8.17
0.13
1.93
51.51
29.05

PaperSample
0.51
0.49
0.84
0.07
0.04
0.07
0.46
0.01
0.54
0.11
0.13
0.00
30.06
8.17
0.13
1.94
51.26
28.94

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

TableI.2.4.ComparisonofMatchedSamplesGrade4Math
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014MathAchievementMean
Spring2014MathAchievementStandardDeviation

OnlineSample
0.51
0.49
0.83
0.07
0.04
0.10
0.46
0.01
0.62
0.11
0.12
0.00
26.07
7.76
0.07
1.88
51.46
28.96

PaperSample
0.50
0.50
0.85
0.07
0.04
0.07
0.45
0.01
0.53
0.11
0.13
0.00
26.08
7.77
0.06
1.88
51.16
28.83

TableI.2.5.ComparisonofMatchedSamplesGrade5ELA
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014ReadingAchievementMean
Spring2014ReadingAchievementStandardDeviation

I-20

OnlineSample
0.51
0.49
0.83
0.07
0.03
0.10
0.45
0.01
0.62
0.11
0.11
0.00
28.57
7.98
0.10
1.92
50.80
29.01

PaperSample
0.51
0.49
0.84
0.07
0.04
0.07
0.46
0.01
0.53
0.11
0.13
0.00
28.57
8.00
0.10
1.92
50.58
28.86

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

TableI.2.6.ComparisonofMatchedSamplesGrade5Math
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014MathAchievementMean
Spring2014MathAchievementStandardDeviation

OnlineSample
0.51
0.49
0.83
0.07
0.03
0.10
0.45
0.01
0.62
0.11
0.11
0.00
22.93
8.32
0.03
1.84
51.17
29.09

PaperSample
0.51
0.49
0.84
0.07
0.04
0.07
0.46
0.01
0.52
0.11
0.12
0.00
22.93
8.33
0.02
1.85
50.66
28.85

TableI.2.7.ComparisonofMatchedSamplesGrade6ELA
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014ReadingAchievementMean
Spring2014ReadingAchievementStandardDeviation

I-21

OnlineSample
0.51
0.49
0.84
0.07
0.03
0.08
0.46
0.01
0.61
0.08
0.11
0.00
28.67
8.57
0.10
1.92
50.67
27.38

PaperSample
0.51
0.49
0.84
0.07
0.04
0.08
0.45
0.01
0.52
0.08
0.13
0.00
28.68
8.57
0.10
1.93
50.61
27.25

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

TableI.2.8.ComparisonofMatchedSamplesGrade6Math
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014MathAchievementMean
Spring2014MathAchievementStandardDeviation

OnlineSample
0.51
0.49
0.84
0.07
0.03
0.08
0.45
0.01
0.61
0.08
0.11
0.00
23.20
8.10
0.02
1.86
50.84
28.91

PaperSample
0.50
0.50
0.84
0.07
0.04
0.08
0.44
0.01
0.52
0.08
0.12
0.00
23.20
8.11
0.01
1.87
50.75
28.68

TableI.2.9.ComparisonofMatchedSamplesGrade7ELA
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014ReadingAchievementMean
Spring2014ReadingAchievementStandardDeviation

I-22

OnlineSample
0.51
0.49
0.85
0.07
0.03
0.08
0.46
0.01
0.59
0.05
0.10
0.00
29.61
8.62
0.15
1.98
50.89
27.23

PaperSample
0.51
0.49
0.83
0.07
0.04
0.09
0.45
0.01
0.52
0.06
0.12
0.01
29.62
8.63
0.15
1.99
50.64
27.17

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

TableI.2.10.ComparisonofMatchedSamplesGrade7Math
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014MathAchievementMean
Spring2014MathAchievementStandardDeviation

OnlineSample
0.51
0.49
0.85
0.07
0.03
0.08
0.46
0.01
0.59
0.05
0.11
0.00
23.05
8.35
0.01
1.88
50.58
28.79

PaperSample
0.50
0.50
0.83
0.07
0.04
0.08
0.45
0.01
0.52
0.05
0.12
0.00
23.05
8.36
0.01
1.89
50.36
28.56

TableI.2.11.ComparisonofMatchedSamplesGrade8ELA
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014ReadingAchievementMean
Spring2014ReadingAchievementStandardDeviation

I-23

OnlineSample
0.51
0.49
0.85
0.07
0.03
0.07
0.45
0.01
0.58
0.05
0.10
0.00
31.30
8.40
0.15
1.95
51.25
27.33

PaperSample
0.51
0.49
0.83
0.07
0.04
0.08
0.44
0.01
0.51
0.04
0.11
0.00
31.30
8.40
0.15
1.96
51.34
27.20

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

TableI.2.12.ComparisonofMatchedSamplesGrade8Math
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014MathAchievementMean
Spring2014MathAchievementStandardDeviation

OnlineSample
0.51
0.49
0.85
0.07
0.03
0.07
0.45
0.01
0.58
0.05
0.10
0.00
22.38
8.65
0.01
1.84
50.96
28.98

PaperSample
0.51
0.49
0.84
0.07
0.04
0.08
0.45
0.01
0.52
0.04
0.11
0.00
22.39
8.66
0.00
1.85
50.62
28.87

TableI.2.13.ComparisonofMatchedSamplesGrade9ELA
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014ReadingAchievementMean
Spring2014ReadingAchievementStandardDeviation

I-24

OnlineSample
0.51
0.49
0.84
0.07
0.04
0.08
0.38
0.01
0.45
0.02
0.09
0.00
27.69
7.24
0.17
1.98
54.42
28.18

PaperSample
0.50
0.50
0.86
0.07
0.05
0.05
0.48
0.01
0.51
0.03
0.08
0.01
27.69
7.27
0.16
1.99
54.56
28.00

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

TableI.2.14.ComparisonofMatchedSamplesAlgebraI
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014MathAchievementMean
Spring2014MathAchievementStandardDeviation

OnlineSample
0.51
0.49
0.85
0.07
0.03
0.07
0.41
0.01
0.47
0.02
0.09
0.00
21.46
7.72
0.15
1.87
54.92
28.45

PaperSample
0.50
0.50
0.86
0.06
0.05
0.05
0.47
0.01
0.50
0.02
0.09
0.01
21.46
7.74
0.14
1.87
55.13
28.27

TableI.2.15.ComparisonofMatchedSamplesGrade10ELA
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014ReadingAchievementMean
Spring2014ReadingAchievementStandardDeviation

I-25

OnlineSample
0.51
0.49
0.84
0.08
0.04
0.08
0.38
0.01
0.43
0.01
0.09
0.00
29.75
8.38
0.20
1.97
55.03
26.24

PaperSample
0.50
0.50
0.86
0.07
0.04
0.05
0.46
0.01
0.50
0.02
0.08
0.01
29.74
8.39
0.19
1.97
55.08
26.27

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

TableI.2.16.ComparisonofMatchedSamplesGeometry
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014MathAchievementMean
Spring2014MathAchievementStandardDeviation

OnlineSample
0.51
0.49
0.84
0.07
0.04
0.08
0.38
0.01
0.44
0.01
0.08
0.00
16.65
6.79
0.07
1.94
55.32
27.26

PaperSample
0.51
0.49
0.86
0.07
0.05
0.05
0.46
0.01
0.49
0.02
0.07
0.01
16.66
6.81
0.06
1.95
55.81
26.95

TableI.2.17.ComparisonofMatchedSamplesGrade11ELA
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014ReadingAchievementMean
Spring2014ReadingAchievementStandardDeviation

I-26

OnlineSample
0.52
0.48
0.84
0.07
0.05
0.09
0.36
0.01
0.40
0.01
0.09
0.00
25.99
7.00
0.27
2.15
57.63
24.57

PaperSample
0.51
0.49
0.86
0.07
0.04
0.05
0.48
0.01
0.49
0.01
0.09
0.01
26.00
7.00
0.26
2.16
57.70
24.55

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

TableI.2.18.ComparisonofMatchedSamplesAlgebraII
DemographicandAchievementVariables
Male
Female
White
Black
Asian
AmericanIndian
Hispanic
PacificIslander
FreeandReducedLunch(FRL)
EnglishLanguageLearner(ELL)
SpecialEducation(SPED)
Migrant
PredictedScoreMean
PredictedScoreStandardDeviation
PredictedScoreSkewness
PredictedScoreKurtosis
Spring2014MathAchievementMean
Spring2014MathAchievementStandardDeviation

I-27

OnlineSample
0.50
0.50
0.84
0.08
0.05
0.08
0.36
0.01
0.41
0.01
0.06
0.00
14.09
6.13
0.45
2.34
63.26
24.70

PaperSample
0.48
0.52
0.86
0.07
0.05
0.05
0.47
0.01
0.49
0.01
0.05
0.01
14.11
6.14
0.44
2.34
64.22
24.59

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Appendix I.3. Analysis of Mode Comparability of AzMERITs Online and


Paper Administrations for Spring 2015

Analysis of Mode Comparability of


AzMERITs Online and Paper Administrations
for Spring 2015.

Arizona Department of Education


Assessment Section
July 23, 2015

I-28

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

In spring of 2015, the Arizona Department of Education (ADE) instituted a new assessment
(Arizonas Measurement of Educational Readiness to Inform Teaching, AzMERIT) in compliance
with state and federal mandates and aligned with the Arizona State Board of Educations (State
Board) adopted values which in part state that It is essential that the new statewide assessment:
- Use 21st Century technology to deliver the assessment, as available infrastructure allows. The
full March 6, 2014, State Board values document is available at: http://www.azed.gov/state-boardeducation/files/2014/03/adopted-essential-assessment-values-6mar14.pdf. In November 2014, the
State Board adopted AzMERIT as its test to measure student achievement in mathematics (MATH)
and English language arts (ELA) in Grades 3 through 8 and for end-of-course use in Grades 9
through 11 for ELA and Algebra I, Geometry, and Algebra II for MATH.
Since the State Board wanted an on-line testing system but many schools did not have the
infrastructure to administer computer based tests to all of their students, ADE developed, with the
assistance of its vendor (American Institutes for Research, AIR), a dual mode assessment
(computer-based and paper-based) for each grade/subject combination. The question of how the
comparability of these two modes of administration should be analyzed was brought before ADEs
Technical Advisory Committee (TAC) during their February 2015 meeting. AIR proposed a
method to TAC which they approved. The TAC, however, suggested that ADE might also examine
the modes comparability using a different method since ADE/AIR planned for Item Response
Theory (IRT) scaling calibration to be performed using the Rasch model.
TAC recommended that ADE use the methodology presented by Wright (1967) first to compare
the AIMS 2014 scales scores students in schools that were assessing students online with those of
the whole state, to determine any differences between the scale scores of these two groups of
schools on last years tests. The purpose of this initial analysis was to have a baseline comparison
of how much discrepancy to expect when the two groups of school went from taking assessments
in the same mode to different modes. The second analysis that was recommended, using the same
methodology, would be carried out once the data from the spring 2015 administration of AzMERIT
was available. This paper presents the results from the first analysis of AIMS 2014 data, and results
for the second analysis, using all data available for both modes as of 7/13/2015.
Method
The method put forth by Wright (1967) to examine the effect of student sample and item sample on
the calibration of a test is as follows:

First perform IRT scaling separately on the two samples of interest using all items that
contributed to each groups total score.
Organize the resulting scale scores (or student ability estimates, Thetas) from least to
greatest.
Graph the ability estimates or scale scores for both groups by number of raw score points
available on one graph.
Compare the amount of distance there is between the graphs of the two groups.
I-29

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Results
The graphs resulting from the first analysis, that comparing AIMS 2014 scales scores from
computer-based schools to that of the whole state, are presented in Figure 1 for MATH and Figure
2 for Reading.1 The examination of the two graphs for every grade/subject combination showed
very little, if any, difference between the independently determined AIMS scales. Where there
were differences, they occurred in the very lowest ranges and in the very highest ranges of the raw
scores where the most measurement error occurs, but even these differences were extremely small.
The graphs resulting from the second analysis, that comparing AzMERIT student ability estimates
(Theta) for each raw score from computer-based schools to that of those from paper-based schools,
are presented in Figure 3 for MATH and Figure 4 for ELA. The examination of the two graphs for
every grade/subject combination showed very little, if any, difference between the independently
determined raw score to Theta estimates. (At this writing, the AzMERIT scales had not been
determined, so the IRT Theta estimates from which the scale scores will be computed were used
for these graphs.) Minor differences were found in some grade/subject combinations (specifically
Grade 3 ELA and Algebra II), however, these differences again were both small and located at the
ends of the raw score continuum where the most measurement error exists.
In both of Grade 3 ELA and Algebra II, while the number of items administered to students was the
same for each mode, there was one less raw score point for the paper mode upon which to estimate
student ability. For Grade 3 ELA, this difference was due to an online two point item, which could
not be translated to paper, being substituted with a single point item aligned to the same content.
For Algebra II, one item was dropped from analysis due to a minor technical issue in the sharing of
data between ADE and the test vendor at the time of the creation of this report. Given these raw
score differences, the resulting raw score/Theta comparisons between the two modes is extremely
small even for these most egregious subject/grade combinations.

__________________
1

For AIMS, only the Reading test was compared using the 2014 data since this test did not have an ELA score, as such.
In 2014 AIMS, as in prior years, writing was only assessed in Grades 5, 6, 7 and high school, so it was never combined
with the reading score to compute an ELA score across all tested Grades 3 through 8 and high school.

References
Wright, B. D. (October, 1967). Sample-free test calibration and person measurement. Paper
presented at the ETS Invitational Conference on Testing Problems, Princeton, NJ, retrieved
from http://www.rasch.org/memo1.htm on 2/18/2015.

I-30

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Figure 1. AIMS 2014 MATH raw score/scale score graphs for schools assessing students via
computer as compared to all students in the state.

AIMS2014Grade4Math

700

700

600

600

500

500

ScaleScore

ScaleScore

AIMS2014Grade3Math

400
300
200

400
300
200

100

100
0 5 10 15 20 25 30 35 40 45 50 55 60 65
TestScore
Computer

0 5 10 15 20 25 30 35 40 45 50 55 60 65
TestScore
Computer

All

AIMS2014Grade6Math

700

700

600

600

500

ScaleScore

ScaleScore

AIMS2014Grade5Math

400
300

500
400
300
200

200

100

100

0
0 5 10 15 20 25 30 35 40 45 50 55 60 65
TestScore
Computer

All

0 5 10 15 20 25 30 35 40 45 50 55 60 65
TestScore
Computer

All

I-31

All

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

AIMS2014Grade8Math

700

700

600

600

500

500

ScaleScore

ScaleScore

AIMS2014Grade7Math

400
300
200

400
300
200

100

100
0 5 10 15 20 25 30 35 40 45 50 55 60 65
TestScore
Computer

0 5 10 15 20 25 30 35 40 45 50 55 60 65
TestScore
Computer

All

All

AIMS2014Cohort16Math
800

ScaleScore

700
600
500
400
300
200
0

10

20

30

40

50

TestScore
Computer

60

70

80

All

I-32

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Figure 2. AIMS 2014 Reading raw score/scale score graphs for schools assessing students via
computer as compared to all students in the state.

AIMS2014Grade4Reading

700

700

600

600
ScaleScore

ScaleScore

AIMS2014Grade3Reading

500
400
300

500
400
300

200

200
0

10 15 20 25 30 35 40 45 50
TestScore
Computer

700

700

600

600

ScaleScore

ScaleScore

800

500
400

500
400

300

300

200

200
0

10 15 20 25 30 35 40 45 50
TestScore
Computer

All

AIMS2014Grade6Reading

800

10 15 20 25 30 35 40 45 50
TestScore
Computer

All

AIMS2014Grade5Reading

10 15 20 25 30 35 40 45 50
TestScore
Computer

All

I-33

All

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

AIMS2014Grade8Reading

800

800

700

700

600

600

ScaleScore

ScaleScore

AIMS2014Grade7Reading

500
400
300

500
400
300

200

200
0

10 15 20 25 30 35 40 45 50
TestScore
Computer

10 15 20 25 30 35 40 45 50
TestScore
Computer

All

All

AIMS2014Cohort16Reading
1000

ScaleScore

900
800
700
600
500
400
0

10 15 20 25 30 35 40 45 50
TestScore
Computer

All

I-34

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Figure 3. AzMERIT MATH raw score/student ability (Theta) graphs for schools assessing students
via computer as compared to those using paper-based tests.

Grade4Math

Theta

Theta

Grade3Math

6
0

10

15

20

25

30

TestScore
Computer

35

40

45

10

Theta

Theta

6
15

20

25

TestScore
Computer

30

35

40

45

Paper

10

25

Grade6Math

20

TestScore
Computer

Paper

Grade5Math

15

30

35

40

45

10

15

20

25

TestScore
Computer

Paper

I-35

30

35

40

45

Paper

American Institutes for Research

Grade7Math

Grade8Math
StandardSettingTechnicalReport

Theta

Theta

AzMERIT

6
0

10

15

20

25

30

TestScore
Computer

35

40

45

TestScore
Computer

Paper

Paper

Geometry

AlgebraI
8

Theta

Theta

10 15 20 25 30 35 40 45 50

6
0

10

15

20

25

30

TestScore
Computer

35

40

45

10 15 20 25 30 35 40 45 50
TestScore
Computer

Paper

Paper

AlgebraII
8
6

Theta

4
2
0
2
4
6
0

10

15

20

25

30

TestScore
Computer

35

40

45

Paper
I-36

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Figure 4. AzMERIT ELA raw score/student ability (Theta) graphs for schools assessing students
via computer as compared to those using paper-based tests.

Grade4ELA

Theta

Theta

Grade3ELA

4
6

6
0

10 15 20 25 30 35 40 45 50 55
TestScore
Computer

TestScore
Computer

Paper

Theta

Theta

Paper

Grade6ELA

Grade5ELA

4
6

6
0

5 10 15 20 25 30 35 40 45 50 55
TestScore
Computer

10 15 20 25 30 35 40 45 50 55

10 15 20 25 30 35 40 45 50 55
TestScore
Computer

Paper

I-37

Paper

American Institutes for Research

Grade7ELA

StandardSettingTechnicalReport
Grade8ELA

Theta

Theta

AzMERIT

4
6

6
0

10 15 20 25 30 35 40 45 50 55
TestScore
Computer

TestScore
Computer

Paper

Grade9ELA

Paper

Grade10ELA

Theta

Theta

5 10 15 20 25 30 35 40 45 50 55

6
0

10 15 20 25 30 35 40 45 50 55
TestScore
Computer

5 10 15 20 25 30 35 40 45 50 55
TestScore
Computer

Paper

Paper

Grade11ELA
8
6

Theta

4
2
0
2
4
6
0

10 15 20 25 30 35 40 45 50 55
TestScore
Computer

Paper
I-38

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Table 1. Demographics of Students by AzMERIT Mode of Assessment ELA.


Demographic Variables
Number of Students
Number of Tests
Male
Female
White
Black
Asian
Native American
Hispanic
Pacific Islander
Multiethnic
Free and Reduced Lunch (FRL)
English Language Learner (ELL)
Special Education (SPED)
Migrant

Online
328,237
329,691
51.1%
48.9%
83.1%
7.6%
3.6%
9.2%
43.1%
0.9%
3.2%
57.0%
5.5%
11.1%
0.4%

Paper
371,867
372,897
50.6%
49.4%
84.1%
7.5%
4.4%
8.4%
44.6%
0.9%
3.2%
50.4%
4.7%
11.2%
0.8%

Table 2. Demographics of Students by AzMERIT Mode of Assessment MATH.


Demographic Variables
Number of Students
Number of Tests
Male
Female
White
Black
Asian
Native American
Hispanic
Pacific Islander
Multiethnic
Free and Reduced Lunch (FRL)
English Language Learner (ELL)
Special Education (SPED)
Migrant

Online
318,208
329,289
51.1%
48.9%
83.0%
7.6%
3.5%
9.3%
43.5%
0.8%
3.2%
57.6%
5.6%
11.2%
0.4%

Paper
353,648
365,516
50.4%
49.6%
84.3%
7.5%
4.3%
8.3%
44.9%
0.9%
3.2%
50.7%
4.7%
10.7%
0.7%

Note: Percentages on these tables were computed based on unique students, however since students were sometimes of
multiple ethnicities, they were counted in each. Also, students were allowed to take tests in all grades and/or subjects
for the courses that they were taking resulting in some students taking multiple tests within a subject area. Multiple
course tests taken by students were especially prevalent in mathematics.

I-39

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Appendix J AzMERIT Vertical Linking Study

Appendix

AzMERIT

StandardSettingTechnicalReport

AppendixJ.AzMERITVerticalLinkingStudy

Introduction
Thepurposeofthisreportistodocumentresultsoftheverticallinkingstudythatwasimplementedto
developaverticalscaleforscoringandreportingstudentachievementresultsontheAzMERITandthat
allowsformonitoringandevaluationofstudentsgainsovertime.
Toemphasizetheacquisitionofnewknowledgeandskillsinthedevelopmentoftheverticalscale,
operationalitemsfromeachgradelevelassessment(g)wereembeddedinfieldtestslotsofthe
assessmentinthegradebelow(g1).Inthisapproach,theresultinglinkagerepresentsstudent
achievementeachyearonthescaleofthesubsequentgradelevelassessmentforwhichtheyare
preparingtoreceiveinstruction.Assuch,thescalescoresforeachassessmentcanbeinterpretedasa
pretestscoreformeasuringstudentacquisitionofacademiccontentinthesubsequentgradelevel.
Whilethisapproachrisksadministeringtostudents12itemsmeasuringcontentthattheymaynotyet
havehadtheopportunitytolearn,itprovidesamoresensitivemeasureofstudentgrowththancouldbe
obtainedbyalinkingdesigninthelinkagerepresentscontinuedgrowthonacademiccontentassessed
inthepreviousyearsassessment.

Linking Items
SincetheverticalscaleessentiallyplaceseachAzMERITassessmentonthescalefortheassessmentin
thegradeabove,wecanbestassurecomparabilityoftestscoresbetweenthegradesbyestablishingthe
linkageusingallavailableoperationaltestitems.Thus,tolinkthegrade4assessmentstothegrade5
scales,alloperationalitemsinthegrade5assessmentweremadeavailableforadministrationinthe
grade4embeddedfieldtest(EFT)slots.Includingalloperationalitemsintheverticallinkingsetensures
thattheitemsetusedtolinktothetargetadjacentgradescalerepresentsfullythemeasuredconstruct
inthetargetgrade,allowingvalidinferencestobemadewithrespecttostudentbaselineperformance
forachievementinthesubsequentgradelevel.
BecausetheAzMERITassessmentsofEnglishlanguagearts(ELA)inhighschoolcontinueasendof
course(EOC)orgradelevelmeasuresofstudentachievementoftheArizonaCollegeandCareerReady
Standards(ACCRS),eachassessmentcanbelinkedtothegradeaboveusingallavailableoperational
items.
However,AzMERITassessmentsofhighschoolmatharecomposedofasetofEOCteststhatarenotas
consistentlyassociatedwithgradelevelinstructionandwhichmeasurespecificsubsetsofthecontent
domain.Forexample,whilemathematicscourseworkinhighschoolfollowsatypicalprogressionandit
wouldthereforebepossibletoembedgrade9AlgebraIEOCitemsinthegrade8mathassessment,
embedthegrade10GeometryEOCitemsintheAlgebraIEOCexam,andembedthegrade11
AlgebraIItheGeometryexam,theconstructsmeasuredacrossthefourexamsvaryconsiderablyand
haveimplicationsfortheinterpretationofgrowth,orlackthereof,acrossassessments.Forexample,itis
notclearwhattheexpectationforgrowthshouldbeinaverticalscaleestablishedbyembedding
GeometryitemsinanAlgebraIexam,sinceGeometryisnotafocusofinstructioninAlgebraIcourses.

J1

AmericanInstitutesforResearch

AzMERIT

StandardSettingTechnicalReport

Analternativeapproach,andtheoneadoptedbyADE,wastolinkthegrade8mathscaletoboththe
AlgebraIandGeometryEOCscales.BecauseAlgebraIIbuildsontheknowledgeandskillsassessedin
AlgebraI,allAlgebraIIitemswereusedtolinktheAlgebraIassessmentstotheAlgebraIIscale.

Item Administration
AIRsfieldtestenginewasusedtoadministerbothfieldtestandverticallinkingitemsintheembedded
slotsintheonlinetestadministrations.Thefieldtestalgorithmrandomlyassignsboththefieldtest
itemsandthefieldtestitempositionwithintheEFTblock,ensuringthat

Arandomsampleofstudentsisadministeredeachitem;and
Foranygivenitem,thestudentsaresampledwithequalprobability.

Thefieldtestalgorithmyieldsarepresentative,randomizedsampleofstudentresponsesforeachitem.
Thefieldtestalgorithmalsoleadstorandomizationofitempositionandthecontextinwhichitems
appear.Fieldtestingeachiteminmanypositionsandcontextsshouldrendertheresultingstatistics
morerobusttothesefactors.

Linking Analysis
Whenfeasible,itisdesirabletoestablishlinkagesusingbothconcurrentcalibrationsandchainlinking
approachestoensurethatresultsareconsistentacrossmethods.Animportantadvantageofchain
linkingapproachesisthat,becauseitemresponsetheory(IRT)calibrationsproceedbyestablishingthe
withingradescale,theachievementconstructintendedbytheblueprintandenactedintheoperational
testformispreserved.Unfortunately,however,ateachstepinthelinkingchain,thelinkingerror
accumulates,sothatlinkingconstantsforgradesmoredistantfromthereferencegradearelessprecise
thanarelinkingconstantsforgradesincloserproximitytothereferencegrade.Concurrentcalibrations
donotaccruelinkingerroracrossgradelevels,sothatlinkingconstantsaresimilarlyprecisebetweenall
gradelevels.However,thecalibrationsresultingfromthisapproachmeasuretheconstructthatis
commonacrossthelinkedassessments,whichmaybedifferentfromtheintendedachievement
constructateachgradelevel,especiallyforsubjectssuchasmathematicswheretheassessedconstruct
maychangemarkedlyacrossgradelevels.Generally,bothapproachestendtoconvergetoproduce
verticalscalesthatoperatesimilarly(Ito,Sykes,andYao,2008;Karkee,Lewis,Hoskens,Yao,andHaug,
2003),andweviewconvergenceasevidencefortherobustnessoftheverticalscale.
Final Linking Set.Tofacilitatethedevelopmentofaverticalscalethatwillbesensitivetostudent
growthovertime,wefirstevaluatedtheperformanceofverticallinkingitemsbetweenthegradelevels
inwhichtheywereadministeredtoidentifyanyitemsthatweremoredifficultforstudentsinthe
intendedgradethantheywereforstudentsinthelowergrade.Formath,itemsthatshowedproportion
correctscoreslowerintheintendedgradethaninthelowergradeweredroppedfromthefinalvertical
linkingset.Thisresultedindroppingonaveragejustovertwoitemsperlinkingset,withamaximumof
sixitemsdroppedforthelinkagebetweengrade6andgrade7mathassessments.
Forreading,theproportioncorrectvaluesacrossgradesweremuchcloser,especiallyatthehigher
gradelevels,sothateliminationofallitemswheretheproportioncorrectvalueinthelowergrade

J2

AmericanInstitutesforResearch

AzMERIT

StandardSettingTechnicalReport

exceededthehighergradewouldresultindroppingmoreitemsfromtheverticallinkingsetthanwould
bedesirableforexecutingarobustequatingdesign.Thus,wemodifiedtheruleforreadingtoexclude
fromtheverticallinkingsetthoseitemswhichshowedproportioncorrectvaluesmorethantwo
standarderrorsbeyondtheaveragestandarderrorforthetotallinkingset(i.e.,itemsthatwerereliably
lessdifficultatthelowergrade).Thisapproachallowedustoidentifyafinalsetoflinkingitemsthat
wouldmaximizedetectionofgrowth,whileretainingsufficientitemstoestablishastronglinkage
betweenthegradelevelassessments.

Table 1. Number of Items Dropped and Remaining in the Final Vertical Linking Set

Linkage
G3G4
G4G5
G5G6
G6G7
G7G8
G8MAlgI
G8ELAG9ELA
G8MGeometry
G9ELAG10ELA
AlgIAlgII
G10ELAG11ELA

Math
DroppedItems
FinalVLSet
1
44
0
45
1
46
6
41
3
47

ELA
DroppedItems
1
3
0
5
2

FinalVLSet
42
46
47
39
46

28

11

30

31

39

32

10

35

ChainLinking. Thechainlinkingapproachproceedsfromthewithingradeitemparametersidentified
intheinitialcalibrationsoftheoperationalandembeddedfieldtestitems.Becauseoperationaltest
itemsateachgradewereadministeredintheEFTslotsinthegradebelow,eachiteminthevertical
linkingsethastwosetsofitemparameters:ongrade(g)andbelowgrade(g1).Thechainlinking
proceedsbyidentifyingthelinkingconstantsnecessarytoplacethebelowgradeitemparameterson
theongradescalefortheitemsinthefinalverticallinkingset.Thelinkingconstantforeachgradewas
definedasthemeandifferenceoftheitemdifficultyestimatesforthelinkingitemsbetweenthelinked
grades.Thechainlinkingbeganbyplacingthegrade3itemparametersonthegrade4scaleforboth
mathandELAandproceededupwards.FormathEOCassessments,thegrade8mathscalewaslinkedto
boththeAlgebraIandGeometryscales,andtheAlgebraIscalewaslinkedtotheAlgebraIIscale.
Concurrent Calibration. Averticalscaleforeachsubjectareawasalsoestablishedbycalibrating
simultaneouslyallitemsinthefinalverticallinkingset.Aswiththewithingradecalibrations,parameters
wereestimatedusingWinsteps.Tocompareresultsfromthechainlinkingandconcurrentcalibrations,
theconcurrentcalibrationswereplacedonthegrade3referencescale.
Table2showstheverticallinkingconstantsresultingfromchainlinkingthewithingradescalesaswell
asfromconcurrentlycalibratingitemsfromacrossgradelevels.Thelinkingconstantsareappliedto
theirrespectivewithingradescaletoplaceallitemparametersonthegrade3referencescale.Tomore
directlyexaminethemagnitudeofgainsacrossgradelevelassessments,Table3showsthedifference
betweenlinkingconstantsbetweeneachofthegradelevelsassessed.Relativegainsarealso

J3

AmericanInstitutesforResearch

AzMERIT

StandardSettingTechnicalReport

representedgraphicallyinFigure1andFigure2formathandELA,respectively,whichplotthelinking
constantsacrossgradelevelassessments.Asthelinkingconstantsindicate,formaththereisrelatively
largeandsteadygrowthacrossthegradelevelandendofcourseassessments.FortheELAassessments,
thecrossgradegainsaremoremodest,andtendtodiminishinthehighergradelevels.

Table 2. Vertical Linking Constants Resulting from Chain-Linking Within Grade Scales
and Concurrent Calibration of Items Across Grades.

Linkage
G3G4
G4G5
G5G6
G6G7
G7G8
G8MAlgI
G8ELAG9ELA
G8MGeometry
G9ELAG10ELA
AlgIAlgII
G10ELAG11ELA

VerticalLinkingConstants
Mathematics
ELA
ChainLinked
Concurrent
ChainLinked
Concurrent
1.32
1.30
0.18
0.16
2.75
2.67
0.81
0.78
3.90
3.73
1.19
1.15
4.48
4.28
1.44
1.39
5.69
5.39
1.76
1.70
6.07

5.76

1.97

1.88

7.15

6.86

2.12

1.98

7.81

7.45

2.32

2.16

Table 3. Linking Constant Differences between each of the Grade Level Scales.

Linkage
G3G4
G4G5
G5G6
G6G7
G7G8
G8MAlgI
G8ELAG9ELA
G8MGeometry
G9ELAG10ELA
AlgIAlgII
G10ELAG11ELA

VerticalLinkingConstantDifferences
Mathematics
Reading
ChainLinked
Concurrent
ChainLinked
Concurrent
1.32
1.3
0.18
0.16
1.43
1.37
0.63
0.62
1.15
1.06
0.38
0.37
0.58
0.55
0.25
0.24
1.21
1.11
0.32
0.31
0.38

0.37

0.21

0.18

1.08

1.10

0.15

0.10

0.66

0.59

0.20

0.18

Linkingconstantsresultingfromthechainlinkingandconcurrentcalibrationapproacharequite
consistent,indicatingthatbothapproachesconvergeonacommongrowthscale.Althoughthelinking
constantsderivedfromtheconcurrentcalibrationapproachmaybeconsideredmoreprecise,thechain
linkingmethodpreservesthewithingrademeasurementconstruct,andwasthereforeselectedasa

J4

AmericanInstitutesforResearch

AzMERIT

StandardSettingTechnicalReport

preliminaryverticalscaleforthepurposeofrecommendingperformancestandards.Wenotethat
ordereditembooksforthestandardsettingworkshopwerebasedonthewithingradescales,soany
modificationstotheverticalscalewillnotimpacttherecommendedperformancestandards.
Theverticallinkingconstantsalsoindicatemuchgreatergrowthacrossgradesandhighschoolcourses
formathematicsthanisobservedforELA.Inmathematics,growthisontheorderofaboutonestandard
deviationperyear,withtheexceptionofgrade6tograde7,whichshowedjustoverahalfstandard
deviationgain.Similarhalfstandarddeviationgainswereobservedbetweengrade8andAlgebraI,
whichsomestudentstakeconcurrently,andbetweencourseworkinAlgebraIandAlgebraII.Gainsin
ELAarelesspronounced,withsomewhatlargergainsintheelementaryschoolyears,withgrowth
attenuatinginthehighschoolgrades.

Figure 1. Vertical Linking Constants Estimated from Chain-Linking and Concurrent


Calibrations: Mathematics

9.00
8.00
7.00
6.00
5.00
4.00

Conccurent

3.00

ChainLinked

2.00
1.00
0.00
34

45

56

67

78

J5

8AlgI 8Geo

Alg
IAlgII

AmericanInstitutesforResearch

AzMERIT

StandardSettingTechnicalReport

Figure 2. Vertical Linking Constants Estimated from Chain-Linking and Concurrent


Calibrations: ELA

2.50
2.00
1.50
Concurrent
1.00

ChainLinked

0.50
0.00
34

45

56

67

78

89

910

1011

Dimensionality
Acentralconcerninthedevelopmentofaverticalscaleiswhetherchangesintheassessedconstruct
acrossgradeslimitsthecomparabilityoftestscores.Wedonote,however,thatbecausethelinkages
werelimitedtooneadjacentgrade,thelinkingdesignsimplyallowstheadjacentgrade(g1)item
parameterstoberepresentedonthetargetgrade(g)scale,withcomparisonsoftestscoresleadingto
veryspecificinferences.Inthiscase,theadjacentgrade(g1)scalescoresrepresentthebaselinefor
measuringtheacquisitionofcontenttaughtinsubsequenttargetgrade(g).Inthissense,thederived
verticalscaleisnotadevelopmentalscalemeasuringprogressonacommonunderlyingconstruct,but
ratherasequenceoflinkedadjacentgradescales,withinferencesaboutstudentgrowthlimitedto
directlylinkedscales.
Nevertheless,theconcurrentcalibrationsaffordanopportunitytoevaluatethereasonablenessofthe
unidimensionalityassumptionunderlyingthemeasurementmodel,andthuswhetheracommon
reportingscaleacrossgradelevelsandcoursescanbesupported.
Toevaluatethedegreetowhichmultidimensionalityispresentintheverticallinkingitems,Winsteps
providesprincipalcomponentsanalysisofresidualsfromthecommonunderlyingachievement
dimension.Thevarianceaccountedforbythefirstprincipalcomponentoftheresiduals,thesecondary
dimension,indicatesthepresenceofmultidimensionality.Theeigenvaluesandthepercentofvariance
explainedbycommonfactorunderlyingitemresponsesitemsandthesecondarydimension,
representingthecommonvariationunderlyingtheresiduals,areshowninTable4.Astheresultsofthis
comparisonindicate,evidenceformultidimensionalityisweak,andtheassumptionofunidimensionality
intheachievementitemsacrossgradesissupported.

J6

AmericanInstitutesforResearch

AzMERIT

StandardSettingTechnicalReport

Table 4. Eigenvalues and Percent of Variance Accounted for by Items and the First
Contrast.
DimensionalityComponent
Rawvarianceexplainedbyitems
Unexplainedvariancein1stcontrast
Rawvarianceexplainedbyitems
Unexplainedvariancein1stcontrast

Eigenvalue
ELA
97.1
2.2
Math
121.4
2.1

PercentofVarianceAccountedFor
16.3%
0.4%
18.9%
0.3%

Winstepsalsoplotstheresidualsoftheitemswithrespecttotheunderlyingachievementdimension
andidentifiesclustersofitemsbasedonthiscontrasttoassistintheidentificationofsourcesof
multidimensionality.ForELA,theconstrastisdefinedbyaclusterofwritingdimensionscores,across
gradelevels,versusotheritemtypesintheELAassessment.Althoughevidenceforthe
unidimensionalityofELAtestscoresisstrong,thatreadingandwritingitemswouldexhibitsomedegree
ofmultidimensionalityisnotsurprising,andcouldrepresenteitherdifferencesintheassessedconstruct
ormethodofresponse,orboth.Disattentuatedcorrelationcoefficientsamongtheabilityestimates
derivedfromitemscomprisingthethreeclustersareuniformlyhigh,withvaluesof0.93,1.00,and0.98,
indicatingthattheitemsinthethreeclustersaremeasuringacommonunderlyingdimension.
Formath,itemclustersidentifiedbyresidualsaredefinedbytheequationitemtypesversusotheritems
inthemathassessments,andtheitemstypesdefiningthecontrastextendacrossgradelevels.This
contrastcouldrepresentamethodfactor,giventhedifferentresponsemodeforequationitemtypes,or
couldevenreflectdifferencesinthecognitiveprocessesassessedbytheseitemtypes.Nevertheless,
disattenuatedcorrelationsofabilityestimatesbasedontheitemsidentifiedinthethreeclustersareall
1.0,indicatingthatitemsineachclusteraremeasuringthesameunderlyingconstruct.

Reporting on the Vertical Scale


StandardsettingworkshopswereconductedtheweekofJuly13,2015torecommendtotheArizona
StateBoardofEducationasetofperformancestandardstoclassifystudentperformanceonthe
AzMERITassessments.Arizonaeducatorsreviewedtheperformanceleveldescriptorsandusedthe
BookmarkingmethodtoidentifythelevelofachievementontheAzMERITconsistentwithstudentswho
justbarelyqualifyforentryintoeachperformancelevel.
ADEintendstoreportstudentperformanceontheAzMERITontheverticalscalederivedfromthechain
linkedwithingradescales.Becauseabilityestimatesofextremelylowandhighscoringstudentsareless
precise,testscoresforverylowandhighperformingstudentswillbemorepronetofluctuateovertime.
Tominimizescalescoreinstabilityforverylowandhighscoringstudents,abilityestimateswillbe
truncatedat+3.5onthewithingradescalebeforebeingtransformedtotheverticallylinkedscale.

J7

AmericanInstitutesforResearch

AzMERIT

StandardSettingTechnicalReport

StudentabilityestimateswillthenbetransformedfromtheverticallylinkedRaschthetascaletothe
subjectspecificAzMERITreportingscale:

2500

30

3500

30

ApplyingtheAzMERITscalescoretransformationstotheperformancestandardsrecommendedbythe
workshoppanelsresultsinthesystemofscalescorerangesforeachoftheAzMERITperformancelevel
classificationsidentifiedinTable5forELAandTable6formath.

Table 5. ELA Scale Score Ranges for AzMERIT Performance Level Classification

ELA
Assessment
Grade3

Minimally
Proficient
23952496

Partially
Proficient
24972508

Proficient
25092540

Highly
Proficient
25412605

Grade4

24002509

25102522

25232558

25592610

Grade5

24192519

25202542

25432577

25782629

Grade6

24312531

25322552

25532596

25972641

Grade7

24382542

25432560

25612599

26002648

Grade8

24482550

25512571

25722603

26042658

Grade9

24542554

25552576

25772605

26062664

Grade10

24582566

25672580

25812605

26062668

Grade11

24652568

25692584

25852607

26082675

Table 6. Math Scale Score Ranges for AzMERIT Performance Level Classification

Math
Assessment
Grade3

Minimally
Proficient
33953494

Partially
Proficient
34953530

Proficient
35313572

Highly
Proficient
35733605

Grade4

34353529

35303561

35623605

36063645

Grade5

34783562

35633594

35953634

36353688

Grade6

35123601

36023628

36293662

36633722

Grade7

35293628

36293651

36523679

36803739

Grade8

35663649

36503672

36733704

37053776

AlgebraI

35773660

36613680

36813719

37203787

Geometry

36093672

36733696

36973742

37433819

AlgebraII

36293689

36903710

37113750

37513839

J8

AmericanInstitutesforResearch

AzMERIT

StandardSettingTechnicalReport

Summary
Verticalscalingwasaccomplishedboththroughchainlinkingofwithingradescales,whichhasthe
advantageofpreservingthemeasurementconstructateachgrade,butmayleadtolessprecisevertical
linkingconstantssincelinkingerroraccumulatesacrosslinkages,aswellasthroughconcurrent
calibrationofallverticallinkingitems,whichmayyieldmorepreciseverticallinkingconstants,butmay
notpreservethemeasurementconstructassessedwithineachgradelevelandEOCassessment.Both
methodsconvergedtoproducehighlycomparableverticallinkingconstants.Withaneyetopreserving
themeasurementconstructineachofthewithingradescales,thepreliminaryverticalscalewas
developedusingresultsfromthechainlinkingapproach.
Theconcurrentcalibrationsaffordedtheopportunitytoevaluatetheverticallinkingsetforevidenceof
multidimensionality.ThevarianceaccountedforbyanysecondarydimensionwasweakforboththeELA
andmathassessments.Moreover,whenthebasisforanymultidimensionalitywasinvestigated,it
appearedrelatedtodifferentiationofreadingandwritingintheELAassessment,whichmaybe
expected,anddifferentiationofequationandotheritemtypesinmath,whichcouldbeduetomethod
factorsorpossiblyevendifferentiationintheunderlyingcognitiveprocessesassessed.Ineitherevent,
themeasurementconstructappearstobepreservedacrossthegradelevelassessments.

References
Ito,K.,Sykes,R.C.,&Yao,L.(2008).Concurrentandseparategradegroupslinkingproceduresfor
verticalscaling.AppliedMeasurementinEducation,21,187206.
Karkee,T.,Lewis,D.M.,Hoskens,M.,Yao,L.,&Haug,C.(2003).SeparateversusConcurrentCalibration
MethodsinVerticalScaling.PaperpresentedattheannualmeetingoftheNationalCouncilon
MeasurementinEducation,Chicago,IL.

J9

AmericanInstitutesforResearch

AzMERIT

StandardSettingTechnicalReport

Appendix K Panelist Evaluation Forms

Appendix

AzMERIT

StandardSettingTechnicalReport

DocumentK.OverallWorkshopEvaluations
Please take your time to carefully complete the information below. It is important that you answer this
evaluation thoroughly as the results will be used to improve the Standard Setting process. The
information gathered from this evaluation will be reported in the Standard Setting Technical Report that
will be available to the public.
Panelist ID Number: __________
1. At the end of the workshop,
Strongly
Disagree

Disagree

Agree

Strongly
Agree

a.

I understood the purpose of this standard setting


workshop.

b.

The procedures used to recommend performance


standards were fair and unbiased.

c.

The training provided me with the information I


needed to recommend performance standards.

d.

Taking the online assessment helped me to better


understand what students need to know and be
able to do to answer each item.

e.

The Performance Level Descriptors (description


of what students within each performance level
are expected to know and be able to do) provided
a clear picture of expectations for student
achievement at each level.

f.

I was able to develop an understanding of the


knowledge and skills demonstrated by students
who are just barely described by the
Performance Level Descriptors.

g.

I understood how to review each page in the


Ordered Item Book (OIB) to determine what
students must know and be able to do to answer
each item correctly.

h.

I was able to interpret having a two-thirds


likelihood of answering an item correctly as
indicating mastery.

i.

I understood how to place my bookmarks.

j.

I found the benchmark data and discussions


helpful in my decisions about where to place my
bookmarks.

K-1

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Strongly
Disagree

Disagree

Agree

Strongly
Agree

k.

I found the panelist agreement data (room


medians and individual bookmark placements)
and discussion helpful in my decisions about
where to place my bookmarks.

l.

I found the impact data (percentage of students


that would achieve at the level indicated by the
OIB page) and discussions helpful in my
decisions about where to place my bookmarks.

m.

I felt comfortable expressing my opinions


throughout the workshop.

n.

Everyone was given the opportunity to express


his or her opinions throughout the workshop.

2. Please rate the clarity of the following components of the workshop.


Very
Unclear

Somewhat Somewhat
Unclear
Clear

Very
Clear

a.

Instructions provided by the Workshop Leader

b.

Performance Level Descriptors (PLDs)

c.

Ordered Item Booklet (OIB)

d.

Panelist agreement data

e.

Impact data (percentage of students that would


achieve at the level indicated by the OIB page)

K-2

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

3. How important was each of the following factors in your placement of the bookmarks?
Not
Important

Somewhat
Important

Very
Important

a.

Performance Level Descriptors (PLDs)

b.

Your perception of the difficulty of the items

c.

Your experiences with students

d.

Discussions with other panelists

e.

External benchmark data

f.

Room agreement data (room medians and


individual bookmark placements)

g.

Impact data (percentage of students that would


achieve at the level indicated by the OIB page)

h.

Interpolated page numbers provided for adjacent


grades

4. How appropriate was the amount of time you were given to complete the following components of
the standard setting process?
Too
Little

About
Right

Too
Much

a.

Large group orientation

b.

Experiencing the online assessment

c.

Review of the Performance Level Descriptors

d.

Discussion of skills demonstrated by students


who are just barely described by each PLD

e.

Review of the Ordered Item Booklet (OIB)

f.

Placement of your bookmarks in each round

g.

Round 1 discussion

K-3

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

5. Please read the following statement carefully and indicate your response.
Strongly
Disagree

Disagree

Agree

Strongly
Agree

a.

I am confident that students classified as Proficient


demonstrate a fundamental understanding of and ability
to apply the content knowledge and skills needed to be
on track towards Arizonas College and Career
Readiness Standards (ACCRS) in Mathematics and
English Language Arts Standards. (Level 3)

b.

I am confident that students classified as Partially


Proficient demonstrate a partial understanding of and
ability to apply the content knowledge and skills needed
to be on track towards Arizonas College and Career
Readiness Standards (ACCRS) in Mathematics and
English Language Arts Standards. (Level 2)

c.

I am confident that students classified as Highly


Proficient demonstrate an advanced understanding of
and ability to apply the content knowledge and skills
needed to be on track towards Arizonas College and
Career Readiness Standards (ACCRS) in Mathematics
and English Language Arts Standards. (Level 4)

6. What suggestions do you have to improve the training or standard setting process?

K-4

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

7. Do you have any additional comments? Please be specific.

Thank you for participating in the Standard Setting Workshop!

K-5

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Appendix L Recommend Performance Standards by


Round

Appendix

AzMERIT

StandardSettingTechnicalReport

TableL.RecommendedCutsByRound
Form

Round
Round1

Grade3ELA

Round2
Moderation
Final
Round1

Grade4ELA
Round2
Final
Round1
Grade5ELA
Round2
Final
Round1
Grade6ELA
Round2
Final
Round1
Grade7ELA
Round2

Grade8ELA

Final
Round1

Panelist
Minimum
Maximum
Median
Minimum
Maximum
Median

Minimum
Maximum
Median
Minimum
Maximum
Median

Minimum
Maximum
Median
Minimum
Maximum
Median

Minimum
Maximum
Median
Minimum
Maximum
Median

Minimum
Maximum
Median
Minimum
Maximum
Median

Minimum

PartiallyProficient
16
19
18
16
19
18

18
3
33
19
15
26
19
19
11
15
15
14
15
15
15
9
21
14
13
26
16
16
15
20
18
15
20
18
18
13

L-1

Proficient
22
26
25
24
25
24
25
25
13
58
33
25
38
32
32
28
33
32
32
32
32
32
21
47
28
28
46
30
30
34
39
36
34
39
36
36
30

HighlyProficient
45
50
49
42
49
49

49
52
63
57
50
59
57
57
46
64
53
52
57
53
53
36
64
51
51
61
58
58
58
63
60
58
63
61
61
54

American Institutes for Research

AzMERIT

Form

StandardSettingTechnicalReport

Round

Round2
Final
Round1
Grade9ELA
Round2
Final
Round1

Grade10ELA

Round2
Moderation
Final
Round1

Grade11ELA

Round2
Moderation
Final
Round1

Grade3Math
Round2
Final

Grade4Math

Round1
Round2

Panelist
Maximum
Median
Minimum
Maximum
Median

Minimum
Maximum
Median
Minimum
Maximum
Median

Minimum
Maximum
Median
Minimum
Maximum
Median

Minimum
Maximum
Median
Minimum
Maximum
Median

Minimum
Maximum
Median
Minimum
Maximum
Median

Minimum
Maximum
Median
Minimum

PartiallyProficient
33
17
16
28
19
19
15
18
17
15
18
17
17
7
16
13
12
15
14
13
13
6
15
13
10
16
12
13
13
6
11
10
6
10
10
10
7
20
9
8

L-2

Proficient
46
32
36
42
38
38
27
35
32
27
35
32
32
21
45
27
26
34
32

32
20
46
29
23
31
29

29
29
35
33
30
35
33
33
23
37
32
31

HighlyProficient
63
59
56
65
62
62
51
59
56
51
59
56
56
45
58
53
52
56
53
59
59
44
61
52
47
56
52

52
50
56
52
51
53
52
52
44
61
56
54

American Institutes for Research

AzMERIT

Form

StandardSettingTechnicalReport

Round

Final
Round1
Grade5Math
Round2
Final
Round1
Grade6Math
Round2
Final
Round1
Grade7Math
Round2
Final
Round1
Grade8Math
Round2
Final
Round1
AlgebraI
Round2
Final
Geometry

Round1

Panelist
Maximum
Median

Minimum
Maximum
Median
Minimum
Maximum
Median

Minimum
Maximum
Median
Minimum
Maximum
Median

Minimum
Maximum
Median
Minimum
Maximum
Median

Minimum
Maximum
Median
Minimum
Maximum
Median

Minimum
Maximum
Median
Minimum
Maximum
Median

Minimum
Maximum

PartiallyProficient
14
10
10
4
10
6
4
6
4
4
3
21
9
7
13
9
9
10
11
11
10
14
11
11
11
17
15
13
25
15
15
15
19
17
16
19
17
17
11
20

L-3

Proficient
36
35
35
26
30
27
26
30
27
27
19
30
26
24
26
26
26
29
33
30
29
30
30
30
24
33
28
28
30
29
29
30
37
30
31
38
33
33
20
31

HighlyProficient
61
58
58
51
58
52
51
58
52
52
42
51
46
45
47
46
46
44
48
46
45
46
46
46
39
52
44
44
51
47
47
51
57
54
53
57
56
56
30
53

American Institutes for Research

AzMERIT

Form

StandardSettingTechnicalReport

Round

Round2
Final
Round1
AlgebraII
Round2
Final

Panelist
Median
Minimum
Maximum
Median

Minimum
Maximum
Median
Minimum
Maximum
Median

PartiallyProficient
13
12
18
16
16
9
30
15
14
16
15
15

L-4

Proficient
21
30
31
30
30
27
43
29
27
33
29
29

HighlyProficient
45
45
54
52
52
48
57
51
48
56
49
49

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Appendix M Convergence of Bookmarks across


Rounds

Appendix

AzMERIT

StandardSettingTechnicalReport

FigureM1.ConvergenceofBookmarksacrossRoundsGrade3ELA
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.
55

Grade3ELA
50

45

40

OIBPageNumber

35

30

25

20

15

10

Round1MedianBookmarkPlacement

Round2MedianBookmarkPlacement

Table1HighlyProficient

48

48

Table2HighlyProficient

48

47

Table3HighlyProficient

49

49

Table1Proficient

24

24

Table2Proficient

25

24

Table3Proficient

25

25

Table1PartiallyProficient

17

17

Table2PartiallyProficient

16

18

Table3PartiallyProficient

18

19

M-1

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureM2.ConvergenceofBookmarksacrossRoundsGrade4ELA
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.
65

Grade4ELA

60
55
50

OIBPageNumber

45
40
35
30
25
20
15
10
5
0

Round1MedianBookmarkPlacement

Round2MedianBookmarkPlacement

Table1HighlyProficient

58

53

Table2HighlyProficient

54

57

Table3HighlyProficient

58

58

Table1Proficient

34

32

Table2Proficient

33

33

Table3Proficient

23

32

Table1PartiallyProficient

21

21

Table2PartiallyProficient

18

17

Table3PartiallyProficient

19

19

M-2

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureM3.ConvergenceofBookmarksAcrossRoundsGrade5ELA
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.
60

Grade5ELA

55
50
45

OIBPageNumber

40
35
30
25
20
15
10
5
0

Round1MedianBookmark
Placement

Round2MedianBookmark
Placement

Table1HighlyProficient

55

55

Table2HighlyProficient

53

53

Table3HighlyProficient

53

53

Table1Proficient

33

32

Table2Proficient

32

32

Table3Proficient

28

32

Table1PartiallyProficient

13

15

Table2PartiallyProficient

15

15

Table3PartiallyProficient

15

15

M-3

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureM4.ConvergenceofBookmarksAcrossRoundsGrade6ELA
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.
65

Grade6ELA

60
55
50

OIBPageNumber

45
40
35
30
25
20
15
10
5
0

Round1MedianBookmark
Placement

Round2MedianBookmark
Placement

Table1HighlyProficient

48

60

Table2HighlyProficient

58

58

Table3HighlyProficient

48

54

Table1Proficient

28

37

Table2Proficient

29

30

Table3Proficient

21

30

Table1PartiallyProficient

20

26

Table2PartiallyProficient

11

16

Table3PartiallyProficient

14

14

M-4

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureM5.ConvergenceofBookmarksAcrossRoundsGrade7ELA
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.
70

Grade7ELA

65
60
55
50

OIBPageNumber

45
40
35
30
25
20
15
10
5
0

Round1MedianBookmark
Placement

Round2MedianBookmark
Placement

Table1HighlyProficient

62

62

Table2HighlyProficient

60

61

Table3HighlyProficient

58

58

Table1Proficient

38

38

Table2Proficient

35

35

Table3Proficient

36

36

Table1PartiallyProficient

16

16

Table2PartiallyProficient

18

18

Table3PartiallyProficient

18

18

M-5

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureM6.ConvergenceofBookmarksacrossRoundsGrade8ELA
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.
70

Grade8ELA

65
60
55
50

OIBPageNumber

45
40
35
30
25
20
15
10
5
0

Round1MedianBookmark
Placement

Round2MedianBookmark
Placement

Table1HighlyProficient

55

63

Table2HighlyProficient

62

62

Table3HighlyProficient

61

60

Table1Proficient

31

38

Table2Proficient

44

40

Table3Proficient

32

36

Table1PartiallyProficient

16

17

Table2PartiallyProficient

30

27

Table3PartiallyProficient

16

16

M-6

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureM7.ConvergenceofBookmarksacrossRoundsGrade9ELA
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.
65

Grade9ELA

60
55
50

OIBPageNumber

45
40
35
30
25
20
15
10
5
0

Round1MedianBookmark
Placement

Round2MedianBookmark
Placement

Table1HighlyProficient

55

55

Table2HighlyProficient

56

56

Table3HighlyProficient

58

58

Table1Proficient

32

32

Table2Proficient

30

32

Table3Proficient

32

32

Table1PartiallyProficient

18

18

Table2PartiallyProficient

17

17

Table3PartiallyProficient

17

17

M-7

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureM8.ConvergenceofBookmarksacrossRoundsGrade10ELA
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.
60

Grade10ELA
55
50
45

OIBPageNumber

40
35
30
25
20
15
10
5
0

Round1MedianBookmark
Placement

Round2MedianBookmark
Placement

Table1HighlyProficient

53

53

Table2HighlyProficient

54

53

Table3HighlyProficient

52

52

Table1Proficient

29

32

Table2Proficient

26

31

Table3Proficient

32

33

Table1PartiallyProficient

14

14

Table2PartiallyProficient

13

15

Table3PartiallyProficient

12

13

M-8

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureM9.ConvergenceofBookmarksacrossRoundsGrade11ELA
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.
60

Grade11ELA

55
50
45

OIBPageNumber

40
35
30
25
20
15
10
5
0

Round1MedianBookmark
Placement

Round2MedianBookmark
Placement

Table1HighlyProficient

52

53

Table2HighlyProficient

56

51

Table3HighlyProficient

51

51

Table1Proficient

25

26

Table2Proficient

31

28

Table3Proficient

30

29

Table1PartiallyProficient

13

13

Table2PartiallyProficient

15

15

Table3PartiallyProficient

11

12

M-9

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureM10.ConvergenceofBookmarksacrossRoundsGrade3Math
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.
60

Grade3Math

55
50
45

OIBPageNumber

40
35
30
25
20
15
10
5
0

Round1MedianBookmark
Placement

Round2MedianBookmark
Placement

Table1HighlyProficient

52

52

Table2HighlyProficient

52

52

Table3HighlyProficient

54

53

Table1Proficient

35

33

Table2Proficient

33

33

Table3Proficient

30

32

Table1PartiallyProficient

10

10

Table2PartiallyProficient

11

10

Table3PartiallyProficient

10

M-10

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureM11.ConvergenceofBookmarksacrossRoundsGrade4Math
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.
65

Grade4Math 60
55
50

OIBPageNumber

45
40
35
30
25
20
15
10
5
0

Round1MedianBookmark
Placement

Round2MedianBookmark
Placement

Table1HighlyProficient

58

58

Table2HighlyProficient

55

57

Table3HighlyProficient

57

57

Table1Proficient

34

35

Table2Proficient

23

35

Table3Proficient

29

34

Table1PartiallyProficient

Table2PartiallyProficient

Table3PartiallyProficient

12

11

M-11

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureM12.ConvergenceofBookmarksacrossRoundsGrade5Math
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.
60

Grade5Math
55
50
45

OIBPageNumber

40
35
30
25
20
15
10
5
0

Round1MedianBookmarkPlacement

Round2MedianBookmarkPlacement

Table1HighlyProficient

52

52

Table2HighlyProficient

53

53

Table3HighlyProficient

52

52

Table1Proficient

27

27

Table2Proficient

28

27

Table3Proficient

30

30

Table1PartiallyProficient

Table2PartiallyProficient

Table3PartiallyProficient

10

M-12

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureM13.ConvergenceofBookmarksacrossRoundsGrade6Math
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.
50

Grade6Math
45

40

OIBPageNumber

35

30

25

20

15

10

Round1MedianBookmark
Placement

Round2MedianBookmark
Placement

Table1HighlyProficient

47

46

Table2HighlyProficient

46

46

Table3HighlyProficient

46

46

Table1Proficient

27

26

Table2Proficient

25

26

Table3Proficient

26

26

Table1PartiallyProficient

Table2PartiallyProficient

Table3PartiallyProficient

13

13

M-13

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureM14.ConvergenceofBookmarksacrossRoundsGrade7Math
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.
50

Grade7Math
45

40

OIBPageNumber

35

30

25

20

15

10

Round1MedianBookmarkPlacement

Round2MedianBookmarkPlacement

Table1HighlyProficient

46

46

Table2HighlyProficient

46

46

Table3HighlyProficient

46

46

Table1Proficient

29

29

Table2Proficient

31

30

Table3Proficient

30

30

Table1PartiallyProficient

11

11

Table2PartiallyProficient

11

11

Table3PartiallyProficient

11

11

M-14

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureM15.ConvergenceofBookmarksacrossRoundsGrade8Math
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.
55

Grade8Math

50
45
40

OIBPageNumber

35
30
25
20
15
10
5
0

Round1MedianBookmark
Placement

Round2MedianBookmark
Placement

Table1HighlyProficient

44

47

Table2HighlyProficient

46

47

Table3HighlyProficient

50

49

Table1Proficient

27

29

Table2Proficient

29

29

Table3Proficient

28

28

Table1PartiallyProficient

14

15

Table2PartiallyProficient

15

15

Table3PartiallyProficient

15

16

M-15

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureM16.ConvergenceofBookmarksacrossRoundsAlgebraI
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.
60

AlgebraI

55
50
45

OIBPageNumber

40
35
30
25
20
15
10
5
0

Round1

Round2

Table1HighlyProficient

56

56

Table2HighlyProficient

54

56

Table3HighlyProficient

52

56

Table1Proficient

36

37

Table2Proficient

31

33

Table3Proficient

30

33

Table1PartiallyProficient

17

17

Table2PartiallyProficient

16

17

Table3PartiallyProficient

18

17

M-16

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureM17.ConvergenceofBookmarksacrossRoundsGeometry
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.
60

Geometry

55
50
45

OIBPageNumber

40
35
30
25
20
15
10
5
0

Round1MedianBookmark
Placement

Round2MedianBookmark
Placement

Table1HighlyProficient

45

52

Table2HighlyProficient

38

50

Table3HighlyProficient

53

53

Table1Proficient

21

30

Table2Proficient

22

30

Table3Proficient

26

31

Table1PartiallyProficient

13

18

Table2PartiallyProficient

15

16

Table3PartiallyProficient

13

14

M-17

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

FigureM18.ConvergenceofBookmarksacrossRoundsAlgebraII
Thefigureillustrateshowvariabilityinpanelistsbookmarkdecisionschangedfromthefirsttothe
secondroundanddisplaysthemedianbookmarkforeachtable,fromRound1ofbookmarkplacement
toRound2ofbookmarkplacement.

AlgebraII

60
55
50
45

OIBPageNumber

40
35
30
25
20
15
10
5
0

Round1MedianBookmark
Placement

Round2MedianBookmark
Placement

Table1HighlyProficient

48

48

Table2HighlyProficient

50

50

Table3HighlyProficient

57

55

Table1Proficient

29

29

Table2Proficient

32

31

Table3Proficient

28

28

Table1PartiallyProficient

12

15

Table2PartiallyProficient

16

16

Table3PartiallyProficient

16

15

M-18

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Appendix N Estimated Percentage of Students at


Each Performance Level for Panelist Recommended
Performance Standards, Overall and by Gender and
Ethnicity

Appendix

AzMERIT

StandardSettingTechnicalReport

TableN.EstimatedPercentageofStudentsatEachPerformanceLevelforPanelistRecommendedPerformanceStandards,Overall
andbyGenderandEthnicity
Test

Grade3
ELA

Grade4
ELA

Grade5
ELA

Grade6
ELA

Grade7
ELA

Grade8
ELA

PerformanceLevel
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient

Overall

Female

Male

White

Black

America
n Indian

Asian

Hispani
c

MultiRacial

Other

44
15
31
10
43
19
33
5
37
33
27
3
39
27
30
4
41
26
29
4
40
27
26
6

40
15
33
12
38
19
36
6
32
34
30
3
34
28
34
5
35
28
32
5
35
29
29
8

48
14
29
9
47
18
31
4
42
32
24
2
45
26
26
3
47
25
25
3
45
26
24
5

28
14
40
18
28
19
45
9
23
33
40
5
25
27
42
6
26
26
41
7
27
28
35
10

57
15
24
4
52
19
27
3
49
32
18
1
49
28
21
2
49
27
22
2
52
26
19
3

68
14
16
2
69
16
15
1
64
26
9
0
66
22
11
1
70
20
9
0
65
22
11
1

22
12
43
23
19
15
47
18
18
28
46
8
19
22
49
11
19
21
46
13
20
23
41
16

55
15
25
5
53
19
26
2
47
34
18
1
50
27
21
1
51
27
20
2
49
28
20
3

35
14
37
14
34
20
39
8
28
35
35
3
31
27
38
4
35
29
31
5
34
28
30
8

56
14
24
5
56
17
23
3
53
28
17
1
51
27
20
2
51
25
22
2
45
24
24
7

N-1

American Institutes for Research

AzMERIT

Test

Grade9
ELA

Grade10
ELA

Grade11
ELA

Grade3
Math

Grade4
Math

Grade5
Math

Grade6
Math

StandardSettingTechnicalReport

PerformanceLevel
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient
MinimallyProficient
PartiallyProficient
Proficient
HighlyProficient

Overall

Female

Male

White

Black

America
n Indian

Asian

Hispani
c

MultiRacial

Other

47
26
21
6
49
21
22
8
54
20
17
8
27
31
27
15
29
29
32
10
29
31
27
13
38
30
21
11

41
28
25
7
44
22
25
10
50
21
19
10
27
33
27
14
29
30
32
9
27
32
29
12
36
31
23
11

52
25
18
4
54
19
20
6
58
19
16
7
28
30
27
15
30
28
32
11
30
30
26
14
40
28
20
11

33
28
30
9
36
22
30
13
41
22
23
13
17
28
33
22
17
26
41
16
17
28
35
21
25
30
28
17

56
25
16
3
61
19
16
3
66
18
13
4
39
33
21
7
41
32
23
4
42
33
20
5
52
29
14
5

70
22
8
1
73
16
9
1
78
15
7
1
44
35
17
4
51
30
17
2
47
33
16
3
59
27
12
3

23
20
36
20
25
19
33
24
30
19
25
25
8
20
33
39
10
18
43
30
11
19
32
37
14
22
30
34

56
26
15
2
59
20
17
4
64
19
13
4
34
34
23
9
38
32
26
5
36
34
23
7
47
31
17
5

39
26
27
8
42
22
25
11
47
21
21
11
22
30
30
19
22
30
35
12
23
31
31
15
32
33
23
12

67
18
12
3
62
18
15
6
52
17
19
12
41
32
19
9
45
29
22
4
30
32
25
12
53
28
13
6

N-2

American Institutes for Research

AzMERIT

Test

PerformanceLevel
MinimallyProficient
PartiallyProficient
Proficient
Grade7
Math
HighlyProficient
MinimallyProficient
PartiallyProficient
Proficient
Grade8
Math
HighlyProficient
MinimallyProficient
PartiallyProficient
Proficient
AlgebraI
HighlyProficient
MinimallyProficient
PartiallyProficient
Proficient
Geometry HighlyProficient
MinimallyProficient
PartiallyProficient
Proficient
AlgebraII HighlyProficient

StandardSettingTechnicalReport

Overall

Female

Male

White

Black

America
n Indian

Asian

Hispani
c

MultiRacial

Other

48
22
18
13
43
24
20
13
45
23
23
9
47
24
24
6
47
24
23
6

48
23
18
12
42
26
21
12
42
24
25
9
46
25
24
5
45
26
23
6

48
21
18
14
45
23
19
13
48
22
21
9
48
22
24
7
48
22
22
7

32
24
24
20
29
26
26
19
32
23
30
15
33
25
33
9
35
25
30
10

65
19
11
5
57
24
14
6
58
22
16
4
61
23
15
2
61
21
15
2

72
16
9
3
67
20
10
3
66
20
12
2
66
21
11
1
70
19
9
1

20
18
24
38
17
18
25
40
17
19
36
28
22
18
40
21
18
21
38
22

59
22
13
6
53
24
16
7
53
24
18
5
58
23
17
2
57
24
17
2

45
23
19
14
39
26
21
14
39
22
26
13
41
26
26
7
42
24
24
9

62
19
12
7
55
17
14
14
69
17
11
2
64
19
13
4
67
17
13
3

N-3

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Appendix O Summary of Panelist Evaluations

Appendix

AzMERIT

StandardSettingTechnicalReport

DocumentO.SummaryofPanelistEvaluations
1. At the end of the workshop,
Strongly
Disagree

Disagree

Agree

Strongly
Agree

a.

I understood the purpose of this standard setting


workshop.

73

b.

The procedures used to recommend performance


standards were fair and unbiased.

24

54

c.

The training provided me with the information I


needed to recommend performance standards.

69

d.

Taking the online assessment helped me to better


understand what students need to know and be
able to do to answer each item.

14

64

e.

The Performance Level Descriptors (description


of what students within each performance level
are expected to know and be able to do) provided
a clear picture of expectations for student
achievement at each level.

42

33

f.

I was able to develop an understanding of the


knowledge and skills demonstrated by students
who are just barely described by the
Performance Level Descriptors.

41

38

g.

I understood how to review each page in the


Ordered Item Book (OIB) to determine what
students must know and be able to do to answer
each item correctly.

11

68

h.

I was able to interpret having a two-thirds


likelihood of answering an item correctly as
indicating mastery.

29

50

i.

I understood how to place my bookmarks.

10

69

j.

I found the benchmark data and discussions


helpful in my decisions about where to place my
bookmarks.

71

k.

I found the panelist agreement data (room


medians and individual bookmark placements)
and discussion helpful in my decisions about
where to place my bookmarks.

12

67

l.

I found the impact data (percentage of students


that would achieve at the level indicated by the
OIB page) and discussions helpful in my
decisions about where to place my bookmarks.

23

56

O-1

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Strongly
Disagree

Disagree

Agree

Strongly
Agree

m.

I felt comfortable expressing my opinions


throughout the workshop.

72

n.

Everyone was given the opportunity to express


his or her opinions throughout the workshop.

72

2. Please rate the clarity of the following components of the workshop.


Very
Unclear

Somewhat Somewhat
Unclear
Clear

Very
Clear

a.

Instructions provided by the Workshop Leader

71

b.

Performance Level Descriptors (PLDs)

18

59

c.

Ordered Item Booklet (OIB)

77

d.

Panelist agreement data

75

e.

Impact data (percentage of students that would


achieve at the level indicated by the OIB page)

74

O-2

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

3. How important was each of the following factors in your placement of the bookmarks?
Not
Important

Somewhat
Important

Very
Important

a.

Performance Level Descriptors (PLDs)

17

63

b.

Your perception of the difficulty of the items

17

61

c.

Your experiences with students

16

64

d.

Discussions with other panelists

12

68

e.

External benchmark data

46

33

f.

Room agreement data (room medians and


individual bookmark placements)

42

36

g.

Impact data (percentage of students that would


achieve at the level indicated by the OIB page)

27

51

h.

Interpolated page numbers provided for adjacent


grades

41

36

4. How appropriate was the amount of time you were given to complete the following components of
the standard setting process?
Too
Little

About
Right

Too
Much

a.

Large group orientation

65

14

b.

Experiencing the online assessment

64

c.

Review of the Performance Level Descriptors

68

11

d.

Discussion of skills demonstrated by students


who are just barely described by each PLD

59

17

e.

Review of the Ordered Item Booklet (OIB)

60

19

f.

Placement of your bookmarks in each round

46

34

g.

Round 1 discussion

71

O-3

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

5. Please read the following statement carefully and indicate your response.
Strongly
Disagree

Disagree

Agree

Strongly
Agree

a.

I am confident that students classified as Proficient


demonstrate a fundamental understanding of and ability
to apply the content knowledge and skills needed to be
on track towards Arizonas College and Career
Readiness Standards (ACCRS) in Mathematics and
English Language Arts Standards. (Level 3)

35

44

b.

I am confident that students classified as Partially


Proficient demonstrate a partial understanding of and
ability to apply the content knowledge and skills needed
to be on track towards Arizonas College and Career
Readiness Standards (ACCRS) in Mathematics and
English Language Arts Standards. (Level 2)

34

45

c.

I am confident that students classified as Highly


Proficient demonstrate an advanced understanding of
and ability to apply the content knowledge and skills
needed to be on track towards Arizonas College and
Career Readiness Standards (ACCRS) in Mathematics
and English Language Arts Standards. (Level 4)

30

49

O-4

American Institutes for Research

AzMERIT

StandardSettingTechnicalReport

Appendix P Independent Observer Report to State


Board of Education

Appendix

AzMERIT

StandardSettingTechnicalReport

DocumentP.IndependentObserverReporttoStateBoardofEducation
TO:

FROM:

StateBoardofEducation
JohnWilson,TempeElementarySchools
JoeOReilly,MesaSchools
JayMidyett,AmphitheatreSchools

StandardSettingObservations

RE:

AsrepresentativestothestatesassessmentTechnicalAdvisoryCommittee,wewereinvitedto
observetheAzMERITstandardsetting.Thethreeofusobservedthroughouttheprocess.At
leastoneofusobservedeachgroup,withmostgroupsbeingvisitedbymultipleobservers.The
firsttwodayswereobservedbyallthreeofus,andoneortwoobserverswerethereforthe
lasttwodays.

THESTANDARDSETTINGPROCESS
Therewereeightgroupsofapproximately12teacherseachthatsetthestandards.Eachgroup
wasdividedintothreetablesandworkedindividually,asatableandasawholegroupasthey
wentthroughtheprocess.Elementarygroups(grades4,6,8)setthecutpointsfortheirgrade
andaninterpolatedgradejustbelow(3,5,7).Highschoolgroupssetthecutpointsforallmath
(AlgebraII,Geometry,AlgebraI)orELA(Grades911)usingasimilarprocesswithninthgrade
beinginterpolated.

Asidefromafewhandouts,thematerialsnecessaryforstandardsettingwerealldeliveredon
computerswhichmadetheprocesseasierandallowedsharingoftableandgroupresults
quickerandclearer.Participantsappearedtofindthesystemeasytouseandintuitive.

Teachersstartedwithalargegroupintroductorytraining.Itwasemphasizedthattheywereto
makedecisionsbasedonthePerformanceLevelDescriptorsandtheirprofessionaljudgment.
TheStateBoardgoalsfortheassessmentwerealsosharedwiththem.

Teachersthenbrokeintogroupsandtookthesameassessmentthestudentstook.Theythen
reviewedthePLDsanddevelopedjustbarelyPLDsordescriptionsofastudentwhowasjust
barelyproficientwouldknowandcando.

Teachersthenfamiliarizedthemselveswiththeonlineordereditembooklet(OIB).TheOIBhad
apageforeachquestionwhichshowedthequestionandresponseoptionsaswellasthe
correctanswer.Theitemsconsistedoftheactualtestitemsandotherfieldtestitemstogivea
fullprogressionoftheitemdifficulties.Tohelpwithcontextforthedifficulty/complexityof
eachitem,thecomparablecutpointsforothertests(AIMS,SmarterBalance,NAEP,ACT
CollegeandCareerReadyIndex,and/orPISAdependingongrade)wereprovided.

P1

AmericanInstitutesforResearch

AzMERIT

StandardSettingTechnicalReport

Next,AIRusedthebookmarkmethodtohavegroupsidentifycutpoints.Teachersdetermined
theitematwhichajustbarelyproficient(orbarelypartiallyproficientorhighlyproficient)
studentwouldgetanitemright,butastudentinthelevelbelowwouldnotbelikelytogetit
right.Teachersthengottoseehowtheothertablesandeachparticipantplacedtheir
bookmarksandtheythendiscussedthebookmarks.Theyweretoldtheexpectationis
convergingjudgements,notnecessarilyconsensus,sotherewasnorequirementthatonehad
toagreewiththegroup.

InRound2theteachersagainplacedtheirbookmarks.Beforestartingtheysawimpactdataor
whatpercentof2015studentswhotookthetestwouldfallineachcategorybasedonthe
roundonecutscores.Theywerethenaskedtosettheirbookmarksasecondtime.

DuringlunchontheseconddaySuperintendentDouglasspokebrieflytothegroupthanking
themfortakingtimefromtheirsummervacationandshetoldthemtheyweredoingimportant
workforallofArizona.Herwordssentamessagetotheparticipantsabouthowcrucialtheir
taskwasandhowimportanttheirprofessionaljudgementwastothesuccessofthestandard
setting.

Attheendoftheprocessparticipantsgatheredinalargeroomandwereabletoseewhere
everygrade/subjectplacedtheircutpointsandtheimpactdata.Atthatpoint,vertical
moderationwasscheduled,butitwasnotneededexceptforonehighschoolgroupthatsetthe
tenthgradestandardslightlyhigherthangradeeleven,andtheydiscussedloweringtenth
gradebyoneitem,whichwasmoreforappearancethanasubstantivechange.

Theprocesswasrepeatedforinterpolatedgrades.Eachgroupwasprovidedacutpointfor
theirgradethatwaspredictedusingapsychometric(statistical)analysis.Thetaskforthese
gradeswastoadjust,asneeded,thepredictedcutpointsothatitreflectedthejustbarely
thresholdbasedoncontent.Again,thedecisionwasguidedbythepsychometricpredictionbut
determinedbyprofessionaljudgementandthinkingaboutthejustbarelystudent.

OBSERVATIONS
Theprocesswasclear,wellorganizedandlogical.Teachersweretrainedtomakedecisions
basedonthePerformanceLevelDescriptorsandthecontentstudentsaresupposedtoknow.
TheywerealsoguidedbytheBoardsgoalsofhavingteststhatcanbecomparedtoother
assessmentsandthatreflectcollegeandcareerreadiness.

Theteachersdrovethedecisions,anditappearedtheyreliedheavilyonthePLDsandthe
Boardsgoals.Teachersweregivenalotofspacetodiscussandmaketheirownchoices.They
werenottoldthatthecutscoreshavetobeatcutpointsforothertestsbuttheyweretherefor
context.InthetrainingtheyweretoldYourdecisionshouldbebasedonyourprofessional
opinion.Therelatedtestsaretogiveyouacontextforyourchoice.Whengivenimpactscores
theyweretoldtheyreallyneedtomakedecisionsbasedoncontent,notbasedonthese
[impact]scores.

P2

AmericanInstitutesforResearch

AzMERIT

StandardSettingTechnicalReport

Teacherstooktheirtrainingtoheart.WeheardthemsaythingstoeachotherlikeIwantto
makesurewearesettingitatcollegeandcareerready,nottoolow,notwhereastudentisnot
reallycollegeandcareerready.Theyalsodiscussedwhytheresultsturnedoutastheydidand
saidthingslikewearesettingthisforwhatwewantstudentstobeabletodo,notwhatthey
candonow,that[itemsresults]isateacherissueandwheretheteacherisonthenew
standards,andthatshouldbeaneasyitemifitwastaughtproperly.

Teachershadveryspiriteddiscussionsaboutitems,whatajustbarelystudentwasandwhat
studentsshouldknowandbeabletodoandwhy.Theytalkedaboutincreasedcognitive
demandandadditionalcomplexityordepthseparatinglevels.Thesewereteacherswhohad
clearlytaughtthesesubjectsandcouldarticulategradientsofperformance.

Inonlyoneinstancedidweseeacasewheretheywereverydisparateratings(e.g.,individual
cutssetfromitem30toitem47).InGeometrytheteacherswouldseeanitemthatonlya
proficientstudentwouldgetrightfollowedbysomeeasieritemsfollowedbyaharditem,and
thatpatternwouldrepeatuntilthesubsequentitemswouldonlybeansweredbyproficient
students.Onceitwasclarifiedthatthecutpointshouldnotbesetatthefirstquestiononlya
proficientstudentwouldgetright,butatthepointatwhichitemswouldconsistentlybethose
onlyaproficientstudentwouldgetright,theratingsbecamemoreconsistentandteachers
wereinagreement.

CONCLUSIONS
Weobservedaverywellorganized,professionallyrunstandardsettingprocess.Itwasavery
goodstandardsettingthatleftus,andtheteacherswetalkedto,withafeelingof
accomplishment.

Thecutpointsweresetbasedonteacherjudgment,andthefinaldecisionwastheirs.The
directionsandtrainingmadethatcleartoteachers.TheteacherstooktheStateBoardsgoals
toheartastheirtargetoutcomeandthecutpointsreflectthat.

AIRandADEshouldbecommendedfortheirhardworkandprofessionalismthatledtoa
productiveandsuccessfulstandardsetting.Theteachersarealsotobecommendedfortheir
hardwork,theirdeepandnuancedknowledgeofstudentperformanceattheirgradelevels,
theiropendiscussions,andtheirengagementthroughouttheprocess.

P3

AmericanInstitutesforResearch

EXECUTIVE SUMMARY
Issue:

Arizona State Board of Education Special Meeting


August 14, 2015
Item 2
Page 1 of 2

Presentation, discussion and possible action to adopt a Move on When


Reading cut score for AzMERIT Grade 3 ELA

Action/Discussion Item

Information Item

Background and Discussion


In 2010, Move on When Reading (MOWR), ARS 15-701 (A)(2), was enacted. This
statute requires third grade students demonstrate a reading level above Falls Far Below
on AIMS or equivalent on a successor test in order to be promoted to fourth grade.
Certain students are exempted from meeting this requirement. For the Grade 3 AIMS
Reading test, students who scored at or above 379 scored above the Fall Far Below
performance level and met the MOWR requirement.
School year 2013-2014 was the first school year that students were retained due to the
MOWR requirement. In that year, approximately 97% of Grade 3 students scored
above the Falls Far Below performance level on AIMS Reading and approximately 3%
of Grade 3 students scored in the Falls Far Below performance level on AIMS Reading.
Not all of the students scoring in the Falls Far Below performance level were retaining in
Grade 3.
For school year 2014-2015, AzMERIT was administered instead of AIMS. Since this
was the first year of AzMERIT, test results were not available before the start of school
year 2015-2016. So, under the provisions of MOWR, no students were retained in
Grade 3 due to their reading test scores. However, once scores are available in school
year 2015-2016, any students who did not meet the MOWR requirement on AzMERIT
must receive appropriate interventions and remedial strategies.
On November 3, 2014, the Board adopted AzMERIT as the statewide assessment to
measure the Arizona English Language Arts and Mathematics standards. The English
Language Arts (ELA) tests measure reading, language, and writing skills. The total ELA
score is not comparable to AIMS Reading. AzMERIT does not report a single reading
score. Instead, to better match the Arizona academic standards, AzMERIT ELA results
include two reading scores: Reading for Information and Reading for Literature.
For MOWR purposes, an AzMERIT Grade 3 reading score, which combines the two
existing reading scoring categories, will be calculated. Using the linking established
between AIMS and AzMERIT, we have determined that an AzMERIT Grade 3 reading
score of 2446 is equivalent to the previous MOWR cut score of 379 on AIMS Reading.
For the spring 2015 administration of AzMERIT ELA, approximately 97% of Grade 3
students are expected to attain a reading score of at least 2446 which is consistent with
spring 2014 AIMS Reading results.
Contact Information:
Irene Hunting, Deputy Associate Superintendent
Leila Williams, Associate Superintendent

EXECUTIVE SUMMARY

Arizona State Board of Education Special Meeting


August 14, 2015
Item 2
Page 2 of 2

Attached are examples of AzMERIT Grade 3 ELA student reports showing how the
Move on When Reading requirement will be reported.
Recommendation to the Board
It is recommended that the Board adopt an AzMERIT Grade 3 Reading score of 2446
as the MOWR cut score.

F A M I LY S C O R E R E P O R T

AzMERIT
SPRING 2015

Maria A. Doe

ABC School (123654)


ABC District (987456)

Grade 3

English Language Arts (ELA) Assessment

About This Assessment

About This Report

Maria took the AzMERIT Grade 3 ELA


assessment in spring 2015. The questions in
this assessment measure the knowledge and
skills taught in this grade and subject area.

Front:
Marias overall score for this assessment includes a numeric score and
a proficiency level.
Her numeric score can be compared with the school, district, and
state averages.
The proficiency level indicates how well students understand current
grade-level material and how likely they are to be ready for the next
grade.
Back:
Marias level of mastery is shown for each scoring category.
Scoring categories represent specific knowledge and skills included in
this assessment.
There is a detailed description of the mastery level for each scoring
category.

Marias score shows how well she understands


Grade 3 ELA content. A student who scores
Proficient or Highly Proficient on AzMERIT
is likely to be ready for the next grade level of
ELA.

Marias Performance on the ELA Assessment


2605

Marias score is
Minimally Proficient.

Highly Proficient:

Advanced understanding,
highly likely to be ready

She shows a minimal


understanding of the expectations
for her tested grade. She is highly
likely to need support to be ready
for ELA in the next grade.

School Average: 2519


District Average: 2514
State Average: 2509

2541

Proficient:

Strong understanding, likely


to be ready

2509

Partially Proficient:

Marias
score in ELA is
2482, which is
Minimally
Proficient.

Partial understanding, likely


to need support to be ready
Has Maria met the Move On
When Reading requirement?

2497

Yes

Minimally Proficient:

Minimal understanding,
highly likely to need support
to be ready

This determination is made based


on the Reading component of the
ELA assessment.

2395
AZED. GOV
Spring 2015 987456-1

AR IZO NA D EPAR T MENT OF EDUCATION

AzMERIT

F A M I LY S C O R E R E P O R T
Legend: Scoring Categories

Below Mastery

At/Near Mastery

Above Mastery

ELA Scoring Categories


Reading for Information

Maria performed below mastery


in Reading for Information.

What was assessed?


Students find the main idea and the supporting details of a text. They connect events, ideas, steps,
sentences, paragraphs and illustrations to one another. They find similarities and differences between
two texts on the same topic.
What do these results mean?
Your student may have trouble answering questions about a text; finding the main idea and supporting
details; using clues in the text (like charts, key words) to find information; using pictures to understand
a text; and telling the authors point of view in a text.

Reading for Literature


What was assessed?
Students ask and answer questions about a text. They tell how characters and their actions affect
a story. They explain how pictures help tell a story. They read two texts by one author and tell the
similarities and differences. They find the central message of a story.

Maria performed at or near


mastery in Reading for
Literature.

What do these results mean?


Your student can often find similarities and differences between the settings or plots of stories written
by the same author, tell how one part of a story affects another part, use key details to retell a story and
find the main idea, and tell the point of view in a story.

Writing and Language

Maria performed below mastery


in Writing and Language.

What was assessed?


Students write to give information or state opinions. They write on a topic giving supporting details
or facts. They use correct capitalization, punctuation, and spelling. They use sentences, a glossary, or a
dictionary to figure out the meaning of new words.
What do these results mean?
Your student may have trouble organizing writing for a purpose (like to give information or give
opinions); using clues in a text to understand the meaning of new words; spelling commonly used
words correctly; and writing simple sentences with correct capitalization and punctuation.

The Writing and Language portion of the ELA assessment requires that each student complete an essay. The essay is evaluated on three criteria.
The chart below shows your childs performance on each criteria.
Writing Essay Performance
Statement of Purpose, Focus & Organization

Evidence & Elaboration

Conventions & Editing

Your student earned 2 out of 4 possible points.


Your students essay is somewhat on topic but
may drift or include unrelated ideas. The opinion
is stated but may become unclear throughout
the essay. It has a weak structure with an unclear
beginning and end. The use of transitions is
inconsistent. Ideas are unclear as the opinion
develops from beginning to end.

Your student earned 1 out of 4 possible points.


Your students essay includes details, facts, and
sources that minimally support its opinion. This
evidence is not integrated into the response. The
words used are not appropriate for audience and
purpose.

Your student earned 1 out of 2 possible points.


Your students essay shows some understanding
of sentence formation and other conventions.
There is inconsistent use of punctuation,
capitalization, and spelling rules.

For more information about AzMERIT, go to azmeritportal.org.

AZED. GOV

AR IZO NA D EPAR T MENT OF EDUCATION

F A M I LY S C O R E R E P O R T

AzMERIT
SPRING 2015

Maria A. Doe

ABC School (123654)


ABC District (987456)

Grade 3

English Language Arts (ELA) Assessment

About This Assessment

About This Report

Maria took the AzMERIT Grade 3 ELA


assessment in spring 2015. The questions in
this assessment measure the knowledge and
skills taught in this grade and subject area.

Front:
Marias overall score for this assessment includes a numeric score and
a proficiency level.
Her numeric score can be compared with the school, district, and
state averages.
The proficiency level indicates how well students understand current
grade-level material and how likely they are to be ready for the next
grade.
Back:
Marias level of mastery is shown for each scoring category.
Scoring categories represent specific knowledge and skills included in
this assessment.
There is a detailed description of the mastery level for each scoring
category.

Marias score shows how well she understands


Grade 3 ELA content. A student who scores
Proficient or Highly Proficient on AzMERIT
is likely to be ready for the next grade level of
ELA.

Marias Performance on the ELA Assessment


2605

Marias score is
Minimally Proficient.

Highly Proficient:

Advanced understanding,
highly likely to be ready

She shows a minimal


understanding of the expectations
for her tested grade. She is highly
likely to need support to be ready
for ELA in the next grade.

School Average: 2519


District Average: 2514
State Average: 2509

2541

Proficient:

Strong understanding, likely


to be ready

2509

Partially Proficient:

Partial understanding, likely


to need support to be ready
Has Maria met the Move On
When Reading requirement?

2497

Marias
score in ELA is
2405, which is
Minimally
Proficient.
AZED. GOV
Spring 2015 987456-1

No

Minimally Proficient:

Minimal understanding,
highly likely to need support
to be ready

This determination is made based


on the Reading component of the
ELA assessment.

2395

AR IZO NA D EPAR T MENT OF EDUCATION

AzMERIT

F A M I LY S C O R E R E P O R T
Legend: Scoring Categories

Below Mastery

At/Near Mastery

Above Mastery

ELA Scoring Categories


Reading for Information

Maria performed below mastery


in Reading for Information.

What was assessed?


Students find the main idea and the supporting details of a text. They connect events, ideas, steps,
sentences, paragraphs and illustrations to one another. They find similarities and differences between
two texts on the same topic.
What do these results mean?
Your student may have trouble answering questions about a text; finding the main idea and supporting
details; using clues in the text (like charts, key words) to find information; using pictures to understand
a text; and telling the authors point of view in a text.

Reading for Literature

Maria performed below mastery


in Reading for Literature.

What was assessed?


Students ask and answer questions about a text. They tell how characters and their actions affect
a story. They explain how pictures help tell a story. They read two texts by one author and tell the
similarities and differences. They find the central message of a story.
What do these results mean?
Your student may have trouble retelling a story using details from a text; telling the feelings of
characters in a story; finding the meaning of words or phrases in a text; telling the parts of a poem (like
a stanza); and using pictures from a text to understand a story.

Writing and Language

Maria performed below mastery


in Writing and Language.

What was assessed?


Students write to give information or state opinions. They write on a topic giving supporting details
or facts. They use correct capitalization, punctuation, and spelling. They use sentences, a glossary, or a
dictionary to figure out the meaning of new words.
What do these results mean?
Your student may have trouble organizing writing for a purpose (like to give information or give
opinions); using clues in a text to understand the meaning of new words; spelling commonly used
words correctly; and writing simple sentences with correct capitalization and punctuation.

The Writing and Language portion of the ELA assessment requires that each student complete an essay. The essay is evaluated on three criteria.
The chart below shows your childs performance on each criteria.
Writing Essay Performance
Statement of Purpose, Focus & Organization

Evidence & Elaboration

Your student earned no score out of 4 possible


points. Your students essay was incomplete or
written in a foreign language. The response was
confusing, off-topic, or unrelated to the purpose.

Your student earned 1 out of 4 possible points.


Your students essay includes details, facts, and
sources that minimally support its opinion. This
evidence is not integrated into the response. The
words used are not appropriate for audience and
purpose.

Conventions & Editing

Your student earned 0 out of 2 possible


points. Your students essay shows a lack of
understanding of sentence formation and
other conventions. There are many mistakes in
spelling, punctuation, and capitalization. These
mistakes make the meaning or point of the
response unclear.

For more information about AzMERIT, go to azmeritportal.org.

AZED. GOV

AR IZO NA D EPAR T MENT OF EDUCATION

EXECUTIVE SUMMARY
Issue:

Arizona State Board of Education Meeting


August 14, 2015
Item 3
Page 1 of 2

NCSC Alternate Assessment Standard Setting Process and Cut Score


Recommendations

Action/Discussion Item

Information Item

Background and Discussion


ARS 15-741 A.7 requires states to establish a fair and consistent method and standard
by which test scores from schools in a district may be evaluated taking into
consideration demographic data.
NCSC Alternate Assessment
At the September 22, 2014 Board Meeting, the state board approved the NCSC
Alternate Assessment to be used as the operational alternate assessment for students
with significant cognitive disabilities in English Language Arts (ELA) and Mathematics.
A presentation to the board included information about the characteristics of the
students that participate in alternate assessments and sample items.
Since this time, Arizona has continued to be very involved with the NCSC State
Partners including acting as steering committee members, participation in the reporting
committee to develop state, district and student level reports, and standard setting
development committee.
Operational Assessment
The NCSC Alternate Assessment was administered in our state during the March 31
through May 15 test window. Over 7,000 students were administered this new
assessment. The overall experiences were positive and there were no major issues with
accessing the tests or a new technology platform. Many educators shared that the test
was more rigorous and included many of the best practices for education for this
specific group of students.
Standard Setting Process
Arizona was involved in the development of the performance level descriptors, and
providing critical input to the standard setting process that will be utilized to determine
the proficiency levels for ELA and Mathematics. Educators and policy makers will
convene to establish three cut scores resulting in four performance levels: Level 1,
Level 2, Level 3 and Level 4. Understanding that distinguishing a difference between
the general assessment and alternate assessment is critical for all stakeholders. NCSC
will be following a very similar standard setting process with the exception that the
NCSC standard setting will be in collaboration with our state partners.

Contact Information:
Audra Ahumada, Director of Alternate Assessment, Assessment Section
Leila Williams, Associate Superintendent Quality Assessment and Adult Education

EXECUTIVE SUMMARY

Arizona State Board of Education Meeting


September 22, 2014
Item 3
Page 2 of 2

The NCSC Standard Setting meeting will take place August 10 13 in Indianapolis,
Indiana. Arizona will have 7 educator panelists representing various grades, content
knowledge and districts from our state and a state representative.
The final process will be to present the recommended cut scores to the board for their
approval at the August 24th meeting.

Recommendation to the Board


This item is presented to the Board for information only, and no action is requested.

EXECUTIVE SUMMARY

Arizona State Board of Education Meeting


August 14, 2015
Item 4
Page 1 of 2

Issue: Presentation and discussion regarding the administration of the Move On When
Reading Program
Action/Discussion Item

Information Item

Background and Discussion


In 2010, the Move On When Reading law was enacted. A.R.S. 15-701 prohibits a
student from being promoted from the third grade if the student obtains a score on the
reading portion of the statewide assessment that demonstrates the student's reading
falls far below (FFB) the third grade level. The law requires school districts and charter
schools to offer 3rd grade students who score FFB on the statewide assessment at
least one of the intervention and remediation strategies listed in statute and adopted by
the State Board of Education (Board).
In 2012, the legislature appropriated $40 million for K-3 reading base support funding to
provide per student funding to schools for students in grades K-3, and prescribed
requirements for the receipt of the funds. A.R.S. 15-211, requires school districts and
charter schools that serve any K-3 grades to annually submit a literacy plan to the
Board. The law further requires school districts and charter schools which either
received C/D/F letter grades or had more than 10% of their 3rd grade students labeled
as Falls Far Below (FFB) on the statewide reading assessment to have their reading
plans approved by the Board before the Arizona Department of Education School
Finance Division may release reading base support funds.
The General Appropriations Act for fiscal years 2013, 2014 and 2015 included a budget
footnote which allowed the Board to use up to $1,500,000 of the $40 million
appropriated for K-3 reading base support on technical assistance and state level
administration of the MOWR program. Additionally, the footnote included authorization
for two full time equivalent (FTE) positions. Over the last three fiscal years the Board
has used this authority and funding to administer the MOWR program.
The fiscal year 2015-2016 General Appropriations Act (Laws 2015, Chapter 8) created
the Board as a separate budget unit and appropriated $39.9 million to the K-3 Reading
base support level, while it removed the footnote that provided the Board with the
authority to use a portion of the funds to provide technical support and administer the
program. One of the two FTE provided in the K-3 Reading base support funding was
transferred to the Boards separate budget, this is the position filled by the Deputy
Director.

Contact Information:
Christine Thompson, Executive Director, State Board of Education

EXECUTIVE SUMMARY

Arizona State Board of Education Meeting


August 14, 2015
Item 4
Page 2 of 2

HB 2479 (Laws 2015, Chapter 310) included session law that allowed the Department
to use up to $500,000 of the K-3 Reading base support funding on technical assistance
and state level administration of the MOWR program. This legislative action shifted the
administration of the MOWR program from the Board to the Department.
MOWR Policy and Administration
The authority to set policy for the MOWR program remains with the Board pursuant to
A.R.S. 15-701 and A.R.S. 15-211. In June 2012, the Board approved the content of
the literacy plans and determined procedures by which the plans would be submitted,
reviewed and approved, thus allowing distribution of funds to those districts and
charters. These procedures were developed in partnership with the Department.
In 2013 the Board, using the FTE authorized in the budget, hired a Deputy Director and
Director of K-3 Reading. The K-3 Reading Director was responsible for overseeing the
MOWR program. The Director oversaw eight consultants which reviewed literacy plans
for completeness. The majority of time and resources spent administering the MOWR
program have been related to literacy plan review and submittal pursuant to A.R.S 15211. This includes building an online portal for districts and charters to submit their
plans online.
As required Laws 2015, Chapter 310, the Department has submitted an expenditure
plan for the MOWR program to the Joint Legislative Budget Committee. The
Departments plan contemplates that the Board will expend $300,000 for administration
of the program, inconsistent with the Board budget unanimously adopted by the Board
on May 18, 2015. The JLBC will review the Departments spending plan at their June 19
meeting.
Update
The transition of the administration of the MOWR program from the Board to the
Department occurred on July 1, 2015. The Board staff created a transition plan to
minimize disruption to districts and charter schools. In order to review the transition
plan, Board staff has requested contact information for the Departments MOWR
Program Administrator as detailed in the JLBC spending plan. As of the publication of
these materials, Board staff has not been provided any contact information from the
Department. Several emails have been sent to the Department informing them of
previous practices of MOWR administration, as well as district and charter expectations.
Specifically, Board staff requested the Department open the window to the online portal
to allow districts and charters to submit literacy plans and data on August 3, 2015.
Board staff does not have access to the MOWR portal, and has not received a response
from the Department. In previous years, the window has been opened in early August.
Recommendation to the Board
This item is for information only.

EXECUTIVE SUMMARY
Issue:

Arizona State Board of Education Special Meeting


August 14, 2015
Item 5
Page 1 of 1

Presentation, discussion and possible action regarding filling Board staff


vacancies in the positions of Deputy Director and Administrative Assistant
for the Investigative Unit, including consideration of the Superintendents
recommendation and those of other Board members. Pursuant to A.R.S.
38-431.03(A)(3), the Board may vote to convene in executive session,
which will not be open to the public, for discussion or consultation for legal
advice with the Boards attorneys.

Action/Discussion Item

Information Item

Background and Discussion


On July 8, 2015, Executive Director Christine Thompson informed the Board via email
that two members of the Board staff had resigned having secured other employment in
state government. In her email, Executive Director Thompson indicated that the
vacated positions of Deputy Director and Administrative Assistant for the Investigative
Unit were critical. Further, Executive Director Thompson specified that she was working
to fill the positions as soon as possible, and to explore interim solutions to ensure the
work of the Board continued as smoothly as possible.
On July 15, 2015, Superintendent Douglas sent a letter via email to Board members
stating that she cannot recommend the hiring of any Board employees as long as the
current employees are working illegally outside my direction. (See attachment)
The Board may wish to consider the Superintendents recommendation not to fill the
positions of Deputy Director and Administrative Assistant for the Investigative Unit, or
other recommendations of Board members.
Pursuant to A.R.S. 38-431.03(A)(3), the Board may vote to convene in executive
session, which will not be open to the public, for discussion or consultation for legal
advice with the Boards attorneys.

Contact Information:
Christine M. Thompson, Executive Director, State Board of Education

State of Arizona
Department of Education

Office of Superintendent Diane M. Douglas

July 15, 2015


Arizona State Board of Education
1535 W. Jefferson, Room 122
Phoenix, AZ 85007
RE: Filling Newly Vacated State Board Employee Positions
Dear State Board Members:
By now you should be aware that two employees of the Board and the Department of Education
have resigned their positions. By email on July 8, 2015, Ms. Christine Thompson indicated that
she is working with the Department of Administration to fill the soon to be vacated positions.
Ms. Thompson cannot fill the positions. Under Arizona law, as the executive officer of the
Board, any hiring of Board employees needs to be performed under my direction. ARS 15231(B) and AZ Admin. Code R2-5A-301. In addition, the Board can only hire employees upon
my recommendation. ARS 15-203(5). I cannot recommend the hiring of any Board employees as
long as the current employees are working illegally outside my direction. ARS 15-251(4).
If the Board wishes to hire any new employees, I will be happy to discuss that possibility, but
please understand that any employees will have to work under my direction, which necessarily
includes my controlling when, where and how they perform the work of the Board as required by
Arizona law.
Sincerely,

Diane M. Douglas
Superintendent of Public Instruction

1535 West Jefferson Street, Phoenix, Arizona 85007 (602) 542-4361 www.azed.gov

You might also like