November 15, 2019
  • 6:51 pm Jermaine Couisnard, Jair Bolden News Conference — 11/13/19
  • 5:51 pm Yurachek press conference on future of Razorback football
  • 5:51 pm FOMC Press Conference March 20, 2019
  • 1:50 pm Trump Impeachment hearings live: Public testimonies from Bill Taylor and George Kent
  • 12:52 pm [Boyfriend ASMR] Can’t i just video call to my beautiful? (Korean Boyfriend JINO)
CAASPP Principles of Scoring and Reporting Webcast

[Music playing]>>Hi.
Good afternoon, everybody. And thank you very much
for clicking in and joining us for our post-test scoring and reporting webcast. My name is Michael McDaniel. I’m from the Educational
Test…Testing Service. And I’m here today with
my colleague, Terran Brown, to deliver the 2018-2019
California Assessment of Student Performance
and Progress or CAASPP post-test webcast,
and show you how to access your scores, reports, resources, and tools. Our in-person workshop
that’s…that are… the results are in,
will show you how to analyze your summative
assessment results to inform teaching and learning. And could be found out
at And if you have any issues
finding that webcast, feel free to contact CalTAC,
and they’ll be happy to help you locate that on We also have with us today
Devin Triplett. He is from the California
Department of Education. And he’s one of our reporting
managers there. And we also have Jenny Lowe
Miller from ETS. She is our reporting manager
on the ETS side. And both of them will be in our
backroom answering questions and going through those
questions, and then we’ll get to as many of them as we can. Oh, I’d also like to introduce
again Terran Brown. Terran, you want to say
hello or…>>Good afternoon, everyone.
>>Good afternoon. Terran is our lead
psychometrician for the CAASPP testing program. So we are currently presenting our 2019 Summative Institute
Workshop Series. They are going very, very well. It’s a very popular series
of workshops that we’re presenting
to our teachers. The workshop
is a two-day in-person professional development
opportunity intended for classroom teachers,
instructional coaches, and teachers
on special assignment, and kindergarten
through grade 12. And if you haven’t,
we only have few more of those workshops left for the remainder of the series. But you can go out
to, if you haven’t attended
or would like to register. Go and check out the schedule, and come by and see us. Today’s webcast is scheduled
for about an hour and half with possible time
to take questions at the end. And we’re going to cover
what’s new, then we’re going to go
into an overview of how scores are derived and the different ways
that the results are reported. So if you have any questions
during the presentations, we welcome you to ask
by using the question box at the bottom of the page.
If you…if you… if you’re using
a smaller device, just use your scroll bar
and pull that up. It’s at the bottom
of the presentation screen that you’re looking at.
And your question might be read, hopefully,
at the… at the end of the time
if time permits. We also have a team,
as I mentioned earlier, that are going to be answering
those questions. And if we’re not able to get
to your question, we’re going turn these over
to the call center and they’ll reach out to you
and answer that question. If we…lots of times questions
are duplicated. And if it’s not,
if it’s a unique question, we’ll get out to you and chat. So now let’s talk
about what’s new for the 2018-2019 testing year. Within the Online
Reporting System you will be able to…I’m sorry. We just jump pretty…no. I’m sorry. We’re okay. So running on the Online
Reporting System, we’ll be able to find writing
of extensive responses scores, starting with the 2016-2017
administration. Target scores relative
to standard met starting with the 2017 and 2018
testing administration. And within
the Public Reporting System, you’re going to find
four years of data for the California Alternate
Assessments, the CAAs, the first year of the California
Science Test results. Those will be released
in late fall. We’ll mention that a number
of times today, but the scores
will not be coming out until very late in the fall. And they’ll present
in an easily accessible tables similar to how the results
for the Smarter Balanced Summative Assessments
and the California Alternate Assessments
for English Language Arts and Literacy and mathematics
are presented. So briefly about
electronic reporting, beginning in the 2018-19
school year, students score reports
or the SSRs are available electronically. And school districts have
options of providing the student score reports to parents
and guardians. And, as you know,
or I hope you know, that there are no printed
Student Score Reports unless the LEA had purchased
those reports from ETS this year. So I have a feeling
a lot of our viewers may be coming back
after the summer break or very soon, and you’re going
to be finding this PDFs in your student
information system. And, again, if you have
any questions contact CalTAC. The primary method
of obtaining electronic student score reports
will be to access the SSRs from your LEA’s parent portal. This can also be done
through a student portal if your LEA is actively
using the student portal instead of a parent portal.
And we talked to a number of LEAs this year
where the student portal is printed next
to primary portal for the…for the family. The electronic student score
report solution promotes increased use
of an LEA’s parent or student portal as part
of a one-stop shop for families to access their
child’s school information such as assignments,
test results, grades, and attendance records. The CAASPP and the CSA
will be delivering a separate student score reports in the late fall of 2019. So both of those reports
will be put on to your student
systems late… later on in the fall of 2019. To provide SIS
or parent portal vendors with the credentials to access
student score reports, LEA CAASPP coordinators
need to generate a username and secret key
or password. This method of providing
these credentials to your student information
system or parent portal, vendor should be announced…or, by your vendor after it has
completed integrative testing. So we’ve already done that,
and it’s okay to go ahead and do that credential
if you haven’t done already. The image that you see on
the screen right now in TOMS from the…is for LEAs to create that electronic reporting
credential for the student information
systems or SIS vendor. When you select
the Generate Credential button, the system generates a username
and secret key or password which you can securely share
with your SIS vendor. When you generate credentials
that have not yet expired, the existing credential expires
24 hours later, so please keep that in mind. You must share the new
credential with your SIS vendor within the 24-hour time period. But beginning in the 19-20 year, the credentialing process
will be valid for one year, and your [inaudible]
will happen in early September, and every year to…along
with the plan software release. So when we do the software
release as an updates and all those changes
at the beginning of the year in September…or academic year, that’s the best time for you
to go in and do that credential,
and you’ll be able to not worry about it until
the following year. Since both LEA CAASPP
and LPAC coordinators generate and regenerate
credentials, we recommend
that you get together with your fellow coordinators to the side who is responsible
for that task. For a step to step on how
to generate credentials for your student information
system vendors, access the “how
to generate credentials for student information
system vendors,” which is available
on the CAASPP portal. For a step by step on how
to regenerate credentials for your student
information system vendors, you can access the how
to generate credentials for student information
system vendors again, and that is on
the CAASPP portal. In addition to the key step
by step guides, be on the lookout
for emails for the… from CalTAC for reminders
regarding regenerating and credentialing
for your SIS vendor. So we’re going to have a lot
more information about that for the next coming testing
and academic year. TOMS will fulfill a request
for LEA for bulk download of student score reports
by sending an email with a secured link from which
the file can be downloaded. For information and detailed
instructions on how to download the student score report
in bulk, get the quick reference guide
from New this year is the Starting
Smarter website. And if you haven’t gone out and checked out this website,
please do. We show the URL
at the bottom of the page. And it’s there to help parents,
educators understand in more depth
the student score reports for Smarter Balanced ELA
and mathematics scores. The site…the site
shows examples of Smarter Balanced CAA
for ELA and mathematics. Sample test questions
by grade and content area and each performance area. And it’s an excellent resource to help support
your child’s learning. So check it out and pass
it along to your parents and to your guardians,
and your teachers as well. It’s a great resource
and a good checkout. Here is the additional
Electronic Reporting Resources located on the Student Score
Report options webpage. These resources are specific
to the four possible options for providing the student
score reports to parents. Our hope is for all LEAs
to utilize parent portals, but we know some LEAs are still
transitioning to this method. And the last three links
that you’ll see other the page provide information
on alternatives to using a parent portal. Also to assist the transition
to electronic reporting is the electronic student score
report webcast, as well as a list
of quick reference guides that provide step by step
instruction on how to accomplish certain activities within the
California Assessment Program. Download the student
score report PDF will help you download
your student reports from TOMS once testing
has completed for all the content areas. Generating credentials
will help you establish the right credentials
so your vendor can make that student data available
in the parent student portals. And then the possibility
you may need to merge some of those files if you’re
going to be downloading those in bulk. All CAASPP reports
have been redesigned this year. Student score reports
now include links to new resources for families, including student residential
address and information for mailing. The three…
the three previous years of school and state averages
are now included on the CAASPP’s Smarter
Balanced student school report. The last page of the SSR
shows the student score history for the current administration, and last two scores for English
Language Arts and mathematics. For students grades five through
eight the score report reflects one year
of testing history. Grades three
and eleven will have no previous signs score history. And the SSRs
are available in English and one of the additional
following languages, Chinese traditional, Filipino, Spanish, and Vietnamese. Data for the students
three year of testing is shown for the CAAs
for English Language Arts and mathematics,
and includes state averages. The student score reports
for the CAAs for English Language Arts,
mathematics are available again in English and the additional language
of Chinese traditional, Filipino, Spanish,
and Vietnamese. So coming very soon,
we’ll have the CAA for Science. The CAA for Science field test
preliminary indicators. Descriptive statements
used in reporting the CAA for Science present correct
results will be provided for LEAs to use in the fall
for 2019. No student score reports
will be provided this year for the CAA
for Science field test. The California Science Test,
or CAST, scores will be reported
for the additional CAST in the fall of 2019. Once the State Board
of Education, which is the SBE, has approved
the threshold scores, the minimum scores
at level two, level three, and level four, that determine
the scale score range for CAASPP
achievement levels. This approval is expected
to happen during the board meeting
in November of 2019. And the CAASPP,
SSR in grades five, eight, and eleven
is separate from the SSRs for the CAASPP Smarter Balanced
Summative Assessments, so you’ll be receiving
two separate documents. For the California
Spanish Assessments, scores will be reported for the operational CSA
in the fall of 2019. Once the SBE has approved
the threshold scores, the minimum scores
at level two and level three that determine
the scale score ranges for CSA achievement levels, this approval is expected
to come next month… or roughly next month,
in September of 2019. The CSA student school report is an individual document that you will be receiving
as well. So let’s talk about
how the data flows. So to understand
how reports are generated, it may be helpful
to understand the data flow. It starts with CALPADS. And CALPADS is very,
very important. And we remind you
of this every year. You know,
to keep updating CALPADS. And you may have gotten
some emails from me recently about updating your
superintendent’s information in CALPADS
and the CDS directory. So it’s always
really important to keep that information up to date. CALPADS is the longitudinal
data system used to maintain
individual level data, including student’s
demographics, course data,
discipline assessments, and other data the State
and Federal use for reporting. Your CALPADS coordinator enters
student demographic data in the CALPADS. TOMS then receives the student
demographic data from CALPADS. Some of the CALPADS’
information is used to determine which test
are assigned to the student. In TOMS,
the LEA CAASPP coordinator or test site coordinator
will enter information on what test settings
the student needs. And students taking
the alternate exam are identified manually in TOMS. The test settings are…
and basic student information are passed on to the test
delivery system, which uses that data
to match the right test and settings for your students. Once a student submits a test, the test delivery system
sends that student responses to the scoring system. And this is where
the constructive responses are scored by human raters. The machine-scored
responses are completed in the test delivery system. And when the score system
has all the scoring components for your student’s test,
the scoring system compiles those results to produce
the different test results that are reported,
such as scaled score and claim level score. And the test results are sent
to the online reporting system in TOMS where you can access
those test results. Please remember to confirm
that your student enrollment and demographic data are up
to date in CALPADS and TOMS, and prior to the end of your LEA
selected testing windows. So it’s another thing we really
want to remind you about, next year when we start
testing, that… making sure that those CALPADS’
information is up and… updated way before
the students start testing. Students who will be returning
to the same school the following year should not
be reenrolled in the next grade before the LEA’s selected
testing window closes. This also brings
another reminder. Do not exit your students prior to the end
of the testing window. It may cause confusion
within the different reporting systems
that we’re using. For example, say a student
leaves on April 18th of 2019 prior to the May 1st, 2019 opening of the LEA’s testing
window. On May 2nd, 2019,
the LEA exits the student from CALPADS
with an April 18th exit date. So if the student
was exited from CALPADS after the testing window
open on May 1st of 2019, the student will include…
be included in CALPADS to the TOMS file, and therefore a testing
registration was created. The student will continue to be
in the completion status report for the remainder
of the testing window. The student is counted
as not tested on the public reporting site
which is misinterpreted as the participation rate. When the CDE screen
[inaudible] students when the official participation
rate is calculated, LEA sometimes get confused
when the… when they received
the student score reports for those students. So that is why it is very,
very important to enroll and exit students in CALPADS on a timely basis. So now I’m going to turn
it over to Terran, and he is going to walk us through the processes
of scoring. And we’ll be answering
any of your questions about scoring later on
in the presentation. Thank you.>>For the Smart Balanced
assessments, I’ll be focusing on the computer
adaptive testing and scoring. I would describe
the relative contribution of the performance task
and CAT on the overall scores. And then I’ll describe
the different types of scores available,
which includes scale scores, achievement levels claims,
and assessment targets. There are several
real world examples in which industries deploy
technology to cater to our unique taste. For example,
search engines that use cookies to recommend products
and services we may like. Smart home software that learns
from our daily living patterns to heat and cool
our homes efficiently. Music applications that allow
us to create our unique music, stations based on the responses
to presented music selections. The notion of this type
of adaptability extends to the assessment world. We can obtain a precise estimate
of an individual’s proficiency or achievement based
on his or her responses to a sequence of test questions. So I like to further illustrate
this concept of adaptability by comparing
computer adaptive testing to a customized music service. So let’s say our customized
music service, XXYYZZ music service allows users to customize
their music around their unique taste. And then comparing it
with a computer adaptive test which is a test administered
on computer that presents test items
for an assessed content area that is at the right
difficulty level of an individual test taker. So in the case
of a music service, in order to be effective in
measuring your true music taste, the music service must
have a large music library. In an adaptive testing
environment, in order to be effective
in measuring a test taker’s true level
proficiency or achievement, the CAT must have
a large item bank. For users of the music service, as well as test takers, a similar set of steps
are followed in order to estimate the user’s
true music taste or estimate a test taker’s true proficiency
or achievement. For example,
the music service will present the user with a variety
of music selections from its library. For a test taker,
the CAT will present them with a collection of test items aligned to the content
being assessed. So, in the case
of music service, to discover your own
unique musical taste, you can respond positively
with…which…with a… by choosing a smiley face
if you like the music selection or negatively by choosing
a frowny face if you dislike the selection. In the case of CAT,
to discover a test taker’s estimate of proficiency
or achievement, test takers provide
responses to items where a green
check mark indicates the test taker answered
the items correctly and a red X indicates
the test taker answered the items incorrectly. The music service and the CAT
both track responses. For the music service,
feedback on the proposed music selections over
an extended period of time is tracked by a series of happy
or smiley faces, as well as unhappy
frowny faces. Whereas for the…
for the CAT responses to test items
at various levels of difficulty in order to hone
in on true proficiency or achievement is tracked by
a series of check marks and Xs. In terms of getting…
arriving at someone’s true music taste,
this can be an ongoing process, but can end when
the listener stops providing responses to music selections. And in a case
of an adaptive test, the final estimate
of proficiency or achievement is based on
the test taker completing a sufficient number
of test items associated with a test blueprint
that’s aligned to the content area
being assessed. Here is an example
of an adaptive test using 10 questions administered to a student of average
of ability. The left axis represents
the student’s ability and test questions difficult…
the test question difficulty. They are on the same scale. The bottom axis shows
the test questions and the number of student… the student’s responses,
correct or incorrect. So the test starts
with a medium difficult… medium difficulty question
since we know nothing about the student’s ability. Following a correct answer
to the first question, a more difficult question
is selected. This is based
on our revised estimate of the student’s ability. You’ll also notice
that there are lines associated with the estimate
of student’s ability. This is called
measurement error. After a second correct answer, a similarly difficult question
is selected. But the lines that represent
measurement error become smaller because
we have a better estimate of the student’s ability. An incorrect answer to question three results in an easier
test question being selected. And then we follow the pattern
that the student’s correct answer to question four leads
to a more difficult question, and so on as we continue
through the test. Know that this
measurement error, the lines that we see
become smaller and the test question
difficulty and estimate of the student’s ability
stabilize. Also note in these example
that half the responses were correct and the other
half are incorrect which is typical
in adaptive test. The adaptive test
that I described so far is the ideal because
there are very few constraints. But, in practice,
the adaptive nature of a test is constrained by considerations
of the following, the content coverage. You got to make sure
that the content measures the full breadth of the content. Exposure to the test questions
to many students. We want a controlled…
we want to make sure that the algorithm…make
sure that the same item is not administered
to students repeatedly. Also there’s…items are
a part of a set…part of sets. For example,
a reading passage, and then all items
must be machine scorable. Each student’s test is
constrained to ensure coverage of the full range of appropriate
grade level content. For example,
the LEA test can’t consist of only reading
informational items. The level of a test question
exposure is constrained to maintain test security,
assess a test questions based on common passages
or stimulus…stimuli, constrain the ability
to adapt within the set. The test question polls
for a particular grade are designed to include an
enhanced pool of test questions that are more
or less difficult for a grade but still match
the grade’s test blueprint. And then again human-scored
items must be combined with the adaptive section
at a later time. As I stated, the student’s
ability to constantly– the student’s ability
is constantly reestimated in the CAT. And their pattern responses
is tracked. Successive questions
are selected based on… based on this reestimation
of the student’s ability. The Smarter Balanced CAT
does include enhanced pools of test questions
that match this test blueprint for the grade
but maybe more difficult or easy for that grade’s general
population of students. These are used to increase
the measurement precision for very low and very high
ability students in the grade. So the resulting ability
estimates are based on the pattern of responses to a particular set
of questions, not the sum of correct
answers, which is different from
the test that we typically see. Each response
is weighed by the statistical characteristics
of the questions. To illustrate the point
of how pattern scoring work, suppose we have two
students taking a test consisting of four items. Notice that each student
answered all of their assigned items
correctly. However,
since the student on the right was presented with more
challenging content, the student on the right
will receive a higher score. For the CAASPP Smarter
Balanced assessments, there are some items
that require human scoring. The adaptive algorithm
will select the items based on the performance
on prior items. Since these items cannot be
scored in real time, the student’s performance
on these items will have no impact
on the next item selected. But after human scoring… after the human-scored items
have been scored, they will contribute
to the student’s overall score. In all Smarter Balanced test,
a performance task or PT and a set of stimuli on a given
topic are administered, as well as the CAT. The CAASPP Smarter Balanced
assessments have PTs. These PTs are administered
at the student level, and these PTs are not adaptive, therefore they’re not targeted to a student’s specific
ability level. The items associated
with the PTs may be scored by machines or human raters. For more information
on the number… of the number of machine-scored
and human-scored items, please refer to the Smart…
Smarter Balanced blueprints, which you can find
on website. After each student completes
the performance task and CAT portions
of the assessment, their responses are combined
for final scoring. The final score is based
on pattern scoring where difficult and more
discriminating questions result in higher scores. More discriminating
in the sense that… in the sense means that more
information is available between those students
who get the item correct and those who get
the item incorrect. Given the size of the item
pool for the test and the infinite…infinitely
many pathways in which a student can receive
and answer items, there will be infinitely
many response patterns that will lead
to the same estimate of a student’s achievement. One question
we routinely receive is the relative contribution
of each of the test sections. These tables describe
the relative contribution in terms of number
of test items administered of the CAT and the PT portions
of the assessment as defined
by the test blueprint. However, it’s important
to note that the relative contribution of the PT
and CAT sections to the overall score
is influenced by the performance
on the CAT, the average ability of the items
from the PT section, and the performance
on the PT section. The relative contribution
of the PT and the CAT sections on the total score
will vary across students since the overall score
is based on pattern scoring. Again, depends on the
performance in the CAT section, depends on the performance
in the PT section, and depends on the average
ability of the PT section. For example,
a PT may have significant… have a significant
negative effect on a student’s overall score if he or she did extremely well
on the CAT but performed poorly on a very
easy performance task section. So let’s illustrate this point. Again, we have two students
that were assigned items of comparable difficulty
on the CAT and both were assigned
easy performance task. Both students answered
the CAT items correctly. So prior to taking
their respective PTs, both students’ ability estimate
should be quite high. And you would expect
that if they were assigned an easy performance task,
they should do well on it. And if they do well in the PT, their estimated ability after taking the PT
should remain high. The student
on the left didn’t attempt any of the PT questions, and hence received zero credit. So when we reestimate
his ability, it will be significantly lower
due to the poor performance on an easy collection
of PT items, which is not expected
given his prior performance. The student on the right
receive full credit. When we reestimate
her ability, it will not likely change
because we expected her to perform well
on the collection of PT items given her prior performance. The guidance provided
by Smarter Balanced is that the PT section
could account up to 25% of a student’s
overall score. So the upshot here
is that all parts of the Smarter
Balance assessment are important. Now, let’s move on
to the Available Summative Assessment Scores. There are several
assessment scores available to schools and LEAs. The first set
of scores available are scale scores. In terms of properties
of the reporting scale, we have a vertical scale that allows
for future descriptions of student progress
or growth over time. The full scale
is about 650 points. It…but each grade’s
reported score scale is about 500 points, as we will go over
in the upcoming slides. Scores exist
on the vertical scale. They are expressed
on a single continuum for a content area. This allows users to describe
student growth over time and across great levels. The score…
scale score ranges on ELA is between 2114 and 2795, and for mathematics, it’s 2189 through 2862. For each grade level
and content area, there’s a separate
scale score range. The second set
of scores available are the achievement levels. The scale scores
on a Smarter Balanced test are mapped
into four achievement levels, Not Met, Nearly Met, Met, and Exceeded. Achievement levels
are set by a process called Standard Setting in which educators from across
the Smarter Balanced states work together
and reach a consensus. Level 3 is considered
the standard by which high school students are considered college
and career-ready. This table describes
the scale score ranges for each
of the performance level… for each performance level. You will notice
that there is overlap in the scale score ranges for a particular
performance level across grades. This figure illustrates
the achievement levels by grade for ELA. Again, you will notice that the range
of the achievement levels overlaps across grades. You may notice that
the achievement level thresholds increase at higher grades
but the amount of increase does become smaller. Here are the scale score ranges and levels for mathematics. This figure illustrates
the plotted achievement levels by grade for mathematics. There are four Claim Areas
reported for ELA, and three Claim Areas
reported for mathematics. For ELA, we report out
Reading, Writing, Listening, Reaching…Research/Inquiry. And for mathematics,
it’s Concepts and Procedures, Problem Solving/Modeling
and Data Analysis, and Communicating Reasoning. On the…on the score reports, we report achievement levels
for Claims. Achievement levels of Claims
are very similar to sub-scores. They provide
supplemental information regarding students’ strengths
and weaknesses. It is reported for all Claims
in ELA and Mathematics. However,
in order to obtain a claim, a student must complete
all items within a Claim so that we can estimate
their performance. This information is reported at individual
and aggregated level. One question we often receive is why are there only three
achievement levels for Claims? So to answer that,
there’s a few considerations. First, there…
since there are fewer items associated with each claim, and hence,
less information is available to reliably provide
supplemental information about students’ strengths
and weaknesses, it’s important for score users,
for example parents, guardians, teachers,
and students, to not over-interpret
this information. The decision to provide
three achievement levels for Claims were made
to prevent over-interpretation. The three achievement levels
for the Claims are Above Standard,
Near Standard, and Below Standard. There was no achievement
level setting for Claims. Instead achievement levels
for Claims are based
on the students’ performance in relation to the Level 3
“standard met” criterion. Students with Claims scores that are substantially
above the standard met criterion will be classified
as performing above standard. Students with Claim Scores that are substantially below
the standard met criterion would be classified
as performing below standard. And students with Claim Scores
that are not far away from the standard met criterion Would be classified
as performing near standard. Let’s move on
to assessment targets. Assessment targets
connect the state… the content standards
to evidence that will be collected
from the assessment. Target maps…targets map
the Common Core Standards onto assessment evidence
that is required to support the content categories
and claims. Targets are used to guide
the development of items and tasks that will measure
the Common Core State Standards. Assessment target reports,
along with other score reports available through CAASPP
Online Reporting System or ORS, is part
of the [inaudible] materials that can be use…
can be useful in determining
patterns of performance to help inform
teaching and learning. Smarter Balanced uses
an evident center designed to develop assessment items. That design results
in a hierarchal scoring and reporting system
with the total score provided at the content domain area, English Language Arts/Literacy, or mathematics
being the broadest and most…and the…
being the broadest and the assessment target square representing a more detailed or granular level
of information. Assessment Target Reports
are group level indicators. For 2019,
there are two types of reports. The first target report,
the Strength/Weakness indicator, describes a group of students’
strengths and weaknesses relative to the overall
test performance of the group. For this indicator,
it does not imply that a group has
or has not met the standard. The Target Standard Report
focuses on areas where performance
indicates standard met. Particularly
for a group of students, it indicates whether their…
they performed above, below, or near
the standard met criteria for a particular
target standard. Target reports
and the Online Reporting System indicate group levels strengths or areas for improvement
for specific standards that are addressed in each Claim in the ELA Mathematics Concepts
and Procedures Claim. So, going back
to our Claim conversation, another question
we routinely receive is whether Claims are weighted. There’s no absolute weighting
that will apply to every student in the same way. A student’s score on the Claims
depend on items that a student is assigned. Students that correctly answer
more difficult and discriminating test items
with any Claim will perform better
on the Claim. Low performance on a Claim
is based on a student correctly answering
less difficult and discriminated test items in the Claim area. So, you may ask,
“Do the Claims factor… is…how do the claims factor
into a student’s overall score?” There is no universal weight that applies to every student
in the same way. For each content area,
the average difficultly and discrimination
for each Claim varies for each grade with some Claims being
more difficult than the others. For every student completing
the Smarter Balanced Assessment, a Claim achievement level
is estimated. It is a broad measure
of his or her performance for all achievement measures. There is a certain degree
of uncertainty associated with the estimate.
Claims with fewer items will generally have more
uncertainty associated with their estimated
achievement levels than those with more items. To compensate
for the uncertainty associated with the estimated
Claim Achievement, an interval is plot…
of the plausible scores is calculated. It is possible for a student’s
estimated achievement level for Claims to exceed
an achievement level threshold, but your interval to cover
multiple achievement levels. Here’s an example
where a student’s estimated achievement
is above standard. However, when you account
for the uncertainty where the students’ achieve… Students’ Claim Achievement, the interval of a plausible
achievement indicates the performance
may be near the standard in the above category,
therefore to be considered… to be conservatively
reported out that the student’s
achievement level is near standard. Here is a similar example
where, again, the estimated achievement level for the student for a Claim exceeds the achievement
levels threshold. However, the interval
of plausible achievement does not cover
multiple achievement levels. Therefore, we have clarity about a student’s
actual achievement. Again…so going back to our conversations on Claims
and then Target Reports, you can find more information
by going back to the and search for the Target Report FAQs. I’m now turning back
over to Michael.>>Thank you very much, Terran. And, you know,
over the last couple of years for…we’ve done
this presentation together, I get a lot of questions
from teachers saying, you know, “Explain scores to me.”
>>Right.>>And it’s much easier for me
to take the presentation from last year and give it… give it to them
and show it to them. And it really answers
all the questions.>>So, thank you very much
>>Okay.>>So, let’s look at
understanding score reports in the Online Reporting System. So, we’re going to take
a minute to look at that and hold on just a second. I think I may… there we go.
We’re on the right screen. So, on the screen’s
a list of the available reports that you’re going
to be able to find in the Online
Reporting System. We have the Initial
Student Test Results, Aggregate Data,
Assessment Target Reports, the Student Score Reports that come in a PDF file, and then
the Final Student Data File is also available on TOMS.
And we show, you know, what reports
are available to each group, LEA, a school, or a parent. So, that’s all the data
that’s available to you in the Online Reporting System. And the ORS system
is a wonderful tool during testing to look up
Student Score Reports and just use that
as one of the many tools that’s available to you. We’ve talked
about how the data flows. Now, we’re going to take a touch
on the web-based system and display score reports
for each student. The Online Reporting System
displays results for students who have taken the following
California assessments. The Smarter Balanced for English
Language Arts and Mathematics, the California
Alternate Assessment for English Language Arts
and Mathematics. And after the release
of the scores late fall of 2019, we’ll see
the California Science Test and the California
[inaudible] Assessment of results will be available
as well. So, those all will be up
on the Online Reporting System. Excuse me. As in previous years,
LEA CAASPP Coordinators and Test Site Coordinators
will continue to have access to test results
for their students. You have the ability
to provide test results for creating rosters
for test administrators and test examiners. So, why is this important? By providing your teacher access
to the student’s test results, they can use the information
to work on areas of improvement with their class
or with each student. This is especially powerful
when used in combination with the digital library
and the interim assessments. The Completion Status
and Roster Management interface is the primary means
of viewing test status and completion data
and managing rosters for students taking part
in the CAASPP assessments. For more information
on completion of status systems and the roster management guide, check out the guide
to CAASPP Completion Status and Roster Management located all on For quick reference on selecting
the appropriate roles for staff, utilizing the User Role Guidance
document use, you could also
find that out at in the…at the portal. But if you’re new
to the process, we know that every year,
we have a lot of new coordinators
who are joining the team. If you’re confused
or need more information about those roles, again,
always lean on CalTAC and they’ll be able
to help you out. This slide includes
all of the features and reports that are available
in the Online Reporting System, the homepage…Page Dashboard. We have the Subject
Detail Report. The Claim-Level Report,
Assessment Target Reports for English Language Arts
and Mathematics. The Student Listing Report,
Student Detail Report, and the Manage Roster Report
is also available. For more…for more information,
you cold access the Quick Start Guide
to access those score reports in the Online Reporting System. The Quick Start Guides
are available on the CAASPP portal. [inaudible] Assessment
Target Reports, more information
on how to access the Assessment Targets
can access the… you can access
the Assessment Target Report Quick Start Guide
which, again, is available at the CAASPP portal. So, let’s go
to the score reports for this year that we’re
going to be looking at. And those test results
that are going to be in TOMS. Excuse me. So, after you log on,
you’ll see the TOMS dashboard. In the left navigation bar,
click on… you’ll select the “Reports”
to expand the “Reports” menu and then select
Student Score Reports PDFs. Through TOMS, you can down…
through TOMS you can download PDFs for your Student
Score Reports. Like test results in the Online
Reporting System, the Student Score Reports
are updated nightly. As your student completes
testing and the responses are processed and scored, they’ll be generated
into Student Score Reports. Those SSRs and PDFs
can be downloaded. This year, you can find PDFs
in two different locations, TOMS and for the first time, your LEA
Student Information System with that information will send
over directly from TOMS into your parent portals for the electronic
reporting process. First, you’ll select
the test administration year that you want to download
to those student PDFs. Default…the default
is the current year. And next, you’ll select
the organization or LEA. This will populate a list
of schools within the LEA and grades for the reports
that you want to download. You can select
one school or grade or multiple schools and grades. The information on the slide
and the directions on TOMS user interface describe how you can select
multiple grades. Finally, you’re going to name
that report and then select the “Down Report” button. When you download the report
and TOMS generates a compressed file in ZIP format that you need to save
on your local hard drive, network,
or other storage device. Saving the file locally, that does depend on the device
and the internet browser you’re…that you are using. This includes reports
you may have requested in additional languages. Please note
that if you would like to download the student’s
individual report, you can do so by accessing
the Student Score Status page and it’s a very quick,
easy way to download this individual student reports
one by one. Continuing with this process of downloading
the student reports, please keep in mind,
files may be very large and take time to process, depending on the number
of students, and the school, and grades that you’ve selected. For the best performance, make sure
that you’re the only user at the time downloading
the files for your school and that you allow time
for that file to finish, processing and downloading
before submitting requests for that next grade. So, it takes a few minutes
to get through that process but that process
has been speeding up year after year, and I don’t think
you’ll have a big problem unless you’re a really,
really large school district. And it’ll just take a little…
a little longer but it won’t be a problem
in downloading those. This slide shows an example
of the file structure that might be included
in the downloadable file. In this example
with the downloaded student school reports
for grades three, four, and five in a demo school. When we open up the folder for grade five, we see the separate PDFs
for each individual student school report that is available. The file name
for the PDF provides help on the information
so that your…for the student whom that report belongs to. You may have noticed
the file named PDF index.html in the last slide,
that file includes which each downloadable file
is available. So it’s a…
it’s a hyperlinked way to get in to look at those
individual student reports. The file contains an index of all the student
school reports including your downloadable file and the index list,
the student school reports in your downloadable file. And on the very far right
of the column lists the PDF files as hyperlinks that contain
the student score reports. Be sure your site coordinators
will have access through TOMS to download those and print the student
school reports if needed. So please pass that along
to your site coordinators, that this year, they do have the ability
to go in and print those student school reports
if needed. The TOMS reporting system
gives the ability to pull reports by through the score report PDFs, through results report, the student score data extract. This is the LEA’s
student score data extract for all the students
who tested during the current or two prior CAASPP
administrations. LEA should download
and save the report since TOMS
will only be able to hold it for the past three years. The organization
percentage report, the organization percentage
report shows current scoring percentages for all local education agencies
or LEAs through test administrations
in California and you’ll enter a range
of the percentage of students’ scores
within all LEAs that will be reported. The LEA in school
student school report distribution report
is available. This report displays
the school, or LEA, that have requested
delivery method of the student school reports as well as the student
school report language on the latest CALPADS
information, the language that’s an addition to the default electronic
students report in English. This report will also show
the reporting selections made for the students
based on whether any paper student score report
selection box is selected
for one more students. If any box is selected,
the LEA opts to purchase paper student school reports
for that test. If no box is selected,
the LEA opted out of purchasing student
school reports for students. So allow me to turn that over
to Terran just for a few minutes.
He’s going to talk about interpreting school reports
or score changes.>>Thank you, Michael. So as previously highlighted, the CAASPP Smarter Balanced SSRs tracks students’
performance overtime. The CDE has produced tables summarizing typical changes
in scale scores from one grade level
to the next. So here’s an example
of the table of the average scale score
change table from moving from
one grade level to the next. This one has the change from… moving
from grade four mathematics to grade five
mathematics. In this…in this column, each row,
the first column is labeled “Scale Score Grade 4.” In this column,
each row provides a range of non-overlapping scale scores
that covers the entire range of the grade four
mathematic scale. The second column provides
the “Average Scale Score Change” observed on this grade five Smarter Balanced
mathematics test based on prior performance
in grade four. So for example,
fourth grade students with Smarted Balanced scores between 2400 and 2499, when they completed
the Smarter Balanced mathematics test in grade five, the average change
in their scale scores was approximately… was 2…2181. The last column provides
the number of students appearing in each of the grade four scale score ranges
in mathematics. So for example, there were…
there were 203,945 students whose scale scores were between 2400 and 2499, that also have this
scale score change. So, let’s go over an example. Madison’s scores on the Smarted Balanced
mathematics test increased from 2467
in grade four to 2530 in grade five for a scale score change
of 63 points. Madison’s prior scale score was 6…prior scale scores was in the middle
performance group between 2400 and 2499. Students whose prior year’s
performance in this range had an average score change
of 2392 points, so overall,
Madison’s score change was well above average. So handing it back
over to Michael.>>Thank you very much. And we do have a number
of questions coming in and we’re going to be, again,
addressing those questions in a few minutes when we get
to the end of the presentation. So now let’s take a little bit
more in-depth look at some of the student
or at the student school reports that are available to you
this year. So moving to the electronic
student school reports, that has allowed to implement
several enhancements to the student school reports based on parent
and stakeholder input. This is the redesigned CAASPP
Smarted Balanced score report for 2018 and 2019. This year’s school report
will be four pages long, we want to remind everybody
who do not know that, we’ve been messaging that for about six
or seven months now, but we want to remind you
that it is four pages long, two pages back to back providing more…
it does provide more information to our parents
and our guardians than in years past. The front page
of the CAASPP reports shows us that parent
and guardian address information and that’s the SSID number
of the student, date of birth,
the student’s grade, test date, school, LEA, and then the LEA county, district, or CDS number. So let’s take a minute
and go over each new report that’s under
the CAASPP umbrella. CAASPP results
give one measure of how well students
are mastering California’s challenging academic standards. The skills called for
by these standards and the abilities
to write clearly, think critically,
and solve problems are critical to preparing
our students for college in the 21st century careers. There are four levels
of scores for ELA, Standard Exceeded, Standard Met, Standard Nearly Met, and Standard Not Met. Standard Met
and Standard Exceeded are the state targets
for all student score ranges per each level of different… for each level of different…
for grade…for each grade. That was really hard
for me to say. The report also shows you
where your student score compared to school average
and the state average of students in the same grade. At the bottom of the page, we have our English Language
Arts performance section showing us
how the student performed in reading, writing, listening,
and research and inquiry with scores of below standard, near standard,
and above standard. For CAASPP mathematics,
the student school reports shows our students’ score
in range levels, there are four levels
for mathematics, Standard Exceeded, Standard Met, Standard Nearly Met,
and Standard Not Met. The mathematics
performance levels show concepts and procedures, problem solving,
and modeling, and data analysis, and communicating reasoning. The front of the California
Alternate Assessment Student School Reports shows
the parent and guardian address along with the student’s
state SSID number, their date of birth, grade, test date, school, LEA, and that county district
school number. And look for language
and messaging that was in the CDE toolkit for the 2017, 2018 year. The toolkit
will be updated very soon. The front of the student
school reports shows the students’ overall score for English Language Arts,
and math, and scores are identified
by Level 1, Level 2, or Level 3. The CAAs give the students
the opportunity to demonstrate their learning
by taking a test aligned for their grade level. So, let’s take a minute
and look at the CAA for English Language Arts. At the top of the page is
your students’ overall ELA and score, a comparison score
for students in the same grade and the state averages
for that grade. The report also shows
the last three years of the student scores. For example, we show
the students’ grades three, four, and five results. The CAA SSR shows the students’ overall
mathematic scores. The report shows a comparison
of the state average student scores in
the same grade as your student and the report also shows
the last three years of student scores. For example, we show
the students in grade four, five, and six results. So regarding the CAA
for Science Reporting for 2018 and 2019, and those preliminary indicators
in the toolkit, we’re going to talk
about that for just a second. So the key messaging about the preliminary indicators
include, will provide
very important information about the science assessment and reporting results for ELAs, schools,
and other stakeholders, reporting assigned
assessment results for 2018 and 2019,
the timeline, and it provides the ELAs,
schools, and then the stakeholders
with information about releasing
the science results to parents, guardians, and the public. Regarding
the science assessments and the transition
to the operational status of the exam,
it’ll provide information about reporting student results
for each school year through 2017 and 2018, and through
the 2019-2020 year. And then you’ll also find
frequently asked questions, provide…that will help you
provide answers to some of those questions
for your LEAs, schools, and stakeholders and may have… and have about reporting and releasing
the preliminary indicators for the science assessment. So for the CAA
for Science Reporting for 2018 and 2019, the CAA for Science
Preliminary Indicator Communication Toolkit is a great resource to explain
the preliminary indicators that parents will receive
about their students’ progress. It provides frequently asked
questions, key messages, and resources for educators
and…educators and parents. And we’re going to be
updating that website, CDE will be very soon
the fall of 2019 and we’ll alert all
the LEA CAASPP coordinators when that update
has been completed. Because it’s the first
operational year for the CAST, scores will be…that’s
the California Science Test, scores will be available
the late fall of 2019. This means that scores
will not be avail at the same time the scores
for Smarter Balanced ELA and mathematics, and if you’re
one of those new schools that are just coming back
from your summer break, we just want to remind you that those will be out
later on in the fall. For 2018 and 2019,
students will receive a separate student score report of the Smarter Balanced
English Language Arts and mathematics scores. The California Science Test
summer results will be publicly released
at the CAASPP, LPAC, website in the fall of 2019. And notice
that we will not be using preliminary indicators this year because the test
is now an operational exam. We’re evaluating on through… the CAST Reporting. They were evaluated
on three science domains, Life Sciences,
Physical Sciences, and Earth and Space Sciences. The California Science Test
reporting will contain an overall score
and achievement level. Reporting at the domain level
will not be a numeric score, but will be a level
either below, near, or above standard. And one of the key messages
we want to mind everybody is the scores are going to be
maybe a little low this year. So, please don’t be surprised
and that’s something that’s…it’s a new exam,
a new assessment, and for the first go-around,
the scores may be a little low. The overall
science achievement levels will describe how well
each students met the performance expectations of the California Next
Generation Science Standards or CA NGSS. So for example,
a student at Level 3 has the met the performance
expectations of the CA NGSS and has demonstrated
an adequate understanding of the standards
and has an adequate ability to apply his or her
knowledge and skills through the California Next
Generation Science Standards in science
and Engineering Practices, Disciplinary Core Ideas, and Cross-Cutting Concepts. The CAST is a new kind
of Science Test in California. This challenging new assessment is for critical thinking
and problem solving skills and the California Science Test
Student Score Report front page will contain the parent address and student information. The same information
we talked about earlier. The student’s overall score, performance level,
and a description of the California Science
Test Assessment. SSR show the standard met
and performance band. The California Science
Test Scores won’t be available until the fall, so that’ll be
a late fall of 2019. And standard setting
was conducted… or will be conducted
in July 30th through August 2nd. Threshold scores
must be approved by the California State Board
of Education in the fall meeting and scores are going
to be low again, so don’t be surprised. Science Area Performance levels
will be available and the parent/guardian
resources as well. This year,
the CDE proudly celebrates the California
Spanish Assessment or CSA. The operational exam, students, and parents,
and guardians will receive their California Spanish
Assessment Student Score Report containing the following
information. Again, the address
of that student, the student’s
identification information such as their SSID number, where they go to school,
the CDS number, and then, as the example
we’re showing you here, that’s his overall score, with the high-degree level
of three, a moderate-degree level of two, and then a limited degree
of one. The CSA SSR shows
the scoring band for Madison, showing Madison’s score
to Level 3, high-degree,
with an overall score of 761. Also, you’ll see the Spanish
Area Performance Levels for reading, math,
and listening as well as an information about
parent and guardian resources. So, here are some examples
of Score Reporting Recourses that are out
on Smarter Balanced’s site and as well. We have also Quick Start
reference guides that are…that are posted and the Student Score Report
options are there as well. So, let’s talk about
the timeline for reporting. The student scores will be
available as PDFs in TOMS within one week
after the student’s last score is reported
in the Online Reporting System. This will be the same
reporting timeline as it was for last year. The student scores are created
during the test administration and once the student
has completed testing. While we make every effort
to release test scores as quickly as possible, there may be situations
where a student’s test results are temporary placed on hold. Some of the examples
are listed on the slide, and as always, make sure
your CALPADS data is up-to-date. So again,
CALPADS is very, very important. When a hold is placed
on a student’s score, our help desk may we…
may reach out to the LEA CAASPP coordinator
to resolve the issue and clear up that Student
Score Report…Reporting. So, you know,
it’s possible you could… you’re going to be getting
some calls. Very few of the LEAs
do get the calls, but you may get a call
from the call center. And then the LEA Dashboard, this is an example
of the LEA Dashboard that you saw in TOMS this year. The Dashboard displays
the list of administrations, the percentage of scoring
completed for students who were assigned
Smarter Balanced English Language Arts literacy
or ELA and Mathematics Assessments based on the local
Education Agency or LEA selected. For the 2018-2019
administration, the 90% threshold delivery
of the Student Score Reports will no longer be applicable. As a result, functionalities
related to these features, 80% email, 90% email, alerts at 80%, 90%, have been removed. Instead, there will be alerts
one week prior to the selected
testing window start date, two weeks prior
to the selecting testing window end date, one week prior
to the selecting testing window end date. And those LEA CAASPP
coordinators will also be alerted
when the remaining small, what we call straggler reports,
become available. Once this email arrives, your LEA will have 100
of their… of their results score. So, we do have
a few questions, Terran, and let’s just go through those
and we’ll answer as many of them as we can. How are we doing on time?
I don’t see a clock. Oh, we’re doing well.
>>We’re have… we’re doing great.
>>Great. Thank you very much. So, “What will the Student Score
Report look like in historical area if a student takes
the CAA one year and the SBAC another year?” The California Alternate exam
will not appear in the student score history
on a Student Score Report, for the Smarter Balanced
Assessment as these test scores
are not comparable. So, again, if you have a student
who took the CAA and then is taking the SBAC,
again, they’ll come… they’ll come out
on separate reports and will not have
any historical information. “Regarding slide 94, is the comparison
of the CAA SSR with other ELA test takers or with ELA CAA test takers?” And the report shows
the comparison of the state average of California Alternate
Assessment Student Scores in the same grade
as your student. “When will the score data
extract and data layout files
be available?” Last year, the file
was available around July 18th. And that preliminary data file
is tentatively scheduled to be at least in mid-August and the file date for the file is tentatively scheduled
to be released in September. And those are about
the same time, end of August, early September, when we have those
on a yearly basis. “What is your requirement
for providing the Student Score Reports
to parents? I’m asking
because we are working on our Parent Portal and we would like
to not mail everyone a Student Score Report.”
That’s wonderful. We don’t want you to do that
if you’re able to do it. “But I know we are required
to notify parents in a certain amount of time. We use PowerSchool
and we’re still working on opening up the Parent Portals
and I know that PowerSchool has those Parent Portals
opened up. So we can plan to open it up by hopefully…
by September 1st.” And this is regarding
this individual LEA. Student Score Reports
need to be made available to parents within 20 days
of being received. If results were received after the last day
of instruction, they can be made available at the start
of the next school year. Parents also need to be notified as how they can access
those student reports. “Since the Student Score Reports
are four pages, will LEAs receive
a peel-and-place label for Students
for their accumulative folders that show basic information
as was done in previous iterations?” Labels will not be provided
this year. School Reports
are no longer provided in paper, so there’s no need to
and there’s no labels, there’s no more paper. LEAs are not required
to keep a hard copy in that Student Score Report, pursuant to the state
regulations 863(c), and you can look that up
if you like. But schools are responsible
to maintain pupil scores with the pupil’s permanent
score records or for entertaining
the scores into the electronic
pupil reports. “When the bulk final student
data files will be available for TOMS?” Again, we mentioned
that the data files will begin about mid-August to the very early part
of September. And, again, we’ll release
that information out to all of the LEA CAASPP
coordinators once that data has been decided. “When will student score
data extract be available for your 2019 scores?” Again, those will be available in mid-August,
early September. And here’s
a really good question. “We’re going to have
our print shop print out these student score reports, but they were
passcode-protected. We have a very…
we’re a very large district and our print shop
is going to have to open each one of those PDFs
individually to print out. Why is that protected?” For security reasons, each of these
student score reports is password-protected. We are currently looking
into the options and we have one
that may work for you. And if you are one
of these schools who are having issues
bulking up those reports to download, send an email out to CalTAC. And in the subject line or somewhere in the text
of your email, just ask them to forward
that over to Michael McDaniel and we’ll look into that,
we’ll give you a call and we’ll chat about
those student score reports and how to get past that. This is coming
from a superintendent of a school district. “On my TOMS account, I’m able to add
and modify users. I’m not, however,
able to access scores in the online reporting system.” And that’s because
the LEA coordinator can grant you access
to your online reporting system. So that person you designated
as your coordinator for last year, go…
contact them. They can set you up
for the online reporting system as a user and you’ll be able
to get in there and look at that information. “If all our students
completed testing in May, should I assume
that the data file download and the online reporting system
includes all student scores?” No. All student scores
are not included, as it does not include students who took the Alternate exams and the California
Spanish Assessment or the CAST test as well. So, as we mentioned earlier, those reports
will be coming out, and data will be coming out
in the late fall, so we’ll have more information
and more email announcements about that for you. And we’re getting
a lot of questions about the data files. So again, the data file
will be available through TOMS at the end of August,
early September, and we’ll reach out to you
for those dates. “Is there a report…
is there a report that would give a sense
if whether students possibly performed less well
on the PTs versus the CAT sections
or vice versa?” While there is no performance
test score available for reporting,
the writing scores for the English Language
Arts Assessment are available through the online
reporting system. All these task…and the… it just jumped on me,
I’m sorry about that. Well, my computer
had a little glitch and your question went away, but that person
who was answering that question,
go ahead and contact CalTAC and they’ll forward that
over to us. And again, “Who takes the California Spanish
Assessment and why?” This is a very good question. The California
Spanish Assessment, CSA, is a computer-based
Spanish Language Arts test available to students
in grades three through eight and high school districts
can administer the test to any student in California who’s receiving
instruction in Spanish or seeking a measure
of student’s Spanish reading, writing, mechanics,
and listening skills. So, it’s out there. We had a wonderful response
this year of students who took the CSA and we’re hoping that grows
over the next couple years. So, if you have any more
questions about that, we always say
contact CalTAC, but please do. “Is there a report…
is there a report that would give a sense
of whether a student has possibly performed
less well in the PT versus the CAT sections of…
or vice versa?” This is the question
that I mentioned earlier that kind of went away,
but now it’s back. It’s amazing
how technology works, isn’t it?
>>It is. Yes.>>So, while there is no PT
scores available for reporting, the writing scores for English
Language Arts Assessments are available through the online
reporting system, as these tasks do not measure all the same claims
as questions in the…in the…in the exam. “When will the CAST test file
be ready again?” The Student Score Reports
will be available in late fall… well, I’m sorry, not fall. Next month or early September. Late August, early September. “Last month, we had the initial
Student Score Reports for ’18 and ’19
were available in TOMS. It seems that those results
are no longer available. Will there be an official
revised students results reporting coming soon?” And we do not have an exact
date, again, for that… the release of that file, but that will be very soon, coming, hopefully,
end of August, early September. “Since we have not received
final summative data, can I share the target reports
with teachers and admins or will they change?” Target reports are based on…
based upon data from the entire testing group
of students, as such as any target reports
prior to the final LEA data file may be subject to change. So just please keep that
in mind. Terran, if anything pops up here
and you want to jump in, please feel free to do so.
>>Sure. Absolutely.>>”What was the Ed Code, again,
that states where you have not to keep
hard copies of student score reports
anymore?” And I did not read
that Ed Code completely out, but I’m going to do it
this time. Pursuing to 5 CCR, Section 863(c), schools are responsible
for maintaining pupil scores with the pupil’s
permanent score…school records for the…for entering
the scores into the electronic
pupil records. So, I’m going to read
that again. Schools are responsible
for maintaining pupil scores with the pupil’s
permanent school records or for entering the scores into the electronic
pupil records. “Regarding slides 46 through 47, how can we…how can one
monitor and determine whether all hand-scoring PTs
were completed for all testing… tested students in the LEA?” You can do a number
of things. You can check
the completion status on TOMS by using the LEA level
Completion Status reports under the Reports tab. In addition,
check the CAASPP Guide to Completion Status
and Roster Management available on and under the Completion Status
and Roster Management Guide for 2018 and 2019. If you were a LEA and you have a number
of students who may not have completed
the PTs or sensed
that may have happened, contact CalTAC, send an email, ask them to forward that email
over to Michael McDaniel and I’d like to give you a call
and we’ll chat about that and do a little
investigative work and see what’s going on. “We’ve told students
that it’s okay to skip a question
and come back to it. However, they have to choose
an answer before moving on and the students often
just pick one and flag the question. The pattern for scoring suggests
that this is not a good idea. Should we stop telling kids
it’s okay to skip in something
that they aren’t sure about?” And I’m definitely going
to let you answer this question. And there’s our answer
for the back but…>>So, really how you prepare
students is a local decision. And while flag for review does not influence
the CAT algorithm, it may still provide
a better chance of answering the…
that question correctly. So, pattern scoring
really impacts the [inaudible] once a full
record has been submitted. So, they have…
they can go back, flag an…hey can flag an item,
continue on to another item, and then return to the item
that they flagged and then they can answer it, but at that point
when their score is calculated, it is the pattern
of those responses to the answer
that was administered that was telling them
that will really determine their final score. So, I really won’t get into how to best determine how the student should proceed
with the exam other that…
other than a student does have options to flag
and return items and, you know,
that’s a good option is if they’re not…
they’re not sure at that moment, continue on, and then return to the item.>>And there are other
adaptive tests that are much more–
Thank you very much.>>Right, right.
>>Rigid. And then, this test is not. It’s giving those students
the ability to change their answers and–
>>Absolutely.>>Just a little bit more
mobility in there.>>It’s more student-centered
and experience-centered.>>Wonderful.
Great way…great answer. Thank you. “We’ve told students–”
>>That’s the same.>>That’s that same question,
so we’ve already gone through that one. “Can districts still print
their own Student Score Reports for their students?”
Yes. Definitely. You just [inaudible]
print theirs… those Student Score Reports. You can find information again
for doing that on the CAASPP portal
at under the Student
Score Report options. But, again, you do have
a lot of other options rather than…other than
just printing those out. And if you’d like
to discuss this, please feel free
to give us a call and we’ll be happy
to chat with you about it. And I just noticed
that we’re getting a lot of questions
from charter schools. That’s really nice to see.
>>Great.>>So I have a feeling
a lot of people are going back to school right now
and thank you for sending in your questions. “Are the CAST and non-PT items going to be adapted this year?”>>And the answer is no.
>>And the answer is no. And I think you mentioned
that earlier on in the presentation. So, no, those PTs
will not be adapted.>>Well, so, the PTs are… so the question is asking
about non-PT.>>I’m sorry. Non-PT.
>>They’re non-PT. So, there’s no adaptivity
with this year’s test.>>Thank you. “Will you be covering
the how to assess… access more reports
for school site to compare years and progress?” So you can download
the current year and historical score reports
from within TOMS by using the Reports tab
and navigating to the scores’ PDF tab. From there, you can download groups of student score reports from prior years
by grade school and by grade. So, this person
is a little confused about some part
of the presentation. “If a student exits our district during the testing window, we have to exit them
so that the next school district can enroll them. How do we not exit them
during this time? Am I misunderstanding
something?” Students should be exited after the testing window
has closed unless the student
is moving to another school in the middle
of the testing window. So, hopefully that answers
your question. Terran, you want to briefly talk about the 20-minute
pause rule and how that impacts
flagging or…>>My understanding about
the 20-minute pause rule that students
have about 20 minutes in which if they pause the test that after that 20 minutes
has expired that they are not allowed
to return back to the… return back to that section.
>>Uh-hmm.>>So, that pretty much
locks them out.>>And, you know, one question
we get occasionally, and I do mean occasionally, is that someone believes
that the school…the… or the students… questions that were given
to them skips. Wow.
They skip the question. And that never happens. To answer question two,
you have to answer question one. There’s no other way
to get to question two. But sometimes what we have
are students who… let’s say it’s the end of the… the end of the period
for testing and Mr. Brown says, “Class, I like you to go back
and review all your testing questions, all the questions
that you flagged.” So, Mike is the student,
Mike goes back and he’s reviewing
these questions and I only got about 30 or 40
more seconds to go before it’s the end
of our bell period and Mr. Brown is going to turn
the system off. And we’re going to move on
to our next class.>>Uh-hmm.
>>And I look… I’m looking at question 15 and I flag question 15. And as I’m doing that, I take the answer off
of question 15 as part of my analysis
of answering the question. And Mr. Brown
ends the test that day and we go on to our next class. The next time
I come into the test, I’m going to be presented
with question 15 because that’s the last question that I did not answer. But yesterday,
I actually went all the way up to question 22,
but I just flagged question 15. Now, I’m going back
to question 15. I left it blank. I’m answering it. Now, the next question
is going to be presented to me as question 23, where I left off yesterday. And sometimes, students
are very confused by that, especially if there’s a large
gap of time between the time
that the students answer those two questions. And when we’ve gone back
and done that analysis, we’ve always seen
that sometimes there’s a week, 10 days, sometimes even more before the student goes back
into the test of the LEA. And when you’re in grade three,
grade four, grade five, after a time period like that, sometimes you’re confused
about where the student is or the student’s confused
about where they are. And they raise a hand and say,
“Look, I was on question 15. I didn’t get to see
question 16 through 22. And now I’m on question 23.
What’s going on?” Because the student’s already
answered those questions. They went back, they looked
at the flagged question. They didn’t answer it. And the next time
they come back into the test, they’re presented
with that next question. So, I hope that answers
the questions about flagging and the skipping aspect of it. And if not,
please give us a call. We’ll be happy to talk to you. “Why do I see a student score
report on TOMS, but not in our SIS? I’ve updated my secure code. The student’s core
was ready later than the majority
of my schools.” This would be something
to contact your SIS vendor about and ask them
if they’ve downloaded all those or accessed all those reports
from TOMS. If not, contact CalTAC
and ask them to either research that and they’ll lean on us
on the program management side and we’ll find out
what’s going on. But if you need
the credentialing, those… and your student score reports
are up and they’re getting merged
into your parent portal, we would want to do
some investigating to find where that other
Student Score Report is. I do want to tell
the field, though… that the field is very happy
with Student Score Reports this year
on the electronic format. We’re not getting complaints
at all about it and the process moved on
very, very smoothly. I…I’m just very happy
how that turned out this year. So, if you’re missing
a Student Score Report and you’ve done credentialing
and you have them available for all your other students,
give us a call and we’ll talk…we’ll talk. We’ll look into that
and see where that student is. “Can you explain the difference
between the scores we see in reports in TOMS
versus the bulk scoring reports that will be out
in late August?” The scores in TOMS
comprise of all students who have been scored so far. However, it is possible that not all of your scores
are…have been completed. The completed information
will be available in the LEA data file. And that might be some
of the straggler students that are still out there. But by the time that data file
comes out in late August, early September, all that information
should be out there. And if it’s not, again,
contact us and we’ll… and we’ll do some investigating
to find out what’s going on. “When will the ’18-’19 ELA and Math Student Score Reports
should be available? Also, when I go onto TOMS
and request a report, it sent me an…
it sent me an email that says the ELPAC…
it says ELPAC. And when I clicked on the link,
it was broken.” For 2018 and ’19, ELA and Math Students Score
Reports are currently available. Please go ahead and try again. If you have any issues,
then please contact CalTAC. And for those of you
who don’t know contacts… or CalTAC’s contact number, it is 800-955-2954. “What’s the best way to ensure
I am on the mailing list for future broadcast or webcast? This link was forwarded to me
by my colleague.” Unfortunately, the only way
we can get you on the list right now
is if you are a LEA coordinator or a backup coordinator. So, please lean
on your coordinator to forward those links to you. And you can also check back
at under our Training button and we have all the dates
of the…of the presentations and when they’re going
to be available, too. “For ELA…for English learners
less than 12 months in a US school
and didn’t take the ELA, do we have their Student Score
Reports with mass scores yet or are they
in the straggler batch?” No. All the scoring
has been completed. Scoring is not put on hold
due to an ELL status. So, while the English
Language student may still not yet have a score
for the assessment, a student’s ELA status does not rely on the scoring
of their assessment. Okay. And we have time
for one more question. “Can you please confirm
my understanding of what was shared on slide 27?” One should not exit students during the testing window. So, again, we’re going to touch
on that one more time. Students who will be returning
to the same school the following year
should not be… should not be reenrolled
in the next grade before the ELA’s
selected testing window closes. Do not exit students prior to the end
of the testing window. It may cause confusion
within the system. And this happened the first year that we did the program
a couple years ago. And when we put it in that rule that we just can’t have
any movement of the students going on
to the next grade or being updated in CALPADS until that testing window closes that your LEA was testing in. So, if that window is closing
on May the 3rd, on May the 4th
or May the 5th, go for it. You can go ahead
and do those updates. But we really want you to wait
until the very last day of your testing window
has been completed, we do some work
in the background that evening, and then you’ll be ready
to go again.>>The pause rules [inaudible]>>And the pause rules
are…we… you know, just…
we do…we want to remind you the pause rules are different
for the CAT and the PT. So, if you have any questions
about that, go back and look
at those pause rules at in the Test
Administration Manual, especially
for your new teachers. If you have new teachers
to your school district or new coming in and may have come
from another state, they’re new to the assessment, please remind them
of those pause rules and…you can practice
those pause rules using the practice test as well,
you know. Flag a bunch of questions,
go back, unflag them, take the…take the answer
off of a question, shut the system down,
bring it back up, and look at that pattern
of how the score or those questions
are then presented back to the student. I think they’ll answer
a lot of your questions. Okay. One more question, I think. “Will our LEA dashboard
get to 100% when all the tests are scored? Is there a timeframe for this?” That’s a very good question. Your LEA Dashboard reflects
the percentage of scores completed for your students. The calculations
for the scoring percentages include only students
who are assigned Smarter Balanced
for English Language Arts and Mathematics Assessment. Your percentage
may not reach 100% if you have students assigned
to Smarter Balanced English Language Arts
and Mathematics that did not end up testing. So, we want to thank you
very much for tuning in with us today. If your question
did not get answered, please contact CalTAC
and ask them that question. And if they’re not able
to answer, they’ll forward it on
to our program management team, for…we will be reviewing
the questions and more of them have come in, we’ll have CalTAC
reach out to you and answer that question
for you. So, on behalf of ETS, the California Department
of Education, we want to thank you
for a wonderful testing year. Things went really, really well across the assessment this year. Thank you, school districts,
LEAs, charter schools for all your hard work. And we look forward to working
with you again next year and we’ll see you in September. Thank you very much
and have a great day. [Music playing]

Robin Kshlerin