What factors are associated with increased quality of CCEE program sites over time as measured in the Quality Rating and Improvement System (QRIS)?

CCEE programs’ quality rating may change over time as they complete the requirements that demonstrate they are meeting quality standards at the next QRIS tier. Answering this question can provide insight into which characteristics, conditions and program management practices may be associated with changes in QRIS rating over time. To answer this question, site level information can be used to identify all programs participating in QRIS, to calculate changes in QRIS scores, and identify programs that have had an increase in quality rating. Once these programs are identified, additional site level information can be used to determine which factors these programs share in common.

Identify Operational QRIS Participating CCEE Program Sites

Select programs with indicated “Active” category in the option set for the Operational Status data element to include only programs currently in operation. The data element QRIS Participation History can be used to isolate program sites (Site ID) participating in the state’s QRIS by selecting the option “Program site currently participates in the QRIS.” The quality level of a program is indicated in the QRIS Score data element.

If the state does not have a QRIS but has another quality improvement system or initiative, use the data element Quality Improvement Participation to isolate program sites (Site ID) participating in the state’s other quality improvement initiative by selecting the option “Program site currently participates in another QI initiative”.

Calculate Changes in QRIS Ratings Over Time

Select the time period desired to calculate a change in score. Rating scores should never be overwritten so there should be a QRIS Score for every time a program site was rated. If program sites are not rated every year, scores will have to be collapsed across time periods such that it is possible to compare the most recent score with the previous one regardless of what year the rating actually took place.

To calculate the change in scores, subtract the previous QRIS Score from the current QRIS Score. Negative scores indicate a decrease in rating while positive scores indicate an increase. Divide the number of program sites that increased by the total number of rated sites, for the percent of sites that increased their quality scores. Average the scores across all program sites to determine the change in scores over time for all sites. Additional elements, such as Early Childhood Setting, can be used to examine changes in QRIS Scores for sub-groups of program sites.

Analyze QRIS Score Change Data by Factors Potentially Associated with Changes in QRIS Scores.

Several categories of data elements may be associated with increases in a program site’s QRIS Score. Some of those categories are Quality Improvement, Workplace Conditions/Policies, and Program Management. Below are two examples of analyses that could be performed examining the intersection of quality scores with other factors that may not be included as indicators when making ratings determinations.

Example 1. Do program sites that engage in inclusion activities have higher quality rating scores? First, calculate the change in QRIS Scores, by subtracting the previous QRIS score from the current one. Scores that are positive indicate an increase while scores that are negative indicate a decrease. Compare these scores between those that do and those that do not engage in any of the activities in the data element Inclusion Activities. Another option is to create a scale of the five inclusion activities and compare the scale scores to the change in ratings.

Example 2. Do program sites that incorporate child assessment into their practice attain higher quality rating scores? First, identify two time points in which to measure improvement, for example the most recent date of QRIS score assessment and the previous date of QRIS score assessment. Then identify programs that use child assessments. These program sites are indicated with a “Yes” for the data element Use of Assessment.  Next, calculate the change in QRIS Scores,by subtracting the previous QRIS score from the current one. Scores that are positive indicate an increase while scores that are negative indicate a decrease. Program sites can be separated into four groups: those that did not and do not use child assessment; those that did not but now do use child assessment; those that did and now do not; and those that used child assessment at both time points. Compare the change in ratings for those that have never used assessment and those that started using assessment to see if the latter group showed greater positive growth in their quality scores.