Understanding eduClimber and FastBridge Rates of Improvement

The ROI values in FastBridge are calculated by determining the amount of growth in weekly units. The formula used for the ROI is:

• ROI = (Ending Score - Baseline Score) / Number of Weeks

For example, if a student had an ending CBMreading English score of 98 [words read correctly per minute], a baseline score of 75, and participated in 12 weeks of intervention, the resulting ROI would be:

• ROI = (98 - 75) / 12
• ROI = 23 / 12
• ROI = 1.91 words gained per week

The same method was used to calculate the goal ROI values in the FastBridge progress monitoring system. Each FastBridge progress measure has a default value that was selected based on research indicating how much weekly growth is possible. The default values can be adjusted when a progress monitoring schedule is created; more information on setting progress monitoring goals is available in Progress Monitoring Goals: Understanding Rates of Improvement for Reading and Math Assessments. Teachers can then compare the ROI goal with the student’s obtained ROI in order to determine the extent to which the intervention is effective. Here is a FastBridge progress graph with the Total Trend and Goal Line Trend ROI data highlighted below it.

The above progress monitoring graph indicates that the student is making a weekly gain of 0.88 words read correctly per minute, but the goal is to be making a weekly gain of 1.15 words read correctly per minute.  In this case, the educator will need to pay close attention to the weekly gain as they collect additional data points in order to determine whether adjustments need to be made to the intervention.

Goals

FastBridge progress monitoring goals are calculated when a progress monitoring schedule is created using the following formula:

• Goal Score = Screening Score + (Rate of Improvement x Number of Weeks)
  --or--
  G = SS + (ROI x N)

The screening score comes from the screener that the student completed in the most recent universal screening period.  If the progress measure is not the same as the screener, then the teacher needs to administer the screening version of the progress assessment to obtain a screening (i.e., start) score. 

The FastBridge system includes a range of rate of improvement (ROI) growth levels for progress measures. The ROI values are based on the average (typical) amount of growth that was made by students in the national normative group. In order to help teachers set appropriate goals for their students, FastBridge categorizes ROI levels into four groups. 

• Very Realistic: growth that is less than 80% of the average
• Realistic: growth that is between 81% and 100% of the average
• Ambitious: growth that is between 101% and 120% of the average
• Very Ambitious: growth that is between 121% and 150% of the average

The FastBridge system default ROI for each progress measure is set at the top end of the Realisticrange of ROI values for that measure. This default value equates to the average rate of growth. Users can change the ROI value when creating a progress monitoring schedule so that a lower or higher student goal will be calculated.  Here are examples for CBMreading:

The Goal is calculated using the formula above and the higher the ROI value, the higher the goal. It is important to consider how likely a student is to reach the goal in relation to the intervention provided.  More frequent and intensive interventions are likely to result in higher growth and goal attainment. The FastBridge Goal score formula is designed to help teachers set goals that students can reasonably attain during the current school year. 

The tables below show the ROI ranges and corresponding national growth percentiles, derived from the aggregate growth norms, for FastBridge reading and math progress measures. 

Using Letter Sounds as an example, we see that the Very Realistic range corresponds to an average weekly gain, ROI, of less than 1.51 letters per minute.  The top end of the range,1.51 corresponds to the 67th national growth percentile. The Realistic range extends from 1.52 through 1.89. The top end of the Realistic range is the default ROI value displayed in the progress monitoring set up to determine an end-of-year performance target. As shown above, users can change the ROI value when creating progress monitoring schedules so that the student’s goal is based on the expected ROI.

Values and Percentile Ranks for Progress Measures in Reading

Values and Percentile Ranks for Progress Measures in Math

*Correct per 10 min on PM report

eduClimber also provides ROI data. Using progress monitoring scores, an average weekly rate of improvement referred to as the “Trend ROI” using the least-squares regression method is calculated. Using the ROI, a trend line is plotted. There are score types that determine which scores are included in the ROI calculation (a) baseline, (b) fadeout, and (c) benchmark. 

• Baseline: Scores that occur before the start date of the intervention to which the probe is attached. If not attached to an intervention, there will be no baseline scores. If enabled, all baseline scores will be included in the ROI calculation and trend line plotting.
• Fadeout: Scores that occur after the end date of the intervention to which the probe is attached. If not attached to an intervention, there will be no fadeout scores. 
• Benchmark: Scores that are entered/imported into eduCLIMBER as benchmark scores, usually three times a year (Fall, Winter, Spring). If enabled, benchmark scores will be included in the ROI calculation.

Using progress monitoring scores, an average weekly rate of improvement referred to as the “Trend ROI” using the least-squares regression method is calculated. Using the ROI, a trend line is plotted.  There are certain variables that determine which scores are included in the ROI calculation.

Configurations

Per Assessment Group (or even down to the Assessment level), certain configuration parameters that affect what scores are included in the ROI calculation exist.  These parameters are not currently able to be changed within the application, but they can be changed by the eduCLIMBER team.

• Include Baseline Scores:  If enabled, all baseline scores will be included in the ROI calculation and trend line plotting (enabled by default for STAR, disabled for all other Assessment Groups).
• Include Benchmark Scores:  If enabled, benchmark scores will be included in the ROI calculation. If enabled along with “Include Baseline Scores.” The most recent benchmark baseline score will be included as well (enabled by default for STAR, disabled for all other Assessment Groups).
• Minimum Score Quantity:  The number of scores required in order to start calculating ROI (3 by default for STAR, 2 for all other assessments).

Trend ROIs

There are two separate, mutually exclusive ways ROI is calculated in eduCLIMBER.

• Intervention ROI:  If a probe is attached to an intervention, the ROI displayed on the Progress Monitoring chart and on the grid below the chart represents the ROI for each individual phase of the intervention. In this case, the ROI is stored on the phase itself, not on the probe.  When viewing an Intervention on the Interventions Summary Page or the Student Profile, the ROI listed is the ROI of the latest phase of that intervention.
• Probe ROI:  If a probe is not attached to an intervention, the ROI is calculated across all scores on that probe, utilizing the configuration parameters above.  When viewing Probes on the Student Profile, this is the ROI that is listed.

Where Trend ROIs are Displayed

• Student Profile > Interventions: ROI of the latest intervention phase
• Student Profile > Monitors: ROI of the probe (if not attached to intervention or not primary)
• Intervention Fidelity Report: ROI of each individual phase
• Progress Monitoring Graph: ROI of each individual phase or the overall probe, depending on the situation outlined in the section above. An Overview of Rate of Improvement in eduCLIMBER - Goal

An Overview of Rate of Improvement in eduCLIMBER - Goal

A single goal score can be entered for a monitor.  Goal dates can be entered, but those dates are currently only used for interventions monitored by point sheet. A goal-line is plotted from the first progress monitoring score (which includes baseline scores) up to the end date of the progress monitoring schedule.  An ROI is then calculated off of this goal line. Note that the intercept (the initial starting point) between the trend line and the goal line do not line up, meaning that each ROI is calculated from a different starting point, so ROIs cannot be directly compared (e.g. a trend ROI value higher than the goal ROI does not necessarily mean the student will achieve their goal score).

An Overview of Rate of Improvement in eduCLIMBER - ROIs Described

Three types of ROIs are displayed in eduCLIMBER:

• Trend ROI
            - Displayed on the Progress Monitoring chart
            - 1st data point within the intervention phase
            - Requires three data points to calculate

• Goal ROI
            - Displayed on the Progress Monitoring chart, Student Profile, and Fidelity Report
            - 1st data point
            - Date in phase to the end date of the probe

• Plan ROI
            - Displayed on the Student Profile and Fidelity Report
            - Trend ROI for the current phase