Progress Monitoring ROI for Academic Interventions

Using progress monitoring scores, an average weekly rate of improvement referred to as the “Trend ROI” using the least-squares regression method is calculated. Using the ROI, a trend line is plotted.  There are certain variables that determine which scores are included in the ROI calculation.

Score Types

• Baseline:  Scores that occur before the start date of the intervention of which the probe is attached. If not attached to an intervention, there will be no baseline scores.
• Fadeout:  Scores that occur after the end date of the intervention of which the probe is attached.  If not attached to an intervention, there will be no fadeout scores.
• Benchmark:  Scores that are entered/imported into eduCLIMBER as benchmark scores, usually three times a year (Fall, Winter, Spring).

An Overview of Rate of Improvement in eduCLIMBER - Configurations

Per Assessment Group (or even down to the Assessment level), certain configuration parameters that affect what scores are included in the ROI calculation exist.  These parameters are not currently able to be changed within the application, but they can be changed by the eduCLIMBER team.

• Include Baseline Scores:  If enabled, all baseline scores will be included in the ROI calculation and trend line plotting (enabled by default for STAR, disabled for all other Assessment Groups).
• Include Benchmark Scores:  If enabled, benchmark scores will be included in the ROI calculation. If enabled along with “Include Baseline Scores.” The most recent benchmark baseline score will be included as well (enabled by default for STAR, disabled for all other Assessment Groups).
• Minimum Score Quantity:  The number of scores required in order to start calculating ROI (3 by default for STAR, 2 for all other assessments).

An Overview of Rate of Improvement in eduCLIMBER - Trend ROIs

Currently there are two separate, mutually exclusive ways ROI is calculated in eduCLIMBER.

• Intervention ROI:  If a probe is attached to an intervention, the ROI displayed on the Progress Monitoring chart and on the grid below the chart represents the ROI for each individual phase of the intervention. In this case, the ROI is stored on the phase itself, not on the probe.  When viewing an Intervention on the Interventions Summary Page or the Student Profile, the ROI listed is the ROI of the latest phase on that intervention.
• Probe ROI:  If a probe is not attached to an intervention, the ROI is calculated across all scores on that probe, utilizing the configuration parameters above.  When viewing Probes on the Student Profile, this is the ROI that is listed.

Where Trend ROIs are Displayed

• Student Profile > Interventions: ROI of the latest intervention phase
• Student Profile > Monitors: ROI of the probe (if not attached to intervention or not primary)
• Intervention Fidelity Report: ROI of each individual phase
• Progress Monitoring Graph: ROI of each individual phase or the overall probe, depending on situation outlined in section above. An Overview of Rate of Improvement in eduCLIMBER - Goal

An Overview of Rate of Improvement in eduCLIMBER - Goal

A single goal score can be entered for a monitor.  Goal dates can be entered, but those dates are currently only used for interventions monitored by point sheet. A goal-line is plotted from the first progress monitoring score (which includes baseline scores) up to the end date of the progress monitoring schedule.  An ROI is then calculated off of this goal line. Note that the intercept (the initial starting point) between the trend line and the goal line do not line up, meaning that each ROI is calculated from a different starting point, so ROIs cannot be directly compared (e.g. a trend ROI value higher than the goal ROI does not necessarily mean the student will achieve their goal score).

An Overview of Rate of Improvement in eduCLIMBER - ROIs Described

Three types of ROIs are displayed in eduCLIMBER:

• Trend ROI
            - Displayed on the Progress Monitoring chart
            - 1st data point within the intervention phase
            - Needs at three data points to calculate

• Goal ROI
            - Displayed on the Progress Monitoring chart, Student Profile, and Fidelity Report
            - 1st data point
            - Date in phase to the end date of the probe

• Plan ROI
            - Displayed on the Student Profile and Fidelity Report
            - Trend ROI for the current phase