Few institutions are performing sophisticated predictive modeling on factors affecting student retention, and given how many factors there are (both within and outside the institution’s control), predictive modeling can appear quite daunting. But you don’t have to go from A to Z all at once.
“Typically,” Jim Scannell, president of Scannell & Kurz Inc, advises, “we encourage institutions not to leap into predictive modeling immediately. Start by doing univariate analysis, collecting descriptive knowledge.”
For example, out of an entire class, you could set out to describe:
- How many men retain versus women?
- How many men versus women achieved higher than a 3.0 GPA?
- How did your transfer students from two-year institutions perform versus your transfer students from four-year institutions?
- Students from public versus private high schools?
- Student cohorts based on race?
Suppose you find that men in that class are, on average, achieving a lower GPA than women. Can you dig deeper? For example, if you have strong athletics, compare both the academic preparation of entering athletes versus non-athletes and the academic success of those two groups during the first year. Do you have a lower GPA for male students because you enrolled 100 football players who were less academically prepared than the rest of their class? Or, if there is no significant correlation, are there other likely factors you can check?
"Descriptive analysis will help you understand what hypotheses you want to test using a predictive model."
Jim Scannell, Scannell & Kurz
The key is to get started.
Moving the Needle
Let’s look at one example of predictive analysis—assessment of students’ non-cognitive skills—and sample tactics informed by that analysis.
"Scores and high school GPA only account for about 20 percent of the variability we see in student outcomes. Some students with a respectable GPA and high scores underperform academically in college and drop out, while other students who appear academically under-prepared then proceed to perform highly. This means that some of the students you are losing are in good academic standing. They don't appear to be "at-risk students." We need better predictors of student success."
Paul Gore, University of Utah
Citing a 2004 meta-analysis, Paul Gore, the student success special projects coordinator at the University of Utah and the past director of the Career Transitions Research Department at ACT, notes six non-cognitive variables that appear to have the greatest impact on an institution's ability to identify those students who are likely to succeed. These are not the only non-cognitive variables that impact student success (for example, communication skills are also important), but these are the six variables that, when assessed together with other traditional, cognitive variables, offered an incremental increase in predictive accuracy.These included:
- Indicators of academic performance, such as academic engagement (the student’s diligence in their studies) and academic efficacy (the student’s confidence in their ability to complete academic milestones)
- Indicators of academic persistence, such as educational commitment (the student’s motivation for achieving a degree) and educational engagement in extra-curricular activities
- Indicators of emotional development and maturity, such as resilience in response to stress and comfort level in social settings
Gore recommends employing a non-cognitive assessment (of which there are many currently on the market) during the admissions process, to help predict which students possess those non-cognitive skills that drive student success.
If yours is a selective institution, this assessment can inform recruiting, by empowering you to make better decisions about which students to enroll based on which students are likeliest to persist and succeed at your institution.
If yours is an open-enrollment institution, this assessment can help you:
- Identify which students in a traditionally “at-risk” cohort are most and least likely to need support and intervention (allowing you to devote limited resources more effectively)
- Identify cohorts of students who would benefit from a summer bridge or first-year program focused on building non-cognitive skills, or from other student services
Here are two resources for digging further:
- Read our article “Predicting Student Success: When SAT and GPA Are Not Enough”
- Order an online training on “Developing Academic Stamina in First-Year Students”
A FURTHER EXAMPLE
In our recent article "DFW Rates and You: Rethinking Support for At-Risk Students," Bernadette Jungblut, West Virginia University’s director of assessment and retention, offers ideas for mining your data in courses with high DFW (D/fail/withdraw) rates to inform a more sophisticated and effective approach to identifying and supporting at-risk students.
In This Issue
A Letter from Amit Mrig, President, Academic Impressions
Reviewing Your Data: What You Might Find
Data-Informed Recruitment and Enrollment
A Data-Informed Approach to Student Retention
Mining Your Data: From Students to Alumni