Caleb Simmons (University of Arizona) on Using Data Partnerships to Better Understand Online Learners

On today’s episode of the Illumination by Modern Campus podcast, podcast host Shauna Cox was joined by Caleb Simmons to discuss how modern data partnerships and internal alignment are key to engaging and retaining today’s online learners.

Transcript

Shauna Cox (00:02):Caleb, welcome to the Illumination Podcast.

Caleb Simmons (00:06):Thank you for having me

Shauna Cox (00:07):So we are here to talk about building effective data partnerships to drive engagement and retention, especially in online education. Online education has seen quite the evolution now to play on the words here over the last couple of years. So I want to kick off our conversation and first ask you what kinds of data partnerships have the greatest potential to transform engagement and retention when we're talking about that online learning environment?

Caleb Simmons (00:35):Well, thank you for the question. Again, thank you for having me. It's a lot of fun to be able to come in and talk about some of the things we're doing in Arizona online and how that applies a little bit more broadly for higher education. First thing I'll start with is the good problem that we have in online is we have a wealth of data from tracking learners and LMS to our student information systems and our CRMs. We just have a lot of data that we can analyze to continue to refine our processes. But while this is a problem because we often don't know what to do with that data or we don't have systems in place to analyze it, one thing we've been trying to do in Arizona online is to make sure that we're asking the right questions to make sure that the data that we're capturing is going to be able to help inform those.

(01:34):So getting a little more directed with our data collection. And as a result, this makes our partnerships very wide ranging. As you might imagine internally we have under the umbrella of Arizona online, five different teams that report up that all have different concerns and collect different data. So making sure that there is an internal partnership and that we're all aware of the questions that the other team is asking and helping one another frame that and think through its implications from the beginning of the marketing funnel all the way to continuing learning after they get their degree here. So thinking about it as a broader ecosystem, but then moving out the way we're centrally located at the university, we also have data partnerships with the colleges where our learners reside and where they earn their degrees and receive their instruction and they're collecting different sorts of data and they're asking different types of questions, but often the questions are overlapping and related.

(02:36):And then to stacked on top of that, the institutional research, which we lovingly call you Air University of Arizona institutional research. They're also collecting data and they have their own set of questions regarding the learner experience, financials, all of these things. So you have all of these different teams asking similar questions, trying to figure out what they're tasked with, but also trying to collaborate and help the other units. So all of these have such great potential because we are not only institutions of learning for our learners and students, but also we need to continue to learn. And so all of these partnerships are really rewarding because the way someone else is approaching a problem using data informed approaches can inform how we're going to be able to approach the same thing. So returning to the meat of your question about engagement and retention for online learners, I think this is where the partnerships can be fruitful for us engaging.

(03:40):I'll speak now for college partners and for institutional partners. We are part of an institution that's a traditional brick and mortar. And so often when we think about data, those are the assumptions from which we work. So primarily retention, IEDs, first time, full-time census date. This is so ingrained into the reporting structures, but it's also rooted in a model that does not reflect the student learners that we have in our population. Part of that partnership is education both ways. Why is it important for the institution that we think about that? Obviously it's because rankings and all these things that we want, but also getting the institution to understand what we need to be able to consider. So we need to think not just about census state, but we need to think about rolling admissions and what this means for a student. How do we understand our population better as primarily part-time primarily transfers?

(04:44):So shifting that conversation of retention more to a traditional customer service model of retaining someone through persistence from term to term. And then over time and figuring out completion rates based on that. Because if you say four years, obviously that doesn't work for part-time students, but then you start looking at the number of credit hours or units that they're taking. Maybe a realistic graduation timeframe is eight to 10 years because just the math doesn't math for anything shorter than that. So it's a partnership to inform our practices, but also to educate one another about the needs and the student populations that we

Shauna Cox (05:24):Serve. Absolutely. There's so many little golden nuggets in there that I definitely want to follow up on, but I'll first touch on the point that you made about the customer service model. Student needs are changing constantly. Higher education isn't what it looked like to five, 10 years ago, and we're always needing to adapt and evolve to what our students customers, however you want to phrase them, what they need and want. So how do you balance that need for actionable insights to keep up with what they're asking for with those challenges of data integration across various systems with various teams as you mentioned?

Caleb Simmons (06:09):Yeah, and those are two big points. The first part about the changing landscape, I mean, you could make a realistic argument that it wasn't today what it was yesterday. And just with all the changes going on in this space with regulations to business practice, and I often introduce the word customer even though I know it's provocative in these contexts because when we say student, we do have that traditional mindset often at least in institutions that are similarly situated as University of Arizona. And as a result, we think about retention in that year to year because that summer months is when students could just choose to leave and not return for online. We've got a seven and a half week model that happens every seven and a half weeks. You just don't have to enroll the next time. You don't have to pack up your dorm and leave town.

(07:07):So we need a little bit more of that approach of there are fewer barriers to leave an online institution. And ultimately the idea then is that introduce something like the customer and then you could settle on learner somewhere between the traditional student and the customer mindset to where everyone feels like through that negotiation, their position has been well represented. So I think that there's a need to get a little bit more of that conversation going at institutions because it helps us unsettle the narrative of who we are and who our students are. But then when you think about the challenges of data integration, I'll give an example of the University of Arizona of how we've built up systems to be able to meet the learner demand, but those systems have in some ways not been, if an engineer was creating it, they wouldn't create it this way though.

(08:05):When you come into the University of Arizona online, we use one CRM for our application phase, and then whenever you matriculate in, you move to another CRM that's integrated within a student information system. And then we have additional ways that we track people using third party vendors and whatnot. So there's a lot of back work that needs to be done to get all these integrated, but then you start talking to different teams and the way the marketing team tracks lead to enrollment is slightly different than how our academic team is going to track an application to enrollment. And then trying to build that large student journey from initial marketing spend through RFI to matriculation on into the completion, it's difficult to really be able to see what's happening and track a student all the way through, but you still have to try because you have to be able to understand what the student journey really looks like and who your student population is.

(09:17):And I almost said typical student, but it's really your typical sets of students where you can kind of make these larger buckets and then start to really target the interventions that they need to be successful. And that's where you can start making actionable insights is where you can disaggregate the whole Right now about 10,000 students. That's just a big group. And what we start to disaggregate, we do have groups that are similarly situated to in different interventions. Our military connected students are adult learners, our traditionally aged learners. There's different communication strategies needed. And then we are very big here into using the data that we can collect to run pilot programs. So anytime we're going to try to move the needle on student success, persistence, retention, completion, we want to measure because we are in a world of limited resources, if we're going to invest in something, we need to be able to see the payoff. And so again, finding the questions we want really using the scientific method, we got to our hypothesis, here's what we need to measure to be able to measure that, run our pilot, see if it was worth it, if it is, invest to a larger degree and move forward. If not, sunset that and move on to the next.

Shauna Cox (10:42):Absolutely. And I think what we're talking about here are partnerships and not partnerships just within the institution itself. I know you mentioned cross campuses and other places outside of that one institution. So what role do cross-departmental or cross institutional collaborations play in unlocking that full value of the learner data that everybody is using?

Caleb Simmons (11:07):Yeah, the cross-departmental in some ways winds up being a little more difficult at times because in institutions, your silos can be very strong. And when you talk about or learners between Arizona online or college partners, one thing that I can always guarantee is that we want what's best for our learners.

(11:31):And so the outcome we know should be the same. They should be complete, they should meet their academic goals. The process to get them there is sometimes we see different routes to get that. So the difficulty in that, again, it goes back to this sort of translation, is like from our perspective, this is what we think the learner needs. From your perspective, this is what you think the learner needs, let's dive into the data and let's understand a little bit more about what that means. And a great example for us has been some of our general education foundations courses and working with the different departments on campus to be able to understand course length in particular was one question we have is how long should a math course be? Can you do it, accelerate it or does it need to be extended? So getting data from that.

(12:23):And then the amazing thing is in online education, it's still a community where people really are trying to build collectively best practices to move forward. So it was nice that I was able to call up a SU online, it's supposed to be our in-state rivals, but they had asked the same question. They had collected data, we were able to have a really meaningful conversation about what they saw because their scale is larger than ours, what they saw at that scale. And it was in alignment with what we were seeing with our data as well. So it really informed us to really be able to push forward with making a decision that isn't going to be harmful to students, but actually is going to help them toward that progress. So it's nice that institutions want to work together to be able to solve some of these larger higher education issues that we're running across.

Shauna Cox (13:18):Absolutely. I love over the past few years really this drive for collaboration among other institutions and sharing playbook, sharing ideas, and having a healthy competition rather than this big rivalry. You can still have some fun and some competition, but I think at the end of the day, we're all here to meet the needs of the learners. And I think that collaboration is key. And previously you mentioned the, I mean the big question that I hear among higher ed leaders is, well, we have the data, but what are we doing with it? Where does it go? What's the next step? And especially in your situation where a program is every seven weeks and there's always that risk of are they going to stay, are they going to go, what are some of those critical data points for identifying and supporting those at-risk online learners?

Caleb Simmons (14:13):If anybody listening has cracked this code, please send it to me because this is what we're constantly trying to identify. And I honestly think it's constantly changing. And this is something where when we're looking at at-risk online learners, we look at our population. I mean, I'm very proud of the population we serve or learners who have tried and potentially had not been successful or they had things come up in their lives that derailed those original plans and they're coming back. This is a large part of our population. There are folks who have resilience, have persistence in the nomenclature of University of Arizona. They know how to bear down. And so we have this really unique population, but this population also has millions of other obligations that they're dealing with. So in some ways, when you talk about the online learner by traditional standards, they are at risk by their very nature of who they are.

(15:20):So supporting them overall is key. But things that we've tried here is to really think about that experience being able to provide mental health counseling. And a lot of this is coming from student feedback. We're polling our students. We have a student experience survey where they're able to identify their own needs when students choose not to enroll, we have an outreach program to where we check on them and see, can we get you enrolled? But really the question that we're asking is, what's keeping you from enrolling? And I think most people listening are going to know the top two or related to finances. And then for us, the next one is work obligations. And so these are things that are very difficult to solve. I know that we can provide emergency funds for those lower financial needs, but lower dollar amount financial needs. But often we're talking about larger issues that students facing.

(16:29):So trying to think of what's within our control. And what we found is after those first three large rocks that are largely out of our concern is availability of advising in a schedule that meets their schedule as well. And then availability of courses in a predictable manner. So that's the feedback that we received directly from students. So that identifies how we need to work to move forward to be able to address those needs. But then other ones were mental health concerns. So we've contracted with a third party vendor to do that. We've heard mentoring as part of a belonging, a need for belonging. So introduced mentor Collective, and we call it Mentor Cats, where we put students in touch with each other, va, SMS relays. And this seems to be really moving it forward a little bit, but there is a whole wealth of other things that we need to implement through getting reports from the LMS about people's turning in assignments, being late logging in.

(17:36):There's a lot that we could do, but we have to balance that all that data we have about learners and where we're helping and where we're actually just causing more anxiety, causing more. And so it's a delicate balance. And luckily we have a wonderful student success team here at Arizona online who collaborates closely with student success and retention initiatives and advising on campus to build an ecosystem where we bounce ideas off of one another and we align communication strategies to really try to find that sweet spot of making sure we're engaged and providing information and resources to students, but also not overwhelming them with being a helicopter parent constantly trying to push them through the institution.

Shauna Cox (18:28):Absolutely. No, that's very fair. And I think within that balance, there can be that need to jump on board, put out everything, all these other things. And sometimes on the other side, you can get very lost in the data. There's so much of it, you don't know what to do with it. It can become overwhelming. So how can institutions ensure that these data-informed strategies that they're creating or implementing remain student-centered, especially when you're looking in large scale online programs?

Caleb Simmons (19:01):It's another great question. And I think the first thing I am really big into having a mission, vision and values that is always looked toward. And we just had commencements this past Friday, and one of the first things that our president ish Guam said was Students our North Star. And I thought it was such a beautiful way to start because that is it. And if you think always students as the North Star, an example that I always remember about a mission centered approach is Southwest Airlines when they wanted to be the top low budget airline, anytime someone would come to the CEO with an idea, his first question was, how does this work making us the best low cost airline? And if it didn't, it's not an idea that needs to be pursued. And so that is the approach we take at Arizona online is when we start a new initiative, the first question is, does this make us more student-centered?

(20:06):Does this help our students? And if it doesn't, then it's probably not the thing that needs to be taking our attention at this time. And once you have that as your north star, your guiding direction, then your strategies automatically start to align with that. And so you set the strategy, you ask the right questions for how you achieve that, and then you're able to turn to your data to inform that approach. And so really it's kind of the design thinking approach. You figure out what you want, you want students to persist and complete, and then you start working it back from there and be able to find at the beginning of the journey, we actually need to introduce this orientation. And then they will have that information then to continue on and persist. Or we find that after they go through their first four terms, which would be equivalent to two semesters, we find a dip off.

(21:01):We need an intervention there, and what do we do there? So I think that that sort of ties us back into the very beginning as you ended the question, it was about at large scale INE programs, this is where we actually have a leg up on traditional brick and mortar. We have way more data about our students and how they're interacting with content, with assessments, with assignments when they walk into their virtual classroom. We know that with the LMS, we know how long they were in there. And so being able to ask and ask those right questions to ensure that our students are getting the best experience that they can, both from a satisfaction level but also from a pedagogical, and then we can collect the data and really execute to make sure that we're fulfilling our promise to

Shauna Cox (21:57):Them. Absolutely. And I love that design thinking approach and working backwards. I think that is really critical when being student-centered meeting the needs that they have and are constantly evolving. Now, those are all the questions I do have for you, but is there anything else that you would like to add? Anything I missed? You guys do have that leg up? Any lessons learned over the years or advice you'd like to share?

Caleb Simmons (22:21):Yeah, I think it's specifically about data. The thing that I would encourage everyone to think about is your big rock questions. What is it that you can identify within your institution are your large things that you need to accomplish? And then start working back from there and think, what are the low hanging fruits? What are the quick wins we can get from morale and moving forward? But it also, it helps you to be able to ask the right questions of your data and then bringing in your folks from institutional research, your college and academic partners to say, what questions am I missing? What data can we collect? And then start really being intentional with the data that you're collecting and thinking through how are we going to analyze this? What are our metrics? And in that way, those big rocks wind up becoming a lot of smaller pebbles that you can tackle along the way. So I think as we continue to do this, we'll grow as a modality. And again, places like OLC ea, we can share this and overall we can start to move the needle even farther on online education because it is the modality of the future.

Shauna Cox (23:45):Absolutely amazing. Well, Caleb, thank you for sharing your insights. Before I let you go, we do need to ask an important question around food because that's important on this podcast. So you are based in Tucson, Arizona. If someone's coming to town, where do they need to go to eat?

Caleb Simmons (24:02):Oh, this is a tough one. Lots of good food here. A little plug for Tucson. It is a UNESCO world, gastronomy city, so we have a lot of fantastic foods. So I would say, I'm going to give you two spots that you have to go to. So the first one is the oldest continuous running Mexican restaurant in the US It is called El Charro, and they're famous for their carne seca, which is beef cooked in the Tucson sun. It was very slow cooked. It's delicious. You have to try it. And then the next is my favorite taco shop, which is Juanitas. It is a wonderful place started by a former boxer, delicious food, and if you're hanging out with me for a very long time, we're also going to go to Buffalo Wild Wings because it is my favorite spot no matter where I am in the world.

Shauna Cox (24:59):Amazing. I love it so much. I love all the recommendations. It's always great to have many options. Caleb, again, thank you so much for coming on. I really appreciate the conversation.

Caleb Simmons (25:09):Thank you, Shauna. Bye.