Joe Levy (Excelsior University) on Amplifying Awareness: The Vital Role of Assessment in Student Affairs

On today’s episode of the Illumination by Modern Campus podcast, hose Amrit Ahluwalia was joined by Joe Levy to discuss the value of assessment of student affairs and how collaboration can bridge some of the gaps across the institution.  

Transcript

Amrit Ahluwalia (00:02):Joe Levy, welcome to the Illumination Podcast. It's great to be chatting with you.

Joe Levy (00:06):Yeah, great to be here.

Amrit Ahluwalia (00:08):Well, you know what I mean. We're talking today about assessment and the role of really the broad role of student affairs. I think you and I have chatted about this topic in the past because it always amazes me how foundationally misunderstood student affairs departments are within the context of strategy and the role of higher ed institutions. So let's start off there. I mean, right off the top, why is it important for higher ed leaders to focus on assessment within student affairs, and how does this focus on assessment start to, validate is the wrong word, but really highlight the work that student affairs units do?

Joe Levy (00:52):Yeah, I think it's hard to, so I think assessment becomes this great mechanism and this great avenue to elevate the work of student affairs because one of the things that I always say, whether I'm presenting internally or externally or consulting with schools is I have a slide that I've used all the time that poses the questions. I ask the room, raise your hand if you don't need to know the answers to these questions. And it's what are students learning? Are we providing needed support for student success? How can we improve quality where necessary? How can we be better stewards of resources and how can we articulate what we're doing to outside parties, right? And assessment helps us do all those things, but also those are questions that everyone needs to know. Whether you're in student affairs, academic affairs, administration, everyone needs to know the answers to those questions.

(01:48):And so assessment really helps become this common language and common ground for student affairs to walk on with faculty and academic affairs that I think in a lot of other spaces, they might not even be invited to the table or they might not even be present to be having those conversations. But assessment allows us, if we're talking about things like critical thinking or career readiness, professional skills, communication skills, that's something that all academic programs are concerned about, employers are concerned about it, and then student affairs can speak to how they're helping support that and demonstrate that for students outside the classroom as well. So I think assessment really becomes this way to help elevate and increase awareness and transparency around student affairs and co-curricular services that I think in a lot of ways people take for granted or that they don't think about counts as learning.

(02:52):They think of learning in a very one dimensional way of in the classroom, and they don't think about things like, I love to use example of financial aid. I mean, people do not know what responsible borrowing means, right? Students don't know how to handle loans. That is not something, I mean, maybe at some point in a business related class or accounting or so you might cover that, but students are first faced with that when they're talking to financial aid, when they're trying to figure out what's going to help support them financially to engage in this academic journey. And so that is knowledge and skills and abilities that exclusively financial aid is providing and encouraging and promoting. And we don't always think about that as learning, but it totally is. And I think that's a big thing. That assessment can help let other people see student affairs in a different light and see them as educators alongside faculty.

Amrit Ahluwalia (03:54):It's so funny you raised the financial literacy piece around roles that student affairs units can really play in delivering a meaningful and holistic experience. There's a meme that goes around usually every September member where it's something along the lines of the government feels I'm not responsible enough to drink a beer. Fortunately, I am responsible enough to take out $300,000 in loans for my higher education. And it's just one of those very, it's a crystallizing moment of how we prioritize. And I want to put some numbers behind the context that you're talking about because I think it's really important. In fact, Grammarly released a research report today on workforce readiness of modern learners, and it's one of these beautiful moments of inconsistency between what we think we're doing as an industry and what our stakeholders think. 98% of higher ed leaders say that their career prep and their work to prepare individuals for the life after college, 98% of higher ed leaders felt that what's happening today is effective.

(05:04):Only 25% of graduates felt that they would choose the same education path. Only 41% of graduates feel their college degree actually signals the skills employers need. Only 11% of business leaders say that college graduates are ready for the workforce. It's a massive, massive variance. And speaking about the importance of the student affairs unit in general, we know that there's a 24% first year dropout rate. We know only about 60% of students who start a degree actually finish it, and we know that only 28, 30 ish percent of students who drop out do so because they lack the academic skills necessary. So I wanted ground this discussion around the importance that comes into the work that these units do, because at the end of the day, these are all factors. These are all opportunities for a student affairs division to play a very meaningful role. So what are some of the challenges that you and your colleagues face when it comes to actually assessing student affairs activities and frankly getting visibility for that work in general?

Joe Levy (06:12):Yeah, I think one of 'em ties back to what I was saying for the first piece is that both individuals who are within student affairs as well as the rest of higher ed, can very much not identify or treat them as educators. They can look at them as support staff or service providers, right? Too often we are reducing student affairs to just a client customer transaction, a transaction component, and not a transformational component. And so I think that's to begin with, because if I'm trying to work with somebody and trying to, again, I go to approach financial aid and say, let's talk about the learning you're building with students. And they say, what do you mean? We're just processing loan requests, we're dealing with fafsa. And it's like, no. Some of it is an education component of recognizing the space you hold and the impact you have with students and the role you're playing here.

(07:15):I think another big aspect to it falls into the world of my dissertation around I looked at motivation around assessment. I looked at this self-determination theory, and the reason why I gravitated towards it is that it speaks to the fundamental challenges with assessment. It's talking about how individuals, the extent to which you're meeting the needs of competence, autonomy, and relatedness, the extent with which you meet those can add to or take away from your ability to self-regulate and engage in the world, engage in an activity, engage in your daily life, put energy towards something, and that speaks directly to assessment. Because if you don't know what it is, how can you be successful in it? If you don't feel like you have the autonomy to make decisions and make changes based off of the data, then you're not going to want to engage in it, right?

(08:12):You're going to feel like you're just going through the motions. And from a relatedness component, if you don't feel like this relates to you or that you have a connection to this work, you're not going to want to do it either, right? You're going to feel like this is just something you're being told to do or do for Joe, the assessment guy asking you to do it. So I really feel like there's a lot of education we can be doing to help people see the impact they can have, but also help them see the data and the impact that we're leaving on the table because we're not accounting for it. And then probably one of the biggest things that cuts across assessment or really any report is taking action. I think just like people maybe don't know how to engage in assessment and analyze data, they know even less about then what to do with that data and how to take that data and translate it to action. And then it comes back to that autonomy piece that they may not feel empowered or that they have the authority to then make this an action item or put this in the strategic plan or get the resources needed to back up what the data's telling them they need to do.

Amrit Ahluwalia (09:27):I do want to dive into best practices in a moment. I think that it's such a valuable topic, but before we get into it, I want to dive a little bit into your personal interest in this assessment arena, because obviously your new role, you're the associate vice Provost for accreditation and quality improvement at Excelsior. You've taught a MOOC on assessment. For those who might not be familiar, the Student Affairs Assessment, readers Association, S A l ran a MOOC earlier this year where Joe was an instructor and a manager in delivering that programming. This is a topic that you've been passionate about for a very long time. In fact, in off the record conversations, you and I have had been very much centered around making assessment a foundational part of how we think and what we do and how we work. So I'm curious, what drove your interest in this space in the first place? Why is it something that you're so passionate about?

Joe Levy (10:24):To me, it helps surface some of those intangibles around student success and student experience. I think about, I was a military bratt, so I moved around all the time growing up and college was a very, college was a very enlightening time for me in life and broadening my horizons, being in a setting much larger than I'd typically been used to in the towns where I'd been. And it had such a exponential transformational impact that I had outside of the classroom. We certainly had a great academic experience, don't get me wrong, but the big things where I learned to connect with people, I learned from my peers. I gained skills in leadership through being involved in student activities and through watching some of the student affairs professionals run offices and run orientation and demonstrate the passion and the work that goes behind all of these experiences and the intentionality to it, it's incredible.

(11:38):And it's so important. I mean, you were just talking about so few students actually stop out of college because they academically can't cut it, right? It's all the other stuff. It's sense of belonging, it's financial support, it's mental health. I mean, it's all sorts of things that are outside the bounds of the academic classroom. And when we think about some of the key primary metrics that higher ed is using to evaluate R O I or success, it's really not surfacing the richness of the student experience and student success because I say all the time, what we talk about, completion being a big measure of success for colleges. Well, we could just give everyone degrees, right? Well, so then, okay, completion is a proxy for something else. We talk about persistence, retention, but we can always just pass students along. Look at the K 12 system.

(12:38):We're passing students along. They're getting to college, they're not college ready. So persistence and retention, again, are proxies for something else. So really we have to surface that messy, complex learning. We have to measure that. And assessment is this wonderful blend of science and art to try and quantify and understand that. And so to me, it's just fascinating, super fascinating. And because having done this for my career, I'm still learning, and you talk about the open course that I teach, been doing it for eight years now, and I always learn something and it's always a good reminder of, oh yeah, I could be approaching it this way. I could think of it that way because students are ever changing. The environment we're operating in is ever changing, and I just feel like assessment really becomes a grounded way and at least an attempt to help understand what is a phenomena of learning and student success that so many people are coming and achieving in one way or another.

(13:48):And so how can we best replicate that? Well, this is just one way to help surface that and do it in a very confined way of just focusing on learning and putting that in context with the other metrics too. I'm not saying those other metrics are bad, but really helps them define and contextualize that yes, students are graduating and they're having the skills and they're demonstrating the competencies that their employers are thanking us for as opposed to saying, yeah, they got a degree, but we're having to do all this on the job thing, or having to teach them on these success skills that they really didn't get there. And so assessment just really helps dig into those things and get beyond some of the other noise that can be in grades or some of these other typical metrics.

Amrit Ahluwalia (14:33):Absolutely. So let's talk best practice. I mean, how would a student affairs leader get the ball rolling on creating an assessment framework for student affairs work?

Joe Levy (14:46):Yeah, I mean, I think you have to start with the learning outcomes, and that's where I start with people is I say, what does your office, what are the interventions you're leading? What do you hope to do with students? When students come in, when they use your service, when they attend your events, what do you want them to walk away with? And then let's articulate that. Let's put that on paper so that we all can agree what these things are. You don't want to have, well, this is what Joe said, and then this is what somebody else said, but we get these consistent learning outcomes just like an academic program. We have those program learning outcomes that are the blueprint from which you align your course and your course learning outcomes, and you think of your key measures to demonstrate competency. So you really have to start with those learning outcomes, and aside from going through the assessment cycle of then picking measures, I think another big component to this is committing to embedding the work assessment is not a one and done activity.

(15:48):It's not something that you do one survey and then you're good, your question's answered. This has to be something that we commit to doing regularly. It has to be something we commit to systematically and comprehensively looking into, because again, people are complex and learning is messy, and we're not going to know our impact and exactly what's happening from just a simple exercise of asking for people's feedback on a survey. So we really need to commit to ongoing embedding this work as well as embedding it in the every day. I mean, too often, okay, well, who in our department's going to do assessment? Great. You're going to do that by yourself. And that's off my list as the director. I don't have to worry about it. I don't have to think about it. I'll just get the report at the end and check the box, but you have fun.

(16:42):Hopefully it works out, right? Yeah, I'm glad we have grad students and we can give this to them, but it's really, I say all the time when people push back on, because I say assessment should be part of a standing topic on your team meeting agendas. And if people scoff of like, oh, but we only do say one survey a year, so how could that be? And it's like you do one survey a year. Okay, so one meeting, you're going to talk about what the survey is about, another meeting you're going to talk about looking through the questions, seeing how it's aligning to your learning outcomes, how you're going to use the data. Another meeting you're going to talk about, okay, and then we talked to students or we shared this with somebody else, and this is the feedback they had, and how best do we incorporate this?

(17:30):Then another meeting is, okay, how are we going to administer this? Are we going to send it out? How can we get a good response rate? Another meeting is, how's it going? Do we need to send reminders? And I keep saying another meeting because all of this can't be done in one sitting. Likewise, all of it shouldn't just be done by this team. You should be sharing this with students. You should be getting their feedback. You should be getting other people's feedback. And so even just for one survey, you can have many, many meetings to talk about, and that's just one survey. Now, imagine if you have an assessment plan that's covering multiple outcomes with multiple measures, you could easily fill your year's agenda with talking about it and giving updates. Then we haven't even gotten to the part of, okay, now we have the data and how do we really think about making meaning of this and taking it into turning it into action.

(18:22):That's a big conversation where you need to talk to then a lot of other people and think about budgeting, think about. So I mean, it really is something that you need to commit to and embed in the rest of your decision making and your strategy for your department. But the good thing about it is it has so many beneficial byproducts, and so often people think of it as this add-on, when really it's this catalyst and it's this exponential benefactor of giving you insights and helping you turn your attention to certain initiatives and giving you data to advance initiatives or answer questions or follow up on things and to bring in other people. So the more you can embed it and commit to embedding it in your regular work, the better. And I think institutions as a whole, as well as specifically with student affairs, can do a better job leading by example of talking about data, referencing data, not just saying, this is a decision we're making. This is a strategy we're doing, but talk about the data that's informing that and how we're going to follow up to measure the effectiveness of it and how it relates to learning outcomes. But

Amrit Ahluwalia (19:32):That's a foundational challenge. I mean, so many student affairs divisions, even being able to get those core level metrics is where folks run into issues. It winds up being a manual signup sheet, like a little piece of paper somewhere. Or you have a student leader going from individual to individual, trying to write down student numbers, just collecting the data in and of itself winds up presenting a challenge.

Joe Levy (19:59):Well, and even having people with formal responsibility for assessment can be a challenge. Now, don't get me wrong that the field has gotten better, but it's still a challenge. I mean, when I came out of graduate school, I knew I wanted to do assessment, and the best advice I got from my mentors was just take any student affairs job you can get. Let them know you're interested in assessment, and they will readily plug you in hand. You work and get engaged. And still now, there aren't a lot of entry level assessment jobs. It's a lot more of like, oh, well, there's maybe a director level if that, right? There's still many institutions because I've been at them where there is one assessment director for the entire university, let alone specifically for student affairs. And so you talk about collecting data and you talk about resources allocated and technology to support this work.

(20:55):I mean, this is where collaboration is critical. And I think that's also where student affairs can elevate their work and overcome some challenges is by talking to their internal colleagues. If there's a director of academic affairs assessment and a director of student affairs, they should be talking on the regular because assessment is assessment and they can be talking strategies. They can be talking about how to share resources for data collection supporting one another's work. But I think, again, this goes back to some of the both imaginary and very real and very felt division between academic affairs and student affairs that exists on institutional, both from a philosophical perspective as well as from an operational one.

Amrit Ahluwalia (21:45):So I mean, looking to that topic, because collaboration ultimately becomes a really important mechanism to overcoming or bridging some of these gaps, you've worked with technologies that have helped to create that foundational space, but how important is policy driven collaboration and just interpersonal collaboration in general? What role does that play in creating a more effective student affairs division, especially as it relates to meaningful assessment?

Joe Levy (22:15):Yeah, it's huge. And I've been fortunate to have been in a position where often I've either been the institutional director of assessment or in my current role where I oversee accreditation and assessment. And a big thing I do is I make that space, even if it hasn't been asked for at that institution, I make it a point to say, we are going to do student affairs assessment and my team is going to dedicate resources, and we are going to engage people in this because it is important, from your point, I make policy at my current institution. We just recently revised charters for a bunch of committees. And so I made sure that in our university assessment council, that the language as we're talking about assessment, student learning and who were engaging was not exclusive to faculty and was not exclusive to academic programs and assessment within the classroom, but assessment in general, student learning in general and holistically across the institution.

(23:19):So I think it's so important that there is institutional leadership who recognizes and carves out that space because if not, then at best it's somebody's pet project. And then you're just clawing for resources and clawing for attention and just using any excuse you can get to get a seat at the table to talk about this. And so the more we can have institution perspective people and positions to again, carve out that space and open the door and recognize the role student affairs plays in student learning in general and student success and the better. And I think, again, it's one of the reasons why I'm so passionate about assessment is it becomes that foot in the door assessment becomes a way that you can get student affairs people at the table to talk about student success because you can align interventions to university learning outcomes or even gen ed learning outcomes.

(24:22):You can talk about how outside the classroom experiences are contributing to that. So then when we are having a larger conversation about general education or these core common success skills our students need, student affairs has a relevant seat at the table and a voice there, they can talk about how they're contributing to those outcomes. And I think without assessment, when they're not participating in assessment, it's a lot harder to make that argument because somebody might say, oh, well no, they're just doing transaction stuff. Or the best data they have is about satisfaction and usage and quality, and they don't have content around student learning. So by participating in assessment and having that data readily available, you can participate in those conversations and you justify more of your presence in that institutional setting and institutional conversation around strategy forward. And two core questions we ask at our institution, two questions all the time for any decision we're making, how is this best for the students? And how is this good for the university? And if we're leaving out learning, then you're only focused on academics or operations. And I think it's too easy to look at academic programs as the sole place students are learning, and that's the key to their success as opposed to all these great and transformational interventions that student affairs is leading.

Amrit Ahluwalia (25:55):Well, Joe, I mean that pretty much does it on our end. Now, as you know, when we close out our podcasts, we like to pivot a little bit to chat a little bit about foods and restaurants. Now you're in an interesting spot because you're based in Chicago. You've been in Chicago for, well, what has to be well over 10 years years, but you're now at Excelsior College based in Albany. You're remote. So I'll ask you for two recommendations. If someone's in Chicago for dinner, where do they need to go? And if someone's in Albany for dinner, where do they need to go?

Joe Levy (26:27):So for Chicago, I'll defer to what's a family and friend favorite called Cafe Baba, and it's in the city of Chicago. It's a tapas, so it's small dishes, amazing menu, a lot of different things that you're likely not going to get in a lot of other places. I personally often choose restaurants based on the quantity of food that I might get. I'm a big eater, and to that point, they have this amazing, both delicious and huge paella that I always get in addition to the small plates that we get. But it's a great time. It's a great both small group spot as well as large group spot. So it's a good, like I said, family and friend favorite of a space to go in Chicago and then pivoting big time in terms of cuisine in Albany. The spot that I'm going to try and sneak up to as much as I can, a little outside of Albany is the Rusty Nail. And it's a bar restaurant that chicken wings, buffalo wings, and I mean, first of all, if you're not in the state of New York or near the Buffalo area or upstate New York, you got to go and get wings somewhere. But it's one of the best spots that I've found in the Albany area. And since I lived in Buffalo for a time, it was a great nostalgic revisit of what good Wings can be.

Amrit Ahluwalia (28:10):Absolutely. Joe, hey, it's always a pleasure chatting with you, man. Thank you so much for your time.

Joe Levy (28:14):Absolutely. Thank you.