Where to Find the Best Evidence with Dr. Alfonso Iorio and Dr. Lori-Ann Linkins

Dr. McGavock: [00:00:00] Welcome to the Emerging Scholar Podcast Season 1. We hope you enjoy this series of great conversations with leading medical researchers that's designed for graduate students, medical trainees, residents, and fellows to help you on your journey towards your first scientific project. Welcome everyone to episode three of the Emerging Scholar Podcast.

Today's podcast is focused on where to find the best evidence, and we are fortunate today to have Doctors Lori-Ann Linkins and Alfonso Iorio from McMaster University. The reason we invited them today is they co-lead a resource that I think all of you should be aware of and currently be using in addition to your other resources, like UpToDate.

It's called ACCESS, A-C-C-E-S-S. This is a repository for, uh, the best [00:01:00] clinical trial systematic review evidence at your fingertips emailed to you, and they're gonna talk to us about how they gather this evidence, how it's systematically reviewed, and then delivered right to you so that you have the best evidence at your fingertips as you start on your journey for your first research project.

Over to Doctors Iorio and Linkins Welcome, everyone. It's my great pleasure to have, uh, two physician scientists from McMaster University, Doctors Alfonso Iorio and Dr. Lori Linkins with us today to talk about an excellent resource that residents and trainees, I think, should be using as they're doing their scholarly work, but actually any time they're trying to look at the evidence and, and look at science and interpret science in the right way.

So before we get started, uh, Dr. Linkins first, then Dr. Iorio next. Do you wanna introduce yourselves and tell us how you got here today?

Dr. Linkins: Thank you so much for inviting me. It's my pleasure. Uh, so I am a hematologist who specialized in thrombosis, so [00:02:00] that's what I do in, in my clinical job. And, uh, actually, I got involved with HIRU, or the Health Information Research Unit, which actually provides this resource that we're talking about through meeting Dr.

Iorio, who also works in the same clinical space as me. I have a, a background in some training with clinical epidemiology as well as some exposure to education and research in the past. At the present moment, uh, my key role with regards to this project is that I'm the knowledge, uh, translation platform lead for McMaster at HIRU.

Dr. Iorio: Alfonso Iorio. I'm a, I'm an internist by training. I was trained in Italy. I moved to McMaster 12 years ago, and I moved, uh, to take on, uh, the, the, the enterprise that is behind, uh, uh, Access and the Plus database. I was working already from, from Italy to help selecting the papers that we will be talking to you in, in a second.

I'm, I'm an internist by training. I'm a clinical epidemiologist by profession, and I've spent [00:03:00] my last 25 years of career trying to shorten the time that, uh, research results ready for clinical practice are brought to the attention of practicing clinician, which is again, what, uh, Access and other resources we will mention, um, we do.

Dr. McGavock: Excellent. So we'll put the links for McMaster Plus as well as the HIRU in the, in the link here. But if you just go to the, the landing page for HIRU, it shows you sort of a, a step-by-step process for how you curate information. But could you maybe just introduce the McMaster Plus platform, either Lori or Alfonso, and tell us why was it designed, what was the history of it, um, and what can trainees look forward to if, if they engage with it?

Dr. Iorio: I can set maybe the scope and then Lori can go quickly into the details. Um, this is a process meant to surface among the many million publication that appear [00:04:00] in the peer review literature every year, those that are, uh, ready for clinical practice, that are new, newsworthy, that are not repeating previous knowledge, and, uh, and that are relevant for clinical practice.

That's the goal. Uh, then the process is a, is a complex process, and I'll leave to Lori to give you some details.

Dr. Linkins: So in order to sort of make that cream rise to the top, uh, when you're starting with all those articles, uh, we have a core set of journals that we're, we're reading on a regular basis. Uh, it usually ranges around about 120 journals, but it can go up and down according to different needs.

And those journals are coming in from PubMed, so every day there's articles being fed in that are being published, and clinicians know that you can't possibly keep up with all of that that's coming in. So our process involves, uh, bringing tho- those articles in through filters that have been specifically validated to look for certain criteria, for example, diagnosis or prevention or treatment, and then passing it through, currently through [00:05:00] machine learning, which is something that we've been developing rather actively over the last couple of years to reduce the amount of articles that have to be manually read.

And then we do have a set of criteria that have been established predominantly by Dr. Brian Haynes. And our expert readers, who are generally masters or PhDs, who also have many, many years of experience at this point in critically appraising the medical literature, will look at these individual articles and decide whether they pass our criteria or not.

And on the website that you've mentioned, uh, we do actually have the list of the criteria available if people are interested. And it basically helps to, to bring again to the top of that, you know, that cream that you're interested in while filtering out the ones that maybe they sounded exciting when you looked at the title, but once you actually started digging in, you realized there were so many methodological flaws that you really couldn't rely on it.

And if there's anything that's a good example of that, it would be COVID, right? So when COVID started, it flooded all of us with information, and unfortunately, most of it was inaccurate or sometimes downright [00:06:00] misleading. So one of the things that HIRU did, um, uh, under Dr. Iorio's guidance was to set up something called COVID Evidence Alerts.

So where we actually did apply our criteria to the data that was coming in in the form of the articles that were being published. And so that's just an idea of how the process works. We have multiple automated steps, but we also key part of our process is we have real human beings who are very experienced in critical appraisal, looking at these articles and helping to decide which ones are the ones that should come to the attention of clinicians.

Dr. McGavock: And so that critical appraisal already happens and is, in my case, it gets emailed to me. Anything related to pediatrics, it pops up, or anything related to COVID, it pops up. And not only does it say, you know, "We've filtered this through, we'll give it a ranking," and then actually s, uh, physicians working in that space who are specialists have little comments underneath saying, "Okay, we reviewed this, and this is what this is what, uh, this article says."

Dr. Linkins: Excellent. Actually, that's exactly what we're, [00:07:00] that's what we're aiming for. So if you're following those alerts, you're doing exactly what we want you to do, because we know you're busy. We want you to spend your time on things that are valuable to you. And then, yes, so what you're referring to are something we call the more ratings, uh, and these are physicians and actually healthcare providers, actually, not just all physicians, um, across the world who have signed up to generously rate these articles.

And they're really just asked it on a Likert scale, uh, is it relevant and is it newsworthy? And you can- they can add additional comments if they want to. They don't have to. It's a completely free, uh, no bias involved, um, service available, and they help us basically to give context to, uh, the, the audience or people looking at these articles within the discipline that they're...

You know, these are their peers. I'm looking at a thrombosis article. It are my-- These are my thrombosis peers who said, "No, I disagree. I, I, I wouldn't apply this. I, I, I don't see how it's practical." And those comments, uh, actually can help shape your opinion on whether this is something that you should be, you know, genuinely paying a lot of attention [00:08:00] to or, or maybe not so much.

Dr. McGavock: And it's delivered to you in your email almost every week, which is unbelievable to me. So, uh, maybe we could just go through that process just briefly. So 120 journals. Which are those journals? Why did you choose them?

Dr. Linkins: The primary reason for the journals we have were there was a selection process. I think Dr.

Iorio is gonna know better than me, because I'm a little bit more recent to the program. But the journals were selected based on their, their merit and their impact. Uh, and, and, and also we're trying to appeal to the largest audience, so a lot of our journals are general internal medicine-focused, uh, with some subspecialties.

You'll find that there is some pediatric content, but we, for example, we know that we don't cover pediatrics to the same degree that we do internal medicine. I'm gonna just let Dr. Iorio jump in there. He probably has more context for you.

Dr. Iorio: No. No, no. You covered it, Lori. Uh, the, the list was, uh, refined and evolved, uh, over twenty-five years.

The starting point was, uh, providing breadth and depth to internal [00:09:00] medicine specialists, including, uh, family physicians. So there was some gynecology there. There was some obstetrics. There was some, uh, pediatrics. And then, uh, over time, we developed specific collaboration with specific groups, and we added, uh, journals.

We also are aware that journals, new journals come up every, every day. And when we see, uh, that, um, uh, there are journals eating more and more the, the headlines in the lay press or in systematic reviews, we keep them under scrutiny for a period, and if they yield a sufficient number of high-quality journal, we tend to introduce them.

But, but it's really an empirical list, uh, built based on the journal that are, uh, usually providing clinically ready evidence.

Dr. McGavock: So that's step number one, so you're already getting a nice filter. So you're making sure that you have journals that have a certain level of criteria for what they accept They have a history of publishing good science [00:10:00] that is relevant for practicing physicians.

But like it says on your website, those hundred and twenty journals might generate thirty thousand articles. So your next step is to try to identify the high-level science that's coming out of those journals that would be relevant to clinicians. Is that correct?

Dr. Iorio: Y- yes, it is. Um, and this is, uh, essentially the steps, uh, the critical appraisal step, uh, that, uh, Lori was, uh, uh, describing before, plus, uh, the fact that we attempt to attach to each article which discipline they are relevant to.

So we say, "This is likely relevant for an emergentist, for a pediatrician, for a surgeon." And then when we ask a practicing, uh, healthcare professional to tell us, "Is this really worth the attention of your peers?" We send them to those in the discipline we think the article is relevant to. One of the messages we get back sometimes is, "No, no, no.

This is not for [00:11:00] me. Uh, this is, this is not relevant to emergency medicine." And, and then we take that discipline away. So essentially, there are two subsequent step. One is a critical appraisal of the quality, and the second one is, uh, are we correct in interpreting who this will be relevant to and how much this will be relevant?

And since I'm talking, let me add a little detail to what, uh, uh, you mentioned before. You said, "I receive an email every week." Correct. Now, you can receive it every week. You can decide you get it daily or you get it monthly. We know that needs of different people are different. So when you, uh, register to receive alerts, you can set, one, how often you want the alert.

Two, you can select which discipline you are interested in, and those for resident may vary. You may have different learning needs at different stages in your trajectory, and you can change the discipline and see different material. It might not apply to pediatrics, but in internal [00:12:00] medicine it does a lot when you rotate in different areas.

And third, you can, um, trim the amount you receive, uh, to the amount of time you have to keep yourself current. So there is a simulator that will show you by selecting the relevance on a scale zero to seven, and the newsworthiness on a scale zero to seven, will tell you how many article you would have received per month.

And if you know that you can read four, then don't do the effort of picking four, um, by r- by s- scanning seven and deciding four, or even worse, don't choose at, at random. If you only have time for four, choose a very high threshold. Get only four, read those. If instead you have more time, then lower your threshold.

You'll get more, and you will have, um, better evidence. Now, I will come back on this later because I think it's good to go through the steps. But, um, I want to comment on is this sufficient? Is this giving you something, and do you have any way, do we offer [00:13:00] you any way of putting this something in a broader context?

We think we have Uh, but I'm happy to delay commenting on this. Let's see before all the steps, uh, that leads to what gets to your table, if, if you agree.

Dr. McGavock: Absolutely. And I would, towards the end, talk about the fact that this is, like you said earlier, large repository, and it feeds into maybe 15 other specialized repositories that you can sign up for.

And then within that, you can select how often you wanna get this, the subspecialties you want. Like, you can curate this right down to your needs as a user, and I think that's one of the beautiful strengths of this, of this resource Lori, can I just ask you about that critical appraisal piece? It's such a huge part of being a resident and being a trainee is, is learning how to critically appraise.

How do we trust your team that's critically appraising it? What training have they done, and how do we know that what they're appraising it is the, is the right stuff?

Dr. Linkins: [00:14:00] Well, those are all excellent questions. Um, so I'm hoping that at least some of your residents will be familiar with, uh, Dr. Brian Haynes, who is considered to be internationally one of the leaders in developing evidence-based medicine.

And basically, the criteria were developed over time and honed by him and his group, um, and down to the level where we are using them now. The people that are actually applying... So basically we have a set of rules. Uh, it's actually a rather long set of rules. Uh, the shortened version, uh, is actually available to anybody who's interested in looking at it.

So if you're a resident and you're thinking, "You know what? I've got this article on diagnosis in front of me, and what should I look at for journal club? What, what do I use to decide whether or not this is actually a good article for me or not?" You can use some of our, our criteria. It's very easy for, you know, for example, some of them will, uh, be ve- for, for diagnosis.

Is there a reference standard that's been used, would be an example. Um, we also look at follow-up, uh, for treatment studies. You know, is [00:15:00] it less than 80%? We would consider that to be a, a strike against that article. So the actual criteria themselves are not at all new. Probably Dr. Iorio can give a little bit more background on exactly how Dr.

Haynes developed them specifically. But in terms of the training, we have a staff of, of people, some of them have been with us for over 20 years actually, applying this criteria. And what they used to do is actually go page by page through the physical paper copy- ... of the journal. So would go down to the library, come back with this gigantic stack of paper- Wow go through it, and, um, and it's-- I just, to me, that's just, that's, that's... I can't even get my head around it. I'm pretty sure your residents won't be able to as well. But that process, they, they would specifically go looking for those particular questions, you know, with the... Or what is the follow-up? They've got to determine what it is, and they're not depending on what the author says is the follow-up in the abstract.

We're digging into the full text. We're looking in that PRISMA figure. What actually happened as opposed to what the author said was going to happen? [00:16:00] And that's when I think one of the key things that sometimes residents might miss is the fact that you sometimes have to get your hands a little dirty.

You gotta get into the nitty-gritty. But that is the benefit of our service, is that somebody else who's already very experienced at that has already done that work for you. But I think one of the key things for residents to learn is how to do that themselves. And, uh, so what, what, what, uh... The other thing that's important though is that after my readers, who are not clinicians but are experts in the clini- critical appraisal, once they're finished, those articles that they think pass then go to somebody who is a clinician Who also has experience in clinical epidemiology.

So both myself and Dr. Iorio, for example, um, are that next part of the chain along with a few others, and we look at it and say we agree or we don't agree. And also we have that extra lens of a clinician saying, "Well, you know, it, it, it sounds all lovely and all, but it, it... there's pieces missing here, and from a clinical perspective, this is not ready to go.

This is too preliminary." And we may actually say, "No, we're gonna reject this from the PLUS database, and this is why we're [00:17:00] rejecting it." So we can get stopped at that level. And also, as Dr. Iorio said, we're looking at it in terms of, is this really of interest? Like for example, an article where it's a, a surgical article talking about different surgical techniques.

Is that really gonna be of interest to a family physician? I would say no. Right? Again, you're trying to focus on that time of that particular individual, whereas my critical appraisers may not necessarily know. It's something that the physician should know. And then we also have another level after that, which would be an editor.

So if there's discrepancy, particularly regarding methodology, uh, we sometimes have a very large, very active group discussion about how to come to consensus. And then of course the MORE process, which you've already mentioned, which is the international raters.

Dr. McGavock: Right. W- the other thing I really like, uh, you talked about the PRISMA figure.

Can you maybe explain the, the PRISMA figure?

Dr. Linkins: So articles, in order to, to meet the standards that have been established by other bodies, certainly not by us, but internationally for the type of article that you're looking at, whether it's diagnosis or prognosis or [00:18:00] treatment, there are reporting standards that have been developed.

And when you go to publish an article, you learn pretty quickly that the reputable journals want to see these particular diagrams. And what they should be doing is walking you through how did they do their search? How did they eliminate articles? Basically, how did they end up with that bottom number? So you're starting with 3,000 articles when they did their topic search.

How did they end up with those 12 articles that they actually included in their systematic review? And all of those steps between that first search and what you chose to include in your paper should be transparent. It should be obvious why those other ones did not get put in. You want to show that you're not cherry-picking, that the...

it's definitely an, an unbiased approach.

Dr. McGavock: And the same probably applies for therapeutic trials, right? The, the reviewers are going through to make sure they're following all the reporting guidelines so that you're not just being sold on an abstract.

Dr. Linkins: Most of authors are obviously, uh, reputable, but sometimes when you have a, a study that has a disappointing [00:19:00] outcome, there might be a tendency to try and spin it a little bit to... and maybe focus on a secondary outcome measure which was underpowered but actually seemed to have more of a favorable response. And so authors can sometimes try and focus on that. And part of our job is to say, "Yeah, but that was a secondary outcome measure. Um, the actual primary outcome measure did not pass criteria.

So yes, it's interesting, and maybe that secondary outcome measure can be used as a hypothesis to actually develop a definitive trial to answer that question, but this particular paper isn't answering that question."

Dr. McGavock: So that's excellent. Two things. It's important, as you said, that residents learn how to do this, right?

But this is also a way to curate that evidence so that it's, it's being done for you. So this is a, a massive repository. There are multiple other sub-projects that go with it, which we talked about earlier. So if you're interested in hematology or thrombosis, there's CLOT-Plus. If you're interested in public health, there's Public Health Plus.

ACCESS is one of these. For a resident who's probably [00:20:00] heard of or, or using UpToDate, uh, Alfonso, do you wanna tell us how this resource compares to UpToDate? Um, there's also DynaMed is one of them. Can you maybe talk about what those are and how a user can use this in collaboration with or as an adjunct to UpToDate?

Dr. Iorio: Yes. Thank you for the question. I, I think it's really important to help people make best use of any resources, to know how they can, uh, you know, m- match and patch together. So the, the McMaster PLUS MORE database is a collection of high quality research articles and is what you have heard, and is very timely.

Our time to inclusion in the database, time to alerting, is usually 14 days. From the publication date is in your mailbox. Wow. Or is available for you to search. Wow. But it's a very limited non-structured database. It's rich. It has discipline. You can search it by disease, but is a [00:21:00] non-structured content, is not a textbook.

So the best way I can think of it is a good way to keep yourself up to date, is a best way to keep yourself abreast of, of novelties. If you are searching for a topic, if you are going to check evidence around any topic, any clinical question you may have, our best offer at McMaster for it is something called Access.

Access is, uh, the database underneath is PLUS, but the interface is a searching interface, and is what we call a federated search. Say you search for gout or, you know, um, gout treatment is not typical for pediatrics, so name a disease and you search for something. What we will give you back is a set of result across many layers.

The first layer is, uh, point of care resources like UpToDate and DynaMed. I'll come back in a second. Immediately below, we give you guidelines. If there [00:22:00] is a guideline that we have indexed that speak to that question, you will see the entry point to the guideline. Then we give you content out of PLUS MORE, and we layer it in systematic review and trials.

Systematic review, by the way, capture evidence from much more than the 120 journals. So even if we read 120 journal, we select in all the systematic review, and one of those can give you really a snapshot of the knowledge at that point in time on that disease. If you don't find anything there, then we have already pre-searched for you PubMed, and we will have classified entries from PubMed, because something maybe was not there for you in all the higher layers by diagnosis, treatment, prognosis, and so on.

So federated search mean you search once and you see a full dashboard of where you can go find evidence, and my suggestion is always start at the top. Now, looking at the top, why we put, uh, point of care textbook [00:23:00] at the top? Well, because they have, uh, chapters, they have, uh, topics that are organized like the human mind thinks.

So, you know, etiology, diagnosis, and the treatment, management, prognosis and so on. And they are built in a way that you don't need to wait until the next edition. They incorporate evidence as it comes. I want to spend a word here. You all know UpToDate. UpToDate, you all get exposed, uh, you know, at undergrad, and then you carry it forward.

DynaMed is something I know better, having been trained outside of North America, and having a strong collaboration as a team with DynaMed. The content we make available in Plus more and through Access, we also feed to them, and we, we really are helping them structure their own internal search. They search about 500 journal, the way we search the 120.

And also the way it's incorporated in new chapter, we help them with machine learning in selecting how to get the content [00:24:00] in. But the reason we think you should start there is because for many question, these kind of resources get already what we would provide you as an alert that might be good in your own area of expertise, because you know already all the rest.

You just need what's new coming through. If instead you are moving slightly off your comfort area, you are approaching a new topic, or it's the first time as a resident, as a fellow you see that disease, then these resources give you a better framing. Check with your institution if you have access to DynaMed, because at McMaster, for example, it's available for free.

It has an excellent app, give you access to Micromedex. You can check, uh, drug interaction with the best gold standard resource that's available, and many of you don't even know they have access for free. So, um, be smart, check what you have, and know that these resources are very similar, very useful, very expensive.

So - Yeah ... again, think smart and check for, [00:25:00] um, content following this hierarchy of evidence. Point of care, systematic guideline, systematic review databases like MORE. And if you ask me, are any other database as good as MORE? No. Plus, MORE is the, is the first in class, but cover that narrow space. If you ask me, is this the only one I would use as a practicing clinician?

I would say no. I would use DynaMed or UpToDate. This is one step remote by having unstructured, novel, valid, robust evidence.

Dr. McGavock: Right. So when I've gone in there to use that as a teaching tool, for example, I'll pull up Access, put it on the screen, and we'll put in whatever, IV fluids for something pediatric, right?

And everything comes up on that search. So it's such a, an incredibly useful search output that it gives you the UpToDate link, then it gives you systematic reviews, then it gives you guidelines, and then it gives you the most [00:26:00] recent RCTs that have been published in that space. So for me, it was a great teaching tool, and I think for residents it's an incredible resource, uh, to add to what they're already doing.

I know we've gotta go, but maybe I could just ask you both one last question is: How are you best using this tool within your own clinical practice or within your own teaching?

Dr. Linkins: I actually use Access regularly. I was on clinical service yesterday and, and I, and I used it because as, uh, Dr. Iorio said, I mean, I... hopefully I'm not using it to look up, uh, key areas in thrombosis. I should know that. But what I'm using it for is looking to see whether some of those things have been incorporated into guidelines yet, or what is the most recent original article that just came out because I, I try and stay current too, but I have the same problems everybody else does.

Uh, and I had an in- interesting case come in yesterday of bilateral renal infarcts, which is not something I see every day. That's unusual. I wanted to see whether there- anybody had come up with anything new in that area since the last time I looked at it. So I use it on a regular basis for my clinical work in particular.

Dr. Iorio: A- and I use it, [00:27:00] uh, same way, and we have it, uh, we had it integrated in our previous system. We can now, uh, use Access directly through Epic. There are info buttons. We can pop it up and search. I mostly use it, uh, when, uh, you go in that debriefing moment, uh, with learners, then you want to double-check what you are doing against evidence.

And yes, you can go back to guidelines that you know of, but I always like to say, "This is how you can find it," and the how you can find it is not I'll send it to you later by email, but let's see how you can get to it through Access. And then, uh, whenever I get those patient that have diseases that I've not recently read on, I quickly scan if there's anything I should know, if there's a recent guideline that I can go to and, uh, orient myself.

So yeah, it's mostly a clinical tool. Now, uh, if you were, uh, choosing a topic for a round, you can look at what was published recently there. If you were planning to do, I don't know, a scoping review, then you might say, [00:28:00] "Okay, let me see what's the most recent advancement in this area." I would not use, uh, Access or more for a systematic review because it's, it would not capture all what you need.

But if you are scoping a topic, then you would see what was, uh, high interest, high impact, uh, in the last two, three years. Uh, that's another way I would, um, I, I would use it.

Dr. McGavock: Yeah, that's a good point. You c- you can't do a systematic review 'cause it doesn't systematically include every, every article, right?

It's already been filtered to, to have sci- kind of the best science, so... Well, I know you both have, uh, very busy schedules. I just wanna thank you. I really wanna thank McMaster. You guys are doing some phenomenal work in this space. All of Canada should be proud of this work you're doing. We're gonna put the links here, and I really hope, uh, more people, uh, start to use Access and integrate it into, uh, not only just doing their research projects but into their clinical care decision-making as well.

So thank you both very much.

Dr. Linkins: Thank you. Thank you. It's been a pleasure.

Dr. McGavock: That was our [00:29:00] conversation with Doctors Lori-Ann Linkins and Alfonso Iorio on where to find the best evidence. Be sure to check out their website at accessss.org and join us next week for episode four where Dr. Lucy D'Agostino McGowan will tell us about the design principles for data analysis to make sure your data is clean and ready to go when it's time to analyze it.

Thanks for listening. The Emerging Scholar Podcast was recorded in the UMFM studios located at the University of Manitoba in Winnipeg, Manitoba, Canada. This is Treaty One territory, the original homeland of the Anishinaabe, the Ininew, and on the national homeland of the Red River Métis. Manitoba is also homeland to the Anishininew, the Dakota Oyate, and the Dene peoples.

We acknowledge that our water is sourced from Shoal Lake 40 First Nation. As a settler scientist working in health research, I acknowledge TRC Call to Action Number 18, that health inequities between Indigenous and non-Indigenous people are the direct result of colonization. Winnipeg is the center for [00:30:00] reconciliation in Canada, and where we are speaking from today is at the crossroads of Canada's history and Canada's future.

We are three kilometers away from one of the most sacred meeting grounds for Indigenous people on the plains and within Turtle Island. We are also several kilometers away from one of the last residential schools in the province. We are working collectively on reconcili-action with youth, elders, and community leaders to heal from the trauma of colonization, past and present, honor the original treaties, and create a Canada that addresses and eliminates the inequities and injustices of the past and present to create a future for Indigenous children to live and thrive.

On behalf of the team at the Emerging Scholar Podcast, we want to thank Osani Balkaran for the great production for our opening and closing themes. We also want to thank UMFM 101.5, especially co-producer Jared McKetiak for technical assistance and production on this project. We also want to thank all [00:31:00] of our guests for the discussion and the insight that they've provided.

Once again, thank you for listening. Be sure to subscribe to the Emerging Scholar Podcast on Apple Podcasts, Spotify, and umfm.com.