The Power of Administrative Health Data with Dr. Eric Benchimol

Dr. McGavock: [00:00:00] Welcome back to the Emerging Scholar Podcast. I'm your host, Dr. Jon McGavock from the University of Manitoba, and I'm thrilled to have you back for season two. Last season, we took you on a journey through your first year of research training, covering everything from crafting great research questions to understanding the foundations of both quantitative and qualitative methods.

Dr. McGavock: We met incredible scholars, explored fundamental concepts, and built a roadmap to help you navigate your early research career. This season, we're going deeper. If season one was about laying the foundation, season two is about mastering the craft. We're taking those methods we introduced and really unpacking them, diving into the nuances, the challenges, and exciting new possibilities that each method has to offer.

Dr. McGavock: Throughout this season, you'll meet a remarkable team of both established and emerging scholars with expertise across a diverse array of approaches, from natural experiments that reveal causal insights into real world [00:01:00] settings, to qualitative inquiry grounded in social justice, the complexities of implementation science, and yes, my personal favorite, causal inference.

Dr. McGavock: Each episode will introduce you to researchers who are pushing the boundaries, asking bold questions, and making novel and important scientific discoveries. First, we're celebrating Canadian research excellence, showcasing incredible work happening right here at home. Second, we're emphasizing the critical importance of patient engagement in shaping research that truly matters across a range of scientific methods.

Dr. McGavock: And finally, we're championing curiosity, the essential spark that drives all great science and keeps us asking what if and why. Whether you're designing your first research study, wrestling with methodological challenges, or simply passionate about rigorous and meaningful research, this season is for you.

Dr. McGavock: So let's dive in, stay curious, and continue building the skills that will shape your own scientific journey.

Dr. McGavock: Welcome back everyone to the Emerging Scholar Podcast. I'm your host, Dr. Jon McGavock from the [00:02:00] Children's Hospital Research Institute of Manitoba. Over the past few episodes, we've been exploring the foundational methods that underpin rigorous health research. Today, we're continuing that journey by diving into one of the most powerful research tools available to emerging scholars, routinely collected health data, also known as administrative health data.

Dr. McGavock: To guide us through this landscape, we're thrilled to welcome Dr. Eric Benchimol. Dr. Benchimol is a pediatric gastroenterologist at SickKids. He's a professor of pediatrics at the University of Toronto and a senior core scientist at ICES. He holds the Northbridge Chair in Inflammatory Bowel Diseases, and has built an internationally recognized research program using large scale health data sets to understand why Canada has among the highest rates of IBD in the world.

Dr. McGavock: What makes Dr. Benchimol's work particularly valuable for our listeners, and why I invited him today, is his leadership in not just using the data, but also improving how we conduct research and report it across the world. He chairs the Canadian Gastrointestinal Epidemiology Consortium, which uses [00:03:00] innovative distributed network methodology to paint a national picture of IBD across Canada.

Dr. McGavock: He also critically co-chairs the RECORD Steering Committee, an international group that has developed essential reporting guidelines for studies using routinely collected health data. In today's episode, we'll explore what administrative health data actually is, the incredible strengths and important limitations of administrative health data, the RECORD reporting guidelines that Dr.

Dr. McGavock: Benchimol helped develop to ensure his research is transparent and rigorous. We're also going to do a deep dive into one of his recent papers published in the CMAJ that used machine learning and routinely collected health data to examine the determinants of premature mortality in persons living with IBD.

Dr. McGavock: A perfect example of how to use these methods to tackle complex real world health questions that'll inform clinicians. So whether you're considering using administrative health data for your thesis, planning a population-based health study, or you just want to understand a little bit more about what administrative health data is and the power of using routinely collected health data for your [00:04:00] own research, this episode's for you.

Dr. McGavock: So let's welcome Dr. Benchimol to the Emerging Scholar Podcast Welcome everyone to another episode of The Emerging Scholar. Again, my name is Jon McGavock. It is great to be here with Eric Benchimol, who I first met maybe 10 years ago. I was learning how to become a more efficient academic, and we're gonna put a paper in there, one of your most cited papers about the academic athlete, I think it was what it was called.

Dr. McGavock: But, uh, Eric is a GI specialist and clinician scientist at SickKids. How would you introduce yourself, Eric?

Dr. Benchimol: Thanks for the invitation, Jon. It's great to be here. Exactly as you said, I'm a pediatric gastroenterologist. I mostly focus on treating children with Crohn's and colitis, inflammatory bowel disease.

Dr. Benchimol: And I'm a clinician researcher, where about 70, 75% of my job is research. And in my research roles, I focus on both kind of the epidemiology and health services research of adults and children with IBD, inflammatory bowel disease, and also [00:05:00] on the methodology of the research that we do, the methods that we use to conduct research with routinely collected health data, and particularly health administrative data.

Dr. Benchimol: That's mostly what I work on.

Dr. McGavock: That's the focus of today's conversation, is the dos and don'ts of using health administrative data, but also why should we use it and how could we use it. I went through your Google Scholar and just saw this plethora of amazing research that you're doing in that space, so looking forward to talking about that.

Dr. McGavock: Let's go back to your residency. I did a deep dive, tried to pull up your first resident research project. Back

Dr. Benchimol: when I was a young boy.

Dr. McGavock: Yeah. A

Dr. Benchimol: young pup.

Dr. McGavock: Young pup. So what was the project? And in- incredibly, your resident research project was actually in IBD. Yeah. And so you've continued on that journey, which I think is amazingly rare.

Dr. McGavock: So maybe- Yeah ... how'd you get into that project, and then why is gastroenterology your, your passion?

Dr. Benchimol: So it's funny because in undergrad I was doing a basic science degree in biology, and my, my research was in Dr- Drosophila genetics, fruit fly genetics. And you kind of reach a, a cross point. I wanted to m- go into medicine, but if I hadn't gone into medicine, I would've gone into a [00:06:00] research-based career, probably lab-based, probably genetics.

Dr. Benchimol: But when I got into medicine, I kind of gave up on research through med school. I'm like, "No, I want to be a clinician." Got to residency at CHEO and University of Ottawa, and knew at that point I wanted to go into gastroenterology. I'll explain why in a second if you want to get into it. Knocked on the door of Dr.

Dr. Benchimol: David Mack. Shout out to Dr. Mack, who ended up being a lifelong mentor for me up until this day. He's really been an amazing mentor and friend. And I said, "I want to do GI. Can we do something in research, you know, clinical research?" And he had this data set looking at calcium and vitamin D and bone health.

Dr. Benchimol: He let me analyze it and write up the paper, and also pointed me in the direction of an endocrinologist, Dr. Leanne Ward at CHEO, who was a clinician scientist. She was the first person in my life that I met with this 80% protected time for research role. She was doing a study on bone health in pediatric IBD.

Dr. Benchimol: That was really my first longer term residency research project. I kind of helped usher that through the [00:07:00] REB and through recruitment and stuff like that. But, you know, those two people, Dr. Ward and Dr. Mack, became mentors and examples, role models for me as a clinician researcher. I never imagined having a job with 80% research when I was in med school.

Dr. Benchimol: I thought I would be- Yeah ... a clinician, and that was fine with me. So yeah, it still sort of boggles my mind that I'm in this role, but I love it.

Dr. McGavock: Did Leanne try to pull you into peds, uh, endo?

Dr. Benchimol: No. By that time everybody knew that I wanted to do GI. That was my thing. I'll explain why I got into GI.

Dr. McGavock: Okay. Let's hear that.

Dr. Benchimol: Uh, it's a story. Yeah. So med school, med students have all these symptoms. Yeah, yeah, yeah. They start to feel things- Right ... that they learn about and they all- Everyone's got

Dr. McGavock: diabetes in, like, year two, right?

Dr. Benchimol: The reality was I was having stomach aches, bad stomach upset, and bloating, and it was really uncomfortable, and nobody could figure it out.

Dr. Benchimol: Went to the family doctor, sent me to a gastroenterologist, investigated me, could not find a reason why. Everything was negative. Eventually, I got sent for an upper endoscopy, and the GI who scoped me didn't [00:08:00] really sedate me very well- Oh ... at all. Oh. Through the whole process of getting the scope put down through my throat, and the gagging and the retching and the horrific feeling of that, I'm watching it on the TV screen.

Dr. Benchimol: I'm like, "Hey, this is like a video game," and I was a gamer when I was a kid. This is kinda cool. It's like going through a first-person shooter, right? I watched it on TV, and I'm like, "I'd like to try to do this," and I ended up doing some electives and, you know, in med school they gave us a lot of freedom to play around, and learned that I really liked GI.

Dr. Benchimol: So it was a very personal thing, and I ended up actually getting diagnosed with a peptic ulcer disease and H. pylori, which was not picked up on blood work or anything else. Yeah. And got treated, and I'm fine now, but that's really why I'm in GI. And the more practical reason is really GI, especially pediatric GI, it's a nice mix of all ages, from birth until 18.

Dr. Benchimol: It's a nice mix of ambulatory and inpatient, and there's like, you know, nice mix of procedures and diagnostics and things like that. So it was a good all-around specialty that fit my personality a little bit.

Dr. McGavock: So your day-to-day's a little bit different. Like, you're doing some video gaming with the patients.

Dr. McGavock: You're working [00:09:00] with little guys, you're working with teenagers, that kind of thing.

Dr. Benchimol: I make sure they're fast asleep when I do it. Right. They're not gonna have the same experience that I do.

Dr. McGavock: Exactly. Good- Not

Dr. Benchimol: the

Dr. McGavock: kids. Good for you. What did David Mack and Leanne... Like, you and I met through this program called CCHCSP.

Dr. McGavock: Season one- Yeah ... we talked to Norm, actually- Yeah, yeah ... about his journey there. But, like, what made a good mentor? I think that's critical, right? Like, your journey to go to this stage that you're at, where you're holding a research chair at a preeminent university, you gotta have someone who lights that fire, right?

Dr. Benchimol: Yeah. I hope I know what makes a good mentor by now, but I'm not sure. So I think it's encouragement. I think it's guidance in helping you see things that your lack of experience would prevent you from seeing, and for me, that was partly the role of the clinician scientist, and Norm played a big part of that.

Dr. Benchimol: The role of the clinician scientist and the passion for research, as well as the impact that you can make through research that, on a broader scale, a large scale, that maybe that clinicians cannot make. So I think that really opened my eyes, both Dr. Mack and Dr. Ward. I, I think somebody who [00:10:00] puts aside their ego and do what's in your best interest, and I'll give you the perfect example.

Dr. Benchimol: In the end, after fellowship and PhD, I ended up going back to Ottawa to become faculty and a clinician scientist, and Dr. Mack was my division head. Mm-hmm. In my first four or five years- Probably only twice in all of that time did he ask me to see any more patients, right? And to do an extra clinic, even when people are ill or the wait lists are long.

Dr. Benchimol: He really protected my time to the detriment really of the rest of the division who had to see lots of more patients. He protected my time so that I could be successful as a clinician scientist, and that's a perfect example of somebody who's sacrificing their own career time to benefit you. I'm very appreciative to both him as well as the other clinicians that I worked with at CHEO because they supported my role for a long time, so that was amazing.

Dr. McGavock: It sounds like a great team effort, but also- Mm-hmm ... when your department head sticks to the original contract, right? They're like- Yeah ... "I see something in you."

Dr. Benchimol: Yeah, it's rare.

Dr. McGavock: Uh-

Dr. Benchimol: There was no doubts [00:11:00] that he had my best interest in mind as number one priority, which felt really, really important.

Dr. McGavock: I think Norm talked about, like, you see something in the trainee that they don't see in themselves.

Dr. McGavock: I think you touched on that there. Like, what did Norm see in you going through CCHCSP where you were like, "I didn't know that about me"? You know, you talk about 80% protected time, but, but there's a skill there. There's something there that you had that they saw that they're like, "Ooh, you're gonna be really good- Yeah

Dr. McGavock: if you keep doing this."

Dr. Benchimol: I don't know what Norm saw in me. So the way CCHCSP works is you kind of apply, get rejected once usually. Yeah. Apply again. Yeah. Get an interview, so you're actually interviewed for this grant, which is unusual in Canada. And then if you get the grant, there's a lot of mentorship and support and personal development so that they helped me see what a clinician scientist did-

Dr. McGavock: Mm

Dr. Benchimol: really firsthand. I don't know what Norm saw in me, if he saw anything in me other than I was doing okay. But I do remember Norm was not one to mince words. I was applying for jobs, and one of them was gonna be at SickKids, and he sat me down probably [00:12:00] two years into my PhD, looked at my CV, and he said, "No.

Dr. Benchimol: You're not gonna get a job here unless- Oh, wow ... you start publishing original research." At that point, I was doing my ClinEpi PhD. I was in courses. Yeah. So I had, like, five systematic reviews. I had a few review articles. I maybe had a small study of original research, and he's like, "No, no, I wanna see your publications from your original research, not review articles."

Dr. Benchimol: So he was harsh but fair, right? Like, he was truthful. I mean, he told the truth, and he helped me see that this was the most important step to take and that to focus. It was gonna happen anyway. It wasn't that I wasn't gonna be publishing my PhD thesis, but I wasn't at that stage yet. So I think it's important to have a mentor as well, right?

Dr. Benchimol: To have truth, to tell the mentee when they're doing well but also when things are not going so well and they need to be aware of that.

Dr. McGavock: You gotta focus on this, and I think your focus is incredible just looking at your Google Scholar, especially in the area of admin datasets. So- But after this resident project have really committed yourself to [00:13:00] working with administrative health data, so much so that you're one of the co-authors or the primary author for the reporting guidelines for RECORD, which I went through yesterday for a paper by the way, so thank you for creating that checklist.

Dr. Benchimol: Glad you find it useful.

Dr. McGavock: Honestly, like for residents, start with a checklist even when you're reviewing a paper, because if they've left something out, th- it's usually really big. Tell me a little bit about, from your standpoint, what is administrative health data? Give me some of the strengths, and then maybe after that we can talk a, a little bit about some of the limitations.

Dr. Benchimol: Sure. Well, I'll tell you how I got into that field. It was mentorship, right? I think that's, in the end, gonna be the story of my career, is I was very lucky to have amazing mentors at every step of the way. When I arrived at SickKids, we do one year of clinical and then two years of research as part of the fellowship.

Dr. Benchimol: I approached Anne Griffiths, who's a leading figure in pediatric IBD in Canada. I said I wanted to work with her. I wanted to do Crohn's and colitis research. I wanted to do clinical research. I was gonna do a master's in clinical epidemiology, but I wanted to work with patients primarily. And we were going through some [00:14:00] projects.

Dr. Benchimol: And she said, "Well, listen, there's this group called ICES who have all of these large population datasets." And really the only people who were doing that type of research in gastroenterology were the Scandinavian countries- Mm-hmm ... you know, Sweden and Denmark and Norway. And Charles Bernstein in Manitoba, who had done some work in adult IBD.

Dr. Benchimol: And she said, "Listen, nobody's done this in pediatric IBD. Ontario's huge, right? It's quite a bit larger than Manitoba. And we have the opportunity to see what's going on with pediatric IBD at a province-wide level." And I initially resisted and I said, "I wanna work with patients. I don't really wanna sit in front of a computer screen and work with data all day long."

Dr. Benchimol: Turns out I did. It worked out okay. But she pointed me towards Astrid Guttmann, who's a- Ah ... pediatrician, clinician scientist at SickKids. Great. And Astrid sort of kindled my interest in both, you know, doing this type of research, but also the methodology, making sure that the methods that I'm doing this research are [00:15:00] sound.

Dr. Benchimol: When I came to her and said I wanted to look at the rates of pediatric IBD in, in Ontario, she said, "Fine, but you better make sure the codes are valid. And you have to do a validation study to see if that happens." And again, I initially resisted and I said, "Well, I don't really wanna validate. I wanna actually do the work, right?"

Dr. Benchimol: I wanna make- Make the findings ...

Dr. McGavock: I wanna make discoveries.

Dr. Benchimol: Yeah. But she convinced me of the importance of validation of the data. And that ended up being my master's project. And then I realized, well, no, I don't wanna just do that as a master's thesis. I want to do the other parts of the paper, epidemiology outcomes, health services use, things like that.

Dr. Benchimol: And so I ended up switching to a PhD so that I could have three or four publishable papers. Yeah. And just stayed for the extra two years or whatever and did a PhD. Without their guidance and direction and pointing me in a way that they felt there was a gap, I wouldn't be where I am today, that's for sure.

Dr. Benchimol: Totally. So we can talk a little bit of administrative data if you want and what we've done. Well,

Dr. McGavock: yeah. And I think that's a perfect segue. Thanks for the plug for Manitoba. Always plug in MCHP.

Dr. Benchimol: Oh, yeah. And I, [00:16:00] and I'll, honestly, I mentioned Charles Bernstein. He's still a mentor today. Oh. He's still doing work with Manitoba datasets at the Manitoba Centre and the IBD dataset there.

Dr. Benchimol: So- Great ... I still work very closely with Manitoba.

Dr. McGavock: Yeah. And if you love IBD, like the range of things that he does across different domains is unbelievable. Okay. So IBD, what do you mean you need codes? Like- ... obviously, like I go in, right? I, I see my physician. They do a scope. They do some blood work, whatever.

Dr. McGavock: I've got IBD. It should be there. Like what are you validating? Yeah. Why would you have to do that?

Dr. Benchimol: And so you think your doctor actually cares what codes he's using? Oh. Is that the purpose of the coding? Right. Or is it maybe to get paid?

Dr. McGavock: Right. Right. Right.

Dr. Benchimol: I'm being a little bit cynical. Right. But the reality is, unfortunately, especially with physician billing visits, the purpose of physician billing data is physicians are billing to get paid.

Dr. Benchimol: And so that one diagnosis that they're assigning to their bill, to their tariff- isn't necessarily accurate, and there's lots of things that can mimic IBD, infectious colitis or, you know, in adults ischemic [00:17:00] colitis or C. difficile, or even irritable bowel syndrome can mimic it, and somebody might code it incorrectly, right?

Dr. Benchimol: They may say rule out IBD, and there's no such thing as rule out in the diagnostic codes.

Dr. McGavock: Right.

Dr. Benchimol: So every visit is attached to a, a diagnosis. In outpatient physician visits in Canada it's one diagnosis. Hospitalizations with CIHI you can get up to 20 diagnoses, and you don't know whether the diagnoses that are being used, assigned, truly reflects what the patient has.

Dr. Benchimol: So you need to make sure that the way that you're identifying a patient with a disease using the diagnostic codes truly identifies the correct patients. You need to know the false positive rate, the true positive rate, and understand what's happening and whether your cohort is as clean as possible. And no diagnostic code is perfect, ever.

Dr. Benchimol: We're lucky with IBD that there are really two major types of IBD, Crohn's disease and ulcerative colitis. Mm-hmm. And each have their own very [00:18:00] specific ICD code, and so at least we have specific codes. Like when you talk about other diseases, those codes aren't as specific necessarily.

Dr. McGavock: Right. Like going back in time- So-

Dr. McGavock: there's no code distinguishing type 1 from type 2 diabetes, for

Dr. Benchimol: example. Exactly. Right? Or- Yeah, exactly, so it makes things very difficult.

Dr. McGavock: Mm-hmm. Uh, and-

Dr. Benchimol: And making things more difficult in Canada is that our physician billing data typically is decades behind. Okay. OHIP runs on a simplified three-digit version of ICD 9, which is probably closer to ICD 8, so we're like 25 years behind in the coding in Ontario for physician billing.

Dr. Benchimol: Right. But IBD we were able to validate the codes and develop an algorithm, a combination of different codes, outpatient physician visits, hospitalization, procedure codes, and in elderly people medication prescription codes because we have medication data in s- people over 65 in Ontario. And we were able to identify and validate algorithms that accurately identified patients with IBD at different [00:19:00] age groups.

Dr. McGavock: So like pediatric hypertension, right? One, you could have hypertension but not actually not be in the system.

Dr. Benchimol: Yeah.

Dr. McGavock: So you could have a... You could have high blood pressure, but if your physician doesn't record it or you don't get medications for it, or you don't come back for a second visit, right? This was my first intro to this, so you have to have those.

Dr. McGavock: And the three things to, I guess, to diagnose a condition there would be the three things you might look for in administrative health data would be an ICD code A code for a visit for that condition, medication, and what else? Or would those three be the keys? So you've seen- Well- ... a physician two or three times for, and they've noted down hypertension, right?

Dr. Benchimol: Yeah. So I mean, if you're lucky, nowadays you can link electronic health records to administrative data- Yeah ... and a lot of that is happening, so you could have the actual blood pressure measurement or, you know, for diabetes, a hemoglobin A1C. As we get more and more routinely collected health data from various sources all being linked together, [00:20:00] the possibility of accurately identifying these diseases grows.

Dr. McGavock: Mm-hmm. But you might be missed if you're living on reserve, for example. Yeah. If you're seen in primary care and someone's not coding. I've also heard about, like, um-

Dr. Benchimol: Shadow billing? ... charging. Yeah, that's an issue in pediatrics. So we, most pediatricians I don't know about across the country, but certainly in Ontario, subspecialty pediatricians who work in hospitals are on alternate funding plans.

Dr. Benchimol: We don't... We're not fee for service. We don't bill to get paid. We have a salary, but we have to account for our time to the government, so we shadow bill. So the billing goes in, it's just assigned a value of zero dollars because they're not paid. There was an issue in Kingston for a while that the shadow billing there in the adult hospital when they were on salary was not actually going into the ministry at all.

Dr. Benchimol: So it just, no idea where it went. It just went into a black hole.

Dr. McGavock: Right.

Dr. Benchimol: And so there's a big gap that's f- been fixed now in Kingston, but there was a big gap in the shadow billing from Kingston General Hospital, I think is, is where the issue was. Seeing a doctor means, like, [00:21:00] you actually have access to care.

Dr. Benchimol: Mm-hmm. You can actually physically go and see a doctor and you're diagnosed, right? So people who are not diagnosed who are sitting out there, which can certainly be an issue with Crohn's, where they can sit out there for years and never be picked up as having Crohn's, then the- we can't identify those people, unfortunately, with any disease.

Dr. McGavock: Right. And so is that one of the major limitations, like we talk about, like a misclassification bias?

Dr. Benchimol: Because they just aren't seeking care, they aren't accessing care the same way as other people might be, but misclassification bias also means that certain groups of people are being... That the code works differently in those groups of people.

Dr. Benchimol: So for example, like you said, in First Nations who live on reserve, the code may go to the federal government and not to the provincial data.

Dr. McGavock: Mm-hmm.

Dr. Benchimol: But First Nations who live in cities, it goes to provincial data. You know, some First Nations on reserve may end up going onto the provincial plan, and so there's intricacies in how we're identifying different groups of [00:22:00] people, which is why it's important to validate and actually validate in subgroups of people if you can.

Dr. McGavock: Which is why you got that recommendation, uh, early in your career, right?

Dr. Benchimol: Yes.

Dr. McGavock: You-

Dr. Benchimol: Very much, yeah ... you

Dr. McGavock: laid down the groundwork.

Dr. Benchimol: A- You

Dr. McGavock: proved this-

Dr. Benchimol: Yeah, and- ...

Dr. McGavock: you can do a whole career in this.

Dr. Benchimol: If I had used one code for IBD to identify children with IBD in the province, for that I'm sure your listeners will know what a positive predictive value was.

Dr. Benchimol: We validated it, and the positive predictive value was 7.9%. So we would have identified, like, 10 non-IBD patients for every one IBD patient. Those 10 non-IBD patients would've actually not had IBD at all.

Dr. McGavock: Wow.

Dr. Benchimol: So a high, a huge false positive rate, so we had to develop codes. And then the other question is why not just use the Manitoba algorithm-

Dr. McGavock: Right

Dr. Benchimol: which was validated by Dr. Bernstein, and we discovered that adult algorithms validated in adults don't function quite the same way in children, whether that's because children obviously access the healthcare system differently and maybe see their doctor at different frequencies, [00:23:00] or the alternate funding plan and shadow billing issue-

Dr. McGavock: Mm-hmm

Dr. Benchimol: or children are more likely to be scoped under anesthesia in a hospital, for example. We actually developed a specific pediatric algorithm i- instead of relying on the adult algorithm, and then we validated the adult algorithm in Ontario, the Manitoba algorithm worked just as well. So different subgroups, like I said, right?

Dr. Benchimol: So age is a subgroup that you need to see whether the algorithm's working properly.

Dr. McGavock: The first line in every pediatrics paper, children are not small adults, uh-

Dr. Benchimol: Exactly. ...

Dr. McGavock: either in the intro or in the discussion. So-

Dr. Benchimol: Yeah ...

Dr. McGavock: another huge advantage, you've got everyone from their birth, right? So we're gonna talk about a paper you just published in CMAJ that has- Mm

Dr. McGavock: you know, retrospectively we can look back to see at what age were you diagnosed, what other comorbidities. So we've got you basically through your lifetime because you enter the system with a health number, right? Yep. In most of our provinces, so we'll think about Manitoba and Ontario as an example. The second advantage is data linkage.

Dr. McGavock: If you're being seen by other physician, you can get multiple [00:24:00] comorbidities by having access. So we don't have to go physician by physician and try to pull charts and link charts from different clinics. It's all there in one spot for all health data. Yep. But then Manitoba, and I think Ontario as well, we've got social data, we've got education data, we've got geographic data, right?

Dr. McGavock: So we can actually look at different measures of poverty or affluence or deprivation, right? So linking all these data sets give us a nice rich assessment of the health and the social aspects of the lives of everyone in the province. And that's the final one. It captures pretty much everybody, right?

Dr. Benchimol: Yep.

Dr. Benchimol: I'll give you two examples from the IBD world. One was done in Manitoba because you have education data. Unfortunately, in Ontario it's not happened yet. Hopefully it's coming soon. But in Manitoba you have education and grade 12 outcomes. Dr. Harminder Singh with Dr. Bernstein looked at education outcome of, of children with IBD.

Dr. Benchimol: So identified from the health data, but looked at education as an outcome.

Dr. McGavock: Mm-hmm.

Dr. Benchimol: And thankfully found that children with IBD did just as well as the general population [00:25:00] or even better because IBD typically is in higher socioeconomic status people. But what they were able to look at is predictors of poor grade 12 outcomes, failure to progress in grade 12, and found that diagnosis of a mental health condition, a mental illness, psychiatric diagnosis, as well as socioeconomic status, were the only two predictors of poor grade 12 outcomes.

Dr. Benchimol: So that allows us clinically to target certain populations who are at risk for bad outcomes as teenagers, meaning children with psychiatric illness as well as lower socioeconomic status, and try to improve how we integrate them into the education system and the health system advocates for them, and so on.

Dr. Benchimol: So that's an important finding for our clinical practice. And then more epidemiology sort of perspective, we in Ontario are able to link immigration data And IBD is a disease that typically has been known as one of the Western world. The rates are very high in Western Europe and North America. The rates seem to be growing certainly after the fall of the [00:26:00] Soviet Union and the Iron Curtain rates sp- skyrocketed in Eastern Europe, and now we're seeing rates skyrocket in places like South Asia as well as North Africa and the Middle East.

Dr. Benchimol: And we linked immigration data and found that immigrants to Canada had fairly low rates of developing IBD similar to rates in their home countries.

Dr. McGavock: Mm-hmm.

Dr. Benchimol: But the younger that they arrived to Canada, the higher their rate. So every decade younger that they arrived to Canada resulted in a 10% increase in the risk of developing IBD afterwards.

Dr. Benchimol: And children of immigrants who were born in Canada from certain regions, particularly South Asia and again the Middle East, had the same very high rates 'cause Canada has one of the highest rates in the world. So those children from South Asia and the Middle East of those backgrounds had very high rates of developing IBD- Hmm

Dr. Benchimol: similar to other Canadians. But children of immigrants from other regions, specifically East Asia, like primarily China, Hong Kong Their rates were low when they arrived to Canada, and their [00:27:00] children's rates remained low. So that tells us that there's certainly environmental risk factors that are happening and triggering risk in people of certain ethnic backgrounds and certain-- from certain regions of the world, but those same environmental factors don't seem to be triggering risks in people, for example, from East Asia.

Dr. McGavock: Right.

Dr. Benchimol: So I think that has led to a number of studies starting to look at things like environmental risk factors and the impact on the microbiome very early in life to try to see if we can predict who's gonna get IBD and then hopefully intervene to prevent who's gonna get IBD later.

Dr. McGavock: Awesome. So thinking about season one, this work is really important for descriptive epi, which you just talked about, right?

Dr. McGavock: Mm-hmm. New immigrant or recently immigrated versus not. Maybe the second advantage of this admin dataset, you can link to your parents, right? Mm-hmm. Yep. So another really nice resource there is you don't have to recruit the parents in.

Dr. Benchimol: Yeah. So you can link to the mother. The

Dr. McGavock: mother.

Dr. Benchimol: Dads don't count in

Dr. McGavock: admin

Dr. Benchimol: data.

Dr. McGavock: No. I don't know if I'm linked to my own [00:28:00] son, which is-

Dr. Benchimol: Yeah. So you're not linked to your... Yeah, you're not... In the data, you're not linked to your own son in Canada. I can't, I can't like- In other places like Denmark, you can.

Dr. McGavock: Okay. Okay.

Dr. Benchimol: But in Canada, it's only the mother child.

Dr. McGavock: Oh.

Dr. Benchimol: I think only if the child is born in hospital because it has to be through CIHI, through the hospitalization data, but that's the majority in Canada.

Dr. Benchimol: It's not perfect, but it's the best we can do.

Dr. McGavock: Thinking about epi too, moving from descriptive epi to predictive epi- Mm-hmm ... which is the paper I think we're gonna talk about right now. You talk about identifying risk factors or classifying individuals based on risk. Maybe we can use this paper that you just published in CMAJ as an example of using the health admin data.

Dr. McGavock: This is the machine learning prediction of premature death from multimorbidity among people with IBD, a retrospective cohort study. So maybe let's start with a question. What question were you asking in this paper?

Dr. Benchimol: So what we were asking is, amongst people with IBD who passed away, who died-

Dr. McGavock: Mm-hmm ...

Dr. Benchimol: what proportion of those people die early?

Dr. Benchimol: And the Canadian definition for early death is under 75 years [00:29:00] of age, and that's sort of the well acce- accepted public health definition. So what proportion of patients died early? And in patients who died, are there clusters of comorbidities, clusters of different diseases that will then predict who's going to die early, which is obviously early death is not a desirable outcome.

Dr. Benchimol: And we know that IBD can affect all systems of the body. It's not just a gut issue.

Dr. McGavock: Mm-hmm.

Dr. Benchimol: We know things like extra intestinal manifestations of IBD, arthritis, psoriasis, skin diseases. There's all sorts of extra intestinal manifestations that psychiatric illness being one of them. I applied that earlier in kids.

Dr. Benchimol: So we wanted to know of the different diseases that they develop, what clusters will best predict who's going to have a premature death. There are extra intestinal manifestations of IBD, but there are also complications of IBD and associations of IBD. For example, IBD is an immune [00:30:00] mediated inflammatory disease, so if you have one of those IMIDs, you're more likely to have others.

Dr. Benchimol: People with IBD are more likely to have asthma, for example, and people with asthma are more likely to have IBD. Mm-hmm. People are, with IBD are more likely to have MS. But also people with IBD are more likely to have cardiovascular disease, hypertension, and even stroke as a complication of the inflammation that's going on in your body and not just an association.

Dr. Benchimol: So all of these different diseases are wrapped together as the associations or complications of IBD, and we wanted to know what clusters of these diseases best predicted premature mortality.

Dr. McGavock: Why was this an important question for you as a clinician and you as a public health official?

Dr. Benchimol: Yeah. So I, I think as a clinician why it's important is that unfortunately our health system works in a siloed fashion.

Dr. Benchimol: Mm-hmm. When you have IBD, you're seen by a gastroenterologist, and with respect to my fellow gastroenterologists, we don't do a very good job at screening for other diseases. We're just starting to become more [00:31:00] aware about psychiatric illness in IBD and mental health, and just starting to develop tools to screen for that.

Dr. Benchimol: But things like cardiovascular disease and kidney disease and even cancer outside of the GI tract, we're not great at recognizing that promptly and addressing it.

Dr. McGavock: Mm-hmm.

Dr. Benchimol: So-

Dr. McGavock: You gotta go down the hall. Go down the hall. You got-

Dr. Benchimol: Yeah. Exactly ... hypertension? Go down. Yeah ... your joints hurt, consult rheumatology.

Dr. Benchimol: Right. Hypertension consult, you know, general internal medicine. So the idea of this is that, first of all, do these comorbidities actually predict premature death? And if they do, should we be looking at restructuring how we care for patients with IBD to provide much more multidisciplinary care in a coherent fashion, working together with all the specialties or having a navigator Who can help you, patient, go through these different specialties and make sure that everybody's talking to one another and dealing with all the issues that you have- Mm-hmm

Dr. Benchimol: surround having IBD. So that was the idea behind the study.

Dr. McGavock: This is something that would be very difficult in a prospective study, right? Within your own [00:32:00] clinical data set, let's say. So what was the advantage or why did you turn to the ICES data set? And we're gonna keep using that term a lot, so ICES is what?

Dr. Benchimol: Sorry. ICES is the organization, the research group, the company that was founded by the province of Ontario, the Ministry of Health, with academic researchers. ICES holds all of Ontario's health administrative data and population data, and allows researchers and the ministry to evaluate how the healthcare system is treating people in Ontario, and encourages academic research as well as practical research for the ministry to try to better the care and outcomes of Ontario residents.

Dr. Benchimol: There's similar organizations in Manitoba, the Manitoba Centre, Health Data Nova Scotia, and so on. BC has one as well. These are kind of organized ways of accessing Ontario health administrative data and doing research with them.

Dr. McGavock: We also have one more nationally, CIHI.

Dr. Benchimol: Mm-hmm.

Dr. McGavock: Which is a little bit different.

Dr. McGavock: Maybe we can [00:33:00] touch on that at the end. But this is- Yeah ... a central repository where you as a trainee can walk in there and access some of these super rich data sets to ask really important questions. Yeah. Okay. So- Yeah I

Dr. Benchimol: was gonna say, wa- walk in there is a bit of a- Yeah, not walk in there ... uh, the security behind these places is quite tight.

Dr. Benchimol: But yeah, if you have a supervisor that works there. And there's ways of actually accessing it from outside where you request the data to be cut. It is not the public library, that's for sure.

Dr. McGavock: It might take two years for you to actually get approval to use some of these datasets.

Dr. Benchimol: No comment.

Dr. McGavock: We're not going down that path.

Dr. McGavock: So what dataset did you create, and then how did you go about answering this question?

Dr. Benchimol: Right. So for this study, it was everybody in Ontario diagnosed with IBD who passed away between the years 2010 and 2020. So it was primarily an elderly population. We have data in Ontario from 1991 onwards. These patients could have had IBD for 20 years or more and then died.

Dr. Benchimol: We looked firstly at the rate of premature death, which was almost half, so 47% of people who died d- with [00:34:00] IBD died prematurely. Wow. It's not that they died because of IBD. They had IBD at the time of their death.

Dr. McGavock: Were you expecting it to be that high?

Dr. Benchimol: I was not expecting it to be that high, and that's a bit discouraging.

Dr. Benchimol: We've made a lot of advancements over the years in the treatment of IBD, especially with the, the development of the biologics and small molecule therapies, which have really revolutionized the treatment of patients with Crohn's and colitis. But again, this is an older cohort, you know- Mm-hmm ... who died, so it's possible that those changes in care didn't filter down over the years yet.

Dr. Benchimol: Right. But 47% is significantly higher than what's reported in the general Ontario population for premature mortality. It's in the range of 20 to 25% of people die before 75- Wow ... in Ontario, so it's quite high. Double. The rate was higher in males than females, so 50% versus 44%. That we do see in the general population for premature mortality.

Dr. Benchimol: Mm-hmm. And then we looked at what chronic conditions they have at the time that they die. The most common ones were arthritis, hypertension, [00:35:00] mood disorders, kidney failure, and cancer, various forms of cancer. What we found was that using various clustering and decision tree analysis and various machine learning models, we found that firstly, diagnosis with these comorbidities at a younger age, predominantly if you were diagnosed with these comorbidities under the age of 50, was a significant predictor of premature mortality, as well as male sex was a significant predictor of premature mortality.

Dr. Benchimol: So that starts to lead one to believe that perhaps we could target this multidisciplinary approach to those people who are starting to be diagnosed with these comorbidities earlier in life. And while their death may be 10 or 20 years away, we could potentially delay their death, improve their lifespan, and improve their quality of life as well.

Dr. McGavock: Yeah. So just taking a step back. For anyone who's interested in admin data, the, one of the reasons I chose this paper is [00:36:00] figure one- Mm-hmm ... is beautiful in terms of mapping us through all the steps that you took, and also showing that it's a timeline, right? Where you will have different diagnoses at different time points between the years where the data are available.

Dr. McGavock: A little diagram there showing the patient's individual data, and then modeling using these different modeling characteristics and saying, you know, using multimorbidity in this context, we can get everything that we are interested in from like hypertension, cancer, diabetes, stroke, COPD. And also from an area level, be- because we don't have the individual level, like income, for example, you can get the neighborhood that they live in and kind of extract from that or extrapolate from that what kind of- Level of deprivation or affluence that they're exposed to

Dr. Benchimol: Absolutely.

Dr. Benchimol: I'm gonna give props where they're due. So this figure one and the whole paper really was the work of Gemma Postill, who's a med student. Absolutely. She's an MD PhD student- Oh, wow. Nice ... uh, who had an interest in machine learning and AI when she came into medical school, [00:37:00] and has just nurtured that and is now doing her PhD with Laura Rosella, who was instrumental for this study as well.

Dr. Benchimol: I'm not a machine learning person. I'm more of a traditional epi analysis type of person. Gemma did all of this work, and she did an amazing job. That figure one was all her. Hopefully, your listeners can see it, but she absolutely did an amazing job. She worked with me over the course of three summers and now has entered her PhD and is doing her thesis.

Dr. Benchimol: So- Nice ... great work for Gemma.

Dr. McGavock: Laura was in season one. Mm. An overview of predictive epi, and so this is a beautiful example of predictive epi where you're saying this isn't a causal question. I don't think you were asking about which of these comorbidities, it's just- Yeah ... are they there? In the discussion you talk about how do we interpret...

Dr. McGavock: I'm looking at figure two now, where you have figure 2C where you're looking at that XGBoost model. Mm-hmm. Looking at age at diagnosis. So it seemed to me one of the novel findings that came out of there, looking at the strength of the admin data, is you can look at the time when they were diagnosed with these comorbidities.

Dr. McGavock: And potentially- Yeah ... the length that they lived with [00:38:00] it is a predictor of premature death. Clinically, is an important finding because From your standpoint, age at diagnosis of a comorbidity, not just IBD, becomes another factor you might want to think about- Exactly ... in terms of identifying the populations most at risk.

Dr. Benchimol: Yeah, exactly. And we have something coming as well. J- again, Gemma has written it, but looking at clusters of comorbidity. Okay. So if you've got two or three of these diseases put together at a certain age, when do you get concerned? So there's lots of work to be done on this, but I think the key is that this type of work cannot be done with traditional epi.

Dr. Benchimol: The interactions between all of these different conditions and variables are so complex, we can't design traditional epi models to be able to account for all of these factors: age, and which clusters of diseases, and socioeconomic status, and all these things put together. So that's why machine learning models in this type of situation are so powerful, because they can find associations and detail, a level of detail that perhaps we couldn't find [00:39:00] with traditional regression models that are defined ahead of time.

Dr. McGavock: And then from a administrative health data standpoint, there's no way we're gonna create a cohort of patients with IBD to follow for 90 years. Yeah. Where we're gonna measure all of these things, ask them to come back every year to do measurement. Resource-wise, it's just impossible, so.

Dr. Benchimol: Yeah. This is the major advantage of routinely collected health data- Mm

Dr. Benchimol: which health administrative data is one subtype. It's being collected, it's fairly cheap to access, and typically large amounts of data are being collected on a regular basis. You can query those data in order to answer important questions that perhaps you couldn't do in a prospective cohort study. The disadvantage is you're restricted to the quality of the data and the variables that were collected and the types of data that you have access to, and so your question may be restricted in terms of what you can answer.

Dr. McGavock: And so that's where I was just gonna go next is so the limitations here. I just got a- Mm ... an email this [00:40:00] morning from a journal rejecting my administrative data study that looked at cardiovascular disease inequities in two groups of people, and they said, "Well, you can't really say anything because you don't have data on alcoholism and smoking."

Dr. McGavock: Mm. So one of the limitations of this data set is we don't know, for those who had early onset multimorbidity, we don't know anything about what?

Dr. Benchimol: It is so unmeasured confounding, right? Right. So we, we're all familiar with confounding. Yeah. A confounder is a variable that affects both the predictor and the outcome.

Dr. McGavock: Mm-hmm.

Dr. Benchimol: Uh, so a confounder, the typical, I'm sure your listeners know about the example that everybody use, coffee causes cancer. They discovered that no, coffee does not cause cancer. People who drink coffee are more likely to smoke. Smoking causes cancer, so smoking is the confounder. In IBD, that's important because smoking is a huge issue with Crohn's disease.

Dr. Benchimol: Smoking increases your chance of developing Crohn's disease, increases your chance of having stricturing disease, so blockages in your bowel needing surgery, and decreases the [00:41:00] chance of the medications working. Again, probably all through the microbiome. In administrative data, we don't collect smoking status.

Dr. Benchimol: Even in electronic health records, if smoking status is there, it's spotty and probably not all that accurate. Smoking is a great example of unmeasured confounding, where, for example, if you smoke, you're more likely to have complicated IBD. And if you smoke, you're more likely to have cardiovascular outcomes.

Dr. Benchimol: But we don't have smoking, so what do we do about that? And there's ways of managing things like unmeasured confounding, ways that people have come up with, such as instrumental variables. But more importantly, I think we're starting to learn that perhaps we can impute those unmeasured confounding factors-

Dr. McGavock: Yeah

Dr. Benchimol: from other data sets. So if you have a data set with the unmeasured confounder and you know about their IBD status, you could then impute what you know from the clinical data set with smoking status onto the administrative data and impute what the smoking rates would be, almost like a synthetic data approach.

Dr. Benchimol: Oh, wow. So there's ways that are happening that we can figure out unmeasured confounding. In addition, there's [00:42:00] statistical ways to measure something called an E-value, tells you what strength of association. So what is the strength of association that unmeasured variable has to have in order to change your result from a true statistically significant association to one that is not statistically significant anymore?

Dr. McGavock: Right.

Dr. Benchimol: And so that gives you an indication, like if your association between smoking and cardiovascular outcome has to be seven in order for the association to become non-significant, it's probably not gonna get there. Like we know from clinical research that it's not that strong an association. So there's statistical ways of managing unmeasured confounding that we're learning about that is very applicable to administrative data.

Dr. Benchimol: And that leads to the idea that ... And this w- was what really got me excited when I started doing this research in terms of the methodology. Really a field with evolving methods. Mm-hmm. This is, you know, work with administrative data really started in the late '90s in Canada And very early [00:43:00] in our understanding of the strengths and weaknesses and how to address those and the methods that we might use and things like that.

Dr. Benchimol: So that's why I got very interested in things like validation and reporting, which I'm sure we'll talk about. I think the methods are still evolving, and this machine learning AI is a new method that w- didn't exist 10 years ago. Mm-hmm. Nobody was using it with administrative data. So that leads to more methodologic issues that we need to hash out and think about.

Dr. McGavock: Not having smoking doesn't affect your predictive epi, right? It's not- Correct. There are no-- There's no confounding in predictive epi, so it might just be another variable that could enhance the, the specificity of your model. Yeah. In my study, it was descriptive epi, so it actually having those unmeasured confounding makes no difference 'cause I wasn't asking a causal question.

Dr. McGavock: And then a beauty of this too, I think administrative data, is you can ask descriptive epi, predictive epi, and causal epi, right? So all of these methodological tools are available in that rich resource at ICES or at the Manitoba Centre for Health Policy. You used AI and machine learning, and maybe we can [00:44:00] talk about that later, but you were really important in reporting guidelines.

Dr. McGavock: And I think just like reporting guidelines for RCTs, they make sure the data we're reading are as easy to interpret as possible, right? So that we're getting the full picture of what the study was and what happened in that study and what kind of data we're looking at. So you were involved for the RCT, that's called a CONSORT statement.

Dr. McGavock: We talked a little bit about that in a previous season. You created a reporting tool for administrative data that would go alongside another reporting tool, like a STROBE checklist, right? And STROBE is for observational studies. So maybe from your standpoint, why is it important to have these reporting guidelines and the RECORD and what does it look for?

Dr. Benchimol: Yeah. So RECORD, uh, RECORD are the reporting guidelines for studies, observational studies conducted using routinely collected health data.

Dr. McGavock: Yeah.

Dr. Benchimol: They're an extension to STROBE. So like you said, STROBE are the reporting guidelines for observational or non-interventional studies, things like cohort studies, cross-sectional research, those types of research.

Dr. Benchimol: And the idea of reporting guidelines [00:45:00] is that they should be general checklists of items, the bare minimum that should be in your manuscript, that you need to report in your manuscript in order to ensure that the reader, the re- peer reviewer, the editor, the stakeholder like policymaker are able to understand what you did, the strengths and weaknesses of the study, the results, and the implications of the results So STROBE was intended as a very general observational research reporting guideline.

Dr. Benchimol: And what we noticed along the way was that it didn't cover important issues for research using health administrative data or routinely collected health data. For example, we talked about validation of the codes. There's nothing about validation in STROBE because it's not about codes and data, it's about cohort studies where it's prospectively collected or retrospectively collected.

Dr. Benchimol: So we felt that it was important to extend STROBE to research using routinely collected health data. This all [00:46:00] came out of a comprehensive project that I did as part of my PhD. That was a systematic review of validation studies. The journal said, "Why don't you turn this into a reporting guideline as well and say how should we validate studies?

Dr. Benchimol: Like what should you be reporting in a validation study?" And then we said, "If we're doing this for validation studies, we should do reporting guidelines for health administrative data studies in general." We went to STROBE and said we'd like to extend STROBE for health administrative data, and they said, "There's this group from the London School of Hygiene and Tropical Medicine who want to do the same thing for the UK primary care data.

Dr. Benchimol: It's all health data. Why don't you all get together and do one reporting guideline for all sorts of routinely collected data?" And so I met with their leader, Sinead Langan, who's an epidemiologist at the London School of Hygiene, and we've now been collaborating for It's 13 years, and we were both junior, and now we're not.

Dr. Benchimol: Awesome. We are still collaborating through the RECORD process. So we ended up working together and producing these reporting guidelines for all research using routinely [00:47:00] collected health data with items that should be reported if you're doing a study using a non-interventional study, right? So not a trial using administrative data or routinely collected health data, but an observational study.

Dr. Benchimol: And published that in 2015, and then we extended RECORD, so an extension to STROBE became an extension to RECORD for pharmacoepidemiology research, RECORD-PE. Yeah. And then in addition, CONSORT was extended for trials that integrate routinely collected health data- Right ... and that's called CONSORT-Routine.

Dr. Benchimol: Those are the major three reporting guidelines for routinely collected health data, and it's been very nicely adopted by journals who require it sometimes, or whose peer reviewers review it. And then it's a great teaching tool for students, right? I think it, it's important to note that these reporting guidelines are about reporting, not doing.

Dr. Benchimol: So they're about what should be in the manuscript and what you should report. They're not telling you what you should do in research. But I think that for students [00:48:00] to read these over, it helps them understand some of the methodologic issues that are present in research using routinely collected health data.

Dr. Benchimol: And then I think it will help them design their study as well, because they're aware of these issues and, and they can design a study that will address them.

Dr. McGavock: Totally. Like, you need to have that data, or you need to have that information, right? So you need to see those reporting guidelines. Yeah. You need to see the RECORD reporting guidelines when you're designing your study at the outset just to make sure you have those things when you- Exactly.

Dr. McGavock: So-

Dr. Benchimol: Exactly.

Dr. McGavock: So what are some of those key things about administrative observational data that we should be looking for when we're reading a paper?

Dr. Benchimol: Yeah. So I think key things that were not frequently reported in the reviews that we did of the literature, so obviously just saying that you're, what type of data you're using, health administrative data, for example.

Dr. Benchimol: Where it is, what sort of jurisdiction, location you're doing the research in. The, who's in the cohort, who's in the database, and then who's in the cohort that you're using to study your, and answer your research question compared to the overall database. So who's missing from the population but is not in the database, and [00:49:00] then of the people in the database, who's in the study cohort?

Dr. Benchimol: Validation is a big part of them as well. In, in, you know, reporting whether or not the codes have been validated for identification of the cohort, but also outcomes, confounders, predictors, and so on. All of these rely on codes, so did you validate these codes? You know, it's not saying that you must, but if you didn't, you have to say that you didn't.

Dr. McGavock: You also expect us to have a reference to that validation

Dr. Benchimol: or something Yeah, yeah. So either report what you've done or provide a reference in the literature to the validation study or say that it wasn't validated. And then it's up to the peer reviewers to decide whether that's okay or not. Linkage is a big part.

Dr. Benchimol: So when we're linking data sets together, if it's not a straight deterministic linkage with a health card number to health card number, if you're using, for example, a probabilistic linkage where you might use like last name, date of birth, you're not 100% sure, you need to know or we need to know how the linkage was done and the accuracy of that linkage.

Dr. Benchimol: Who dropped out because they could not be linked? [00:50:00] Reporting where your code is housed, the open science framework of where's your code, where's your analytic code housed, can we access it? Where's the raw data housed, can we access it?

Dr. McGavock: Mm.

Dr. Benchimol: And so on. All of those are important components of reporting transparency.

Dr. McGavock: Nice. In this paper, you also talk about reporting AI machine learning. I think administrative data is really ripe for using AI and machine learning. Where do you see AI playing a role in this space, in this kind of epi?

Dr. Benchimol: And there are reporting guidelines for AI research out there. APPRAISE AI is one of them.

Dr. Benchimol: Um, I think that AI and machine learning are tools that we apply to routinely collected data. They're methods, so I think it's very important to know what is being done when you're applying those methods. It shouldn't just be a black box. You need to have an understanding of how the AI is working and what the strengths and weaknesses are.

Dr. Benchimol: Unfortunately, 2015, RECORD, AI was just coming in, and I remember having that discussion a little bit, but it really was not predominant [00:51:00] like it is now. Maybe it's time for an update to RECORD, and we may end up doing that so that we modernize it a little bit. One thing that we are doing at this stage, the RECORD group has focused primarily on reporting and transparency and not on the methodology, so we got a CIHR grant to start to develop a standard protocol template tool.

Dr. Benchimol: So when you're doing a research project, you're hopefully writing a protocol ahead of time so that you know exactly what you're going to do, what your hypothesis aims, your methods and so on. There's been a few. For example, the CONSORT statement led to the development of SPIRIT, which is a protocol template tool for randomized control trials.

Dr. Benchimol: So we've now gotten a CIHR grant to develop a protocol template tool for research using routinely collected health data, and part of that is surveying stakeholders, experts in the field who will say what should be included there. And we have a stakeholder group [00:52:00] specifically for AI and machine learning experts who are gonna be integrated in that.

Dr. Benchimol: So if you're using AI in your research, what should be included in the protocol so that you know that you're doing it properly? Nice. We hope to come out of it at the end with a template that people can use to design their studies. And not only that, the idea of this is also an e-learning curriculum, and I think this is particularly important to students That there's lots of people now who have access to administrative data and routinely collected health data because of the rise of EMRs primarily.

Dr. Benchimol: So what we're aiming for is making an e-learning curriculum that'll be widely available to students, to people in developing nations, for example, who can train and identify and understand how to design a study using routinely collected health data. That's still a few years off, but we're working on it hard.

Dr. McGavock: Oh, awesome. Reach out to us 'cause we are creating this repository for trainees. Yes. Just so that people are aware, we were funded to do a couple of randomized control trials, and we're just now publishing the protocols because you put the protocol out there, you tell everyone what you're gonna do, it saves you [00:53:00] time, publish your RCT, right?

Dr. McGavock: Yeah. So this is a huge movement in the space of administrative data where you can say, "We have a grant. We're gonna answer this question. This is how we're gonna do it," right? And we need protocols to follow. The one for RCT is called SPIRIT, so if you're publishing a protocol paper, you have to follow the SPIRIT reporting guidelines for what's going in your protocol.

Dr. McGavock: This would be outlining protocols for administrative health data, which is really valuable also when you're starting your own project. You can go read Eric's beautiful protocol for how he's gonna be doing his next analysis, and that really helps you inform your steps towards creating your own research project within administrative health data

Dr. Benchimol: Yeah.

Dr. Benchimol: And we helped develop HARPER, which is getting to the administrative data, the protocol template tool for pharmacoepi research, the drug assessment using real world data, which is routinely collected health data. The idea of HARPER is to standardize the protocols that you can then submit to the FDA or the EMA [00:54:00] so that all of their criteria are met and they know exactly what's being done.

Dr. Benchimol: It's very long and complex, so it's not really as much of a teaching tool as what we're hoping to develop. But i- if you're doing a pharmacoepi study, a drug related study using admin data, I would look at HARPER as one of the tools that you can use.

Dr. McGavock: Most of the time we talk to trainees, we say, "You need to say no to people," but it looks like every time you've said yes to someone it's turned into something really valuable.

Dr. Benchimol: The wrong time in my life to point out that I never say no. I'm regretting some of that. But the collaborations that I've, I've been really fortunate to work with some amazing people, and even very senior people who have encouraged my career and done it selflessly. I mentioned Charles Bernstein as one.

Dr. Benchimol: We developed a national network called the Canadian Gastrointestinal Epidemiology Consortium- Mm-hmm ... of researchers who are using admin data in multiple provinces and trying to work together to produce a national picture of what's going on with GI illnesses using admin data. Charles was the father of all of this in Canada, and could have said, "Okay, I'm gonna [00:55:00] lead this," but instead said no, and I was like second year faculty.

Dr. Benchimol: He's like, "No, this is your idea. You lead it. Go for it." Nice. And he's been hugely supportive since then. And even in with RECORD the late Doug Altman. Oh, yeah. I don't know if people, listeners know his name. Wow. He was, you know, the Bland–Altman curve- ... biostatistician with huge impact. He turned later in his career towards more research waste, and as part of that reporting guideline, avoiding research waste, and reporting guidelines as part of that transparency is important to avoid studies being repeated over and over or junk science being done.

Dr. Benchimol: So he helped us develop RECORD. He's passed away since then, but like in retrospect, again, Dr. Sinead Langan and I were- Third, fourth year faculty. This person who is a huge figure in biostats and epi just helped us. Nothing really to gain for him. He just helped. So I've been really lucky with collaboration.

Dr. Benchimol: I've learned to say yes, I guess.

Dr. McGavock: You're also a really nice guy, so it, it's easy for people to say yes- Thank you ... yes to you. Yeah.

Dr. Benchimol: Thanks. I [00:56:00] appreciate it.

Dr. McGavock: So for residents out there who heard this and heard what you're doing and were getting really excited, you can look at Eric's lab at SickKids. You can look at the Canadian Gastrointestinal Epi Consortium where you could work across provinces.

Dr. Benchimol: You have to know what the acronym is, CanGIEC.

Dr. McGavock: CanGIEC. All right. '

Dr. Benchimol: Cause we're all data geeks. You're all- So Ca- It's actually C-A-N-G-I-E-C.

Dr. McGavock: Yeah.

Dr. Benchimol: And the website is cangiec.ca.

Dr. McGavock: All right.

Dr. Benchimol: But yeah, yeah, we're all data geeks. And you can also go to record-statement.org.

Dr. McGavock: Mm-hmm.

Dr. Benchimol: Uh, that's the, the site for the RECORD Statement.

Dr. McGavock: Is there anything that we missed to talk about within this space that you think is really important that trainees should know about?

Dr. Benchimol: I think just trainees need to remember to focus. Like, don't-- Everything is cool and interesting and fun when you're young. Try to develop some degree of focus on one or two areas where you're gonna focus your early career, and then things open up as long as you've established yourself.

Dr. Benchimol: So don't try to do too much, try to focus, but keep your mind open to things and be willing to change. Like I said, [00:57:00] RECORD was kind of a side project for me in my PhD. It was more the methodology was more of a comprehensive project because I had to do it, and ended up being the first CIHR grant that I got, and a major stream of research for me later.

Dr. Benchimol: And so I was a bit flexible, like I shifted from IBD to that and then back, but those were my two major areas of focus. I didn't do everything.

Dr. McGavock: Although I see now you're doing patient-oriented research.

Dr. Benchimol: And I'm doing a clinical trial- Yeah ... a pragmatic clinical trial with implementation science component.

Dr. Benchimol: Oh. It's funny that, yeah, your responsibilities go up just as you get older and your energy starts to go down. So we'll see how that works out for me.

Dr. McGavock: You're exercising every day, which I'm sure that's gonna help. Yes. The last question, like a go-to resource, like if I wanna get into this space or a go-to paper, something that you really relied on that helped you either understand or navigate this space or get you excited about it.

Dr. McGavock: Births being published in the '90s

Dr. Benchimol: Yeah. Exactly. Any validation study. Yeah So listen, for validation studies, in American Journal of Epi this [00:58:00] year, we published a review article which really is the aim was to be almost like a book chapter in some ways of guiding people on how to validate- Hmm

Dr. Benchimol: administrative routinely collected health data, how to validate these disease cohorts, and so on. And that's published in the American Journal of Epi, and we can certainly provide that. I think that's a really nice overview of validation of these types of data. You know, like I said, CONSORT routine are very nice overviews of what should be included in these studies.

Dr. Benchimol: If you're interested in the idea of reporting guidelines, the EQUATOR Network is really where you should turn. EQUATOR-network.org. And they catalog all the reporting guidelines and have the major ones on the front page. There's training courses about how to use reporting guidelines when you're writing and when you're peer reviewing for journals.

Dr. Benchimol: The EQUATOR Network is an invaluable resource in this field.

Dr. McGavock: We have links to that on our central repository for trainees, but we'll pump that as well. And maybe I'll just pump one up that is Manitoba. Pat Martens and Leslie Roos, they've done some historical, like where did MCHP come from, and it's sort of the history [00:59:00] of administrative health data in Manitoba.

Dr. McGavock: You know, a, a nice narrative if you're Manitoba. Yeah. It also gives you a sense of- Yeah ... how hard this was or how hard this is to get started, but also what kind of a rich resource this is if you're asking the right questions.

Dr. Benchimol: Yeah, absolutely. I read that paper. It's really interesting. If you like history, right?

Dr. Benchimol: Yeah Like science history is cool for me. Yeah,

Dr. McGavock: Canadian

Dr. Benchimol: science- Uh, we haven't done anything like that with ICES, but I know that the person who really started and spearheaded ICES was David Naylor, who ended up becoming president- Oh ... of the University of Toronto. I know, I, I've heard how much work it was to get the Ministry of Health on board with all of this, and we're very lucky because other ministries I won't name in the country have not been as willing to work with academics to get this type of resource on board.

Dr. Benchimol: So- Yeah ... you know, Manitoba, BC, Nova Scotia, and Ontario, as well as Saskatchewan's pretty good as well. Mm-hmm. But they don't have a central repository necessarily. But yeah, so we've been very lucky with all of these resources.

Dr. McGavock: So if you're a Can geek, Can Epi geek, we'll put some links into there for the history- Sure

Dr. McGavock: of Canadian Epi and then we'll go from there. Thank you so [01:00:00] much, Eric. I know you're really busy. You got lots going on. Even though you say you're a yes man, I know you're really strict with your time, so I'm so pleased you could make it. My pleasure,

Dr. Benchimol: Jon. It was great to chat. I like chatting with you in general, but in this sort of format, I hope it helps some of the trainees and the listeners out there.

Dr. McGavock: Me too. That was awesome. Thanks again, and we'll put links to all this, and good luck in the next couple of CIHR grants, and look forward to seeing your next publications.

Dr. Benchimol: Thanks, Jon.

Dr. McGavock: Thanks.

Dr. McGavock: That was a fascinating and informative conversation. A huge thanks to Dr. Eric Benchimol for joining us today and for sharing not only his expertise in routinely collected health data and the RECORD guidelines, but also his deeply personal journey from being diagnosed with IBD as a medical student to becoming one of Canada's leading clinical scholars in the area of pediatrics and child health.

Dr. McGavock: I think many of our listeners might relate to Dr. Benchimol's story, how a personal challenge can profoundly shape your research interests and career trajectory in clinical research and clinical medicine, how the transition from bench science and undergrad to clinical research in your own grad work or in your residency can be very challenging.[01:01:00]

Dr. McGavock: It isn't always linear, and how transformative having the right mentors can be at the critical moments in our training. His path reminds us that our lived experiences, our curiosity about different methods, and the guidance we receive along the way all contribute to the health researchers that we become.

Dr. McGavock: I was super excited to dive deep into the 2025 CMAJ paper that Dr. Benchimol published on machine learning, on how to predict premature mortality in people with IBD. This paper beautifully illustrates how these new methods can tackle complex real-world questions that matter deeply to patients and clinicians.

Dr. McGavock: It's also fascinating to see how having the data there, having that administrative health data that's in a repository that we can use for our own research can shape the way that we think about clinical practice, especially in this case for people living with IBD. If you want to learn more about Dr.

Dr. McGavock: Benchimol's work, I encourage you to read that paper in CMAJ. I encourage you to take a look at the RECORD statement and how it was developed at record-statement.org. You'll find links to all of these in our show notes, along with information about the Canadian Gastrointestinal Epidemiology [01:02:00] Consortium and Dr.

Dr. McGavock: Benchimol's other work. Thank you again for joining us on the Emerging Scholar Podcast today. This is an intro to administrative health data. Next week, we'll be talking with Dr. Jennifer Walker, who also works at ICES, an indigenous scholar out of McMaster University and a Canada Research Chair in Indigenous Health Equity, where we talk about race, administrative health data, indigenous data sovereignty, and how data are not numbers, they're stories.

Dr. McGavock: If you enjoyed this podcast with Eric, I really hope that you'll come back for the next episode with Dr. Walker

Dr. McGavock: The Emerging Scholar podcast was recorded in the UMFM studios located at the University of Manitoba in Winnipeg, Manitoba, Canada. This is Treaty 1 territory, the original homeland of the Anishinaabe, the Ininew, and on the national homeland of the Red River Métis. Manitoba is also homeland to the Anishininew, the Dakota Oyate, and the Dene peoples.

Dr. McGavock: We acknowledge that our water is sourced from Shoal Lake 40 First Nation. As a [01:03:00] settler scientist working in health research, I acknowledge TRC Call to Action number 18, that health inequities between Indigenous and non-Indigenous people are the direct result of colonization. Winnipeg is the center for reconciliation in Canada, and where we are speaking from today is at the crossroads of Canada's history and Canada's future.

Dr. McGavock: We are three kilometers away from one of the most sacred meeting grounds for Indigenous people on the plains and within Turtle Island. We are also several kilometers away from one of the last residential schools in the province. We are working collectively on reconcili-action with youth, elders, and community leaders to heal from the trauma of colonization, past and present, honor the original treaties, and create a Canada that addresses and eliminates the inequities and injustices of the past and present to create a future for Indigenous children to live and thrive.

Dr. McGavock: On behalf of the team at the Emerging Scholar podcast, we want to thank Osani Balkaran for the great production for our [01:04:00] opening and closing themes. We also want to thank UMFM 101.5, especially our executive producer, Jared McKetiak, for technical assistance and production on this project. We also want to thank all of our guests for the discussion and the insight that they've provided.

Dr. McGavock: Once again, thank you for listening. Be sure to subscribe to the Emerging Scholar podcast on Apple Podcasts, Spotify, and umfm.com.