AASHTO re:source Q&A Podcast Transcript

Season 4, Episode 24: The Value of Internal Audits - Part 1

Recorded: October 25, 2023

Released: November 7, 2023

Hosts: Brian Jonson, AASHTO Accreditation Program Director; Kim Swanson, Communications Manager, AASHTO re:source 

Guest: Tracy Barnhart, Quality Manager, AASHTO re:source

Note: Please reference AASHTO re:source and AASHTO Accreditation Program policies and procedures online for official guidance on this, and other topics. 

Transcription is auto-generated. 

[Theme music fades in.] 

00:00:02 ANNOUNCER: Welcome to AASHTO resource Q & A. We're taking time to discuss construction materials testing and inspection with people in the know.  From exploring testing problems and solutions to laboratory best practices and quality management, we're covering topics important to you. Now here’s our host, Brian Johnson.

00:00:21 BRIAN: Welcome to AASHTO resource Q&A. I'm Brian Johnson.

00:00:25 KIM: And I'm Kim Swann.

00:00:26 BRIAN: Today on the podcast, we're going to talk about internal audits with Tracy Barnhardt, our quality manager. Tracy, welcome to the podcast.

00:00:34 TRACY: Thank you. Hello.

00:00:36 BRIAN: Yeah. So, I wanted to cover this topic because it's an issue that laboratories really struggle with. So, I'd say the two big things that that come up time and time again are management reviews and internal.  That's which are two different activities. We're not going to get into the differences today, but we are going to do a deep dive on internal audits and the best way I think we can cover this is by talking about our very own internal audit that takes place for the Ashe accreditation program. So I guess that's open it up, Tracy. I mentioned it's the audit.  To the Ashe accreditation program, how many internal audits do you perform every year?  On different programs, or do you just do one big one?

00:01:17 TRACY: When we first started doing internal audits many years ago, we did attempt to just do one huge internal audit to cover everything, and that did not work out well. There's way too much information to cover. We decided to break it into programmatic internal audits, so we do one for the laboratory assessment program, the accreditation. Program the proficiency sample program. We do one for the quality program where someone else audits my activities. We do one for the IT depart. The administrative departments we have a lot of different audits going on per year and helps us laser focus on those specific programs.

00:01:59 BRIAN: As you said, per year, does that mean you're doing every single one of those that you just mentioned every 12 months?

00:02:07 TRACY: That's the goal. Yes, our goal is to hit that 12 month interva12-monthimes I intentionally schedule it at slightly different intervals depending on activities that are going on within those different programs. But yes, the ultimate goal is to not exceed the 12-month interval and we're auditing to ISO 9001 internal audit. Requirements in that particular standard does not specify. A specific interval for internal audits like AASHTO R18 does.

00:02:37 KIM: I was just going to ask that, Tracy, what are you actually auditing our programs to against what standards?

00:02:44 TRACY: ISO 9001 and our own quality management system requirements, and that's a piece that many of our customers aren't thinking about. They're focusing on AASHTO R18 requirements. And are we doing all of these things that AASHTO, R18. Requires us to do, but they're missing the piece of auditing their own policies and procedures unique to their organization. And as Brian and Kim know, we have hundreds of policies and procedures specific to AASHTO resource, and I sample those during internal audits to ensure that we're actually doing what we say. We're doing.

00:03:21 BRIAN: Yeah, that's a good point. We do have so many policies and procedures and they aren't necessarily reflective of the standard we're being audited against. So we didn't necessarily develop our quality management system. Line by line to match ISO 9001, we developed these policies and procedures based on need and the need, and the idea is that we're all not going to be working here forever, even though it feels like it sometimes, right? So, Tracy, Tracy, you've been it and AASHTO employee. How many years?

00:03:53 TRACY: Coming up on 34 years.

00:03:56 BRIAN: 34 years. So it's been a little while and eventually Tracy is going to want somebody else to take over and be able to carry on the way she has and actually continually improve upon. What she's done over the years and I have. I'm coming up on 25 years myself, so I also would like somebody to be able to take the baton for me and run with it and fix all my mistakes and improve upon everything that I have done as well as I can do here. So that is the value of the quality management system. I guess that's just a summary of why we even have it. But when we're auditing what Tracy was saying is you know, we're auditing against our own requirements and 9001, Tracy, our labs are probably listening to this one.  Ring, why aren't you auditing against R18 like we are?

00:04:44 TRACY: We do get that question fairly regularly from customers. The reason is we're not a testing laboratory. We're a service provider. We provide accreditation services, laboratory assessment services and proficiency testing services. The AASHTO R18 requirement doesn't align nicely with that.

00:05:04 TRACY: But ISO 9001 does. It's a more general set of requirements for organizations that provide products and services to customers.

00:05:14 BRIAN: Now do you? Think it's possible if there was a testing firm in our.  Program that was also interested in conforming to ISO 9001. Do you think that that would be a good match based on their services that they provide?

00:05:29 TRACY: I do and we have a handful of customers that do have ISO 9001 accreditation or recognition and I I think it definitely is something to look into to help them improve. Yeah, as you mentioned a few times, the focus should always be on continual improvement and they can raise the bar even higher for themselves by pursuing ISO 9001 recognition.

00:05:49 BRIAN: You know, you mentioned that R18 doesn't necessarily apply to us. But 9001 does, but then you could have a testing firm that is in conformance, Larry Teen. They could also confirm 9001, right?

00:06:00 TRACY: That's correct or ISO 17025, which is even more specific to testing laboratories. The Asha accreditation program requires that all laboratories be in conformance with ASH. Two are 18. That's the minimum set of requirements for accredited laboratories.  So that's why you have to start with AASHTO R18 first and then you can branch out to these other standards.

00:06:23 BRIAN: Right. And the ISO standards are very more generally written correct. So like our team might say, well you have to train all of your testing technicians, they would say you have to perform competent services or ensure that you're competent and I'm.

00:06:29 TRACY: That's correct.

00:06:40 BRIAN: Guessing at what the line would be in 9001. I'm.  Not quoting it.

00:06:44 TRACY: That's correct. It might just say the organization shall ensure that staff are competent to perform the tasks that they perform where R18 is more prescriptive about how that needs to be done for specific testing standards.

00:06:59 BRIAN: Yeah, thanks. That's what I was trying to get at it and try to uncover is what is that specificity. So thank you for that. Alright, let's talk about the audit itself.  How do you prepare for an internal audit?

00:07:13 TRACY: Preparation is key to an effective internal audit. I do spend a good bit of time preparing. First and foremost, you have to schedule an internal audit and the past couple of years I've been trying to schedule audits. When we go into a new year and schedule them for the entire year, work with the program managers to ensure that we have a mutually agreeable. Audit date for each internal audit. So that's the first thing that you have to do.  And then about two weeks prior to each internal audit around that time and I'm actually getting ready to do this for the IT internal audit that's coming up in. A couple of weeks I'll send an e-mail to our staff and I'll let them know to the entire staff. I'll let them know that this internal audit is coming up. I'm seeking a volunteer to help.  Me with the internal audit. I will train you and I I lay out how much time is going to be involved and I and I wait and see what who volunteers to help me. And of course the people that help me have to be independent.

00:08:13 TRACY: Of the program that we're being audited. So today we're talking about the AASHTO accreditation program, internal audit. Obviously, Brian, you would not be part of the audit team for that because you are the director of the program. Also your quality analysts would not be part of that internal audit because they're not independent of the activities. And while we're talking about. Volunteers for internal audits. There seems to be a misconception at times that audits have to be performed just by the quality manager or someone in a management position, and that is not correct or actual, or 18 does not specify. That it specifies that the people performing internal audits must be independent of the activity being audited. That being said, I have had administrative people help me with internal audits. I have had laboratory assessors help me. I have had managers help me. It just really depends on who's available and who's interested. So that's part of the preparation. Once I determine who is going to help me with the audit, typically it's one or possibly two people.

00:09:16 TRACY: Well, then I sit down with them prior to the internal audit and go through a training presentation with them. I have a PowerPoint presentation that I created on internal auditing. What that's all about and what they're the expectations are with me as the lead auditor and the audit team. And who's doing what and how we're going to to go through all of this. I spent some time with them. If it's a laboratory assessor that's helping me doing an internal audit, the training required is minimum. Because they're already trained to be auditors, so they know about the auditing world. They know how to approach these types of things. However, if I'm training, for example, someone from our administrative staff who has never done an internal audit, I would spend even more time with those individuals. So that's part of the preparation. And then I just start gathering.  Documentation I I start figuring out what do I need to review. Of course we need to ensure that the requirements in ISO 9001 are met, so I do have a set.

00:10:20 TRACY: Of worksheets set. Up for the ISO 9001 requirements, which a laboratory could also do for AASHTO R18 requirements. Just check. This is what we call them or worksheet. And then I also look at previous internal audits and take a look at what policies and procedures in our own QMS that I have reviewed in the past two or three years, because auditing is an exercise in sampling. We want to get a good random sample of policies and procedures to review.  Over the years, it isn't necessary to look at everything. Every time, there is no time to do that. Nobody wants to do that kind of picking and choosing some things that maybe we haven't covered in the past couple of years. There's some things that I always investigate in training records and.  Equipment calibration records for the laboratory assessment program. So I I do a good bit of research into that to determine what we're actually going to be looking at. And then I start developing questions based on reviewing those documents. So if we have a requirement that says.

00:11:31 TRACY: Calipers have to be calibrated every three years. I maybe make a matrix for equipment to make sure that we are calibrating the calipers every three years and I go and talk to the person who's actually involved with. Making sure those activities are being done and I have a set of questions for that person. It just takes a good bit of time to do all of these things. Typically, an entire day, at least on the preparation, not including the training.

00:12:01 BRIAN: OK, so for the preparation that seemed like an awful lot of work and. A lot of time spent. How did you learn how to do this? And did you come up with these check sheets on your own, or did it the accreditation body that certifies our quality management system for 9001 conformance? Did they provide any of that information to you? How did how did you get here?

00:12:23 TRACY: They did not provide any of that information. I did come up with it on my own. I started out at Asho as a laboratory assessor, so I had the auditing experience from the get go. And we use worksheets similar to the worksheets that I use for internal audits. When we do laboratory assessments. So that's where I got the idea to create the worksheets, because I found that very helpful as a laboratory assessor. But no, I created the worksheets myself based on the ISO 9001 standard, and again, we're focusing on continual improvement. When I first started doing internal audits, which I think was back in 2006, I I wasn't doing half of these things back then. With each audit I'm learning. Hey, there are some things that I could be doing better. There's some questions I could be asking that I didn't ask this time. Often times the person helping me doing the audit will give me improvement ideas to improve the audit program. So it's it's just developed over the years and I know that next year when I do internal audits, they're going to be even better. And what I'm doing this year because we really do focus on always, you know, sharpening the.  Law and making sure that we're going in that continual improvement direction.

00:13:36 BRIAN: That you mentioned that. Your quality program gets audited every year as part of the internal audit process.  Do you learn more from being audited this way or by performing the audits of the other programs about improvement of the internal auditing process?

00:13:56 TRACY: I'd say it's about 5050 on that. I'm completely hands off on that audit of the quality program.  I'm not involved in any of the preparation creating of the questions. The worksheets are there for ICE 9001, but beyond that I'm not involved and there have been a lot of times where people have asked me. Pointed questions about things that I'm doing that I would have never thought to ask somebody, so I'm always impressed when others are doing the audits and their thought processes on creating these questions. Absolutely. I learn from being the oddity and the auditor.

00:14:34 KIM: Goes back to one of the points you made earlier about the value of having people external from the processes be removed because if you're auditing yourself, you're not going to ask those same questions. Just like you said, like you're like, oh, that's a great question. I would definitely have not asked myself if I was doing this and just auditing my own. Tasks then processes.

00:14:55 TRACY: Absolutely. And of course, when you're looking at your own stuff every day, you kind of become a little bit blind to things. You see it every day. Nothing's really jumping out at. The page that you maybe.  It's something that is not in conformance with requirements, so it's always great to have a fresh set of eyes on those things. And Kim, you, you got me thinking about something else that's a benefit of doing internal audits. It's really a win-win. It's a win for the organization because of the continual improvement opportunities that will hopefully be identified.  And the assurance that you're in conformance with requirements in Nash 2 or 18, I said 9001, your own QMS.  But it's also a win for the auditor because you're learning a lot more about what's going on in your organization that you may have never been aware of. I get that comment a lot from the audit team members. If we're doing, for example, a proficiency sample program audit, most of the people here.

00:15:56 TRACY: Rightly so, they're not working in that program. They have no idea what goes into.  Sample production for the proficiency sample program, for example. So they're learning a ton of information behind the scenes information that they weren't even aware of. So this really helps them grow as an employee to learn more about the organization. And the same can be said for me, absolutely.  Brian, I told you I've been here going on 34 years. I still don't know everything that we do here. I never will. I don't know everything Kim does. I don't know everything Brian does. We just have so much going on here. That's why I love doing audits, cause I'm always learn. Thinking more about what we're doing and the quality of what we're doing.

00:16:39 KIM: You know, I learned a bunch from just.  Reading the reports. That you sent out after the internal audits? Because I'm not often involved in the audit process of things like I'll be asked a question when you're interviewing people or something like that of like, hey, what's your process here? But. I'm like maybe 45 minutes Max and maybe involved in the process, but then reading the report I learned so much just as a member of the organization of like, OK, That's what's happening. And this is what's that. So like, I know a little bit about stuff, but the reports that you share are just so helpful for me from that prospect.

00:17:14 TRACY: Yeah. And I'm glad you brought that up, Kim, because I do share those reports with the entire staff. Once they're finalized. We're very transparent about our internal audits and and the reports and the findings. So, yeah, thank. For sharing that I'm glad somebody's somebody is reading them and Brian going back to the preparation question, I also as part of that set up interviews with different people on staff as part of the internal audit. And I think that's another thing that some of our customers are missing.  That talking to people.  It's pretty easy to just look at documents and check off some things on a on a check sheet, you're going to find out a lot more information if you actually talk to. People about these processes, policies and procedures talk to people about the record, keeping much of the internal audits involve talking to people. In fact, I would say the majority of it involves actually talking to people and then something might come up and I go and talk to another person about whatever it is that we're talking about.

00:18:14 TRACY: So I let people know before the audit even starts approximately what time I'm planning to talk to them. I typically schedule it and. Outlook calendar so people are aware we try to do these on site but we can do them through teams and then I just spend that as Kim mentioned 30 minutes, 45 minutes talking to people and and Brian for the accreditation program, internal audit I would set up two or three or four interviews with you at various times. Over those two or three days that we do the audit for that program. So I kind of keep you aware of what's going on with the internal audit. A little recap at the end of each day, but also sit down with you and do a deep dive on some things that you're responsible for or your team is responsible for.

00:18:59 KIM: Now we've talked about the.  AASHTO Accreditation program internal audit. I'd love to dive into that more because as we were chatting, I was looking at all of the other materials and things that we have available. About internal audits so I can put it in. The show notes. And we have like 6 other episodes about internal audits.  And how to do them and we have webinars and articles and I'm sorry if I'm jumping the gun. Brian, if you had a an order planned, but I would love to just like kind of dive in and really discuss what the value of the internal audit process and what came out of it specifically for you Brian, for the most recent one that happened and.

00:19:38 TRACY: Uh, I think Brian's going to answer this, but Brian, we did receive customer feedback about internal audits recently. We received a comment from a customer that they don't think that internal audits should be required because. Apparently not seeing the value or the benefit of doing them.

00:19:54 BRIAN: That's right. It that actually prompted. This concept of us getting into our internal audit to show what the value is, I think we've already mentioned a little bit of it and if the point wasn't driven home clearly enough about how important preparation is, I would rewind this about 10 minutes and listen to that again where Tracy gets into all the preparation. Preparation is so important in anything you do. Practice and preparation. Those two things lead to success most of the time, and even though I did prepare a plan for this interview, Kim, I'm fine deviating from it for the good of the.  Show well I.

00:20:33 KIM: Was just keeping an eye on the time, and I know Tracy is so knowledgeable. In internal audits and I know we get frequently asked questions, but I think we covered a lot of the basics in the that stuff already. So I would love to use this time. So, we're not having. Three Hour podcast episode because I. Can guarantee nobody wants that.

00:20:53 BRIAN: Yeah, you. You're absolutely right. So, let's end this one now and then we will have Part 2 next week.

00:21:00 KIM: All right. Stay tuned to Part 2 where we get really into the specifics about the recent AASHTO Accreditation Program, internal audit that happened earlier this year. So thanks Tracy for joining us for part one and stay tuned for Part 2.

00:21:13 TRACY: See you next.

[Theme music fades in.]   

00:21:15 ANNOUNCER: Thanks for listening to AASHTO re: source Q & A. If you'd like to be a guest or just submit a question, send us an email at podcast@aashtoresource.org or call Brian at 240-436-4820. For other news and related content, check out AASHTO re:source's social media accounts or go to aashtoresource.org.