AASHTO re:source Q&A Podcast Transcript
 Season 3, Episode 7: Internal Audits - Back to Basics

Released: June 21, 2022

Hosts: Brian Johnson, AASHTO Accreditation Program Manager and Kim Swanson, Communications Manager at AASHTO re:source

Guest: Tracy Barnhart, Quality Manager at AASHTO re:source

Note: Please reference AASHTO re:source and AASHTO Accreditation Program policies and procedures online for official guidance on this, and other topics.

Transcribed by Kim Swanson and Descript

 [Theme music fades in.]

[00:00:00] Announcer: Welcome to AASHTO re:source Q & A. We're taking time to discuss construction materials, testing, and inspection with people in the know. From exploring testing problems and solutions to laboratory best practices and quality management, we're covering topics important to you. 

[00:00:18] Kim: Welcome everyone. I'm Kim Swanson and in this episode of AASHTO re:source Q & A, we're going back to the basics of internal audits. In season three, episode three, we discussed who can perform an internal audit. And in this episode, we'll cover what an internal audit is, and some of the requirements. In these clips, you'll be hearing from myself, Brian Johnson and of course, AASHTO re:source's, Quality Manager, Tracy Barnhart.

[Audio Transition.] 

[00:00:41] Brian: All right.  So, Tracy, just the basic question. What is an internal audit? 

[00:00:46] Tracy: An internal audit is an independent and objective evaluation of criteria. The point of it is to ensure that you're in compliance with requirements and that you're actually implementing those requirements and activities. 

[00:01:04] Brian: Okay. So what does that look like when an internal audit is going on? What is that experience like for the people involved?

[00:01:12] Tracy: The experience involves a lot of review of records and documentation. It also involves, if you're doing it properly and effectively, it would involve observation of ongoing activities within your laboratory. As well as talking to people, interviewing them about their activities and their experience with the quality management system and things of that nature.

[Audio Transition.] 

[00:01:37] Brian: R18 talks about internal audits, in that they are a verification of laboratory's operations to make sure they continue to comply with policies and procedures and the requirements of R18. It also indicates that you have to indicate the frequency of the review, identify who conducts a review, which we talked about, and it has to address all elements of the quality management system.

[00:02:06] Brian: And I said the frequency has to be identified well already and also says it has to be done every [Tracy: 12 months.] 12 months. So, the guess work is removed, but you still have to indicate that in your own quality management system, and then you have to record the findings of the internal audits. So, let's get into the what, so all elements of that quality management system have to be audited.

[00:02:29] Brian: So, it sounds like there's two aspects here that right. There's a making sure that it's in conformance with R18 requirements and probably any other quality management system, like C1077 or E329, or any of those other standards, ISO 17025. If that's something that you're concerned about as well.

[00:02:49] Brian: So you've got that element, making sure your QMS, quality management system, is in conformance. And then if it's being implement in conformance with the way it's written. Should those be split out into two activities? Or can you do that at once? 

[00:03:03] Tracy: You can do that at once. Internal audits are basically three-dimensional, you're saying what you do with documented policies and procedures.

[00:03:13] Tracy: Making sure that those are in place, doing what you say. So you're watching activities, making sure that the activities are being performed in accordance with the requirements. And also when you're reviewing the records, you want to reflect back on what the documented procedures are and making sure the records line up to what you're actually seeing, and then proving it with objective evidence of what is actually happening as a result of that activity, whether it's an equipment calibration record or a technician training and evaluation record or whatever that may be. And I've actually forgotten your question now, Brian, after I, after I explained that.

[00:03:53] Brian: That's okay, that was a really long question. But my question is when you're evaluating the, that your quality management system for conformance with standards, and then you're evaluating implementation of your quality management. Is it best to have those as two separate activities? Or can you do that all at once or what's the best process?

[00:04:10] Tracy: You can of course do everything all at once. For me as an internal auditor, I find it a lot easier and it's a lot more effective for me if I do, what's called a desktop review of the QMS documentation before I even do an internal audit.  So, I'll go through randomly sample a lot of the procedures that we have in our own QMS as part of my preparation for an internal audit.

[00:04:36] Tracy: So, I'm doing a lot of that prep work ahead of time. So, then I'm populating, my worksheets with questions. I want to ask about specific things when I'm actually conducting the audit. I don't like to review documents at the same time as the internal audit, because there are so many for me to review. We have, of course we don't have to be in compliance with AASHTO R18, but we have to be in compliance with ISO 9001, ISO 17043 for our proficiency sample program and our own quality management system.

[00:05:07] Tracy: Which literally has hundreds of policies and procedures in it. I don't want to have to do all of that during the audit. So the prep step really helps me streamline that audit. And then I can focus more during the audit on talking to people and reviewing records because I've already reviewed the documented policies and procedures before I even got started.

[00:05:30] Brian: Yeah, that makes a lot of sense. I think that's the preferable process to really capture everything and to do this correctly. If you already have done the audit part of your QMS to make sure it's in conformance, you're going to get familiar with all those policies and procedures. Maybe even some that you forgot about. And then you may, you may realize that there are ones that you should. That you don't have. 

[Audio Transition.] 

[00:05:55] Brian: Now, now this one's got an interesting can, can internal audit be a sample of tests only, or does it need to be every record for every accredited test that gets reviewed during the internal audit? 

[00:06:06] Tracy: We get that question a lot. Auditing is an exercise in sampling. We are not looking for laboratories to look at every single test report or test record, or every test, every equipment record. Definitely just make sure that you're doing some random sampling to get a good cross-section of things that you're auditing. 

[Audio Transition.]

[00:06:27] Kim: What makes an internal audit the most effective it can be?

[00:06:30] Tracy: Effective internal audit for me is a very thorough audit, making sure that you're hitting all of the key points of your laboratory's quality management system, all of the key requirements and AASHTO R18. Doing a random sampling of records to make sure that you're kind of hitting all the high points on equipment records, technician training, and evaluation records, things of that nature.

[00:06:57] Tracy: Also, to me an effective internal audit does result in identification of improvement opportunities. That you're really taking the time looking at things and thinking of ways that you could do them better in the future. 

[00:07:10] Kim: Brian, do you have anything to add that makes it effective for the accreditation side of what makes it effective for laboratories?

[00:07:17] Brian: It's effective. If you get something positive out of it. The most ineffective ones I've ever seen are the ones that say everything is fine. I always, when I see that, I question whether they even performed it because even a laboratory that is really doing an amazing job, keeping up with things, will identify an improvement opportunity, or will find something that is not ideal.

[00:07:40] Brian: And I can tell from looking at an internal audit record how seriously that laboratories taking the process, you can tell right away. And there are times where this actually happened yesterday. We were having a meeting and talking about looking at an internal audit record, and I said to the quality analyst, I would send it back to them and say, now perform one. Because there was an alleged record that was submitted, but there is no way that it was really conducted. I mean, the laboratory had a fair amount of nonconformities in the report that would have been obviously identified if an internal audit was conducted because typically an internal audit is going to show way more than an external audit.

[00:08:25] Brian: It would because the people actually know the people that are doing it. They typically open up to them more about what's going on. Like, I'll give you an example. Let's say somebody was uncomfortable with a process they were supposed to carry out and the internal auditors, says, okay. You know, Kim you're supposed to be, doing this one activity every six months and Kim looks at the internal auditor says, I don't even know what that is. Well, they're probably not going to say that to the external auditor. But the internal auditor is going to hear it. And Kim will say, listen, I don't know what that is. I don't know why I'm listed for that, either we need to stop doing that or somebody else needs to be assigned it, or I need to be trained on how to do that.

[00:09:10] Brian: And then all of a sudden you say, okay, well, let's get, let's figure out what we can do to fix this and do it the right way. And it kind of opens up possibilities for how to do things the right way that that's the benefit of the internal audit. Is having that kind of candor associated with the audit process and that leads to more effective corrective action.

[00:09:29] Brian: So that's what I would say makes it truly effective. 

[00:09:32] Tracy: I completely agree, Brian. And to touch on that a little bit more, as far as Kim's question about effectiveness, you mentioned this briefly. If you're finding that AASHTO re:source or CCRL, they come in for a third party, external audit, and they're finding all kinds of things that are not in conformance in the quality management system or the R 18 requirements.

[00:09:52] Tracy: But internally you did an audit leading up to that and nothing was identified. Everything was in conformance. I think that is a reflection that the internal audit probably wasn't as effective as it could have been or should have been.  So, I would reevaluate the internal audit process, making sure that there was ample preparation involved, making sure that there were random samples of activities that were evaluated.

[00:10:19] Tracy: The preparation part for me, that can take a day or two before I even start doing an internal audit. You don't want to just wake up in the morning and say, oh, I'm going to do an internal audit today and with no preparation whatsoever. And I've seen records like that as well, Brian, where there are no findings at all and then I've asked for, well, can you show me the check sheets that you've used for the internal audit or any sort of evidence that this audit has been conducted? And they're not able to provide me anything except a one sheet of paper that says everything was in compliance. So again, taking a look at it, reevaluating the process, even the procedure for conducting internal audits would be beneficial to lead to an effective audit.

[00:11:05] Tracy: There really shouldn't be any surprises when you have third-party audits. If you're doing a thorough and effective internal audit, you shouldn't be really surprised by anything. 

[Audio Transition.] 

[00:11:14] Kim: The actual policy is on our website and the document is the AASHTO Accreditation Policy and Guidance on Internal Audits and Management Review for New Laboratory Facilities. So that's the name of the document. If you're looking for it.

[00:11:27] Brian: The policy is that laboratories have six months from the time that they get accredited to send in their first internal audit and their first management review. That way we can see that the policies and procedures that have been implemented and we're seeing how they actually do.

[00:11:44] Brian: And this is in line with a continual improvement process that we're trying to espouse in our program and that we really want all the laboratories that are AASHTO Accredited to adopt. And most of them have some are still coming along and a lot of times new laboratories are just kind of learning about those concepts, for the first time when they go through our process.

[00:12:07] Brian: So, it's been really working out well, getting this cause they can do a more meaningful internal audit. 

[Audio Transition.]

[00:12:16] Brian: One thing I want to make sure that laboratories understand is that we are not going to be following up with them on their findings, on their internal audit. I don't know if that's a problem for people or if that's a worry for people.

[00:12:28] Brian: But we would encourage them to write findings on their internal audit because if they write those findings on the internal audit and they correct them, that means they're not going to show up on the external audit because there's nothing to find it's already done correctly. And that's the, that's the true benefit of the internal audit.

[00:12:44] Brian: And we see it when we go through it on our end because Tracy Barnhart, our Quality Manager, her internal audits are so great and they're so detailed and they're so thorough and she finds stuff that an external audit would never find. But because of her efforts, when we do have an external audit for ISO 9001 conformance, it's usually an extremely clean report because we've taken care of all these things in the auditor can actually see our internal audits.

[00:13:14] Brian: They can see our corrective action tracking on our resolutions, and it helps us. Well, it helps us fundamentally, but it also helps us with the external audits. 

[00:13:24] Kim: Yeah.  So, there's no real surprises during the external audits because we've covered everything, or I've already resolved most of it for the internal audits. There shouldn't be right. 

[00:13:33] Brian: That's right. That's the whole point. Right. And I think that's something people don't necessarily embrace. Cause they, you know, some laboratories that are new to quality management, they may not see the value. You know, you do hear from time to time, people say, oh, these are just administrative hoops that we have to jump through.

[00:13:49] Brian: But if you do it right, they're not, and it'll be really apparent to you that they are not, and they're actually making your organization better. It's better for your employees because people will know what to do and they'll know where to find information if they don't know what to do at certain times, and you can get improvements to your work environment through that process. Improvements to your clients, which is very important for your business to continue and, and be profitable and grow. So, there's just, I mean, endless benefits to the internal audit process. 

[Audio Transition.]

[00:14:24] Brian: The positives that come out of the internal audits.  So, you don't only talk about improvement opportunities and nonconformities, you also talk about things that are going especially well, uh, in efforts that should be pointed out and congratulate people, congratulated. This step really makes the internal audit special because the external auditor may not realize all the efforts going in, but certainly the internal auditor should, do you want to comment on that?

[00:14:51] Tracy: When you're doing an internal audit and something really jumps out at you as like, wow, this is great. They're doing a really great job with this. Whether its record keeping or the way they're training people or whatever, it might be.  Even customer feedback. If the customer feedback has been excellent.  I think it's great to point those things out, because we don't want to just be focusing on negative things during an internal audit.

[00:15:13] Tracy: The idea of an internal audit is improvement, but why not also give kudos for things where warranted. 

[00:15:20] Brian: I agree. And I'd say one of the things that is often missed when, you know, when you're going through this process, lot of the laboratories in our program are performing these internal audits because they have to. I would say if you are, if you are thinking that right now that you do them out of necessity, I would encourage you to think about some other aspects of the internal audit that may be a more worthwhile to you if you feel like you're just kind of going through the motions. Uh, your customer focus at your laboratory. Are you serving the needs of your customer the best way possible? Are there some things you can do better? Communication. Are you communicating internally and externally well? Safety, safety is a big priority. That's not addressed in R18, but you better believe it's important. 

[Audio Transition.]

[00:16:09] Kim:You’ve just heard clips from Season 1, Episode 39, FAQ - Policy for New Lab Internal Audits and Management Reviews - Season 2, Episode 24 – Taken from Tech Ex: Internal Audits, The Focus on Improvements, and Season 2, Episode 12, Conducting Effective Internal Audits.  I encourage you to listen to those complete episodes for a full discussion on this topic. 

[00:16:33] Announcer: Thanks for listening to AASHTO re:source Q & A. If you'd like to be a guest or just submit a question, send us an email at podcast@aashtoresource.org or call Brian at 240-436-4820. For other news and related content. Check out AASHTO re:source's, Twitter feed, or go to AASHTOresource.org.