AASHTO re:source Q&A Podcast Transcript

Season 3, Episode 5: World Accreditation Day

Recorded: May 19, 2022
Released: June 7, 2022

Hosts: Brian Johnson, AASHTO Accreditation Program Manager and Kim Swanson, Communications Manager at AASHTO re:source

Guest: Bob Lutz, AASHTO re:source Manager

Note: Please reference AASHTO re:source and AASHTO Accreditation Program policies and procedures online for official guidance on this, and other topics.

Transcribed by Kim Swanson and MS Teams

 [Theme music fades in.]

[00:00:00] Announcer: Welcome to AASHTO resource Q & A.  We’re taking time to discuss construction materials testing and inspection with people in the know. From exploring testing problems and solutions to laboratory best practices and quality management. We’re covering topics important to you. Now, here’s our host, Brian Johnson.

[Theme music fades out.]

[00:00:20] Brian: Welcome to AASHTO resource Q&A. I am Brian Johnson.

[00:00:24] Kim: And I'm Kim Swanson and we are celebrating World Accreditation Day that is June 9th. And how are we celebrating Brian?

[00:00:32] Brian: Well, the way everyone does.  They have Bob Lutz, the AASHTO re:source Manager, over to talk about accreditation. Now, we are lucky enough to be able to do that in person. I know many of you out there wish you could do the same, so welcome to the podcast, Bob.

[00:00:46] Bob: That's quite an introduction. Thank you, Brian. And Kim, I'll do my best to make this a party.

[00:00:49] Brian: That's good because you are going to be on the hot seat for a little while. I have some questions for you, so let's get started.  Bob, the World Accreditation Day is what we're talking about today and just to set the stage for it. Let's talk about accreditation in general. If you're talking to somebody who maybe doesn't understand what that term means, can you give me kind of a layman's definition of accreditation?

[00:01:13] Bob: Sure.  Accreditation is a...First of all, it's a process and in layman's terms, I would say it's a process that determines that some organization is good at doing what they're doing. More specifically, though it means that they’re competent to perform specific tasks. We talked about accreditation a lot in our industry, but there Are lots of organizations or types of organizations that seek accreditation and get accreditation. Learning institutions. A lot of universities are accredited.  Building departments.  Even elementary schools, not just colleges.  Even police, some police, and fire institutions as well.  So, there are accreditation, believe it or not, even though we don't hear about it much, it really is all around us in all of the things that we do and the services that we rely on. So that's that's it in a nutshell.

[00:02:26] Brian: I like that definition and I think it's good to know that when you talk about accreditation that there's so many different fields that get covered and in different aspects of our life are touched by the accreditation process.

[00:02:14] Brian: Now you talked about accreditation being very specific to certain activities conducted by entities, right? What are the limitations of accreditation though, because I think sometimes people think of accredited and they have this idea that, oh, that means it's good or that means this or that means that it's like a seal of approval or something like that. But what are the limitations of accreditation?

[00:02:52] Bob: Well, there can be many.  You know, going back to talking about universities, for example, a lot of times it's not that the whole university and everything that they do is accredited or is going to be good or meet a certain need. Oftentimes that accreditation applies only to particular programs within the universities or particular fields of study, particular curriculum. And the same is true in, in our industry as well. When we're talking about testing materials and then inspecting materials.  The accreditation applies to very specific activities, certain tests, or certain inspections. There are a lot of companies that we deal with that do many, many things, but their accreditation is limited to only some of the things that they do.  So, it's very specific in that regard.  

[00:03:49] Bob: And I imagine that you're probably going to ask a question coming up too, but I won't get into it now, but there are limitations in the fact that it's not a perfect process as well. It's not a guarantee that everything is always going to be good all the time.

 [00:04:04] Brian: Like you said, it's not just the entity being accredited, it's the accreditation provider and all the other underlying services.  So good point there, but now with that being said, there are limitations. There are imperfections…with all of that being known. Why might somebody want to become accredited? Or why might it agency, a federal agency or state or local, want to require accredited entities for whatever it is they're doing?

[00:04:37] Bob: As I said, there are no guarantees and it's not a perfect process, but it is a good process.  And it is a sign that an organization has a working quality management system, and it has some competence staff, and that they're keeping up with equipment calibrations and things like that.  And that they have a good understanding of testing and inspection.  So, it's a good starting place. Again, it's not a perfect process.  But it is something that people can rely on to know that a certain bar has been met.

[00:05:20] Bob:  It doesn't mean that the bars been set at the very top, but a certain bar has been set and that organization, that laboratory, has met all of the requirements to meet that bar.

[00:05:33] Kim: As you were talking, I think there's an important distinction and I think we covered it, but just in case, that accreditation applies to an organization.  So, in our case it laboratories, we don't accredit people or technicians and that's not what accreditation is. That's more of a certification, is that correct? 

[00:05:50] Bob: That's one of my pet peeves. If I'm going to be honest.  We hear that all the time. People talk about labs being certified.  Certification is similar, but it typically applies to people or products. You know in our industry, technicians can be certified. Inspectors can be certified. But accreditation is a process that applies to an organization and not to just a specific person or product.

[00:06:22] Brian: Yeah, that is. And. And I do expect Kim to know all this stuff because she does enough communication-related tasks for the Accreditation Program and for AASHTO re:source, including writing presentations about what we do, where I would expect her to fully understand these concepts. So I don't give you any extra credit for that, Kim.

[00:06:41] Kim: But I do know that is a sticking point that people often get that confused. I see the people getting confused about that, and I know it is a pet peeve of Bob’s when people say that, and honestly, it is for me too. Like when people say oh, we're AASHTO certified, it's like that's no, you're not. That's not the thing. So at least regards in regards to laboratory testing. So yeah, just like that's not it, but anyway.

[00:07:06] Brian: Yeah, that's right. That is a risk that I, I see out there in the way people communicate their accreditation to, especially a supplier of materials.  You know, they may have an accredited laboratory, but somebody might just see that that supplier is accredited and not understand what that means.  So, they might think that all the materials coming out of there have some sort of like AASHTO approval and therefore it can be used for this, that and the other thing…without checking them to make sure that they are what they are supposed to be. Bob, do you have any comments on that misuse of or proper use of accreditation?  Seal or statements about accreditation.

[00:07:53] Bob: Yes, I do. And I thought you were actually going to talk about location as well.  Accreditation is limited to particular sites and locations. One of the things that we see a lot is that an organization may have one lab accredited in one location, and either they make the assumption, or others make the assumption, that all of their locations are accredited. Going back to the educational example let's say the University of Maryland is accredited. It might be in College Park, MD, but they have many campuses and so maybe not all of their campuses or all of their programs are accredited.  So, people need to remember that that there are limitations that way. It's generally it is very site-specific and only applies to that location.

[00:08:49] Bob: Regarding the use of accreditation, logos and symbols and things too. Again, it you mentioned limitations. It's limited to again very specific tasks and sometimes a lab may present or produce a report that has a lot of information on it. Maybe they ran 20 different tests, but maybe they're only accredited for 10 of those tests.  Because, again, its accreditation is very specific to that.  That would be a misuse of the AASHTO Accreditation logo or whoever's logo.  A lot of times I don't think labs do that intentionally. It's just a misunderstanding of exactly what the limitations of accreditation are.

[00:09:33] Brian: Yeah, that site-specific issue kind of reminds me of another topic that comes up sometimes.   When somebody goes to get accredited for the first time and maybe they are doing it because they're required to, and they go in with a feeling of dread.  Thinking about how unfair it is that they need to go through this and in thinking about all of these uh, issues like scale, for example, that's one of the things I hear about sometimes is, you know, comparing themselves to a large company that has multiple locations and how it must be easier for them to get it because they may be accredited at the one place and have it filtered down to these other locations, but.  That doesn't really work that way. It is site-specific.  So really it's a more level playing field than some people might think. Now. Granted there is a scale issue because you know a big company might have the resources to have a separate corporate quality management system or something like that. It helps make things easier. But really the challenge is take place in implementation at the site, and that's really when somebody's getting assessed, or audited, or whatever term you use.  Inspected if you're CCRL.  Then there's an evaluation takes place at that site, and everything needs to be done the same way at all of the different accredited locations. 

[00:10:54] Brian: So, a very good point to make there, Bob. One other thing I want to talk about, while we're talking about the model that we use that being site-specific and some of the other aspects of it, let's talk about the model of accreditation. What's the typical strategy for an accreditation body?  And open it up to whatever you know, global processes work, you know whether it be. Uh, a university or a manufacturing plant or calibration agency or testing lab?  What's typical out there?

[00:11:27] Bob: Well, it certainly is different than the model that we have at the AASHTO Accreditation Program, not totally different.  But it is different it focuses, very much on an organization’s quality management system.  Do they have all the policies and procedures for what they do in their business? And of course, in our business, it would be you know, how do they receive and identify samples?  How do they review data? How do they calibrate equipment?  How do they train their people? How do they deal with measurement uncertainty? So, all of those documents, all those policies, procedures, it's very heavy on, OK do they have it or are they using these policies and procedures? Are they using them correctly? Do they understand them?  There's also a component of how competent are the staff to perform those specific tasks, whether it's testing or inspecting or calibration?

[00:12:30] Bob: But our model is a little bit different, but that's the international model. It's a sampling of the technical things in an organization does, and then a really deep dive on the quality management system to see that they have a good system in place and that they're meeting those requirements.  That is the typical model worldwide for just about any kind of organization, any kind of accreditation.

[00:12:54] Brian: Yeah.  So, while we're talking about that model, how can an agency or accreditation body that focuses so much on the quality management system implementation…and maybe covers a lot of different types of businesses that are engaged in different activities, how can they be expected to be proficient or competent in some of the technical aspects of what those agencies are doing day to day?

[00:13:22] Bob: Well, they do that through technical experts.  I don't know that this is a difference really, but in accreditation worldwide, typically there is a lead auditor and there's a subject matter expert or a technical expert.  They hire people who are technical experts in those industries to be part of the audit team, to be part of the accreditation team.  So, they rely on the expertise that is out there in the industry.  It would be impossible probably for them to keep all of that expertise in-house. But they do a good job of finding the expertise, finding the people who are experts in metrology, for example, to serve as the technical lead on an assessment.  So, they really rely heavily on resources that are outside of their organization.

[00:14:15] Brian: Yeah, and that seems like that could work just fine.  You could have somebody who's just a quality management expert that's performing the audits and then get your technical expertise from someone else that is trusted. So yeah, that sounds good. Now, what do we do that's different than that?

[00:14:32] Bob: Well, we…we maintain our expertise in-house, that's number one. Our assessors have the quality management system experience and the technical expertise, and this is one area where our model is much different, so we hire people and make them into technical experts, subject matter experts.  We have a really long and rigorous training program. Now keep in mind that the scope of the AASHTO Accreditation Program isn't as wide as most accreditation bodies. We're focused very, very much on testing materials and inspecting materials, so we don't assess, and we don't accredit calibration labs or any other types of labs.  That is another difference is that we are very, very specific in what we do.  So, we hire people to become subject matter experts in the fields of testing materials and inspecting materials and quality management systems, whether it's AASHTO R 18 or all of the ASTM standards or ISO/IEC 17025.  

[00:15:45] Bob: We maintain that expertise in-house. Now we do associate with industry a lot and collaborate with industry a lot and learn things that way. But we maintain it all in-house.  Another way though is that and I mentioned earlier, that accreditation is a process and it truly is a process. There's a lot to it.  And a couple ways that our process is different is the I mentioned how we train the assessor as well the assessors when they go out in the field or these days a remote assessment.  They are observing that laboratory perform all of the testing and inspecting that that lab wants to be accredited for. So in the international model, they might look at a sampling of what the lab does. We look at everything that they want to be accredited for. So if they want to be accredited for 50 tests.  We're going to watch them perform those 50 tests and make sure that they have demonstrated competency in all 50 of those tests.   And that's not done in the international model, but that is our approach. 

[00:15:57] Bob: Another way that it's different is in proficiency sample testing. Again, in other industries and other models, there is less proficiency testing.  They might have a scheme that says you have to… let's say, perform PT, proficiency testing, on soils maybe every four years. Well, for us, it's every year.  So, if a lab that's in the AASHTO Accreditation Program, if they're accredited for soils and aggregate and concrete, for example, they're getting several samples every single year and must test them and get acceptable results and ratings on those to maintain accreditation. So again, the process is... It's not as wide, but it's much, much deeper.

[00:17:49] Kim: As you were talking about that, Bob, about how our assessment staff…they're watching the technicians run every test that they want to be accredited for, that actually did get me thinking of like, so I can kind of see how people think, maybe that the technician that they're watching is then getting certified because we are watching a technician run it. But that's not what's happening. Can you explain that a little bit more and clarify that a little bit?

[00:18:16] Brian: Yeah. So, Kim, I'll jump in and handle that one for Bob. The you're right that what happens when we perform this assessment is we do watch one technician run each test, not necessarily the same technician for all the tests though. So we're not necessarily certifying a technician or evaluating or assessing the laboratories conformance to requirements solely based on the work of 1 technician. But that is, just like the other model is limited by their doing a sampling of the procedures.  The observation of those procedures, we're doing a sampling of the technicians performing those procedures. So we have some assurance that the laboratory has systems in place to train and evaluate the competency of all their technicians. But we are in effect sampling their effectiveness and their quality management system and the competency of their laboratory-based on one person running one test.  For each test that it is accredited.

[00:19:18] Kim:  I don't think I've ever framed it quite that way. I knew that's what it was. But that was an interesting framing.  So, thank you for that answer.

[00:19:24] Bob: But Brian makes a good point.  It is a sampling. I don't want to imply, that although I did say that we dive deep and we do, but we don't go all the way to the bottom if you know what I mean because.  We would have to be there for a month or more to really observe everything and everyone. And we don't do that. We try really hard to balance and find the right balance between spending enough time and not enough time and getting some people say we're there to get a snapshot of what's going on and we take a sampling and we then infer that if this person's doing a good job and that person's doing a good job and these procedures look right and these records look good, that the organization is doing...that on a wider scale and hopefully all the time as well.

[00:20:15] Brian: Yeah. Now there's another aspect of the way we do things. It's a little different than the international model, I think, and that's in the equipment checks that we perform in the field.  And that has been kind of a little outside of our, uh thoughts...the last couple years as we've been performing a lot of these remote assessments, even though we did incorporate some methodology for evaluating the laboratories, ability to check their own equipment, which they do regularly as well.  But do you want to talk about that? Why are we checking equipment when we're in the field when we perform the assessments, Bob?

[00:20:50] Bob: We have learned over the years, one of the big activities that labs are supposed to be doing on a routine basis is checking their equipment, measuring certain items and standardizing items and calibrating items.  And we have found that that's either not always getting done when it should get done, or it's not getting done the way in which it should get done. So for years and as I think, as long as we've ever done this since the 60s, when we go out to do an assessment, we have measuring equipment that we take with us and we're there to verify, number one, that the equipment does meet the specs listed in the standards, you know.  It's the right dimensions or the right tolerances, whatever.  But also to see is that lab keeping up with doing those activities that they're supposed to be doing when we are not there. And it's a really good indication of how well they're quality management system is working because that's the guide that says this is how you should be doing these things and how often and what you should be doing with a piece of equipment when it's out of spec.  

[00:22:06] Bob: If we go out there and measure a bunch of items and say that, you know, this isn't quite right and this isn't quite right, and then leads to the question of what are they really doing these things when they're supposed to be doing them? Because, ideally when we go in there, we shouldn't find too many pieces of equipment out of spec.  That's been a big part of what we do and another difference. I will say that I think that we've seen some improvement by the labs and doing this over the years. So that's been a good thing and we talk about it a lot more as well.  As a matter of fact, it's my understanding from some of the feedback that we've gotten that from the technical exchange.  That that, that's been one of the most popular sessions that we do and people want to see more of that. They want us to teach them how we check equipment and how you do this and how you do that. So that's a good sign. And I think there are really has been some improvement in what the labs are doing in this regard.

[00:23:06] Brian: Yeah, I think so too. And and I know that sometimes people feel like it's kind of redundant to make the laboratory check the equipment regularly and then we do it as well. But I think in some areas we know when you're talking about safety when you're talking about quality, redundancy isn't necessarily bad thing all the time, right? Because then you have some assurance that things are what they are supposed to be. Uh, Kim?

[00:23:30] Kim: I didn't think I realized that other accreditation bodies don't check the equipment. That is kind of a big deal for the reasons that you mentioned that like that why we do it and why it's important for we do it. I just don't think I've realized that that is something that sets us apart.

[00:23:45] Bob:  It's not and just to add on one other thing, going back to a question that you asked earlier, Kim, it same thing applies to the equipment that we check. It's a sampling where we don't measure everything that they have. Again, it would take too much time, but if we do a good job of sampling, then those results will be a good indication of the overall results.

[00:24:05] Brian:  Yeah. So we talked about a lot of these differences. So with that being the case that we do things a little differently than the typical international accreditation body model, why do we do things? Why is there a difference? What's our problem? Why are we doing this?

[00:24:19]  Bob: [Laughing.]  This model that we have was developed with all the state departments of Transportation and the Federal Highway Administration, to meet their needs specifically.  The types and the number of specifiers that use our program has gone up dramatically since then. But those were the people that developed the program initially, and they knew that that was going to be an issue that that was going to be a problem. They knew that. Those things weren't getting done the way they should be getting done.  They had a lot of insight and a lot of experience that went into the development of the program and the decisions on what type of model we were going to create. And so they they knew that that was something that they wanted us to be doing to give them, and here's the word, Brian, to give them confidence in the results of the testing and in the results of the accreditation.

[00:25:25]  Brian:  Yeah, that's right. And I would say that this has come up several times throughout the years. This model and whether the DOTs still support this model and we're constantly asking for feedback about how we do things and you know how our processes are working.  And time and time again we get reinforcement of the value of accreditation and the value of the way the AASHTO Accreditation Program operates.  When we ask those questions, the kind of responses we get makes me feel good that we're on the right track. But even with that. You know, as time goes on, things change. People's priorities change. Do you think, Bob, that at some point you will see the AASHTO Accreditation Program gravitate toward adopting the more international model than doing the things the way we do them right now?

[00:26:25]  Bob: That's a tough question.  I think not. I mean we have made some changes and and in some ways we have gravitated more towards the international model. For example, offering accreditation for ISO/IEC 17025. That's one example, but our owners, our oversight group, the Departments of Transportation.  Are I think, very adamant and insistent about the value that that brings to an assessment and that that brings to accreditation.  Being, you know, watching all the tests and checking equipment, all those things, all the proficiency sample testing, all those things that make us different.  They see a lot of value in that and since those are our Members and the people that we primarily serve, I don't see us totally adopting a different model. I will say, though, that you know there have been improvements in the industry and in the the services that support our industry. For example, back in 1988 when the accreditation program started. I don't know if accredited calibration labs existed.  I don't even know if that if there were any at at that time and that today there are many and there are many that are accredited. And they do a lot of that work now instead of the labs doing a lot of that equipment checking and equipment calibration. And I think that's been a big improvement.  Therefore, that lessens the burden on us a little bit to check equipment as maybe as much as we used to because the labs aren't doing it. There is a whole competent and accredited industry that is responsible now for doing those things. So that's been a good change. And there are there's also a bunch of. Technician certification programs as well. So the whole industry has evolved and because of that, we have made changes to our model and and tweaked it, but whether we will ever…Go a totally different route and adopt the international model by I don't think so.

[00:28:31]  Brian:  I'm glad you brought up the accredited calibration providers because it is interesting for us as an accreditation body that is specifically dedicated to working with the testing laboratories that hire calibration agencies that are accredited. Seeing the difference between the accredited calibration providers and the non-accredited calibration providers can be a pretty stark contrast at times. Now, that's not to say that all non-accredited calibration providers are not on the same level with the accredited ones, but you can see the standardization of records. You can see the detail that goes into the accredited providers work compared to the typical non accredited calibration provider and I hope, and I trust that the specifiers see the difference when they're working with laboratories as well. The accredited versus the non-accredited. All the policies and procedures in place, all that you know, like I said, standardized reporting. All these things that are important when you're getting technical information should be shining through when they go with the accredited testing laboratory. Now, what about accrediting the accreditation bodies? Is that a thing?

[00:29:46]  Bob: It is a thing.  There are more bodies, more organizations that exist to examine, evaluate and give a seal of approval on the competency of accreditation bodies as well.  Like how do we know that we're running a good accreditation program? How do we know what we are doing? Of course I don't. I'm not sure where it ends. Then you know, as far as I know, there aren't. There's not another level of bodies that evaluate those that evaluate accreditation, bodies that evaluate conformity assessment bodies and so on. But there is an internationally, the. Group is ILAC the International Laboratory Accreditation Cooperation.  I believe I should know this. I believe that's it.  So you will see accreditation bodies across the world that have recognition for the standards that ILAC sets on an international level and that has to do with things like the competency of their assessors and of course, their quality management system as well. Again, how are they competent to do what they're doing and how does their accreditation process work? Do they have appeals process and how does that work, and do they have integrity and good judgment and those things? So yes, there is another level and I bet you're going to have a follow up question.

[00:31:15]  Brian:  You are correct. I do have follow questions.  So, you know the AASHTO Accreditation Program, I know from working here for so long that we have a lot of those things like we have we have a quality management system, we get audited regularly. We have all of these measures that you would expect are responsible accreditation body to have in place. But we don't maintain an accreditation. Our quality management system is registered to be in conformance with 9001 ISO 9001, but we are not an accredited accreditation body. Why is that? And do you see that ever changing?

[00:31:50]  Bob:  That's kind of related to the question that you asked a minute ago about what we ever adopt the international model. Well, let me first of all say why that is.  Because we don't fit into that international model, to get the international recognition may require us to change the way we do things, and I can tell you with a lot of certainty that the state DOTs and the Federal Highway Administration, who again who built this program and rely on it so much, want us to operate a program, the way we do.   A certain way that is different.  And we have a ton of oversight from them.  And they are comfortable with the model that we have and that's the way they want it to be. And now there is a Federal Highway Administration document that basically says…for any of the highway work that involves federal dollars that the lab must be accredited by AASHTO or a comparable program. What that means to me anyhow is that in terms of the eyes of FHWA is that.

[00:33:06]  Bob: We are the standard to which others are measured and compared. But will we ever change? Possibly. I mean, I think there is a way that we could maintain the things in our model that we like, but still be able to get international approval and recognition. There is a standard of course which you are familiar with and ISO standard ISO/IEC 17011 for accreditation bodies specifically and talks about some of those things that I mentioned. I think there's probably a way if we need to and I in my very fuzzy crystal ball looking down the road, I think there perhaps will come and day when we want to have that international model recognition.  I think we can have the best of both worlds and maintain the things that make us unique and provide a lot of value to our specifiers. But at the same time, meet the requirements for that international recognition.

[00:34:13]   Brian:  I appreciate that answer. And one thing that I would look forward to if we ever did…Look toward accreditation and getting involved in the international stage a little bit is educating the rest of the world on the way we do things and showing them the value and showing them that it can be done because I know I've heard from others that work in these kind of fields that the way we do things is just it's not possible. It takes too much time. It's too comprehensive to cover, but it can be done and we know how. So I think it would be very interesting to participate in those kind of conversations with people who espouse the other model. Anyway, I could go on and on, but we are out of time. Bob Lutz, thank you so much for coming out of the podcast, today talking about accreditation, handling all of our questions.

[00:35:06]  Bob: It was a pleasure. Thank you both. Happy June 9th World Accreditation Day.

[Theme music fades in.]

[00:35:12]  Announcer: Thanks for listening to AASHTO re:source Q & A.  If you would like to be a guest or just submit a question, send us an email at podcast@aashtoresource.org or call Brian at 240-436-4820.  For other related news and content, check out AASHTO re:source's Twitter feed or go to aashtoresource.org. 

[Theme music fades out.]