AASHTO re:source Q&A Podcast Transcript

Season 1 Episode 10: Management Reviews

Released: September 22. 2020 

Host: Brian Jonson, AASHTO Accreditation Program Manager, AASHTO re:source 

Guests: Tracy Barnhart, Quality Manager, AASHTO re:source; Bob Lutz, Manager, AASHTO re:source; Steve Lenker, Director, Construction Materials Reference Laboratories (CMRL)

Note: Please reference AASHTO re:source and AASHTO Accreditation Program policies and procedures online for official guidance on this, and other topics. 

Transcription is auto-generated. 

[Theme music fades in.] 

00:00:02 ANNOUNCER: Welcome to AASHTO resource Q & A. We're taking time to discuss construction materials testing and inspection with people in the know.  From exploring testing problems and solutions to laboratory best practices and quality management. We're covering topics important to you. Now here’s our host, Brian Johnson.

00:00:21 BRIAN: Today on the podcast I have some special guests with me. We've got our quality manager, Tracy Barnhart.

00:00:27 TRACY: Hello.

00:00:30 BRIAN: We've got our AASHTO resource manager, Bob Lutz.

00:00:34 BOB: Hi, Brian.

00:00:35 BRIAN: And we've got our CMRL manager, Steve Lenker.

00:00:40 STEVE: Hello, Brian.

00:00:41 BRIAN: So, what activity brings such heavy hitters in one room? As Bob shakes his head, it would be the management review. Of course, we just had our management review over the last two days for the AASHTO Accreditation Program. And I thought, wow, this is a great opportunity to talk about. One of the most confusing things for our laboratories, which is how to conduct a management review effectively. So, I've brought you guys together to explain what goes on from your perspective. And I hope that it's useful to people. [Laughing] So first, let's talk about the process I mentioned. This is the AASHTO Accreditation Program management review that we just had. Why are we splitting that out from the AASHTO Resource on site assessment or assessment program and the AASHTO Resource Proficiency Sample Program, Tracy?

00:01:37 TRACY: Brian, that gives us an opportunity to focus exclusively on one program at a time. Years ago, we used to combine the management reviews into one and it was just entirely too much information to review during one meeting because we have these three major programs. We found that to be a lot more effective to focus specifically on each one at separate management reviews.

00:02:02 BRIAN: Now this puts it takes a toll on top management I imagine. So, Steve, you've got CCRL and AASHTO resource to deal with. This means that you're participating in several management reviews throughout the year. How does that work with your schedule?

00:02:18 STEVE: Well, it's important enough that you make time for it. It's a lot of time. But it is worth the effort. You know, it's the way we step back. And see that what we are doing is effective. And I mean the only way. To do that is you have to take the time to do it.

00:02:40 BRIAN: Yeah, one thing that we like to focus on during our management reviews is setting goals, making sure that they are suitable for getting the outcome that we want. Bob, let's. What is the outcome that we want when we're discussing these issues during a management review?

00:03:00 BOB: I had to get that question right. What is the outcome that we want? Boy. We don't have. Certainly if we look at the 9001 requirements or if you look at R 18 requirements or 170525 requirements and there are. Identified outcomes that they want you to have, they call them outputs, inputs and outputs to the management review. We don't really look at it that way. I I think that's Tracy's responsibility to make sure we're hitting all those. But the outcome that we want really is. A big picture view of the program and what's going on, and I think another outcome is just. Focused communication pointed discussions between all of us about what's going on. I think one of the things I said yesterday during the management review, Brian, was you and I talked a lot. We talked frequently and probably 99% of the time we're talking about.

00:04:12 BOB: Issues and problems related to the program or something else. This gives us an opportunity to step back a little bit and have a big picture discussion and not only talk about those issues and what the common threads might be, but actually some of them more positive things as well. So it. I think we have found that it's effective and it's a positive experience as well.

00:04:38 BRIAN: I'm glad you mentioned that one aspect because it is very different to sit down in the management review and talk about quality issues rather than just dealing with it as they come to you. You can never hope to make any real progress for your organization if you're only focused on handling issues when they arise. So, we do a lot of planning in these management reviews and I mentioned goal setting. We spend time thinking about not just what should those goals? Be but how can we get there? Are they the right goals? Do they make sense and all with the objective of what, Tracy, what are we trying to do here?

00:05:26 TRACY: We want to determine if we're actually meeting our goals. And in order to do that, we're making a determination if our quality management system is effective. If we find that we're not meeting those goals, then we do what we can to implement improvement opportunities to try to steer us down the right path to actually achieving those goals. And many times, during the management review. And I believe this happened yesterday a couple of times. We tweak our existing goals, we maybe raise our expectations, raise the bar a little bit higher on certain goals, perhaps delete goals that no longer serve us well. And a lot of times we implement new goals. For the future, because it's a look back and a look ahead when we're doing. The management review.

00:06:12 BRIAN: And I and I know that one of the main objectives that we have in our quality management system and just in in the services that we provide is providing effective customer service. So we spend a lot of time. Talking about customer service in these management reviews, what's the most challenging part of that?

00:06:36 STEVE: I mean, it might get back to the initial goals of. Why you do it? As far as how I mean really you want. The program to operate well. And what goes into that is, as Tracy said, are we looking at the right things? Are we measuring the right things within our quality system? Is our quality system working well and ultimately goes. Running a program that works well and provides good customer service, so ultimately it's good for the customers if we take the time to look at what we're doing and to improve. What are the outcomes really you want? I hope that you do find things.

00:07:26 STEVE: To improve and we always do. You know, I I would feel we hadn't done a very thorough review if we went through. And just said, well, everything's fine. You know they. Don't have to change anything, we're just fine.

00:07:42 BRIAN: What the the outcome you want is not. Hey, everything's fine. Keep doing what you're doing, but what I was trying to get at earlier is that we've got a structure or a backbone to our quality management system that we're constantly reevaluating in the form of pack. Right. Tracy, tell us about PACT.

00:08:06 TRACY: Yes, that's our quality policy statement. It's an acronym that the letters stand for, professional, accurate, competent and timely. And we have measurable goals associated with each of those headings. I think at last count, we have 22 measurable goals. Many of which are measured through customer feedback. Probably I would say about ten of those are measured through customer feedback because we're so focused on customer satisfaction and as a service organization, that's extremely important to us. So that is something that we spend a good deal of time talking. About at our. Management reviews.

00:08:49 BRIAN: Now, Bob, when we talk about these customer surveys, what are you, what are you focused on in those discussions?

00:08:58 BOB: I am thinking about and focused on the common threads. The good and the bad so that we can capitalize on what we're doing well and do more of it. But I'm looking also looking for what are the common types of complaints from our customers and how do we address that? I mean, every business gets complaints and sometimes you don't agree with them and sometimes you just flat out think the customer is. Wrong or misguided, but when you see. A common theme among them then you really have to stop and think what what's really going on here and try and be. Not so defensive and really look at it from the customer's point of view and think about how we can fix it and make it better.

00:09:46 BRIAN: Yeah. Now, Tracy, back to you on this goal setting concept. So we obviously want to get excellent customer survey results. You want to have 100% customer satisfaction. And is that a reasonable expectation and what is reasonable, especially let me also frame it this way in the context of the accreditation program that sometimes delivers bad news to customers.

00:10:15 TRACY: 100% customer satisfaction at all times. I don't. That's a very lofty goal. I'm not sure any organization could achieve that. Is it possible, I suppose? Well, but we have. We're dealing with over 2000 customers and. We set our. Bar pretty high though. We're looking for 95% customer satisfaction for each of our programs, not just the AASHTO accreditation program. So that bar is set quite high. Sometimes we don't meet that goal. But we strive to do so and we make changes based on the feedback. As Bob had mentioned earlier, those common threads, if there are specific issues that customers are complaining about, we do what we can to correct those and hopefully we'll get going in the right direction.

00:11:05 BRIAN: Yeah, Steve, we focused a lot on customer service like we mentioned during these management review. How do we balance providing excellent customer service with our goal setting with meeting the expectations of specifying agencies like the dots and federal agencies that want us to provide not just a customer friendly program, but also one that has some teeth to it.

00:11:27 STEVE: What really, you know, it's really. The same thing. In some ways. You know, we the specifiers that use our programs, our customers also. And so we're trying to satisfy both of these sets of customers. We have the direct customers we were going out to visit and we're being accredited. And I mean what they want from us is they. Want the service that we provide? In other words, they want they're getting this service for a reason, usually because somebody is asking them to be accredited. They want to get through it in a reasonable fashion and they want to, you know, people do treat them well. That doesn't mean that you don't have any deficiencies or you do whatever they want. But it means that you do it in a professional manner that puts them in a position where they can meet the program requirements.

00:12:27 STEVE: And for the specifiers of, you know they're counting on us to do. Our job because in the past they had to go visit all these places in many cases. And So what we provide is a service to them so that the work we do lessens the work that they have to do. Because we can do this very thoroughly and well.

00:12:55 BRIAN: Right now, Bob, during the management review, when we discussed satisfaction from the specifiers perspective. What what's meaningful to you? As an output of this audit or not audit but management review. How do you? How do you know if the specifiers are satisfied based on what Tracy and I deliver during the management review?

00:13:18 BOB: Well, we do ask them from time to time what they think and we have. Close working relationships with them. But specifically to a management review, what output am I looking for in terms of satisfaction for the specifiers? They want a program with integrity, a program with real requirements, a program with some teeth. I think as somebody mentioned earlier, I'm trying to stand back also not only think about our laboratory customers, but as you mentioned the specifiers. And think, OK, you know what, what kind of? Action or decisions would they? Would they want us to be making? One of the things we talk about is how we handle suspensions and the rules that we have and. The rules that we have for reinstatement.

00:14:14 BOB: But I would say for satisfying all of our laboratory customers, 100% of the time then that's probably an indication that we're not satisfying what the specifiers. So certainly again, a lot of the. The feedback that we get from our customers is important too. I mean, we don't like to get complaints. But I I think it really is an indication that we're running a serious accreditation program.

00:14:47 BRIAN: I'm going to back up here and cover a question that I probably should ask in the very beginning we we've got you here today on the call because you participated in the management review. Who are the right people to participate in a management review at a laboratory?

00:15:04 TRACY: I can take that one. Brian and AASHTO R 18, it specifies that top management. Be involved with the management review and we often get that question from customers. What is top management? You want to make sure that your decision makers are involved in the discussion. People who are actually making decisions about the budget, policy changes, hiring and. Recruiting in general. Because those are the types of things you're going to be talking about at management review if they're not there, you're not going to have a complete discussion about where to go with those topics. And with our management reviews, we actually have a requirement in our QMS that Bob and or Steve have to be present at our management reviews. Well, we don't do them. Ideally, we'd like both of them to be. Or Bob is the manager of Asher Resource as you mentioned and Steve is the director of AASHTO Resource and CRL.

00:16:06 TRACY: So those are our top decision makers and then of course the program manager for each program that we're doing the management review for must be in attendance in this case, Brian, that would be. So we make sure those people are there and then I, as the quality manager. Act as a facilitator and I also plan the activities and keep the conversation rolling when we're actually discussing everything.

00:16:29 BRIAN: And the elements that you bring into it, we talk about as we know customer service goal setting, programmatic metrics like errors that we've conducted, timeliness things that people who are participating in our program care about. We also talk about what's going to happen in the next year. Where are we going? What activities can we get? What are the priorities that we identified that we can share with any of our listeners for next year? Bob, would you like to mention any of them that strike you as being potentially interesting either? Plans for next year that we've identified in our management review.

00:17:11 BOB: Next year, I know we're all thinking about next year and waiting for this year to come to a swift end.

00:17:18 BRIAN: I imagine leaving your house is one of the things you have to do.

00:17:23 BOB: Having a management review in person instead of. Over the web. There were a couple of things that we talked about yesterday that are IT related that stick out to me and we are pursuing these things I think to make the program better to make it better and easier to navigate for our customers. But for our employees as well, so Speaking of our own quality management system, we have definitely some improvements to make there. And so we are. Right now in the middle of implementing a new digital platform that will take everything electronic and put it all in into one nice space. But we also have a much bigger IT project that is basically rebuilding the whole website, the internal workings, the external facing site.

00:18:23 BOB: There's going to be a lot of enhancements and improvements for the customers and one of the big things that we're focusing on is especially since we've changed our name to. To resources, we want to do a better job of being a resource and providing good. And timely communication and information to our customers and relevant. Information to our customers. So those are a couple of really big things that we're hoping to both roll out in the next year.

00:18:51 BRIAN: We also talk about other interested parties that they care about AASHTO Accreditation other than specifiers, one of them being CCRL. Well, Steve, how do you deal with it when you're sitting through an Astro resource, either assessment proficiency, sample or accreditation program management review and there are discussions about CCR URL, what do you do with that to make sure the communication is effective from your perspective from between the two organizations?

00:19:19 STEVE: Well, I mean. I'm the go between the two. I'm in charge of the two. So I mean it's a. It's useful to have the perspective. That comes up when we discuss these things in a management review because you know I'm just the decision maker on for CCRL also. So things that come up during our management you may have. Impacts on CCRL as an interested party and vice versa. You know both organizations, I think work together pretty well. And but they don't always know everything. That the other organization is doing so any more information I can find out about how the two are relating and whether we're accomplishing what we intend to is useful.

00:20:13 BRIAN: And now, now, Tracy, we've completed this management review over the last two days and I imagine that you have documented a report. I think I have a copy of it in my e-mail somewhere. At this point, what are the important items to document in a management review? Do you need the document? Something in your management.

00:20:37 TRACY: You know, I love to take notes first of all, but I do think it's very important to document the key talking points for each subject that we're discussing. It's especially helpful when we go into next year's management review to reflect back on the notes from the previous management. Review we identify action items in the notes and assign them to people. So we want to follow up on those action items to make sure those things have been accomplished. If you don't keep notes of these things, it would be very difficult. To remember exactly what we discussed and who was supposed to. Do what? So it's very helpful from that perspective and it just helps us jog our memory, is it even throughout the year, we might say in a meeting. You know, I think we discussed that at the management review for the actual accreditation program, let's go look and see if we did and I can go back and say, oh, yeah, we actually did talk about doing something with that. Particular topics, I think we've all found it pretty helpful to. Have detailed notes about the activities that we discussed.

00:21:43 BRIAN: Yeah, you mentioned action items. Do you do corrective actions or not conformities ever result from a management review?

00:21:51 TRACY: They do and they have for us, particularly if we have specific goals that we're not meeting, whether they relate to customer service or equipment calibration activities or anything like that. Corrective action report would be issued for that and that would be considered a non conformity and then. You do have these. Action items that I keep track of in a spreadsheet and we circle back to those typically monthly at our monthly management reviews and our monthly quality meetings just to make sure that we're staying on top of those items and kind of checking those off our list and not forgetting about them.

00:22:30 BRIAN: And when we get to the next management review for 2021. Do we have a regular set time in that management review already discussed action items and nonconformities? Is just is that the bulleted item that we cover every year?

00:22:47 TRACY: Yes, yes, that's usually how we start the management review as we go back to the previous management review notes and discuss those action items and to see where we are with those. If they've been accomplished or not, and if not, why we may decide we don't need to pursue those. Many more, but that's a good way to keep us accountable for all of these items that have been assigned to people to complete.

00:23:13 BRIAN: And I'm glad you mentioned accountability. That that's one of the words I use when laboratories ask questions about the management review the value of the management review, the purpose of the management review. Why top management needs to be involved? That accountability aspect is really important, and it's also critical for top management and know about resource needs. You know, if there are expenditures coming up, that's one of the questions that always comes up during our management review. Because Steve's in the room, and if I say I really need that espresso machine. I mean to hit my optimal effort, then he can ask me questions about why I feel that way and what other possibilities there may be for us to do that more cost effectively.

00:24:08 STEVE: Unfortunately, I'm in the room. For these reviews, so I can say no to. West, but uh. Get on that general topic. I mean we've. Discussed and made some pretty significant decisions as part of the management review. I mean, one of the things you mentioned was personnel. You know often when things come up and we're trying to figure out what the solutions. Or sometimes, maybe we need to hire another quality analyst. Or maybe we need to buy something to help us with our quality system, or buy something that will help the quality analyst and Brian do his do their jobs. Something other than the coffee espresso machine that wouldn't hurt. I deserve to have one of those first before you get one, but.

00:25:09 BRIAN: Yeah, that's only fair.

00:25:12 STEVE: I don't know how to use them anyway so, but yeah, it's important. This is why you need. The decision makers in the room because often. You know these? Things that come up are resolved by personnel actions. You know by. Looking at your staffing and seeing how you can best use the staff you have or whether you need more, you know that's part of this.

00:25:43 BRIAN: And it can be tricky. Also add you know you talked about staffing and personnel. These are really intended to be focused on systematic issues and improvements. There are times where you might start going down a path where you start talking about an individual or a person's performance, and you really have to make sure that you don't spend too much time on that. You have to get back. And focused on that systematic or company wide improvement and save that personnel, that's specific personnel issues for another discussion probably with a more appropriate group of quality managers, probably not. Somebody who needs to be involved, unless it's an issue that's related to their quality management responsibilities, Tracy, you know we talked about these action items or nonconformities. You're conducting these for the Astro accreditation Program Resources assessment team and the proficiency sample program. That could be a lot of action items, a lot of possible corrective actions that you have to track and monitor.

00:26:46 BRIAN: How do you keep up with that? Is there a good tool that you can use or or some kind of program or? Is it just the old fashioned notebooks or folders? What do you do?

00:26:58 TRACY: Well, initially when I'm taking notes during the meeting, I tend to take paper notes and then transfer those into a Word document so I can circulate them. As far as tracking action items and corrective action reports, right now we're using Excel spreadsheets for that, which works fairly well. But as Bob mentioned, we're. Moving towards a digital platform for our quality management system activities and we're going to have one stop shopping with that where everything's going to be handled through the software. So we're really looking forward to fully implementing that we've. Done some implementation. We're not quite there yet, but hopefully by the end of the year we'll have everything implemented and we can report back later to see how that's going. But I am very. Hopeful that it's going to be a lot better way to track all of these quality related activities.

00:27:52 BRIAN: Yeah, Bob. There are some requirements in AASHTO R 18 for our laboratories to follow as to what needs to be included in their management review. I'm not going to ask you or quiz you on what exactly those items are, [BOB: Thank you.] but I I think and I know from seeing some of the ones that that come across my. The risk is that the laboratories might be suited well to add some other elements to that that are specific to items that they know are meaningful to their management team. Are there any line items in our management review that you are particularly interested in? It might not be. Just the comment every day, boilerplate QMS issues.

00:28:38 BOB: What I might defer to Tracy on this one I I should know that agenda by heart. Yeah, we do have a list of things that we follow for inputs and outputs from 9001. And labs can do the same for R18. But I really do think that that's. Just the beginning. Well, we spend a lot of time talking about customer expectations and surveys. I know sometimes labs say, well, we don't get any complaints. We don't get any feedback that is hard to believe and maybe so or maybe they're just not asking the right questions or any questions at all. We have found we get a lot of feedback, good and bad. Mostly good, thank goodness, but good and bad when we ask questions and we ask the right questions and we ask them often, we spend a lot of time on that and I find that to be one of the most meaningful things.

00:29:40 BOB: Tracy, help me out here a little bit. I'm trying to think of the agenda it was. More than 24 hours ago, Brian so. Yeah, know. I can see the agenda on my I had it on my screen the last two days, but I. Don't have it on there now.

00:29:57 TRACY: I know earlier we discussed interested parties, organizations that are interested in the activities that we're doing, specifiers Brian being one of those interested parties. That's something above and beyond Dash 2 R 18 that we spend a lot of time discussing because we do want to please as much as possible. The people that are interested in what we're doing. Saying another one is performance of external providers, we do hire calibration agencies that doesn't apply to the accreditation program, but that's something that we use for our other two programs. We have calibration agencies come in and calibrate equipment for us, so of course we're interested in what we're getting out of them and their performance. And we talked about employee feedback during our management reviews. Those employees are equally important to us as our customers are. So there are some things that we go discussed above and beyond what are in R 18.

00:31:01 BOB: In the laboratory setting, they're their primary product is data or a test report, and two of the big things that go into that that can affect that are. The competency of the technicians and the the quality of the equipment, and so I think in in a laboratory there should definitely be some focus on training of employees and how do you train them and how do you reevaluate them and when changes are made to standards. How do you communicate that and making sure that it's done well and then on the equipment side just? As Tracy mentioned. Reviewing calibration results, but also the I think really looking closely. That those calibration reports that show that something was out of spec or out of calibration, as they say, and trying to make some adjustments there so that that doesn't happen so. I do think R18 calls them. Out, but I I think. Those are two really, really important areas.

00:32:07 BRIAN: Yeah, Steve, what about you? Are there any parts of the management review that you find particularly revealing?

00:32:14 STEVE: I I would say to me. One of the most important. Arch is when we are reviewing and what our goals and objectives are really reviewing. Are we measuring the right things because I mean we have audits and the audits determining, OK, do we, you know, are we doing what we say we're doing? Are we following our quality system? But for the management review, you need to see OK are we doing the right things? Are we measuring the right things in our quality system? Because if you're measuring the wrong things, it really doesn't. It doesn't matter if you're meeting the goals, so we spend a lot of time looking at. The goals what we're measuring whether what we're measuring is the right thing, whether the levels or we determine whether it meets or not or. Right.

00:33:15 STEVE: And you know, I think that's very important. It's not just. Yeah, we have a quality system, but doing the right things is what matters and what makes a good organization.

00:33:28 BOB: One of the things that we do at the very end of our conversation when we do a management review is what we call a big picture discussion. And I mentioned this a little bit earlier. But I think this is the most important part of the management review. You could talk about all the inputs and outputs you want. You know what? What our customers think what employees think, what are our calibration records look like those kinds of things, what are our audits look like? But at the end you really need to take some time and have a frank discussion about, OK, what, what does? It all mean. If you look at it in its totality, what? Doesn't mean. Are we doing well? Not so well. Where again? Where are the common threads? What can we change in our QMS or in our operations? To to be better, to do better, and to prevent some of these things from recurring. So I think. For me personally, that's. That's the part that I and look forward to the most and think is the most meaningful is the big picture discussion.

00:34:39 TRACY: And just to tack on to that, I completely agree with what Bob said, not just because he's my supervisor, but I also think the big picture look is very important at the end. We're wrapping up and and taking a close look at everything. The management review is. Similar to state of the Union address, it's basically, you know. How did we do? Last year, not only that, focus on last year, but where are we going in the future? It almost acts as strategic planning for an organization. Because you you definitely want to focus on where you're going in the future, and I think that is something that really distinguishes management review from an internal audit.

00:35:22 BRIAN: Now, how do we tie all these things together? So we've got the three different management reviews that go on. There is a risk or a danger in compartmentalizing. Those big picture, I mean, you really want the big picture, right? Not just the big small picture or the small big picture I guess would be the right word. How do you, how do you avoid that, Bob?

00:35:50 BOB: We avoid that by having. The same upper management involved in all of them, so Steve and I are involved in every. Single one. And as we mentioned earlier, Steve is also involved in the CCRL ones as well. Yeah, I had that. I have had that concern and I had some hesitations about going away from doing everything all at once and moving to programmatic management reviews. That was my biggest. We do have the benefit of being a relatively small organization and in addition to these management reviews, we have weekly almost weekly management meetings where the four of us as well as some of our other managers get together and talk. But it is.

xx: It is something to be concerned about and I I do think we have managed that risk, but it definitely is. Risk, Tracy.

00:36:54 TRACY: And in January, we actually have an all programs recap of all of the management review activities from the previous year. And in that management review, everyone in the management group is present. So we have all the players from the different programs available and and so we can talk about all of those. Common threads to kind of tie everything all together. Rather, in addition, I circulate the notes from the management reviews the individual ones to the entire management group so that everyone is aware of of everything that we're discussing. And occasionally, Brian and I know this has been true for the app management review and the Laboratory Assessment Program Management review. Sometimes you and or Maria attend. The opposite management review, because those two programs work so closely together, so that kind of helped helps ties things together as well.

00:37:48 BRIAN: Steve, I've got one last question today and I'm going to direct it to you being the uppermost top manager involved in this discussion, what would you say to other owners of companies or top managers? Who may have. Not gotten involved in the management review before, but to encourage them to participate them in the future.

00:38:19 STEVE: I think that as a manager you want your company. To run as well as it possibly can run and. To do that. Uh, you really have to. Take a step back, look at what you're doing. Have to look at the. Quality system, which hopefully they have one to begin with. But look at it and is it doing the right things? And is it telling you and giving you the telling you what you need to know and giving you the information? You need to know in order for your company to thrive. That's the function that serves.

xxIt's a step back. So you can. It's one thing to do the actions of your quality system and that's all very important to do all those things, but you must step back and determine. Are you doing the right things and is it meeting the needs of your company or your customer?

00:39:23 BRIAN: All right. Well, thank you, Steve. Again, our guest today. We're Steve Lanker, director of CMRL.

00:39:29 STEVE: Thank you.

00:39:30 BRIAN: Brian Bob Lutz, manager of AASHTO Resource.

00:39:33 BOB: Thanks for the great questions, Brian. Thank you.

00:39:36 BRIAN: And Tracy Barnhart, quality manager at AASHTO resource.

00:39:40 TRACY: Thank you, Brian. I enjoyed today's discussion.

00:39:42 BRIAN: Yeah, Tracy, one last thing before we sign off, where can people find more information out about how to conduct? A management review.

00:39:50 TRACY: Good timing on this podcast. I'm actually planning to deliver a webinar specifically focused on management reviews and then look for a policy and guidance document to be published on our website for management reviews and internal audits later this year, which will have a lot of details, many of which. We discussed today.

00:40:10 BRIAN: All right, that'll be great to look out for that listeners and TuneIn to the next episode of the.

[Theme music fades in.]   

00:40:16 ANNOUNCER: Thanks for listening to AASHTO re: source Q & A. If you'd like to be a guest or just submit a question, send us an email at podcast@aashtoresource.org or call Brian at 240-436-4820. For other news and related content, check out AASHTO re:source's Twitter feed or go to aashtoresource.org.