AASHTO re:source Q&A Podcast Transcript

Season 3, Episode X: Making the re:source Proficiency Sample Program Work for You - Q & A

Recorded: November 2021

Released: August 2, 2022 

Host: Kim Swanson, Communications Manager at AASHTO re:source

Guests: John Malusky, Proficiency Sample Program Manager and Ryan LaQuay, Laboratory. and Testing Manager at AASHTO re:source.

Note: Please reference AASHTO re:source and AASHTO Accreditation Program policies and procedures online for official guidance on this, and other topics.

Transcribed by Kim Swanson and Descript.com

 [Theme music fades in.]

[00:00:00] Announcer: Welcome to AASHTO resource Q&A. We’re taking time to discuss construction materials testing and inspection with people in the know. From exploring testing problems and solutions to laboratory best practices and quality management. We’re covering topics important to you.

[00:00:18] Kim: Hello, I'm Kim Swanson in today's episode, we're sharing audio from the questions and answer section of our November 2021 webinar making the resource proficiency sample program work for you. In this audio, I'm joined by John Malusky, Proficiency Sample Program Manager and Ryan LaQuay, Laboratory and Testing Manager.

[Audio Transition.]

[00:00:39] Kim: All right. The first question we have, and I'll address this to John from an anonymous attendee, with the performance chart. Is it possible to download the data in a CSV, or an Excel ready file? We want to add a lot of this data to a spreadsheet for ongoing analysis but haven't seen the option. 

[00:00:59] John: So, at this point that option is not available. However, we have been working on an IT solution for a few years now. And that will be a functionality that we're going to have within the program. So you'll be able to do a data dump, CSV, Excel, uh, XLS, and you'll be able to take that information and use it to how you see fit. We are also hopefully getting some additional features within the performance chart. And also, with the Youden plots themselves.

[00:01:29] John: Uh, so if you're a current participant in the program, you are probably aware that we ask a lot of questions about things like pieces of equipment. What type of procedure you used, did you use procedure, A procedure, B procedure C, whatever it was, and we are going to hopefully have the, the capability where you'll be able to filter the data by those specific parameters or properties. So, it will not do a quote-unquote mini analysis on specific individuals or laboratories who reported that data, but it will show you what laboratories did provide that information. It'll just show you them in the Youden plot. We're not at a point yet where we feel comfortable enough to go ahead and, have that data be exportable. To all participants, we will have the capability to provide that information to specifiers, standards, developing bodies, or who we need to, to hopefully improve the standards and the technical process. But at this point we are planning on doing some enhancements. So hopefully it'll work out for, for you, but that's, that's a great question. 

[00:02:30] Kim: Yes. Thank you for that. Our next question is with the test results being so close as 0.01. How can we possibly determine a root cause for being out by 0.01, especially on a test that do not require, two decimal places; how can you really give a grade on something like? 

[00:02:56] John: So that comes down to the accuracy of the tests and, you know, the one thing that, we try to do, and I know that some laboratories, you know, it's a challenge, when we do things like sieve analysis, we typically ask for one decimal place beyond what is in the test method. And that's just simply to try to get that accuracy requirement within the standard. So, I, it is difficult. It is challenging. You know, when it's something so finite, it can be tough. It's like looking for the needle in the haystack. And I guess my, my bigger, thought is, is that happening over and over again? And could that be something that's seen in the performance charts or is it a one-time event? It says we've constantly received a two or three. So, if you have a three, there's really nothing to worry about, but if you're constantly receiving a two, then there is something that's a consistency issue. So you might want to look at those performance charts and try to investigate and see what's going on with that. Ryan, I don't know if you could explain anything there as well. 

[00:03:55] Ryan: Yeah. I mean, it's a very important line. You know, if you're on the three side, you get the pass. If you're on the two, you know, you get the fail. So, if you're, but if you're bouncing around between the two and a three, you know, there might still be something there.

[00:04:10] John: Yeah. I mean, like I said, my, my biggest thing is the consistency part. If it was just a two at random, um, you know, the way that statistics work it's going to happen sooner or later, but if it's a consistent thing and you're seeing it happen, then, I would take a look at the performance charts and see, how you're trending from year to year and see if there's any kind of bias. That would be my suggestion. 

[00:04:30] Kim: All right. Thank you. Now I had a question and, Tara, I'll get to your question too, but what are some of the other things that you may be asking for? Cause I know sometimes our proficiency sample data sheets has more information and more spots for that. So, what's other types of things do you ask for and why?

[00:04:47] John: Yeah. So, we've, you know, we do this very, very regularly asked for things like pieces of equipment. And as I kind of mentioned, like a method A or method B. Uh, things like mechanical washing versus manual washing for a sieve analysis. And then we'll ask for the time that the laboratory washed for, and we're collecting all that information and all that data to provide to AASHTO or the, ASTM, technical subcommittees to make revisions on the standards. Or we might be, you know, the specific procedure, we might also be looking for an update to the precision estimates within standards. We provide a lot of data for that purpose. One other thing, I, this, this pass round, if anyone here is part of the slurry and micro program that just, wrapped up, I think the report was issued today.

[00:05:30] John: We solicited the laboratories to actually provide the critical dimensions on a majority of the testing equipment. And that is, you know, we're not going to touch that data. No one is going to look at. Uh, from the accreditation program and pass any kind of judgment that is simply meant to provide to the international slurry surfacing association. So they can look at the equipment and actually provide tolerances on their equipment and the pieces of equipment to make sure that people can meet specification. And they're reasonably assigning the values that they need to. 

[00:06:03] Ryan: And also, we have some line items where it's just informational. One that's currently out right now, the hot mix solvent ignition. We're asking for your initial weight because if there's an error with that, we want to know that first before we go and put everything through the system. So, we're using as a another catch all, on the back.

[00:06:21] Kim: Oh, thank you for that.

[00:06:23] John: Yeah, that's actually really good. Ryan, you kind of, kind of sprung another thing into my brain. Um, for, for this round, with the gyratory testing, we actually got some feedback with labs, why are you asking us for what our sample weight was prior to doing the bulk testing? And we found out that through our experiences and through experiences with the laboratories that dry weight to the specimen has a pretty strong correlation with how well you scraped your sample bowl, which has.  Pretty good correlation with your compaction height. Um, so we're using that as, as you know, no, one's no, one's going to get a, get suspended for the test method because of that, but we're just simply using that as information to provide, to laboratories, to say, Hey, here's something else that you might want to go ahead and look at to see why you got a low rating on a different portion of this PT. So. 

[00:07:08] Kim: Thank you for that. Now Tara would like to know, could you explain the type of sample that we will receive for the aggregate gradation and gravity samples that are replacing coarse agg and fine agg will these samples be a blend of coarse and fine agg or will it be. Just one. So, explain that more

[00:07:29] John: Okay. Sure. Yeah. So, you know, this is a, a new program that we are releasing, and basically, you know, it's not really new, I guess I want to say it's reorganized. So, we are changing from a coarse agg and fine agg sample program to aggregate degradation and a aggregate gradation and gravity sample program. So, the program is going to be split. The degradation sample is going to come out in January. And the gradation and gravity sample is going to come out sometime around the end of June or July. You kind of need to see how the production schedule's going to roll out. Right now. But both samples will have combined aggregate of coarse and fine. The one difference that we're going to have this year with the gradation and gravity sample is that we will have a sieve analysis sample, which is a mixture of coarse and fine.

[00:08:12] John: The coarse portion of that mixture is what is going to be used for the specific gravity portion for, uh, ASTM C127. There will be another bag of fine aggregate, specifically fine aggregate that's passing the number four that will be used for the C128 or AASHTO T 84, fine agg specific gravity. So, yep. You'll have one separate gradation sample that gradation sample will be used, you know, for that portion of it and for the coarse portion of the specific gravity, but then there'll be a separate baggy in there for the fine portion for the fine specific gravity.

[00:08:49] Kim: Thank you for that, John and for laboratories already in our programs and were previously enrolled in fine aggregate, or of coarse aggregate that that change has already been made in their enrollment. Correct? 

[00:09:02] John: That's correct. We went ahead, and we tried to do our best to automatically do the enrollment for the laboratories. We used a, a lot of the information that we had in our database from accredited laboratories. And we actually looked at what test methods, individual laboratories submitted data for. And we did our best to try to pull that information. And our IT manager was awesome, and he did a great job in sorting it and putting it together and making all of the enrollments. So, all of the laboratories did not have to do that on their own. And, I think we did a pretty good job with it.  We only had a handful of laboratories who had any kind of issue with it. Uh, and it was because of simple data sorting, but I think we pretty confident that roughly 95% of the labs got the switch over without any kind of major issues. That's, that's awesome. When you're talking about, uh, 2200 participants, so it's worked out well for us.

[00:09:49] Kim: Yes. And if you notice a problem with that, let us know if you look at your enrollment and you're like, oh, that's not what I expected. Um, definitely reach out to us now.

[00:09:58] John: Absolutely.

[00:09:59] Kim: Nathaniel has a question. For some of the tests, there are many labs who participate in proficiency tests that they only maintain for accreditation. They do not run these tests daily. For example, the fine aggregate specific gravity. I also know larger companies might share data internally between multiple labs before they submit, how do you account for these types of issues? 

[00:10:26] John: So that's the one, one of the tricky parts about the, the sample program. We try to do our best to make sure that all of the laboratories are testing with integrity and you. We do know that laboratories call each other, and they talk, I'll get phone calls from laboratories who say, Hey, I talked to the lab down the street and this is what they're getting. Can you confirm that? And I'll say, no, I can't confirm it. Uh, but it, it seems to be within what's reasonable for the testing data. So, I, you know, I would say that, uh, while we can't really prevent collusion, uh, we try to do our best to minimize it. Now I will tell you that when I am conducting a data analysis or Ryan's conducting a data analysis, we will sit there and look at the, scatter plot. We will look for clusters of data.

[00:11:11] John: And if we see a cluster of data, we can easily identify the laboratory number that it came from. And if we see that cluster containing several laboratories that have the same name, we will notify the accreditation program of possible collusion. So, while I can't say we can completely prevent it, I can tell you that we do screen for it and as it comes to the part where if you're never performing that test, if there's no reason for you to do it, then don't do it. I mean, you know, especially if it's, unless you want to use it for training purposes. I mean, if you don't need to do it, don't do it.

[00:11:42] Ryan: Right. Look into why you're accredited for it. Um, was it, we need it for one project five years ago and that's why we got it. You're not doing anymore. Don't do it.

[00:11:51] Kim: Yeah, that is a really good point. In a recent podcast episode, Brian and I discussed that similarly of, you know, an accredited laboratory was like, what, why are we accredited for this? We don't want that. And it's like, well, we don't know unless you tell us, but we also don't want to charge you for being accredited for a test if you don't really need it. So 

[00:12:09] Ryan: Yeah, I just had to dive down that hole with the lab, regarding the, the new aggregate sample, like, well, you know, we're enrolled in this and this. What do we need for the next year? A day later, we find out that they were enrolled for something, you know, five or six years ago. That was it. They didn't need anymore. 

[00:12:26] Kim: Oh, that's rough to know that you've been paying for that for no reason.

[00:12:29] Ryan: Great training the last five years. 

[00:12:31] Kim: True. Very true. So, I want to thank John and Ryan here for answering all these questions and sharing all this information. Was there anything else that you guys maybe forgot to say, or wasn't really included part of the learning objectives, but really want to make sure you let everybody know about before we wrap. 

[00:12:48] John: I just appreciate the time and having so many people come onto the webinar and listen to Ryan, and I have this discussion, hopefully, it helps all of you out. You know, he just. Cannot stress enough to, if you've got a question or a concern, feel free to email us or call us, we're here to help you. We're here to do our best to make sure that the testing is getting done, uh, correctly and appropriately within the guidance of the standards. And, you know, like I said, don’t hesitate to contact us. That's what we're here for. Um, you know, and by all means, you know, we've got TechEx coming up in a, in a couple of months, uh, you know, it might be a little cold in Minnesota, uh, but this is the kind of content that we're going to be going through there. So, you know, please by all means, if you can attend, try to sign up and, and see if you can get yourself to Minnesota in March. 

[00:13:31] Kim: Yes. And details on TechEx and other upcoming events are on our event's page. We do have one other question and that is, would you all be willing to put together documents to help laboratories troubleshoot low scores on specific tests?

[00:13:47] John: So that is, that's a great question. And that is actually one of the requirements of our ISO 17043 accreditation. We are recommended to provide suggestions and information as to how to help resolve deficiencies and corrective actions. And what we had been doing in the past is we have been having a compilation of findings from the laboratory assessment program, and we've had direct links within the PSP section of the website to link to, that portion in the laboratory assessment program as a method of guidance to help resolve issues with proficiency sample program. But as I mentioned, you know, uh, one of the best things you can do is, is shoot us an email or call us and we'll do our best to help you out. 

[00:14:28] Ryan: Yeah. We're willing to, uh, do a deep dive there with you. 

[00:14:31] Kim: They're not just saying it, everybody. They really would like to do a deep dive into these types of things. So, um, with all of that. Think we're going to wrap this up. So, thank you, everybody. Thank you, John. Thank you, Ryan, for your time today, and have a great day, everybody. 

[00:14:51] Ryan: Thanks, Kim. 

[00:14:53] John: Thank you, Kim. Appreciate it. 

[00:14:55] Ryan: Thanks, everybody.

[Audio transition.]

[00:14:58] Kim: You can watch our full webinar on our YouTube channel and from the video gallery on our website. 

[Theme music fades in.]

[00:15:05] Announcer: Thanks for listening to AASHTO re: source Q & A. If you'd like to be a guest or just submit a question, send us an email at podcast@aashtoresource.org or call Brian at 240-436-4820. For other news and related content. check out AASHTO re:source's Twitter feed or go to aashtoresource.org.