Support Group: IDC TA Providers Talk Differentiated Monitoring and Support (DMS) 2.0

A Date with Data

Resources:
DMS Resources for Grantees - Individuals with Disabilities Education Act DMS 2.0 Document Review & Request Template

SEA Data Processes Toolkit | IDC - IDEA Data Center

TA Center Differentiated Monitoring and Support (DMS) 2.0 Adapted Protocols

Reach out to us if you want to access Podcast resources, submit questions related to episodes, or share ideas for future topics. We’d love to hear from you!
You can contact us via the Podcast page on the IDC website at https://ideadata.org/.

### Episode Transcript ###
00:00:01.52 >> You're listening to "A Date with Data," with your host, Amy Bitterman.

00:00:07.34 >> Hey. It's Amy, and I'm so excited to be hosting "A Date with Data." I'll be chatting with state and district special education staff who, just like you, are dealing with IDEA Data every day.

00:00:19.50 >> "A Date with Data" is brought to you by the IDEA Data Center.

00:00:24.66 >> Welcome to "A Date with Data." Today, we are changing it up a bit, and I am so excited to be joined by two of my colleagues. We have two IDC TA Providers: Mary Watson and Amber Store. And they have been really involved with several of the states who are in Cohort 1 of OSEP's Differentiated Monitoring and Support, or DMS, and they are joining us to talk about the experience that they've seen and lessons that they've learned from their perspective as TA providers, and to get started, could each of you tell just a little bit about yourselves and your background? And, Mary, do you want to go first?

00:01:04.63 >> Sure, Amy. Thank you. My name is Mary Watson, and I started working with the IDEA Data Center, IDC, in its first year of being launched, and prior to that time, I had worked with the North Carolina Department of Public Instruction in various roles in special education. In my last years there, I was in the State Director of Special Education.

00:01:28.96 >> Thanks, Mary. Amber, how about you?

00:01:31.61 >> Sure. I'm very happy to be here with you all today, and my experience is a little bit opposite of Mary's, where Mary was here at the very beginning of IDC, I've been with IDC for just over 1 year. But prior to that, I was a research and data specialist at the West Virginia Department of Education for almost 10 years, so I come with a very strong background in evaluation and data and analytics.

00:02:04.91 >> Thanks. Yeah, I forgot, both of you have that strong history in the states before you became TA providers, so I know that probably helped out a lot in the work you've been doing with states, especially with DMS.

00:02:17.13 >> Absolutely.

00:02:19.38 >> So this first question, I'm just going to ask Mary because she, I know, has been involved with some of the states who are in Cohort 1, really, since Phase 1. Can you tell us how you have been supporting the states that you've worked with during Phase 1? And just as a reminder for folks who may not really be familiar with DMS, that's the phase in which states are providing relevant information back to OSEP, documentation responding to questions and that sort of thing. So, Mary, what was that first phase like for you in working with the states?

00:02:54.90 >> Well, Amy, when we first started working the states in Phase 1, and I advised this even moving forward, it's important, I think, to build your team as TA providers, and we did that across centers with IDC, partnering within CSI and Cipher to get to know each other to support the state and to get to know the context of the state. So you're kind of building your TA team, but you're also getting to know the state team that's going to be working with the DMS because depending on the size of the state, every single staff person may not be directly involved, so you spend a little bit of time getting to know who's going to be involved and how the state is going to approach their DMS visit. And I think from there, many conversations about: This is what the state needs to be doing every day. The DMS visit is about checking the state's systems, but it's not above and beyond what the state should already have in place. And I think that's important for all of us to understand. In saying that, in the actual technical assistance, we looked at their systems, an overview of their systems. And using some of the OSEP provided resources like the Document Review and Request Template and the OSEP Protocols, we started to kind of help the state organize themselves with the people that were going to be involved around those protocols and then reviewing their website or helping them to review their website to make sure that they have the documents that are being requested and/or also to make sure that things on the website are current and accurate, that they are not out of date, and in many instances, we haven't done that many yet, but as states start reviewing their websites, they do find things that need to be taken down from the website.

00:05:02.86 >> Hmm.

00:05:04.14 >> We supported the state during the OSEP calls and interviews, and during that phase, we're mainly listeners and notetakers, but immediately following the calls, we would follow right back up with the state for a debrief to help them get ready for the next call or help them strategize how they're going to respond to some of the questions or things they were asked in the call. So really, just being kind of partners with the state, silent partners, for the most part, while they were working with OSEP but then helping them to prepare their materials, helping them to be ready for the next interview, maybe asking some questions to help them prepare for responding during those interviews that were taking place during Phase 1 that may be a little different moving forward. And encouraging the state to feel comfortable and let OSEP know when they need to have some time to get back to them on certain questions. If they're in the Phase 1 interviews, and they really aren't sure about how they should respond, they need to just say that, and then we'll help them draft a response, or they can look through their documents and make sure they have what OSEP is asking for.

00:06:22.72 >> Yeah. That's really great advice. I think in the midst of it when you're kind of nervous or anxious, and you want to just respond and give them an answer, you just have to keep remembering to give yourself that time to take a breath and think, and you can always say, "Let me have a chance to pull that together and get it back to you." So then kind of moving in to Phases 2 and 3, which is now called the engagement phase, and also during the visits themselves, Amber, can you start off by talking to us about the support that you provided in those Phases 2 and 3?

00:07:00.30 >> Yeah, absolutely. So having come on board after Phase 1, Phase 2, and then of course, Phase 3 is when I really kicked in, in working with my state. So one of the first things that the state did, we had a really strong state team, and they had already submitted a lot of their documents, all of their documents to OSEP, and so one of the next things that we did was update all of their Data Process Toolkits. The Data Process Toolkits are IDC tools that allows states to capture their processes for the different data collections, knowing the time lines to the people responsible to the file specs and locations, and this is really ... It was really an internal activity for the state. It wasn't something that they did to share with OSEP. It was more an exercise to make sure that all of their protocols and websites and data displays were all up to date so that they did have everything in place as OSEP was going through and checking their documentation and checking their website links. And so the state just found that as a really, really helpful exercise to get them prepared.

00:08:24.04 >> Mm-hmm.

00:08:25.26 >> Then a couple of other things that I was more heavily involved with is, whenever we would have a meeting with OSEP, we would always come together with different TA providers and the state team and have a debriefing call, and it was usually very, very brief, but it was a really good way for everybody to kind of synthesize what was going on, what the next steps were so that nothing was missed. I found that incredibly helpful because as not part of the state itself, I was able to observe and perceive things in a different way than when you're the state being put on the spot at that moment.

00:09:09.19 >> Mm-hmm.

00:09:09.85 >> And so it was just a really great time for everybody to be able to reflect, and then the other big piece was a site visit that we did in preparation for their OSEP site visit, and this kind of goes back to what Mary was saying in that we were really lucky in that the state invited TA providers from not only IDC, which was myself, but also from two other technical assistance centers, the NCSI and Cipher, and so we had three different TA providers from three different organizations with three very different perspectives that were able to all help coach and suggest and keep thoughts in the forefront and provide some techniques that really helped the state become a lot more comfortable and confident, and they had all of their pieces in place, but, again, just the practice and these different perspectives that we brought to the table, it really put them in a position to succeed. They were a lot more comfortable and prepared to talk to OSEP during the site visit.

00:10:24.62 >> Mary, what about you?

00:10:26.79 >> Well, I certainly agree with everything that Amber shared about Phase 2. I think being that friendly face that they're used to seeing during Phrase 2, especially their own site, was more helpful than I realized. They articulated that, that having a friend in the room was helpful to help them relax. Listening, again, and taking notes so that you can, at breaks, or when you debrief, as Amber talked about, the debrief times being so valuable, you can help them remember some of the things they may need to mention. The states may hear the question from OSEP and not really answer directly, as many things that they're doing that they need to share, I don't know if I said that clearly, but I think the TA providers, in listening and taking notes, can help the states articulate. Remember, you are using this strategy, and you need to tell them about that, so you can do that coaching that Amber was referring to. And also during Phase 2, when they were on-site, I started, along with others that were there, to just jot notes about how we would support the state following the visit as you hear the questions and you know what materials they have and maybe where they need more support. It's a good time to just start drafting out some things that you can offer in support in a follow-up manner.

00:11:52.20 >> All the TA providers that have been involved have been supporting states. I know I've heard from a number of states who are maybe in Cohorts 2 or later for the DMS visits that are completing those data process protocols that you mentioned, or updating them if they've already done them. You mentioned already a couple of the tools and resources, I know, that are available. Mary, what have you found to be the more helpful for states in preparing and taking part in the on-site or virtual visits?

00:12:24.21 >> Well, definitely states that have completed or are in process of documenting their data processes has been extremely helpful, and Amber spoke to that as well.

00:12:34.35 >> Mm-hmm.

00:12:35.00 >> It's helpful to have that documentation, but the conversation that that process elicits is so important in helping states prepare to be able to articulate their system to anybody else. There's just lots of rich conversations as you document all of the 618 Data and the 616 Data, so it has the great outcome of having your data processes documented, but it also has the unseen outcome of helping states talk about their system and understand it better, and then along with that, the OSEP-provided materials, the document review and request template, I found, as a TA provider, to be very helpful. It gave me a starting place on, "Oh, these are the documents they're going to want to see," so when you meet with the state, you have kind of a list of what documents they're going to be looking for. So I thought that was helpful. And then OSEP has protocols for each of the areas like compliance, the data area, the fiscal area. They have protocols with questions that they may ask. If it's not guaranteed, they'll ask 100 percent of those questions, but those are extremely helpful, and then the TA centers took those protocols, working together with the TA centers, and made them into a checklist format so that it's the same questions, same material, but on the right column, there are areas where you can check off: If you have this additional information, the state may want to record, so they're kind of working documents. I found those to be very, very helpful.

00:14:13.89 >> Yeah, and we'll put links to all of these different resources too in the notes for the episode.

00:14:18.86 >> That's great.

00:14:20.61 >> Amber, any other resources or tools that you want to mention?

00:14:25.50 >> Beyond the things that Mary has mentioned, I would just reiterate that we used the protocols from OSEP as well in our preparation, and we really leveraged those to make sure that everybody kind of knew who was going on the be taking the lead in speaking, and then we would have someone who would be kind of the secondary speaker, and the state actually, for those different sections within the protocols, they actually kind of took the time to write out almost script, and they didn't read that verbatim, but they did read it out loud to each other so that they could make sure that they were providing the best possible response to each of the questions, to each of the prompts, and I think that really ... Not only did it help with the comfort and confidence and actually articulating their responses to OSEP, but they became more practiced at it, and the team together collectively became more knowledgeable about each other's specific roles within their larger system, so I think that that allowed everybody just to have a much better idea about the big picture as well as understanding of their individual responsibilities and roles.

00:15:51.21 >> And finally, what would you say are some of kind of the key lessons learned, advice or tips that you would share with states who are maybe in Cohort 2 or Cohort 1 and are still in kind of those earlier phases? Amber, do you want to kick that off?

00:16:09.23 >> Yeah, absolutely. One of the things that we've informally been working on, and it might maybe, knock on wood, someday turn in to a formal tool, is kind of putting together a check sheet, a cheat sheet, a quick facts or things to keep in mind type of document, and allowing the team to set forth their own protocols for the things that they were going to do in the meeting. Again, it's kind of like going back to, who's going to be the primary speaker, and who's going to be the secondary speaker? And then when possible, what we found was an extremely helpful tool that goes along with that is having some sort of live chat, whether that be through a computer program or on people's mobile or cell phones, having a group text going during calls and meetings, as long as it's appropriately done in a way that's not going to be distracting or offensive, but that allowed all the team members to say, "Don't forget to say this. Don't forget to say that," and that's where the secondary speaker really came in, because the primary speaker, you can't expect them to keep up with the chat. They're the ones that are being put on the spot in that moment, and you can't always expect that person to remember 100 percent of everything that they want to mention.

00:17:36.78 >> Mm-hmm.

00:17:37.17 >> And so that's why the secondary speaker would be able to follow along with the chat and be able to insert themselves in an appropriate place and say, "Oh, so and so, don't you remember that this, X, Y, Z, is part of the system?" or whatever it is that they're trying to explain and really be able to kind of make sure that there weren't gaps.

00:17:59.57 >> Mm-hmm.

00:18:00.34 >> And so that was really, really helpful. We found that practice does make perfect. We did mock interviews, and that seemed to help a lot. We stressed that it's best to try to be as succinct as possible and sort of only answer what is asked, and that is not, in any sense, a way to not share as much information as possible. It is that you're really ... The state is really, really trying to paint that picture of their systems for OSEP, and as we all know, they're very complicated systems, and you have so many interchanging working pieces that we can equate to puzzle pieces or cogs. There's just so many moving parts that it can be kind of hard to paint that picture for outsiders, and so that was one of the things that we kind of stressed is: Answer exactly what is being asked. And if you have a good example, or a lot of times, you want to be able to show evidence or proof.

00:19:08.88 >> Mm-hmm.

00:19:09.28 >> And so being able to point to where that exists online, if it is a publicly available document. Things like that are awesome, so please add those things in. But go too far beyond what they're asking because you don't want to muddy the waters. You want to be able to help them truly develop that overarching picture of what the state system is like and where the evidence exists.

00:19:34.94 >> Something I've heard too is, it's not enough just to make a statement or say, "Yes, we do this," or ...

00:19:40.16 >> Mm-hmm.

00:19:41.42 >> Even explaining kind of how, you really have to have that documentation that's either publicly available or internal or the evidence to show, "We have these letters that we send out for notifications. Here's what they look like."

00:19:55.18 >> Yeah. And if you were to see my head right now, I'm nodding emphatically that a lot of it is being able to show, exactly what you're saying, to show the evidence, and finally, I would say ... And I think most people are really excellent about this. I know that the states that I've worked with are ... You have to be patient. Monitoring is not a simple process, and a lot of times, even if a state were to look through the OSEP comps ...

00:20:26.81 >> Mm-hmm.

00:20:27.15 >> ... you can see that a lot of the questions are kind of repetitive, but they're also almost tiered, where you have the overarching question, and then it goes into ... The next question goes into a little bit more detail. The question after that goes into a little bit more detail beyond that, and while it feels that it's repetitive, again, I just think that everybody has to keep in mind that you're trying to provide an overview and then also some very specific information about complicated systems, and just to be patient and to kind of expect some of that repetition but to just always remember that it is in an effort to see the big picture. It is in the effort to be able to say, "Yes. The state has this process and this evidence in place," so they're just trying to get just the big picture, and so, yeah, my advice would be to be patient.

00:21:29.37 >> Yeah. You might have to answer what seems to be the same question multiple times or in slightly different ways, but it's these ... OSEP isn't familiar with all the details, and they're just trying to understand and learn as much as they can. Great. Mary, what tips, lessons learned, key advice do you think is good for states to know?

00:21:53.29 >> Well, I would definitely reiterate what Amber just ended with: answer what you're asked and no more. Don't elaborate because in elaborating, you may get off topic, but just stay with the question and be as positive as you can be and do not hesitate to ask OSEP to restate the question ...

00:22:12.78 >> Mm-hmm.

00:22:13.31 >> ... to make sure that you heard it clearly. You don't have to do that every time, but if you need additional time to think, or if you want them to restate the question, it's fine to ask, and you can just tell them if you need more time, or you want to get back to them at another time on that topic. That's absolutely fine. When you know the date that OSEP is coming, I would advise ... A tip is to talk to your stakeholders that are going to be involved and your state advisory panel, your EC directors, other groups that you believe OSEP may be calling to let them know that you are going through this differentiated monitoring and support process, that you're excited about it as a state, or you're preparing for it as a state, and you want them to be as aware as they can be. I would hope those partnerships are ongoing, but in any case, I would notify your stakeholders and be as positive about the message as you can be. And I think it's helpful if you think about ... If OSEP is coming on-site, if you think about logistics ahead of time, where are you going to position them in the room, and where are you going to have your staff in the room? Who's going to be talking, and does everybody need to be there all the time? Or do you need people going and coming for some of the parts? I just think, thinking logistically about how it's going to work, where people can hear the best, looking at the room you're going to be in, and is OSEP going to have people that are calling and part of the virtual conversation? So that brings in a technology aspect. Does your room have Internet access? Are you going to be able to have the Zoom application or Teams application so that OSEP can participate virtually? If that is true, and even if it's not live, they pretty much always have a virtual component or somebody who's still in DC who didn't come on-site. But it's helpful to have a technology person right there because if you have to stop because somebody can't get on the Internet, or you can't ... The Zoom isn't working, it's really handy if your technology person can be on-site. That's just another tip that popped into my head. As far as the interviews, also, when OSEP is on-site, and you have designated person, and as Amber said, maybe a follow-up person to answer questions, if you ... As a state, you should not have contradictory ... It's not a good time for you to argue about something, so don't override or contradict what somebody else has said. That's what your TA facilitators are there for. We can go over that after the fact in those debrief sessions we talked about, but you should not contradict each other during the interviews, and you think that would be just the norm, but if somebody hears something they disagree with, sometimes they just want to say something right in that moment, and it's not typically the best time. And back to the logistics part, is there an area where people can get coffee or water? And just those creature comforts that you may want to think about when you have guests coming to visit your state. You want your state to look good. You want to receive OSEP in a comfortable atmosphere and perhaps give them suggestions. Sometimes, they tell you what they want to do for lunch, but if not, you may want to provide a list of restaurants that are close by that can help facilitate their needs. Those are the main ones that I can add to what Amber said. She gave some great ones also.

00:25:48.56 >> Yeah. Now, I'll piggyback off of that a little bit because I think it's kind of funny that the logistics of in-person meetings are something that we kind of have to bring forefront again, but after several years of mostly having virtual meetings, I think those are some really great points, Mary. And I would only add to that, just be flexible because the state that I worked with, the on-site visit had to be from ... It had to transition from a face-to-face to a completely virtual, and so the state had to pivot because they were already really kind of set up for the face-to-face, and it all still went really, really well, but just being flexible to whatever the meeting situation ends up being. That's just kind of the world that we live in today, and remember that you might plan for one way, and then it might go another way, but just to keep rolling with it.

00:26:49.18 >> Those are some great tips, and kind of some takeaways I heard is, like you said, the flexibility and just all the preparation, hours and hours of TA providers' time and of course the state, just to really get ready and to make sure that they're very thoughtful and have everything laid out and discussed and just that planning and just gave us, I think, so many great ideas and things to consider. Thank you, both, so much for sharing your wisdom, and love to have you on again maybe when you've gone through a few more of these and have even more that you've learned as things change. We know this process is evolving too. So thank you, both, so much.

00:27:35.05 >> Thank you, Amy.

00:27:35.99 >> Yup, my pleasure. Again, all I can say is practice makes perfect, and it pays off, and it pays off in dividends. It really does because of the pressure for the situation. But like playing a sport or doing anything else, the more practice you have, the better you are at it, and the better your results are going to be. So thank you for the opportunity for us to speak with you today.

00:28:01.16 >> To access podcast resources, submit questions related to today's episode, or if you have ideas for future topics, we'd love to hear from you. The links are in the episode content, or connect with us via the Podcast page on the IDC website at ideadata.org.