Will: Hello everyone and welcome to Data Isn't Just for Science 5+ Secrets to Writing Evidence-Based Grants with Amanda Faye Lipsey. This workshop is being recorded and slides will be shared afterwards. So please keep your eyes peeled for a follow-up email later today, in case you want to review anything that we go over. In case this is your first time here, this free grant workshop is an Intrumentl partner webinar. These are collaborations between Intrumentl and community partners to provide free educational opportunities for grant professionals. Our goal is to tackle a problem that grant professionals often have to solve for while also sharing about the different ways that Instrumentl’s platform can help grant writers win more grants.
Intrumentl is the institutional fundraising platform. If you're looking to bring grant prospecting, tracking, and management to one place, we can help you do that. And you can set up your own personalized grant recommendations using the link on the screen here and also put it in the Zoom chat in case you need it. If you have any questions along the way, we ask that you include three hashtags in front of your question for today, just since there are a lot of folks in the audience so it's easier to spot those. And now with that housekeeping out of the way, I'm very excited to introduce Amanda Faye Lipsey.
Amanda is the owner, founder, and founder of Amanda Faye Consulting. And she's a skilled grants professional health education researcher and patient advocate with more than 20 years of experience in fund development and communications. Her research has been featured in peer-reviewed journals such as the BMC Nephrology, Patient Education and Counseling, Progress in Transplantation, and the Journal of Medical Internet Research. That was a lot. She has co-authored a book chapter on patient advocacy and end-stage renal disease and a transplant toolkit for ESRD providers available through the ESRD and Networks online. She's also a recent approved GPA trainer. So love to see that as well. And Amanda, feel free to take it away from here.
Amanda: Well, thank you, Will. I didn't realize you’re going to read the whole thing. And that's definitely a mouthful. So as Will mentioned, I have participated myself in a lot of research. And of course, my main focus is as a grant writer. So I've had the opportunity to be on both sides of using research to support a grant but also in actually generating that research that we need to support a grant proposal. And so, I'm really excited to bring to you today some of those strategies and some insights about how to do that. Just a little introduction, reiterating what Will has said, I've been in the field for a long time, raised a lot of money, GPA-approved trainer, GPC-approved education provider, and grant professional certified.
And I always encourage all of you grant writers and training, who mentioned that you're in training in the chat, to look into the GPC credential. So today, I've broken this into about six parts here: we're going to talk about why you should care about being evidence-based; how to show your work as evidence-based; where to find that evidence; how to understand it; integrating it into your grant proposal, and then we'll have time for questions and answers. And I encourage you to ask general questions as well as very specific to yourself questions. I love to really get into helping your problem solve. So let's get right into it.
Why should you care about being evidence-based? Well, here is an example. It's not uncommon especially with federal funders to ask in the NOFO about the evidence-based strategies you'll use, or what the evidence-based is for your current program. And you can also use evidence to establish the needs of your community or the needs of your project in the community. And we're seeing this more and more not just from federal funders but also, here's another example, but also from foundation funders. It's becoming a more common question for you to show evidence-based and they're asking this because they want to know if they invest in your program, What's the likelihood that it's going to work? And that's based on other prior evidence.
So we're going to do a little poll and I think Celia was going to put that up. Thank you. So true or false, being evidence-based means to show proof that your program or project works? Let's give you a moment to click true or false. Just come up with the first thing that comes to your mind.
Will: And while folks are submitting that, I wanted to share from our earlier poll, some quick results here. It takes 56% of you, one to three hours to write a grant executive summary; 25% of you, four to six hours; and then the rest are kind of scattered throughout there. And then people's favorite part of the grant application to write—it was really tough and really close—but 25% said, problem statement; and 22% said introduction, abstract, or summary. So those are the two leaders from the earlier poll from today. And now we've got about 75% of folks responding to this poll. So I'm going to go ahead and share the results here.
Amanda: Oh, excellent. And Will, I would love you to keep those other results on hand because I'd like to talk about them in another couple of slides. So 70% of you said it's true, that being evidence-based means to show proof that your program or project works, and 30% said it's false. And I am going to make it to… this was actually a trick question. That 30%—yes, sorry, I know, quizzes and trick questions—but 30% of you were right, it is actually false. You need to show evidence, not proof. And so we're going to talk about the difference between evidence and proof. So I want to put that on hold for a second just to give you a quick definition of evidence based. Being evidence-based means your program or project is built upon the foundation of theories, interventions, curricula, practicum, or whatever that has been tested through a scientifically designed study.
A common misconception is that quoting a statistic such as population data means that you're citing evidence, but that's not evidence-based, that is you'd be inferring things from raw data. So to our point about it, is it evidence if you're showing proof? Is evidence-based showing proof or evidence? You see on the slide here, this image of the target population and the sample size, and this is why we are showing “evidence” and not ”proof”. So scientific studies are based on a sample of a population, we're comparing that sample to the whole population. So we're making an association or correlation or inference. But unless we actually looked at that whole population, we wouldn't be showing proof. Now, of course, I would say there's always the exception where you might be doing a very specific kind of research that actually very clearly looks at something and you can actually say proof. But in science, we generally say we have evidence of something and not proof. So Will, for the first question of your poll, can you tell me the stats on that again or the responses on that again?
Will: Yeah, sure thing. Now let me pull it up, actually. And I can just share the results.
Amanda: Oh, wonderful. So okay, how long does it take you to write the grant executive summary? So 56% said, one to three hours and then 25% said, four to six hours. So we could make a statement from this, essentially saying something like, there is strong evidence, we talk about the level of evidence. So is the evidence strong? or is it weak? or is it inconclusive? In this instance, we could say something like with one to three hours and four to six hours, you could say there's strong evidence that writing a grant executive summary is a lesser involved time constraint. And but to be careful about what we infer from data, right? Even that I would not be sure that's exactly the right way you want to say it. But let's look at our second question.
What is your favorite part of the grant application to write? So even here we can say things about that there's strong evidence that, for example—this is about our favorite part—we could say that writing problem statements and program goals. We can say there's strong evidence that grant writers prefer writing the problem statement and program goals over other parts of the grant. But we wouldn't say we have proof that it takes one to three hours to write a grant executive summary, for example, because our sample size is only the people on this call, and while we're all grant writers, they're still a much larger population. So I hope that helps you understand the difference between evidence and proof, and why we don't typically talk about proof, but we talk about evidence.
And that also may help you increase the scope of what you're looking for when you need to show evidence. So let's go through these bullets really quickly. A scientific study means that it was designed to be rigorous and reproducible. Rigorous simply means held to a high criteria, and reproducible means if I followed the protocol you outlined exactly, I could reproduce your results. The results will have to have been statistically significant. And I'm not going to get into statistics, or what a P value is, or an alpha, or any, or you know, false positive or false negative, or anything like that. If you are interested in more information on that, let me know, because I do have some resources and training about statistical analysis for grant writers. But that is going to be a little bit beyond the scope.
So but we do need to know if you want to look for the words that the findings were statistically significant, meaning it reached a high enough level. And then again, we want to look for evidence, not proof. So in summary for our first section, more funders are asking for you to show how your work is evidence-based, evidence does not equal proof, and evidence-based means it's based on the analysis of data from a scientifically designed study.
Moving on, so how can you show your work is evidence-based? There's a… I don't know if you all have seen this. There's this meme—oh, it's a couple of years old now—and it's a young girl, I really should have put it here. But a young girl tells her mom that she wants to be an astronaut. And the mom says, “Well, you have to, you have to study hard, you have to get good grades, and you have to go to a good school.” And the little girl says, “Well, that's just like three things.” And this slide makes me feel like—well, that's just three things. The process for incorporating evidence-based data into your grant may seem very straightforward and simple, but as usual, the devil is in the details. So we will talk about how to find the scientific literature for relevant studies that you need, how to read and understand the study, and how to integrate that into your case statement or needs assessment whatever your language is.
We look at finding evidence. I often get questions like, we do this program, let's– Oh, we do a summer camp. And I can't find any studies on summer camp. But one thing you could do, you could certainly look for studies that show data about your programs as a whole. But the other thing you can do is say, well, what are all the things you’re doing in that summer camp? Maybe they have music classes, maybe they get math tutoring, maybe they do leadership training, and all of those things you could look at the individual components of the program or the project and say, what is our evidence-based, programmatically component-wise? You can also look for similar or the same demographics. Also geographic location, in terms of things that would be relevant to show it.
Some different places to find evidence. Some of these should look familiar. Finding evidence is very similar to looking for grant prospects. You want to find a good database that will allow you to search based on different keywords and variables. And it's like Intrumentl will allow you to do that to do your grant prospect research. Many of these will allow you to do the same thing and they focus… Some of them focus very broadly on any kind of research. And then some of them focus really clearly on certain areas. So Semantic Scholar, for example, is a very broad research tool. Just like Google Scholar, that allows you to search the breadth of all or any peer-reviewed journals. And then PubMed is very health focused. And ProQuest Criminal Justice, obviously, is focused on criminal justice. So they each can have their own focus, some of them overlap.
And you know, you can find which tools work the best for you. If you have another search tool that you like that you use to find evidence-based data for your grants, please share it in the chat box. I'm sure we know we all love to grow our lists if there are any others that you use. Oh, NCES, yes, it’s a great one as well. And so, you know, I'll just tell you my favorites. Well, I tend to work in the health field. So I use PubMed quite a bit. I also like Semantic Scholar. Google Scholar actually is quite powerful. I was a little down on Google Scholar when it started up but more and more have actually been finding it quite useful. So these are public tools that don't require subscriptions. PubMed does not require any subscription either.
Now, of course, the challenge can be, when you need to get past the paywall of the journal. And that's when, you know, visiting a local library may be helpful to get you access to that. Many of the articles you look for will be publicly available, and you can download the PDF and read it. Another thing that you should know, as a researcher myself, I can tell you this, if you can't get it through the paywall, right? You pull it up, there's just an abstract there and it says, join, pay $50 a month to get this one journal, which is, you know, pretty pricey. You know that you can reach out directly to the researcher and ask them for a copy. And most of the time the answer will be yes because A, we want people reading our research; and B, we want people building on it and learning from it.
So, you know, whenever anybody emails me and asks for a copy of anything I've published, I just ask, you know, sometimes it's still under this, after a certain time, there's a thing called an embargo. Usually, it's about a year. So during the year period, you can't share it publicly, but you can share it privately. So if you email me, I could send it to you, but I can't post it on my research page for people to download. So I would email it to you and say, please just don't share it publicly. Of course, you can share it with your colleague. All these are great, great questions. I would love to… Will, maybe pause in just a minute and look at some of these questions before we get too far ahead.
So if you can't find anything in any of these places using your search terms and how you've narrowed it down. Another place to look is your funders or your prospective funders. A lot of times they will have published recent press releases where they cite research that they find interesting. They will, maybe, in their blog posts they’ve mentioned studies or initiatives that they're interested in. And you get the added benefit if you're reading and citing research that they've cited is that you're further aligning your programs with their own published work. As an aside, can you think of any funders off the top of your head—you could answer this in the chat box as well—that you know of that published findings from studies?
A good example that many of you will know about is Robert Wood Johnson Foundation. They publish research frequently. AARP. Yes, Pew, good. Right. So a lot. I mean, these are just a few examples. United Health Foundation, yes. Kaiser is another one carried in that field. So if you look at your funder, you can see where they're looking for the latest research, and that can help you see where you need to look. If you can't find anything, you can also look at your own preliminary data. Now, this may or may not be the strongest level of evidence, but it is some evidence, and you can build on that. So do you have findings from evaluations or assessments? Do you have your own surveys done?
That question is really great. Sorry, you guys are asking great questions. I'm going to pause here as soon as we get to the end of this section, and we can recap.
So these can be used to demonstrate evidence-based, your own preliminary data. Again, you want to be careful with raw data and counts. It can be really helpful to work with a statistician to make sure that you're assessing your preliminary data or your raw data correctly so that you're not inferring something that's maybe not true or maybe not accurate. It is a better way of saying it. But preliminary data works really well as well. Okay, I actually want to pause here. Oh my goodness, so many good questions. Will, can we pause and address some of these? Yeah.
Will: Mm-hmm. Of course. So the first question is from Lupita, who asked, “Where can I get evidence on religious-based questions? So for example, how many people attend church?”
Amanda: Oh, that is a really great, great question. Lupita, I can't think off the top of my head where you can find that specifically, but I do think it is a good question to use… Actually, you know, you could use any of these databases really actually to, you know, pop that question in because there is a lot of research that's done that correlates people who attend church and different things, like different other variables to look at associations between, for example, attending church and overall well-being, or attending church and how they recover after surgery. Things like that. And that's just an example. So you could use, you know, Google Scholar or Semantic scholar, or even PubMed, and type in the variables that you want in terms of keywords. It's not going to be something of just pulling it automatically.
I'm not sure if the census data covers church membership. But another place to look would be– there are associations, professional associations, for example, that cover memberships of churches like, I can think of like the Southern Baptist Convention, for example. And you could look to them to see if they have data on church attendance. Maybe not that one specifically but other overarching, you know, like the Archdiocese or they may have some data on actual attendance.
Will: Yeah. And it looks like there are quite a few suggestions from the chat as well, whether that's Barna or Gallup, World Council of Churches.
Amanda: Exactly.
Will: If you research. It looks like Lupita got some direction from there. Catherine then asked, “As a nonprofit, how do you deal with citing people who serve data when our internal record keeping has not been adequate?”
Amanda: Say it again? How do you deal with…?
Will: How do you deal with citing people served data? So data around how many people you've served when internal records have not been adequate?
Amanda: Well, that oh, goodness, that's a problem we all deal with, right? At one point or another. So there are a number of ways, but I think the best way that I would deal with it, personally, is to actually admit that and take the opportunity in the proposal to establish how you want to keep records moving forward, and how you'll use that data moving forward and explain the checks and balances you'll take to make sure that you get the data that you need. Yeah, I mean, I would deal with it as both explaining the challenge of maybe why it's been difficult to maintain that data and just, you know, detail how you're going to overcome that to make sure that you are moving forward by collecting the data and able to use it to show the outcome.
Will: Jean asked, “What are the best sites for educational data research?”
Amanda: Yeah, I would… I think ERIC is on the list. It's a good one. But honestly, I… I find that starting with Semantic Scholar or Google Scholar is a good place to start and then it can be more fine-tuned. Some of the sites that I mentioned, like Embase, may require subscriptions. So Google Scholar and Semantic Scholar are both free.
Will: Marisa asked, “I struggle with finding evidence-based programs or practices for food pantry programs addressing food insecurity. Do you have any recommendations?”
Amanda: So, I do. Okay, I want to come back. Can you mark that to come back to that?
Will: Sure.
Amanda: Later. Okay. Thank you.
Will: Two more to go. One asked, “What if your evidence research does not support your claim?”
Amanda: Oh, I want to come back to that one too. I love that. That's a great question. Can you mark that I want to come back to that later because we're going to go through some things and get to that? Yeah.
Will: The last one for now. Christina asked, “If a funder presents research funded by themselves, is it okay to use that data and a proposal to them as a funder?”
Amanda: Yes, and with the caveat that understanding there is somewhat of a conflict there. And that you may want to also find something else to support it.
Will: Great. That’s all I got for now.
Amanda: Okay, great. And we will flag those two to come back to later.
Will: Yup.
Amanda: Great. So I'm going to ask you to bear with me because the next couple of slides are text-heavy and a little technical, but I want to get us on the same page about appropriate evidence and the different levels of evidence. So as it says, of course, there are different types and levels of evidence, not all are created equal, but depending on how you strategically use them, they can all be used to support your case. And we mentioned just a minute ago, some different kinds, but preliminary data, surveys, or kind of data counts, you know, how many people you've served is a type of data. The kind of evidence depends on the study design. We have both qualitative studies and quantitative studies. To be clear, qualitative obviously doesn't have any numerical output to it.
You're describing something that is narrative in nature. Quantitative is where we're describing something statistically—numbers. Quantitative data in the world of science is stronger evidence than qualitative data. But don't count qualitative data out. Qualitative data can be very useful in supporting a case. It can also show, and I have an example later on, of where qualitative data contradicts earlier quantitative data which points out something that needs to be investigated further and that sets up the case for support for that particular grant. So don't count qualitative data out, even though quantitative data is typically considered stronger. Studies, it's a little bit of scientific nomenclature that we have observational and experimental. Observational, meaning that you don't intervene, you are just watching something.
So many things can be observational that, you know, you can like a longitudinal study where you just record things over time, but you don't actually do anything to the population. Experimental is where you actually if you're providing a program, or you're doing something to intervene upon the subject. A really easy example that we're all getting familiar with is getting a vaccine, right? That's an experimental study. The gold standard for quantitative data. And really all data is the randomized controlled trial where you set up a control group against a group that's going to get whatever it is that you've created: the vaccine, a program, and intervention; tobacco smoking cessation program; or housing program. That's your intervention group but randomized control.
So if that's the gold standard, why wouldn’t we always use that to show evidence, right? Why wouldn’t we always just look for that? And the thing is, it's not always ethical, it's not always practical. It’ll cost a lot of money to run a randomized control trial. So there are a lot of reasons why it may not be available. In terms of why it may not always be ethical, if you're withholding treatment or you’re withholding something that has some evidence that it works, but you're trying to get stronger evidence or maybe withholding it would put that person's life in jeopardy or have a negative impact on withholding this intervention even though you're testing it, that would be unethical to do. So you have to find some other kind of study design that allows you to not withhold a treatment that could help someone.
Another example of a quantitative study is the simple before and after study with a single group. You have a single group, you have to come in at baseline, which is the beginning of your study, and you record. So, you know, let's say you have students that are coming in for tutoring, and you have them take an assessment test. Let's say it's math tutoring, and they take an assessment test at the beginning. They go through the tutoring and at the end, you give them the same test to assess their understanding. And based on that, so you have your before, your pre, and your post, which is the assessment test. And that’s a very common study type.
Here are some definitions, I'm not going to read these to you, but your slides, well, they will be in your slides. There will be a separate handout. I have a full list of probably three or four pages of definitions of study designs for you. So that you can just have it on hand, as you're looking through to see what if you're ever curious what any of these things mean. But I'm sure some of these do look familiar. Cost-benefit analysis is a type of quantitative study as well. Cohort Study. These probably are familiar terms that you've seen even in the news. So as I said, don't count qualitative studies out. These can include your focus groups, your interviews, your surveys, case study, or case series.
So the difference, of course, is rather than being analyzed statistically, qualitative studies are more narrative in nature, like a case study certainly has a very narrative format. And they can often be analyzed using this methodological thematic analysis. And that can be where you go in saying, I want to know what people think. I want to know what people think about evidence-based grant writing, and I may ask you some somewhat leading questions that set you up to answer that. Or I could ask even broader questions with no prior question that I want to answer, but I want to see what comes out of a discussion. So you have a priori and a posteriori which, you know, are big scientific words, basically. Did I decide before what I really want to think about asking you or did I decide after?
Another thing you can do in your search is when you go into your database, right, or when you go to Semitic Scholar, or you go into Google Scholar, you can select things like I only want to look at quantitative studies, or I only want to look at randomized control trials. Another thing you can do—and I highly recommend this—is looking for systematic literature reviews within the last 10 years. A systematic literature review can actually pull all the studies done on a particular topic and give you an overview in one article, and that can help you figure out where you need to start. So that's a really good tactic to start just right at the beginning if you don't know what you're doing or you don't know what exactly to look for, or where exactly to look.
If you can find a systematic literature review that's close to your topic or directly on your topic that can really help you start your search. To someone's point earlier, lack of strong evidence is an opportunity. If you can't find strong evidence, even a case study, incorporating some basic research methods into your assessment or evaluation process is an opportunity for further program growth. So to summarize the section: evidence can be for part of your programs or the whole, it can be the theories that you base it on, or it can be evidence-based for the actual methodology of the program or project. So you can have theories like the Stages of Change theory, which is for behavioral change, and you're basing it on stages of change.
Or you can have an actual RCT that showed that this particular intervention worked. So let's say we stopped smoking, right? Stages of Change theory, we know works on smoking cessation, and we did a program on that and we have evidence that it works. Not all evidence is created equal. RCT is the gold standard. Quantitative, generally, is better than qualitative. Don't ignore the qualitative though. The best case scenario is to use both. If you have the opportunity to cite a qualitative study and thematic finding, and then also follow that up with quantitative findings that show something worked. That's a really strong case. When in doubt, look to your funders to see what they cite. Another strategy is to look at your peer organizations and collaborators, and don't forget your own data.
All right. So we're going to look at understanding the evidence and we'll break this into looking at some research articles and how it's broken down. So, in general, I say, in general, because this is going to be, oh, I'm going to throw like a totally non-evidence-based stat at you right now and 90% of all research articles are written this way. I don't have a citation for that. So that's my anecdotal evidence. So, but yes, about you know, generally, most research articles follow this outline. There are some that don't and definitely in different fields. You may get a little bit of a difference and they may come in different orders. So we're going left to right, top to bottom.
So first you'll have your Abstract, these are generally 200 to 300 words with a couple of sentences explaining what their objective was, what their methods were, the findings, and the discussion. It's very to the point, so you can get an overall idea of what they did in this study. The introduction? This is very– what I love about research articles they actually will often follow how a grant might work. So an introduction in a research article is sort of like their case, it's the scientific case of why they decided to do the study, and what their goal was in doing it. Usually referred to as aims or objectives, or our scientific aims or scientific objectives were to, and whereas they may not state specific goals like our aim is to improve XYZ rather will be our aim is to investigate whether this program improved math scores, right?
So then you'll have your Methods, this is your nuts and bolts, it's very straightforward: exactly how they did it; for who they did it (that's their target population); and their sample size that they chose; when they did it; and where (the setting). So the setting is important to note, because if they did this at a, you know, clinic in San Diego, the population is going to be different in San Diego than if you do it at a homeless shelter in New York, right? So setting can be an important factor in determining if this is really relevant or if I include it, do I need to mention something about that? Findings and results. In a quantitative article, the findings and results will be the shortest section, because in a quantitative study, it's going to be very straightforward.
Statistically, you know, they'll give you demographics and participant data, and then exactly what they want to extrapolate. In a qualitative study, your findings and results will be very long, because they're going to go into the themes, what came out of those themes? You might get quotes from the actual surveys, or the focus groups, or interviews that were taken. Certainly, in a case study, it follows a different, somewhat different format. The Discussion section is what they think it means. I often find that for me looking for evidence based, this is often where I want to start, like what do they think it means in the grand scheme of things? And they'll usually summarize the findings in less statistical terms. So if you're not comfortable totally understanding what the statistics mean, the discussion will repeat it in a more plain language that you can understand.
And then finally, you have your limitations, and these are very important because those factors that they outline are limitations of this study, it always says the limitations of this study. Those factors could contribute to skewing the results which may mean that this study might not be good for you to use. And I need to speed along here. So I'm going to speed myself up; get to the end of this section very quickly and I want to go through more of your questions.
So how to qualify research? If any of you have a background in… and I'm assuming since many of you are grant writers and training and many of you have many years of experience that you know how to do grant prospect research, use a very similar process when you get your list back from Google Scholar or Semantic Scholar or whatever database you've chosen, and it pulls you back a list of citations that you can then either read the whole abstract or the whole article, you can use a same, a very similar process, you know, you use steps of disqualify. No, not this one. No, not that one. So I always start with, did the title totally disqualify it? Sometimes the title will just tell you, okay, yeah, no, this is not even close, then the abstract disqualifies it.
Now if the abstract is vague, it doesn't give me the exact information that I need, I'll put it in there, Okay, I need to look further. But the abstract may actually fully disqualify it. And I want to give you an example right here, the abstract that is shown. If I pulled this, I said, you know, I'm looking for quantitative evidence about education programs for kidney transplant patients. This particular article that has this abstract would come up, and I would read it and I go, Oh, maybe. But if you actually read this abstract, it tells you that it is what is called a protocol, which means this study hasn't happened yet. They are publishing what their protocol is going to be ahead of time, but they have no findings yet.
So this doesn't really give me any evidence, it just tells me that someone's about to do a study. So that I go through my list, this is how I break it down. You may find as you go through calling that a different order works for you. And I certainly, I’m always a proponent of finding the best way that works for you. But this is how I handle it: I go to the title; I look at the abstract; then I go straight to the findings; and go, let me read the findings in full. Okay, still good. I'll go back to the methods, is there anything in the methods that doesn't work for me in what I'm trying to, you know, show? I call down and then with the ones that I have left after that. I'll read the full article and either say, these are the ones that I want, or these the ones that I don't want.
Now, the process that I'm giving you is also casting a really wide net, right? You may find really good evidence within just looking for a few articles and ideally that's the case. But if you want to do a full search and see what's out there to find it. So let's move along. So just to look at what this looks like, quickly, this is an abstract. You see here, I pulled it out here, something that stuck out to me if I was looking for this, you know, it talks about recommendations on early introduction to transplant. Here's an example of an analysis. This is a qualitative analysis using a grounded theory study design. So that's telling me how they did this, they did it a posteriori which means they decided what their themes were afterwards.
Result. This is– I remember I said qualitative will often include quotes in the results section. So the theme was they were confused about how to get waitlisted and then these were quotes supporting that theme. And the discussion section. Now, remember, I said that oftentimes the discussion will summarize the findings for you in a way that makes it a little bit easier to understand. So here's one instance of that. Our findings show that patients with CKD three and four are interested in information about managing their CKD diagnosis and avoiding transplantation. And they want information about transplants even when their kidneys fail. So the big bullets will be in that discussion.
So coming back to Limitation, why say, why that article may not mean what you think it means? So this is from that same article that we've been perusing, actually, and you'll notice that they say it is exploratory by nature which means they didn't have any specific aims going into it. So it may be less targeted. And maybe the results may be more skewed, or we could be interpreting them in a way that, you know, because it's exploratory may or may be some bias. Also, I've had it in here, also the relatively small sample size made it difficult to isolate differences in educational needs and preferences by subgroups. Small sample size is, it's probably like every paper written ever, there are common study limitations are small sample size and being able to extrapolate that small sample size to the whole population.
A big limitation is also, for example, something like, well, our sample size was all college students. So it's like it's, you know, a completely homogeneous sample versus a heterogeneous sample. And you can't really extrapolate the behaviors, for example, of college students to the whole population, right? So, it's important to look at limitations. You want to be careful that if you're using a study for evidence that you aren't misrepresenting what the study found by ignoring the limitations or stretching the meaning of the study by ignoring the limitation.
So summarized. I know that was a lot. That was kind of our very technical section of this. Call down your list just like you would qualify a list of grant prospects. Don't forget to check for conflicts of interest from the authors. Someone mentioned earlier about, can you cite the funder's research back to them? Yes. Again, it should be careful about conflicts of interest. Conflicts of interest will often come up in industry funding. So if a pharmaceutical company commissioned a study to show that their drug works, well, that would be considered a conflict, but you may want to be very careful in reading to make sure that it was rigorously done and that it was peer-reviewed. Abstracts are helpful, but the devil is in the details. After you've pulled down your list, make sure you read the whole thing. And make sure as we just said, pay attention to specific study limitations, so that you don't end up misrepresenting.
So into our next section, integrating the evidence-based findings into your case statement. There are definitely a few ways to do this, I'm going to show you a roadmap that I particularly like and use often. And again, it's not the only way and certainly not the only way I do it but it is one way. And I want to come back to Will, there was I think the first question I asked you to hold on.
Will: The question that was held earlier was, I struggle with finding evidence-based programs or practices for food pantry programs, addressing food security, do you have any recommendations?
Amanda: And there was another one, something about…
Will: What if your evidence research does not support your claim?
Amanda: So one way to look at… so okay, that's the second one, the second one is the one I want to do. So let's… we're going to come to that as we go through the roadmap. So this probably looks familiar, right? Because many of you will have hope, I’m... most of you. Again, there are my statistics. It's going to make you really think about when you say: many of you, most of you, the majority. If you start to really think about what those words mean. But most of you will have written a case statement or needs assessment. And this is a pretty typical format, right? You start with stating what the problem is. Half a million people in this country have end-stage renal disease. That's not the right number, but let's pretend it is. And then we talk about what are the current solutions.
Now, if your evidence points to… if your evidence says that current solutions don't work, this is a place to say, okay, these are the things that we're currently doing. This is why it doesn't work. It's not moving the needle. So there's one way to talk about things that don't work. And you kind of want to do that, it's good to say, all these things are being tried and they don't. And here are all the reasons they don't work. And here's research that shows that they don't work. And then you can pull up more evidence that points towards your solution. Now, when there is no evidence that points to your solution, or the evidence points away from your solution, that's when you want to start to look at all those things about limitations, or were there things that could have confounded the work? Or what other support can you show that you should still be doing that work?
Now, if evidence points away from your solution, think of that from the reviewer's perspective, I'm going to say, well, the evidence doesn't support this. So why should you continue to do that? And that's a question that you have to consider. If the evidence is showing that this program isn't helping move the needle, then why are we continuing to do it? And I know that's a hard question to ask. But that is also why funders are asking about the evidence-based because they want to fund programs that they can see will actually move things forward and not sort of hold the status quo or, you know, kind of be in a band-aid in a way, right? If we don't address the root of the problem, right? Were you going to have to continue to do this?
So I would really consider looking at that. And then I would also look at, okay, will this evidence point away from our solution? Why does the evidence point away from your solution? What's different about that evidence? And for the person that specifically asked that question, if you have a specific scenario, I would invite you to actually… can they… Will, can they unmute their…?
Will: I think for the sake of time, I want to be respectful at the top of the hour.
Amanda: Oh, I'm sorry. You're right.
Will: What we're going to do is we're going to try to go through this section and then go through some of the held questions from earlier and then I've shared your contact information in case we don't get every question in today. So I mean if we can get to that in the next few minutes. That'd be great.
Amanda: Yes, thank you. And thank you everyone for– your participation has been so wonderful. Yes, if I don't get to it let… you follow up with me. I love these questions. I want to be able to give you good direction. So again, you get your problem. What's the current solution? Why doesn't the current solution work and the evidence in your research that shows why that evidence doesn't work? Evidence that starts to point toward your solution, this may come from directly in your field or other fields that are using something similar to what you're saying, and you want to try it in your field, and then what your solution is and summarize it. Some quick examples of citing evidence-based data. This is one that I've used frequently. This is—I mentioned it before—the transtheoretical model of behavior change, using this model as a framework.
So that's one way of building and saying you're basing it on a specific model as a framework. That's a way of showing your evidence based. This is, again, showing this– notice this section is your theoretical foundation. You can actually incorporate into your needs assessment, how you've built the design with theoretical foundation and cite those theories. Theories can be an evidence base. Here's another example. This is a foundation grant. We said and we pulled these studies that showed research has shown that children with developmental disabilities who spent time in this multi-sensory environment demonstrated a shorter duration of anxious behaviors.
And we were, of course, trying to support that we wanted to build these multi-sensory environments. This is one that shows evidence current solutions aren't working. Studies reported that students… This was when COVID happened, and everybody started doing online learning. And there were some studies right away that reported that the students on the courses were not effective. They felt unmotivated, impaired, and they experienced problems with online course platforms. And they just disproportionately affected underrepresented minorities and students of color when learning from home. So that's evidence. Its current solution though, it's much better to do online learning. Well, there are some problems. But then they use their own preliminary data to support that.
These best practices also close achievement gaps for marginalized groups of students. So they tried some different things and gathered some preliminary data and showed that. This is a really good example. So summarize. Evidence for use integrates it in various ways, you can use the roadmap I provided for you or develop your own. Of course, every grant flows differently. You can find evidence to show current solutions are insufficient and need new approaches. So that's a good option if you can't find anything directly. Whether or not you specifically call out details of the study. So some different ways to build that and say researchers found in one study, study show, or you can actually, you know, simply cite at the end with an endnote or footnote.
And of course, depending on the space needs, I just like I'm running out of time, you running out of pages, you may have to go with endnotes and footnotes versus being able to describe in more detail what the study found. Oh, goodness, I'm going to turn this over to Will.
Will: Sounds good.
Amanda: Thank you so much. Sorry about that. Go ahead, Will.
Will: Yeah, no worries. I've got a couple of questions saved about anything else that's coming in, please do email Amanda, the email that I shared in the Zoom chat. I'm going to go over some of the next steps from here. In case you want to get some freebies, you can submit a feedback form. To do so, Amanda has a 50% offer on one of her respective workshops in case you liked learning from her and you want to dig deeper with her. You can skip this next slide and I'll just go to the next part, Amanda, for the sake of time. Next week, we are going to be having a big summer release for our plus and above plans, including our calendar view, Salesforce integration, application cycles, and consultant plans. And so if you go to the next slides, Amanda.
I'll just give a short preview for folks about these. Or we're going to be having a launch event next Wednesday. If you want to go to the next slide for me, Amanda. And then one of the things that we're going to go over is our new calendar view, this is going to turn your trackers into a calendar in just one click. If you've ever wanted to just level up your trackers or your spreadsheets, this will be something that you can now do on Instrumentl. In the next slide, Salesforce integration, if you want to stay in sync with all of your data, we will have a new integration that will be available to our pro and custom plans. We are also going to have a new release in terms of application cycles in the next slide. And that is going to allow you to track multi step grant applications.
So that's going to cover, for example, if you're tracking towards an LOI deadline, as well as for a full proposal. That will be available for all customers. And then in the last update that we’ll have, we'll have consultant plans fully released as well. And that's where if you are a consultant in the audience today and you want to set up different profiles for each of your clients and create unique grant searches for each one, and maybe they all run on different fiscal years, you can do so now on Instrumentl as well. And so if you haven't already RSVP for our Summer Splash event, you can definitely RSVP there. We'll have a few other customer-only events too, that you can definitely check out as well. And then if you want to go into the five learning takeaways, Amanda, and then we'll see what questions we can cover from there as well as some of the other feedback from stuff.
Amanda: Right. Yes, it's five takeaways. Any proposal can be an evidence-based proposal. And remember, you're looking for evidence, not proof. Be careful about using raw numbers as evidence, you want to look for studies designed with sound science and findings that are statistically significant. Specifically looking at quantitative, obviously. If you can't find directly related evidence, break your program or project into pieces and look to support components rather than the whole. And you can use my roadmap for guidance on different places where you can seek evidence to support your grant.
Will: Awesome. And with that, here is our upcoming workshop schedule, you can check that out as well in the Zoom chat or you should have gotten an invite if you're already a past attendee. These are all free once more. And so we'll be back next Friday as well for How to Write a Compelling LOI with Arnisha Johnson. The next slide has Amanda's contact information which I've also shared in the Zoom chat. And then you can also use Amanda's Instrumentl link. And I believe that her code actually, it's not on the slide here, but we'll make sure to include it in the final version. The FAYE100 will save you $100 off your first month of instrumentl as opposed to the standard $50 there. And then in the last slide, Amanda, if you go ahead and submit your feedback form in the next slide, do that by the end of the day, on Friday, you'll be able to get that code for Amanda's workshop as well as our 10 Best Lessons from 10 Grant Writing experts. I believe we have time for maybe one question right now. So I'm going to try to tackle Catherine's question, which is, “Do you feel that research grants should be left to hospitals or universities?”
Amanda: No. Absolutely not. And they're not. There are lots of research organizations that are nonprofits that are not universities or hospitals. There are also small businesses that are focused on research. And I absolutely believe that it's hard for all nonprofits to participate in research. But I absolutely think you should be looking at how you can turn the data you collect to show in a scientific way that you are actually moving the needle and what you're doing in the community. The problem of evidence not being available and data not being available is because only universities and hospitals publish and only they do the research. So it's a challenge and I would say it's definitely something that more nonprofits would behoove, it would behoove them to take part.
Will: Awesome. And then one more question is, “What would the best method be to quickly read multiple research articles for our proposals?”
Amanda: Find a systematic review, search specifically for review articles, systematic reviews, systemic mapping reviews, and scoping reviews. They will summarize all the recent literature. You want to look in the last 10 years, older than that will not be as relevant. And they will summarize all the recent research. And from there, you can say, oh, I want to read that study more, or I want to actually pull that, and they have a long reference list. So you can actually find this citation and then go find that article that speaks to you.
Will: Awesome. And we actually have time for one more question, which I'll take from Elaine, which is, “Where can I find statistics about how many military members have pets when they are deployed and need help in fostering their pets while being away?”
Amanda: I would look to, well, certainly I would look at DOD sources. The DOD publishes research. And so I would look to the DOD and see where they, you know, what they've published in terms of staff about the military. Families, also military families, support groups or associations. And the VA may also have some data regarding those kinds of things.
Will: Great. And Catherine, I know we had a second question from you, if you want to email Amanda that question that'd be great regarding the COVID question. And then that wraps up our time for today. So thank you so much everybody for attending today. I hope you enjoyed the workshop. All of the feedback forms as well as slides and replays will be signed later today. And be sure to join us next week for either Summer Splash or for How to Write a Compelling LOI with Arnisha Johnson next Friday.
