How can we know if our advocacy has actually improved outcomes? The mystery of SB93

Capturing the impact of advocacy and legislation intended to improve outcomes for people with ASD is critical, but complex.

September 19, 2019

ASD advocacy has led to tremendous investments in research and related programs over the past decade, yet the return on these investments (ROI) remains unclear. In this essay, I consider the challenge of determining the ROI for one such project in Delaware, Senate Bill 93 (SB93). SB93 was designed to improve outcomes for people with ASD in Delaware through better planning and coordination across agencies, with a special emphasis on training and family navigation.

The mystery? A quick assessment of all publicly available information raises important questions about the ROI for SB93 since it was passed more than 3 years ago.  Only one component - navigators hired to helped families access services - appears to be meeting some of the initial goals.  If SB93 was a project undertaken in a traditional business, there is little question that funding would now be cut off, based on the information available.  Or, at the very least, funders would make it clear that future support for this and other projects would depend on much better documentation of the ROI.

But ASD is (as always!) different, and the process of creating new programs of ASD services and training is unlike any traditional business. So what really is the ROI for SB93? How do you establish what progress has been made? As it turns out, funders face significant challenges trying to determine the ROI for projects like SB93. In this essay, I offer some clues to help solve this mystery, re-framed as some lessons that the story of SB93 might offer to advocates, funders, and project leaders. Some clues lie in the barriers and disincentives to publishing detailed progress summaries, the lack of relevant research on implementation of new practices (and the leaders experienced in doing so), the confusion about what constitutes progress, the fragile network of stakeholders willing to take on system change, and so on.

Some might ask, why even bother? One reason is that these investments to date have not been insignificant. Consider the  small state of Delaware, where the total investment for SB93 alone would have surpassed the $2 million mark later this fiscal year, had the expected progress been made. More importantly, similar questions can be - but are rarely - asked of every other initiative in Delaware, and indeed across the country.  Taken together, these investments have been very significant, especially for a sector that is otherwise chronically under-funded: almost $30 million of publicly and privately funded ASD initiatives over the past 4 years in Delaware, more than $2.5 billion in research funding alone across the US over a 10 year period, and so on.

To be clear, readers in search of some exciting tale of program leaders misleading investors to line their own pockets should turn elsewhere (the gripping story of Theranos told by John Carreyrou in Bad Blood fits that bill!). There is every reason to believe that the individuals and agencies involved in the ever unfolding story of SB93 are doing their best given that traditional research has have not taught us a lot about how to build statewide capacity in ASD services. Nonetheless, a shift in focus by funders on the ROI of programs currently underway could have dramatic consequences, especially because future investments may need to be even more significant. For example, legislators did not allocate any new funding for SB93 for FY2020, and instead encouraged project leaders to draw down on the substantial unspent funds accumulated over previous years.  Were legislators just being fiscally prudent? Or does this signal that legislators are questioning SB93's impact and may move to continue these funding cuts in subsequent years, if not shelve SB93 and other related ASD initiatives like HB292 entirely??  Stay tuned - up to $1 million in funds made available each year by local and state leaders for these kinds of initiatives may hang in the balance. Maybe we do have a cliffhanger of a story after all!!

Regardless, this kind of thoughtful accountability - recognizing the challenges inherent to initiatives like SB93 and balancing these with reasonable expectations of progress - are absolutely essential to any future case for funding. The answers offered in this series of essays might help assure advocates and funders that future initiatives will offer the best possible ROI.  But the questions posed here only begin to scratch the surface of what might be learned from a critical re-examination of our advocacy.  Irrespective of the progress that initiatives like SB93 might have achieved, have these efforts even focused on the most important needs? Has our research even yielded practical tools that typical practitioners might find useful in addressing the most important needs?

?