A learning system that measures what matters and builds an evidence base of what works

Measurement of participant, social and economic outcomes is poor and the NDIS Outcomes Framework is not fit-for-purpose
Significant NDIS policy and operational decisions are not made on a transparent and independent basis
The NDIS has not been set up in a way that enables continuous learning
The Panel’s vision: An evidence-based NDIS can promote an investment approach and optimise outcomes
Recommendation 23: Measure what matters, build an evidence base of what works, and create a learning system

The NDIS currently supports over 610,000 participants though a $35 billion investment in 2022-23.³⁶⁹ Such a significant investment that has such a large impact on the lives of Australians should be backed up by a rigorous commitment to evidence-based practice, evaluation and continuous improvement.

At present there are also significant data gaps that limit the ability to measure what matters for people with a disability across the entire disability support ecosystem, both within the NDIS and beyond in the foundational support systems.

Future research, evaluation and data infrastructure should cover both foundational and

NDIS disability support systems. This is critical to provide an evidence-base that underpins an effective disability support ecosystem for all people with a disability.

Under current data and evaluation frameworks, it is very difficult to link supports, investment and outputs to the achievement of participant goals and outcomes and broader societal benefits. In short it is difficult to demonstrate what works, for whom, and why.

The scheme currently measures aspects of participant and family member satisfaction and outcomes but does not capture the entirety of the impact of the NDIS on the lives of participants. This makes it hard to know what matters to participants and their family members.

The Review encountered and identified substantial gaps in data availability and linkage which meant a complete assessment of participant, social and economic benefits was not possible. For instance, it was difficult to measure the costs incurred previously by families and charities. Accurately evaluating the benefits of previously unpaid care for people that are now supported through paid care was also not possible.

The data gaps and evidence limitations have most likely led to a significant underestimation of net social and economic benefits. Addressing these limitations in data collections and quality is crucial to accurately measure the true impact of the scheme.

The current NDIS Outcomes Framework (the Framework) is limited in its coverage and does not include whole of system interactions between the NDIS, mainstream and other service systems. The outcomes it does measure are not directly linked to scheme costs. As a result, it is difficult to determine value and the impact of the scheme on quality of life and wellbeing for participants and their families. The Framework relies on participant satisfaction surveys, draws on insufficient data to measure participant outcomes and scheme effectiveness adequately.

Analysis of the appropriateness of the current Framework found it has few objective measures, limited measurement of outcomes outside the Scheme and difficulty measuring progress when evaluating whether participant goals were met.

Significant NDIS policy and operational decisions are not made on a transparent and independent basis

We have heard concerns that NDIS participants are able to access therapies that may not be able to demonstrate evidence based high quality outcomes for people with disability and in some cases cause harm (such as some interventions and practices under Applied Behavioural Analysis).³⁷⁰ Within the Australian Government there are examples of evidence based benchmarking for public funding for other systems - for example, under the Medicare Benefits Schedule. These approval processes require robust evaluation of therapies and supports based on their validity, effectiveness, safety and the cost-benefit assessment of their efficacy in maintaining or developing the wellbeing of the person.

High-quality, open and independent research and evaluation activity, alongside knowledge translation of best-practice evidence into policy and provider actions, are critical to optimising outcomes, safeguarding participants, inspiring public trust, and supporting scheme sustainability.

The NDIS has not been set up in a way that enables continuous learning

All governments should invest more in better research and evaluation of the disability support ecosystem. This should include increased sharing of data in a safe and secure manner so researchers, service providers and other governments can support improved evaluation. These investments would help build the foundations for a culture of continual learning and innovation in the NDIS and the broader disability ecosystem.

Governments recently agreed to establish a National Disability Data Asset (NDDA), which will link NDIS, social security, tax, health, hospital, housing, employment and justice data. It is a major opportunity and follows many years of work by Australian, state and territory governments, especially New South Wales, with the disability community. It will enable governments and researchers to examine the impact of a very broad range of policies and develop new data insights. However, despite the importance of the NDDA for long-term policy development and research, funding for this initiative has only been provided for two years.

NDIS data is currently structured to support actuarial analysis, because this has been its central purpose to date. However, a much broader use of NDIS data is needed. To enable this, the data should be structured according to the International Classification of Function. It is also important that data is made more easily available to researchers (with all important safeguards) to ensure independent scrutiny of the scheme.

Providers and workers are a critical part of the disability support ecosystem. Their approach to delivering supports depends on up to date knowledge of what works and what is best practice. Despite some initiatives like the NDIS Commission’s Workforce Capability Framework and Provider Alerts, as well as some National Disability Insurance Agency (NDIA) policy documents, overall there is an inadequate systemic focus on knowledge translation to update providers and workers.

Research funding

Funding of research to underpin development of the NDIS and the disability ecosystem is hampered by insufficient funding. Funding is required to support research projects and building research capacity in universities, research institutes and disability community organisations.

The lack of a dedicated research funding source is striking given the significance of expenditures on the NDIS and disability and contrasts with medical research which is supported by the Medical Research Future Fund and the National Health and Medical Research Fund. Commitment to improved funding has enabled Australia to become a world leader in medical research and there is a similar opportunity in disability research.

The Australian Government has committed to establishing a National Disability Research Partnership (NDRP) under Australia’s Disability Strategy. It brings together the Australian Government, people with disability, families, carers, representative organisations and researchers. It is developing an NDRP research agenda. This is a small but important initiative with significant potential to grow.

The Panel’s vision: An evidence-based NDIS can promote an investment approach and optimise outcomes

All governments should invest more to enable better research, evaluation and knowledge translation. Increased and improved research and evaluation will help demonstrate the value of the NDIS to governments and taxpayers and help participants meet their goals through innovative and evidence-based supports.

Australian governments should replace the current NDIS Outcomes Framework with a new Disability Support Outcomes Framework. The new framework should be focused on clear objectives and key outcomes covering both foundational and NDIS supports. The new outcomes should also identify data gaps and linkages to other related research and analysis.

The Department of Social Services should establish a new NDIS Evidence Committee modelled on the Pharmaceutical Benefits Advisory Committee and the Medical Services Advisory Committee. The Evidence Committee should evaluate and recommend evidence-based therapies that warrant NDIS support, including having regard to their cost effectiveness. A stronger evidence base providing clear regulation and guidance on effective therapies and supports should provide better quality, improve outcomes, and make navigating the scheme for participants and their families easier.

Australian Governments should establish a new Disability Research and Evaluation Fund (DREF) to coordinate and fund research and independent evaluation activities. The DREF would help address critical knowledge gaps in Australian and international disability research that is most relevant to the Australian disability and service delivery context. The DREF would fund work to strengthen disability data availability and use, and build the evidence base to better support people with disability outside of the NDIS. The DREF should also promote participatory disability research and knowledge translation of evidence-based practices to policy makers and service providers.

All governments should promote higher levels of disability data quality, linkage and sharing through additional investments in critical data infrastructure. The NDIA should ensure that its data is structured to align with the International Classification of Function, to significantly enhance its usefulness for research. An evidence base reflecting the actual lifetime experiences of people with disability will improve the quality, effectiveness and responsiveness of the disability support ecosystem and foster a culture of continuous learning and improvement.

Australian Governments should build on and secure the long-term funding of the NDDA. A more comprehensive evidence-based picture of the disability support landscape would help the scheme demonstrate value for money to governments and taxpayers, and most importantly help participants improve outcomes and meet their goals in the most effective manner through innovative and evidence-based supports.

Recommendation 23: Measure what matters, build an evidence base of what works, and create a learning system

* Legislative change required

Action 23.1: National Cabinet should agree to replace the current NDIS Outcomes Framework with a new Disability Support Outcomes Framework. The new Framework should be focused on objectives and key results covering both foundational and NDIS supports. Accountability for disability support outcomes should include measurement against agreed objectives and key results to drive an agenda of continuous improvement. Development of the Framework should be designed with people with disability and led by DSS (see Action 20.5). The Framework should be included as a schedule in the new Disability Intergovernmental Agreement (see Action 20.1).
Action 23.2*: The Department of Social Services, in consultation with the National Disability Insurance Agency, the new National Disability Supports Quality and Safeguards Commission and the Independent Health and Aged Care Pricing Authority, should establish and manage an NDIS Evidence Committee to provide guidance on reasonable and necessary disability supports. The Evidence Committee should evaluate and recommend evidence based therapies that warrant NDIS support in the budget setting process or funding as foundational supports with regard to benefits, quality, safety, and cost effectiveness. The Evidence Committee should also advise agencies and governments on other evidence related dimensions of NDIS operations. The Evidence Committee should address gaps in scientific and economic insight by assessing and determining the efficacy and cost-benefit of supports for NDIS funding. Practice Standards for providers would ensure they deliver supports that are safe and evidence-informed, supported by regulatory oversight by the new National Disability Supports Quality and Safeguards Commission. The Evidence Committee’s annual work plan will be approved by Disability Reform Ministerial Council. While Evidence Committee deliberations on the merits of supports will be independent and transparent, any recommendations concerning the eligibility of supports for NDIS funding will require final approval by relevant decision-makers.
Action 23.3*: The Department of Social Services should establish a new Disability Research and Evaluation Fund to coordinate and fund research and independent evaluation activities. The Disability Research and Evaluation Fund (DREF) should be in the order of $40-50 million per year and consideration could be given to drawing on income from accumulated unspent funding from jurisdictions’ contributions to NDIS transition (the proposed ‘Reserve Fund’). The DREF should fund the development of a robust evidence base to inform the operations of the disability support ecosystem. This should include identifying and addressing knowledge gaps in Australian and international disability research relevant to the Australian context, and supporting the work of the NDIS Evidence Committee (see Action 23.2). The DREF should also fund greater data investment and research on the experiences of people with disability not eligible for NDIS to facilitate collaboration in the disability sector and improve the evidence underpinning effective support for people outside the scheme. The DREF should support and promote knowledge translation of best practice evidence by policy makers and service providers. It should support leading research initiatives and seek to build disability research capacity, so that Australia can become a world leader.
Action 23.4: All Australian governments should agree to jointly invest in actions to improve disability data quality and sharing. This should build on the Disability Reform Ministerial Council agreement for the NDIA to improve the quality, availability and use of disability data through additional investments and fostering a culture of continual learning. The NDIS data structure should be aligned with the International Classification of Function. All governments should invest in developing a robust evidence base on the lifetime experiences of people with disability. This should include investing in the National Disability Data Asset, a triennial Australian Bureau of Statistics Survey of Disability, Ageing and Carers, improved longitudinal insights into labour market barriers, and other disability related data. Governments should also commit to greater collection and sharing of data regarding foundational and mainstream services with linkage to other administrative datasets.
Action 23.5: The Australian Government should ensure that all disability reporting mechanisms facilitate the collection, analysis and publication of intersectional indicators. All people with disability should be able to easily access and understand information that is relevant to them. This means that all data and reporting mechanisms across the disability ecosystem must begin collecting, reporting and publishing against intersectional characteristics including First Nations, culturally and linguistically diverse, LGBTIQA+SB, gender and age as a minimum standard. This approach should be taken across the National Disability Insurance Agency, National Disability Data Asset (NDDA), ADS and the future Australian Bureau of Statistics’ Survey of Disability, Ageing and Carers (including population data, NDIS participants and complaint, government investments, outcome measurements and implementation tracking) (see Action 23.4). Where data is not collected or is not able to be published, a gap should be publicly identified along with a commitment to develop mechanisms to address the gap, through the appropriate data improvement plans. Funded efforts should also be made to adopt this minimum standard across the disability ecosystem including the NDDA.

A learning system that measures what matters and builds an evidence base of what works

On this page

Measurement of participant, social and economic outcomes is poor and the NDIS Outcomes Framework is not fit-for-purpose

Significant NDIS policy and operational decisions are not made on a transparent and independent basis

The NDIS has not been set up in a way that enables continuous learning

Research funding