“Novelty wears off” states Gina Ricker, Senior Program Officer with the Bill & Melinda Gates Foundation, in a recent webinar (1). Bringing new technology such as AI to the classroom is not enough to get investors interested nowadays. Even an early stage EdTech start-up has to show evidence of impact, a demand that gets more stringent as the company matures. The type of evidence obviously depends on the nature of the product, and it doesn’t necessarily involve a two-year, full scale Randomised Controlled Trial. In fact, it hardly ever does.
Investors in EdTech are well aware that very few new products introduced in education actually succeed. The ones that do succeed are the ones that have taken great care to make sure that they improve learning. That’s why VC’s specialised in edtech like Owl Ventures and Learn Capital want to see evidence before they invest, and help their companies expand on that evidence post-investment. VC’s and other investors do not expect early stage start-ups to present full scale, academic RCTs to qualify for funding. However, start-ups are required to present a logic model that specifies what outcome they expect, how this is achieved, and on what research it is based. The advantage of having a logic model early in the development process is that the product can be designed to monitor the metrics necessary to validate efficacy claims. It also allows for features like A/B-testing to be built in.
Proximal outcomes like student engagement and support of teachers can serve as early indicators for investors of the product’s effectiveness. Because improving learning outcomes is known to be hard, start-ups are encouraged to run trials during the development process and improve their product iteratively.
Once the start-up matures into a scale-up, it is supposed to produce experimental evidence backed by third parties. Even then, investors like multiple, short-cycle research studies rather than one large RCT. As Libby Hills of the Jacobs Foundation noted in a recent podcast : ‘evidence is a process, not a single shot. Evidence of effectiveness is gathered in a portfolio that is constantly evolving, giving rise to insights that can be used to improve the product and the way it is used’ (2).
The micro-RCT’s of WhatWorked Education fit in neatly with the idea of a dynamical, evolving evidence portfolio. A micro-RCT is easy to set up and the process is simple enough for a teacher to complete the trial without in-person guidance. The evaluation is run for 6-8 weeks so they can be completed within one term. Typically several evaluations are run in parallel. The results from these individual evaluations are aggregated to generate an accurate assessment of the effectiveness. The results become available as soon as the last trial finishes as the statistical processing is completely automated. This quick evaluation of effectiveness allows for the rapid-cycle development that experienced EdTech investors are looking for.
1. Investing in Success: Leveraging Evidence to Secure EdTech Funding - webinar by Instructure, October 16, 2024
2. Sights on EdTech with Luigi Morino - podcast with Libby Hills and John Soleanicov from the Jacobs Foundation, February 7, 2024
