EdReports is pivoting. Here's what you need to know.

Under fire for its flaws, EdReports announced changes in March, 2024. Has anything changed since?

Last January, we launched the Curriculum Insight Project, partly in response to shortcomings in curriculum reviews by the K-12 standard-bearer EdReports. A wave of journalism about the issues followed, and by March, EdReports had signaled a pivot.

What has happened since then? EdReports has been mostly-quiet. Its planned change have not, in fact, reached the world, as I predicted.

Fortunately, savvy states have moved on from using EdReports as a standard. In fact, EdReports has almost become irrelevant when groups like the CCSSO IMPD curriculum leaders gather. Still, most states continue to rely heavily or entirely on its reviews.

I’m republishing my update from March, 2024. It’s still reflects the state of things at EdReports.

---

Recently, EdReports, the widely-used curriculum review site, has been under fire over inconsistencies in its reviews.

This has spawned a great deal of discussion – almost none of it defending EdReports. In particular, we have seen a dearth of EdReports defenders with regard to its reviews of basal programs and Bookworms. Educators aren’t chiming in to say, “Basal reading programs are actually high-quality.” In fact, the opposite.

Turns out even EdReports acknowledges the critiques.

In a recent column, Eric Hirsch of EdReports announced plans to shift its review strategy, saying, "We’ll be evaluating how to make our reports more responsive to the rapidly evolving curriculum space and considering stakeholder feedback on topics including usability and volume of content." Of note: critiques about "volume of content" are at the heart of concerns of basal programs.

A memo to its reviewers is more pointed: “We’re most vulnerable to criticism around our reviews of basals / big box programs. We need to be particularly intentional in this area.”

At this point, it's a consensus position: EdReports got its reviews of basals wrong.

Literacy experts think EdReports got Bookworms wrong, as well. And close watchers should note that Fishtank ELA earns a recommendation from the Knowledge Matters Campaign, but failed to earn all-green from EdReports. It's hard to miss the daylight between experts and EdReports.

Are these issues clear to the states whose adoption lists mirror EdReports, and the many districts who’ve been told to consult EdReports as a gold standard?

Because these admissions come just after EdReports was the main input into the state list in Ohio, and its all-green list nearly carried the day in Wisconsin.

Here’s what folks need to know about this quiet pivot.

What are the specific issues with EdReports ELA reviews?

In brief:

EdReports has given better-than-deserved ratings to a number of basal programs, including Into Reading, Savvas MyView, Wonders, and Benchmark Advance.

EdReports has given a lower-than-deserved rating to Bookworms – the curriculum with extensive evidence that it improves reading outcomes – and to Fishtank ELA, many would say.

Most likely, this points to design flaws in its process. EdReports reviews are conducted by 4-5 educators who don’t use the materials, and receive as little as 25 hours of training. That’s… not a lot of training, when you consider the complexity of curriculum.

Also, the overrated basals generally came from big publishers whose older curricula earned poor reviews. It appears that they figured out how to game the EdReports review system with bloated programs of mixed quality. Abby Boruff expained it perfectly.

We haven't seen concerns about the reviews in the "all-red" or "all-yellow" category. When EdReports reviewers see big problems with a program, the field generally agrees.

There are broader concerns about the EdReports approach. The organization only reviews curriculum for standards alignment, which may be a dated lens on quality, by any estimation. EdReports doesn’t evaluate for usability and efficacy. And frankly, it’s hard to use its reviews to compare programs. They aren’t particularly clear or detailed.

Natalie Wexler’s comprehensive reporting unpacks all of these layers (and more); it's a must-read if you want to understand the issues.

Are these critiques new?

The national conversation about these issues is new, but the concerns aren’t new at all.

When EdReports started giving all-green reviews to basal programs, a fair amount of shock reverberated in the curriculum community. By 2021, Student Achievement Partners had published a critical review of Wonders, intended to illustrate broader issues with basals. It drew a fair share of attention in high-quality curriculum circles.

Pushback on its Bookworms review began years ago. You can find literacy experts praising Bookworms in 2018, in the wake of one middling review. Critiques of its Into Reading review intensified when New York City adopted Into Readingas part of the NYC Reads initiative.

The critiques have been more intense in private. A veritable Who’s Who of literacy curriculum experts and researchers can share stories of reaching out to EdReports about their concerns – and being dismissed or ignored. I’ve spoken with numerous well-regarded leaders who brought concerns to EdReports, in the years leading up to the public outcry.

Today, EdReports writes that its critics are being “adversarial, rather than collaborative,” yet the organization has a demonstrated pattern of resistance to feedback, even from the leading experts in the field. That should give us pause as EdReports announces plans to make revisions.

What will change – and When?

EdReports has said that the organization will be “revising all of its core comprehensive review tools in 2024.” That work is just starting, and I’m not betting that it will be swift.

After revising its review tool, EdReports will need to complete new reviews with that tool. That should add another six months, minimum, before any better reviews reach the field.

The most essential question: will EdReports be re-reviewing programs?

EdReports has no history of re-reviewing a curriculum unless its publishers request a re-review and submit updated materials. Underrated programs like Bookworms have an incentive to request re-review. All-green basals do not.

If EdReports does re-review ELA curricula: well, there are currently 11 all-green programs, plus others like Bookworms and Fishtank that need another look. If EdReports decides to re-review more than a dozen programs, at its current pace, we are looking at 2026 (or later) before we begin to see real review coverage of the ELA landscape. And EdReports has announced that it will also be revisiting math and science reviews, raising additional questions about its ability to focus on the mess in ELA. (I should note that many point to a similar mess in math.)

In any case, “2026 at best” feels faraway, given the current legislative push for improved reading instruction, which spans 45 states (and growing).

What can we use in the meantime?

Today, there are emerging alternatives – which suggests that multiple organizations had already concluded that EdReports reviews weren’t an adequate guidepost.

In my opinion, the Knowledge Matters Campaign currently has the best short list for K-8 ELA. Its mission is to advance the cause of knowledge-building curriculum, but that’s an expansive goal. Its curriculum review tool crosses multiple factors, including text quality, text-centered lessons, writing instruction, and more, and its curriculum reviews do cover foundational skills.

The Curriculum Insight Project will soon release reports on six programs, and I hope you are following its work.

The Reading League also has five curriculum reviews underway, to debut in May, 2024.

But I’ll be honest, I don’t think we have the resources we need to really support schools and districts with understanding the current curriculum landscape. My best evidence is my texts/emails/DMs, which are full of excellent questions that honestly aren’t well-answered by existing resources.

How can we better support 14,000 districts in a moment of significant change in the curriculum landscape, as well as local efforts to move away from balanced literacy programs? How do we make sure schools can find options for professional learning that align to their new curricula? These questions genuinely keep me up at night – more so each year, as we see basal programs surging in market share – a situation enabled by flawed reviews.