Nearly nine months after its rollout, the University of Washington’s $340 million finance system upgrade is still getting mixed reviews.  

On the plus side, the backlog of unpaid vendor invoices snarled by Workday, UW’s new “enterprise” software platform, has been cut from $90 million shortly after the July 6 launch to $43 million as of Feb. 29.   

But Workday, which replaced hundreds of legacy systems, is still disrupting UW’s portfolio of research grants, which cover nearly a fifth of its $10.4 billion annual budget and funds everything from experiments in advanced physics and climate change to cancer research at UW Medical Center.

Officials say the university, the second biggest recipient of federal funding behind Johns Hopkins University, hasn’t permanently lost any of that money.

But hundreds of new grant awards are still hung up in processing due to Workday issues. Existing funds, meanwhile, now are often hard to track, making it difficult for faculty to cover lab expenses or plan for future research — a mortifying outcome for one of America’s top research schools and the fifth biggest employer in the state.

“For such a major research institute, this seems crazy,” said Olivia Bermingham-McDonogh, a professor in the department of biological structure who relies on National Institutes of Health funding and who has struggled with staffing and planning under the Workday system

Faculty say problems are even worse than for the 2017 upgrade of UW’s payroll system by Workday, which is made by a California firm of the same name.

Frustration is so high over Workday that 39% of grant-dependent faculty have applied for fewer grants due to Workday, according to a survey late last year by the UW Faculty Senate.

UW officials say most of current Workday upgrade, formally known as the UW Finance Transformation, remains under budget and on track and they expect grant and vendor backlogs to fall to “normal” levels by July.

Sarah Norris Hall, newly appointed senior vice president and chief financial officer who oversees Workday stabilization efforts, said there’s no indication faculty are applying for fewer grants.

But UW has also acknowledged what many faculty had long suspected: that Workday went “live” before many critical processes, including those related to grants, were fully completed or tested, according to a January report by a management consultant.

The consultant, Bluecrane, puts part of the blame on the sheer scale of the eight-year project, which aimed to replace or modernize around 800 legacy systems, many of them decades old, all while coping with a global pandemic and an IT labor shortage.

But the 58-page report also details numerous missteps in the massive project.

Virtually from the start, the Workday upgrade struggled under ill-defined priorities, delays in critical decisions and an overreliance on “experts” and “salespeople,” according to Bluecrane.

More broadly, UW and its primary consultant, Deloitte, never adequately mapped out how Workday would interact with the many legacy systems UW planned to keep, Bluecrane said.

That made it hard to anticipate impacts on key operations, such as grants, or to prepare UW employees, vendors and others for what was expected to be a “once-in-a generation” transformation.

UW resolved some of those issues in a 2021 strategic “reset” of the project, which brought in new leadership, added $71 million and postponed the launch by a year. 

But the overhaul didn’t solve another issue: a mismatch between what UW needed and what Workday could readily deliver, according to Bluecrane as well as current and former UW officials. 

Although Workday is the top seller of so-called enterprise resource planning software to universities, these complex products were originally developed for corporate customers, whose businesses processes and organizations tend to be more orderly and thus more amenable to automation. 

They were less suited to the intricacies of large grant-funded universities, which often feature a hodgepodge of technologies and decentralized management, according to UW officials and industry experts.

“There isn’t any [enterprise resource planning] system in higher education that’s particularly good at grants, especially for very large universities,” said Vince Kellen, chief information officer at the University of California-San Diego, which completed its own finance upgrade in 2020. 

Perhaps not surprisingly, that was also the verdict of Chris Mercer, who oversaw the UW finance upgrade from 2021 through 2023, and who noted soon after the chaotic launch that Workday’s “higher-education specific modules such as those for grant management are less developed.”

Early on, UW had raised “concerns about whether the proposed [software] configuration would truly meet UW’s business needs,” but ultimately relied on “advice and counsel” from Workday and Deloitte that Workday could be adapted for UW’s finance system, according to Bluecrane.

Workday didn’t dispute concerns over its grant-based capabilities, but in a statement last week noted that, “as with any deployment of this size, there are numerous processes and resources that help support the stabilization effort.” 

Workday also noted it has 400-plus higher education customers worldwide, including a third of “the top research universities — and a significant number that are successfully live on Workday Grants Management.”

A Deloitte spokesperson said UW had asked “to handle all media inquiries about this project.”

In the end, despite the budget increase and delay, and the hiring of another technical consultant, the Workday upgrade fell short in four of eight main areas: procurement and supply chain; financial reporting; budgeting and forecasting; and the grants process. 

Further, UW wasn’t fully aware of the severity of these deficiencies because there hadn’t been enough time to fully test every new or updated element in UW’s “Workday ecosystem” before the July 6 “go-live” date, according to Bluecrane and former UW officials.

Compounding problems, staff and faculty hadn’t been adequately trained on the new Workday systems. “When the new system went live, many staff realized that, even though they may have attended some training, they did not really know how to perform their day-to-day duties,” Bluecrane said.

UW officials considered postponing the launch a second time, but ultimately rejected it as too costly, said Norris Hall, the UW chief financial officer.

UW isn’t the only school to struggle with an upgrade. In October, the University of Wisconsin postponed its Workday launch by a year following a program that “has been marked by delays,” according to the college newspaper. 

In 2020, Ohio State University delayed parts of a Workday rollout by six months and in 2021 “walked away” from plans to install Workday’s student services software. 

Faculty and staff have also reported Workday issues at Washington State University, which went live in 2020, and the University of Ottawa, which launched last year. 

These rollouts “are really difficult on any platform,” said David Kieffer, an analyst at The Tambellini Group, which advises colleges on technology issues. “Frankly, we haven’t seen very many that handle that kind of [grant] volume well without issues and customization.”

Those issues aren’t limited to universities. UW’s Workday upgrade is part of broader move to modernize by Washington state’s aging IT infrastructure, a nearly $2 billion effort that has struggled with missed deadlines and rising costs.

In May, the Office of Financial Management, the state’s budget office, said its Workday upgrade of the state’s accounting and reporting system, initially expected to launch in 2022 at a cost of $144 million, would be postponed to July 2025, with a cost $284.5 million.

Since UW’s rocky launch, the school has pulled out the stops to fix problems. It shifted leadership, cut backlogs with temporary staff and more automation — with a dashboard showing progress — and created ways for faculty to “escalate” Workday issues threatening research or other key activities.

Faculty say that’s brought some relief, but they’re still seeing problems in key areas.

For example, because of the backlog for the “setup” of new awards in UW’s systems, researchers may wait several months to actually use the funds, said several UW faculty members, including Bermingham-McDonogh, the researcher with the department of biological structure. As of Mar. 18, the setup backlog stood at 659. In 2023, UW had a total of 5,506 awards representing $1.87 billion.

UW officials cautioned that not all backlogged awards have affected research spending — in some cases, for example, expenses can be invoiced later or paid out of other departmental funds, though UW officials noted some departments might lack those funds or not be comfortable using them.

Reporting problems, meanwhile, have left some faculty unable to track funds, or be certain of covering lab expenses or salaries for the graduate students and others who do much of the research.

“We’re talking about students here, junior research scientists who need this funding for their families,” said Robert Wood, a professor in the department of atmospheric sciences and former associate dean for research. “You just don’t really know what funding we’ve got, because it’s so difficult to get reports.”

And while faculty and staff have devised some “workarounds” for Workday deficiencies, many of those hacks will eventually have to be reconciled when Workday is eventually stable.

“Most staff still think that the higher-ups at UW don’t see the anxiety they have created,” said Rick Keil, director of the school of oceanography.

UW officials acknowledge that Workday problems remain “highly impactful and challenging.” But their commitment to upgrade appears unshaken.

Simply put, a 21st century research institution can’t operate with a 20th century financial system, which was not only vulnerable to security threats and accounting errors, but had also fostered an inefficient organizational structure. 

“For a university of this size, scale and scope, not to have a unified financial system was a tremendous risk that just couldn’t be accepted,” said Tricia Serio, UW’s provost and executive vice president for academic affairs, during a February town hall.

The job now “is to find the path forward so that we can realize the benefit of having this unified system,” said Serio, herself a grant-funded researcher. “And I’m really confident that we will get there.”

What’s not clear is how many of UW’s faculty and staff are willing to wait. According to the faculty senate survey, 59% of grant-dependent faculty had considered leaving the UW because of Workday.

“I [hear] it all the time — ‘I’m getting a job elsewhere.’ ‘I’m retiring early,’” said Eric Steig, chair of UW’s earth and space sciences department, of Workday frustrations of faculty members. “It really, really worries me.”

Paul Roberts: proberts@seattletimes.com;