SAP Says Joule Made Its Own Staff 20% More Productive — Measure Your AI Gains Before You Buy the Number

SAP Says Joule Made Its Own Staff 20% More Productive — Measure Your AI Gains Before You Buy the Number

2026-10-07

At its SAP Connect conference in Las Vegas on October 6, SAP said its new AI interface, Joule Work, is live with 110,000 of its own employees, “with the company seeing 20% productivity gains across finance, HR and procurement” (SAP News Center, October 6, 2026).

That’s a big, clean number. It’s also a number with no published method behind it.

The takeaway for operators: vendor productivity figures are a reason to run a test, not a reason to sign a contract. The only AI gain that matters is the one you measure on your own work, against your own baseline.

What SAP actually announced

Joule Work is an AI front end for SAP software. Employees ask for things in plain language, and it pulls answers and completes work across SAP and third-party data. Alongside it, Joule Assistants carry out operational steps inside business workflows using existing permissions and audit trails, and SAP says customers control how much autonomy they get (SAP News Center).

The customer stories are real but early. Novartis describes a newly launched pilot. Peter Wied, Director of Complexity Management at Maschinenfabrik Reinhausen, says work that “once required up to an hour of manual effort” is now “accessible in seconds.” Keith Smith, SAP Platform Architect at CONA Services, says his team looks forward to “exploring and testing Joule’s capabilities in greater depth over the coming months” (SAP News Center).

Timing is also mixed. CIO reports that general availability of the assistants, Joule Studio, and Joule Work begins this month (CIO, October 6, 2026). Constellation Research notes that much of what was announced lands in SAP’s Early Adopter Care program (its early-access track) in the first quarter of 2027 (Constellation Research, October 6, 2026).

What independent voices said

The reaction was split, which is healthy.

  • Thomas Randall, research lead at Info-Tech Research Group, called the announcements “fast-following” rather than a breakthrough, while noting SAP’s real advantage: its systems are already trusted to move money and release orders (CIO).
  • Jason Andersen, principal analyst at Moor Insights & Strategy, was more upbeat about role-specific agents for areas like procurement and supply chain (CIO).
  • Carmi Levy, an independent technology analyst, warned of “a wide gap between plans and reality” and said SAP didn’t explain how it handles attacks that trick AI agents with malicious instructions (CIO).
  • Jens Hungershausen, chairman of DSAG, the German-speaking SAP user group, said there is “still a gap between the vision, the available solutions, and actual customer benefits,” and the group said reliable customer experience with Joule Work is still lacking (Constellation Research).

When SAP’s own user group says “show us,” that’s your cue too.

Why self-reported gains deserve a second look

The best-known cautionary tale comes from METR, a nonprofit AI research group. In a 2025 randomized trial, experienced open-source developers took 19% longer on real tasks when allowed to use AI tools, yet afterward still believed AI had sped them up by 20% (METR, July 10, 2025).

To be fair, METR now says those results are out of date for current tools, and it studied software development, not finance or HR. The lesson isn’t “AI makes people slower.” It’s that how fast work feels and how fast work is can point in opposite directions. A 20% figure without a method could sit anywhere on that spectrum.

How to measure AI productivity on your own team

  1. Pick three to five real workflows. Choose specific, repeatable tasks: closing a month-end reconciliation, answering a supplier inquiry, building a headcount report. “Finance” is not a workflow.
  2. Record a baseline first. Before anyone touches the tool, measure time per task, error rate, and how often work gets redone. No baseline, no ROI.
  3. Count the review time. If the AI saves 20 minutes and a manager spends 15 checking its output, you saved five. Rework and corrections belong in the math.
  4. Run a side-by-side. Have part of the team use the tool and part work as usual for a few weeks on comparable work. It’s not a lab trial, but it beats a survey.
  5. Measure outcomes, not logins. Adoption counts and “time saved” estimates from the vendor dashboard are activity metrics. Track cycle time, accuracy, and cost per task instead.
  6. Ask the vendor for the method. When you hear “20% productivity gains,” ask: which tasks, measured how, compared to what, and including review time or not?

Soft next step

SAP may well be onto something, especially for back-office work its systems already run. But a headline number from a vendor’s own deployment is a hypothesis about your business, not a fact about it. Measure before you scale, and let your own baseline decide.

Yellow Coop helps owners and operators design AI pilots, set up the metrics, and pressure-test a vendor’s ROI claims before they commit — including a clear answer on who should own AI ROI. See the measured results in our track record, or start at contact.

Internal links: Innovate, What We Do, How We Engage, CIO vs CTO vs CISO, Track Record, Insights.

Sources