Price lists, the curse of B2B platforms

In B2C, a product has a price. In B2B, the same drill costs 142.00 for one customer, 127.75 for a second and 118.40 for a third. Showing the right price, to the right buyer, on every page, is the quiet curse of B2B commerce.

Every platform on the market has picked a way to deal with it, and each one hides a cost. Nobody publishes what those costs are, so we measured them.

Why it is hard

Each customer’s price comes from a stack of lists. Their own contract comes first, then their customer group, then a base list that prices everything. The first list with a price for that product wins.

Now multiply. A listing page asks that question 48 times. Sorting a category by price asks it thousands of times. And the answer has to stay right while prices and contracts keep changing.

There are three common ways to do it. We built each one in its most favourable form and ran them on the same data, at 20,000 products and 2,000 customers, all on contract, and on larger sets where noted. Real platforms add queues, layers and network hops, so treat our figures as lower bounds. These are our own prototypes, not any vendor’s product. The details are at the end of this post.

Precompute: prepare every price in advance

Think of a printed price book for each customer. Customers who share the same lists can share a book, and a customer with their own contract gets their own.

The buyer gets the fastest answer of all: 0.015 ms to sort a category by their price. The bill arrives elsewhere, because every book holds a price for every product.

Small changes are fine: 67 ms for a new contract customer. The problem is the change that touches every book. A 3% change on the base list, with only 2,000 customers, means rebuilding about 2,000 books, which takes 44 seconds.

That is why these systems plan ahead. One platform ships with a job that runs every 5 minutes and starts rebuilding 12 hours before a scheduled price change. Its documentation warns that recalculation can take “multiple hours for several hundred thousand product prices”. On another platform with a similar design, a public bug report describes 30 million prices taking 4 hours, and multiple days after a regression, according to its author.

Request time: work it out when the buyer asks

Nothing is prepared. Each time a buyer opens a page, the platform checks the lists in order, then keeps the answer in a cache for next time.

Recording a change is quick, 89 ms for a whole group list, because the real work has not happened yet. Buyers keep seeing the old price until the scheduled job runs, and the first page after that pays the full price again.

A few milliseconds look harmless, but this is the floor, with nothing else on the page. A cache that covers everyone is also large: 40 million entries at 20,000 products and 2,000 customers, 500 million at 50,000 products and 10,000 customers.

There is one more catch. The search engine only knows the public price, so when a logged-in buyer sorts search results by price, they get the order of the public price, not their own.

Search index: put every price in the search engine

This fixes that catch. Every product in the search engine carries one price per customer or contract, so search can sort on the buyer’s own price.

The search engine pays for it. Its index grows 540 times, from 0.8 MiB to 415 MiB, with 20,000 products and 2,000 customers.

Search engines are not built for this. The one we used refuses more than 1,000 fields per index by default, to prevent what its makers call a “mapping explosion”. We had to raise that limit. In an earlier single-pass run at 50,000 products, on Elasticsearch 8.19.3, adding one customer crashed it with 2 GB of memory, and took 52 seconds with 4 GB.

Others have hit the same wall. One platform considers only the first 100 price rules by default, and sorting can be wrong beyond that. A hosted search service needs an extra copy of the index per customer group for each price sort. Another platform has to move prices to a separate index past 1,000 contracts.

Side by side

Each of the three strategies picks which operation will hurt, then moves the pain somewhere a demo does not look. That is the curse. ANOM, in blue, is the subject of the next section.

And ANOM?

We ran the same scenarios, with the same data and the same prices, on ANOM, on the same server. Unlike the prototypes, ANOM ran with its full production data model and all its constraints, and we checked every price it returned.

The first question is the one buyers feel: after a price change, how long do they keep seeing the old price?

Smaller changes shrink the window without closing it: 522 ms on Precompute and 739 ms on Search index when one group’s list changes, 67 ms and 20 s when a new customer signs a contract. These are best cases, because our rebuild starts the instant the change lands. Real platforms run it from scheduled jobs or queues, so the real window is that delay plus these figures.

ANOM serves the new price from the moment the change is saved, with no scheduled job to wait for. While it finishes absorbing the change, prices stay correct and pages take longer to serve.

The times below measure how long ANOM takes to fully absorb the change.

With 2,000 customers, ANOM absorbs it in 3.1 seconds, about 14 times less than Precompute and 4 times less than Search index. Request time records the change faster, in 231 ms, but keeps serving the old price until its job runs. With 50,000 products and 10,000 customers, ANOM takes 2.8 seconds, about 35 times less than Precompute’s 99 s.

With 50,000 products and 10,000 customers, ANOM is ready in 1.5 seconds, against 93 s for Precompute, and needs 130 MiB against 3.2 GiB. With 20,000 products and 10,000 customers, all on contract, ANOM is ready in 12 seconds and needs 1,067 MiB.

Not every line goes ANOM’s way. A price already stored or cached reads faster, and a book written in advance absorbs a small change faster: 77 ms for a new contract customer at 50,000 products, against 260 ms. That is what those strategies were built for. ANOM leads where their bill comes due: preparing, storing, and changing prices in bulk without leaving buyers on the old ones.

Many buyers at once

So far, one buyer at a time. On a busy morning, hundreds of buyers price their baskets at the same moment. We sent baskets of 10 products, from 1 up to 10,000 buyers at once, at 20,000 products and 2,000 customers, all on contract. The search index ran on Elasticsearch, in two releases: 8.17.0, and 9.5.6, the latest at the time of testing.

From 10 buyers upward, ANOM prices more than 1.1 million baskets per second, and still 1.2 million with 10,000 buyers at once. Precompute levels off at about 310,000. The search index tops out below 4,000, then gives way at 10,000 buyers.

ANOM answers each basket in 8 µs at the median, whether 10 or 10,000 buyers are waiting. On the search index, the median climbs from 2 ms with one buyer to 263 ms on 8.17.0 and 291 ms on 9.5.6 with 1,000, and Elasticsearch 9.5.6 reaches 535 ms at 10,000. At 10,000 buyers, Elasticsearch was using about 4 of its 6 cores and 4.6 GB of memory.

ANOM and Precompute were called directly, while the search index can only be reached over HTTP, from a container with 6 of the server’s 12 cores.

The podiums

All four ran together at 20,000 products and 2,000 customers, all on contract. Here is the podium for each test. Request time is not ranked, because its cache keeps serving the old (stale) price until its scheduled job runs.

Four questions for any platform

• Is a new price served as soon as the change is made, or after a job, a rebuild or a cache expiry?
• What does a 3% change on everything cost, with your number of customers?
• Do search and sort use the buyer’s own price, or the public one?
• Does storage multiply with every contract you sign?

Ask for the numbers, on your catalogue and your customers. As we found, nobody publishes them. Now you have ours.

How we tested

• Data. Synthetic. A customer can have a contract list, belongs to one of 50 group lists covering 20% of the catalogue, and sees a base list covering everything. Typical scenarios: one customer in five has a contract, covering 2% of the catalogue. Worst case: every customer does, covering 10%.
• Requests. Listing pages of 48 products, on categories of 4% of the catalogue (typical) or 20% (worst case). Reads are warm: after a warm-up pass, each request is played 3 times and the best kept, then the median is reported. Writes are the median of 3.
• Load. 2,000 different baskets of 10 products, one warm-up pass, then 5 seconds at each step: 1, 10, 100, 1,000 and 10,000 buyers at once.
• Price change. The general change we timed was a +3% on the base list. A -3% changes just as many prices. On Precompute, we count it as a full rebuild of all the books, the most favourable way to do it.
• Machine. One Apple M4 Pro server, 12 cores, 24 GB. The search engine is Elasticsearch on a single node, in a container with 6 CPUs and 8 GB. Build, storage, reads and writes ran on 8.17.0 with 4 GB of memory, on default settings except the field limit. The load test ran on 8.17.0 and 9.5.6 with their default memory. Only its own time is counted.