Hire Software Developers 7
Back to blogs

The double-booking bug: optimistic concurrency and the race you only see in production

Two vivid blocks jammed into a single socket among pale blocks that each have their own, representing two writes claiming the same record

The double-booking bug: optimistic concurrency and the race you only see in production

Two customers book the last slot, eleven milliseconds apart. Both requests read available = 1. Both decide the booking is allowed. Both write remaining = 0. Both get a confirmation email, and on Thursday morning two people show up for one appointment.

The code passed review. There was a transaction around the write. Nobody asked whether it also enclosed the read the decision was based on, because that question has no shape you can see in a diff.

Optimistic concurrency control is a family of schemes that take no locks. A transaction reads freely, writes to local copies, then validates at commit time that nothing it read changed underneath it; a failed validation aborts and restarts. [R2] [R6]

EF Core states the bet out loud: optimistic concurrency "assumes that concurrency conflicts are relatively rare," and "arranges for the data modification to fail on save if the data has changed since it was queried." [R59] Good bet, most systems, most of the time. This is about the hours when it isn't.

Key takeaways

  • The reviewer checks that a transaction exists; the defect is almost always in what it encloses. All 22 of the attacks ACIDRain verified manifest under the default isolation guarantees of popular transactional databases, and 17 of those 22 manifest even under the strongest transactional guarantees those databases offer, because of incorrect transaction usage. [R72]
  • Your default isolation level is probably not what you assume: Read Committed in PostgreSQL and Oracle, Repeatable Read in InnoDB, READ_COMMITTED_SNAPSHOT off on SQL Server but on in Azure SQL Database. [R35] [R46] [R48] [R50]
  • The 1981 paper everyone cites for optimistic locking bounds its own recommendation to systems "where transaction conflict is highly unlikely," names starvation as the standing cost, and measured nothing. [R10] [R7] [R12]
  • Reaching for a lock is the reflex and often the wrong fix: about 73% of the examined non-deadlock concurrency bugs in Lu et al.'s study of 105 real-world bugs were not fixed by simply adding or changing locks. [R15] [R20]
  • Nobody has published a rate at which lost-update races happen in production. The three figures people reach for are a stress test, a vulnerability count, and a sample of filed bug reports. [R68] [R71] [R16]

The lost update problem: the bug that passes code review

Look at the handler the way a reviewer does. A SELECT that counts remaining capacity. An if. An UPDATE inside a transaction, committed properly, with error handling. Every line is correct, and still correct when a second request runs the same three steps at the same moment. None of the failure lives in any single line.

The gap is between the read and the write. A reads 1. B reads 1 a millisecond later, before A has written anything. A decides yes and writes 0. B decided yes on a value that was true when it read it and false by the time it acted, and writes 0 too.

Berenson and colleagues named this shape in 1995. [R27] P4, Lost Update: "The lost update anomaly occurs when transaction T1 reads a data item and then T2 updates the data item (possibly based on a previous read), then T1 (based on its earlier read value) updates the data item and commits." [R29] Their worked history, r1[x=100] r2[x=100] w2[x=120] c2 w1[x=130] c1, makes the point: "even if T2 commits, T2's update will be lost." [R30]

There's a second shape, subtler, and it survives the fixes people reach for first. A5B, Write Skew: T1 reads x and y, T2 reads x and y and writes x and commits, then T1 writes y. "If there were a constraint between x and y, it might be violated." [R31] Their own example isn't exotic. It's a bank, "where account balances are allowed to go negative as long as the sum of commonly held balances remains non-negative." [R32] Two withdrawals, each individually legal, jointly illegal. Nobody overwrote anybody. The invariant broke anyway, because it spanned two rows and neither transaction could see the other's write.

Both shapes have the same review signature: the code reads, decides, writes, and the decision is the unprotected part. A reviewer is scanning for missing error handling, an N+1, a missing index. "Does the transaction around the write also cover the read, and does it stop a concurrent transaction changing what that read returned?" is not a question a diff invites.

Nor is this a distributed-systems problem. One Postgres instance, one table, one integer column. Two users, not microservices.

Why it survives review — and why raising the isolation level doesn't save you

In 2017, Warszawski and Bailis published ACIDRain, which did what this field mostly doesn't: it went and looked. [R69] They pointed a prototype analysis tool at 12 popular self-hosted eCommerce applications, written in four languages, deployed on over two million websites and covering "over 55% of eCommerce sites on the Internet," WooCommerce alone accounting for 39% of all online stores. [R71] [R73] They found and verified 22 critical attacks, ones "that allow attackers to corrupt store inventory, over-spend gift cards, and steal inventory," across all but one application tested. [R71] [R73]

The finding that should change what you ask for in review:

All 22 vulnerabilities manifest under the default isolation guarantees of popular transactional databases including Oracle 12c, and 17 vulnerabilities — due to incorrect transaction usage — manifest even under the strongest transactional guarantees offered by these databases. [R72]

Seventeen of ACIDRain's 22 verified vulnerabilities are not isolation-level bugs. They're transaction-scope bugs. Turning the dial to maximum leaves most where they were, because no database setting fixes a boundary drawn in the wrong place. That's the empirical answer to "how did this pass review": the reviewer checked for the presence of a transaction, and presence was never the variable.

And here's why staging doesn't catch it. ACIDRain's abstract carries the sentence that explains every one of these incidents:

While low transaction volumes mask many potential concurrency-related errors under normal operation, determined adversaries can exploit them programmatically for fun and profit. [R70]

Volume masks the error. Your integration tests run one request at a time. Your staging environment has three users, two of whom are you. The interleaving that breaks the booking needs two writers on the same row within a few milliseconds, and the only place that reliably produces it is the environment with real customers in it. The bug isn't rare in the code. It's rare in the places you'd see it.

An attacker, of course, doesn't wait for the coincidence. They send the requests concurrently, on purpose, which turns the same defect into a repeatable exploit.

Database isolation levels: your default is not what you think

Ask five engineers what isolation level production runs at. You'll get three answers, one naming a level their database doesn't default to.

PostgreSQL: "Read Committed is the default isolation level in PostgreSQL." [R35] InnoDB: "The default isolation level for InnoDB is REPEATABLE READ." [R46] Oracle: read committed, where a query "sees only data committed before the query — not the transaction — began," and "this isolation level is the default." [R48] SQL Server defaults to read committed with READ_COMMITTED_SNAPSHOT off, but "READ_COMMITTED_SNAPSHOT ON is the default on Azure SQL Database." [R50] Same queries, different behaviour on-prem and in Azure, and that difference isn't in your repository.

PostgreSQL's manual is honest about what its default doesn't give you. An updating command "can see the effects of concurrent updating commands on the same rows it is trying to update, but it does not see effects of those commands on other rows," making Read Committed "unsuitable for commands that involve complex search conditions." [R36] Underneath sits the re-evaluation rule: when a second updater unblocks, "the search condition of the command (the WHERE clause) is re-evaluated to see if the updated version of the row still matches. If so, the second updater proceeds." [R37] Your UPDATE ... WHERE remaining > 0 doesn't fail because the first writer got there. It re-checks, then proceeds.

A vocabulary note, because it costs teams time. PostgreSQL's documentation never uses the phrases "lost update" or "write skew," in either the transaction isolation or the application-level consistency chapter. Its term of art is serialization anomaly, a committed group of transactions whose result is "inconsistent with all possible orderings of running those transactions one at a time." [R45] Search the manual for Berenson's words and you'll find nothing; they cover it under another name.

One level up, PostgreSQL's Repeatable Read "prevents all of the phenomena described in Table 13.1 except for serialization anomalies" and "is implemented using a technique known in academic database literature... as Snapshot Isolation." [R40] Stronger than the standard requires, and still not enough for an invariant spanning rows: "Attempts to enforce business rules by transactions running at this isolation level are not likely to work correctly without careful use of explicit locks." [R42] Berenson agrees from the other side — snapshot isolation "admits history anomalies that REPEATABLE READ does not," write skew among them. [R33]

MySQL's Repeatable Read is a different animal again: the snapshot rule covers non-locking reads, while locking reads, UPDATE and DELETE follow a separate index-locking regime. [R47] Oracle's SERIALIZABLE permits a row change "only if changes to the row made by other transactions were already committed when the serializable transaction began." [R49] Oracle never calls that snapshot isolation; saying it is snapshot-based in substance is analysis, not an Oracle quote. And ACIDRain reports that "Oracle's flagship offering" does not offer serializability as an option at all. [R85]

Then the trap that catches almost everyone who has "fixed" one of these. At non-serializable levels PostgreSQL tells you to use "SELECT FOR UPDATE, SELECT FOR SHARE, or an appropriate LOCK TABLE statement" [R38] — then prints the sentence that quietly invalidates the usual fix:

SELECT FOR UPDATE does not ensure that a concurrent transaction will not update or delete a selected row. To do that in PostgreSQL you must actually update the row, even if no values need to be changed. [R39]

Someone on your team has shipped a SELECT ... FOR UPDATE and closed the ticket. That lock is doing less than they think. Berenson's group diagnosed the root of this thirty years ago: "The three ANSI phenomena are ambiguous... In particular, lock-based isolation levels have different characteristics than their ANSI equivalents." [R28] The level names are shared vocabulary for unshared behaviour.

What the 1981 optimistic concurrency control paper actually says

Optimistic concurrency control has a canonical citation: Kung and Robinson, "On Optimistic Methods for Concurrency Control," ACM TODS Volume 6 Number 2, June 1981, out of Carnegie-Mellon. [R1] Design docs cite it as the authority for "we'll use optimistic locking here." It says something narrower than the folklore version. Their methods "are 'optimistic' in the sense that they rely mainly on transaction backup as a control mechanism, 'hoping' that conflicts between transactions will not occur," and during the read phase "all writes take place on local copies of the nodes to be modified." [R2] [R6]

Their motivating argument is worth stealing, because it's about where locks earn their cost. Of five listed disadvantages of locking, the fifth is labelled "Most important for the purposes of this paper": "locking may be necessary only in the worst case," since with n roots in a graph and two processes running transactions at the same rate, "locking is really needed (if at all) every n transactions, on the average." [R3] [R4] That holds under the conditions they state — graph nodes very large relative to those involved in running transactions, and a small probability of modifying a congested node. [R5] The conclusions then bound the recommendation explicitly: "These methods may well be superior to locking methods for systems where transaction conflict is highly unlikely. Examples include query-dominant systems and very large tree-structured indexes." [R10]

Highly unlikely. Query-dominant. Large tree indexes. Now think about the last appointment slot on a Friday, or the inventory row for the product that just went viral. Contention there isn't an accident, it's the business.

Their word for what it costs is starvation, and it's in the introduction rather than a limitations section: "Since locks are not used, it is deadlock-free (however, starvation is a possible problem, a solution for which we discuss)." [R7] Their remedy is the part nobody quotes. A transaction whose validation keeps failing gets restarted "without releasing the critical section semaphore. This is equivalent to write-locking the entire database." [R8] The fix for optimism failing is to stop being optimistic, globally, until the starving transaction gets through. Hence the symmetry they state as a conclusion: "The major difficulty in locking approaches is deadlock, which can be solved by using backup; in an optimistic approach, the major difficulty is starvation, which can be solved by using locking." [R9] There's no free position.

There's one more reason to discount the citation. The paper contains no experiment, no benchmark, no simulation and no measurement of any kind. Seven sections, no evaluation section. [R12] Its only quantitative content is a derived probability bound for B-tree insertions, openly modelled: "Lacking theoretical results on the distribution of the number of keys in B-tree pages, we make the conservative assumption that the number of keys in any page is uniformly distributed." [R13] Analysis, not evidence.

The authors would have agreed: their closing paragraph is a request for work that hadn't been done — "Some techniques are definitely needed for determining all instances where an optimistic approach is better than a locking approach, and in such cases, which type of optimistic approach should be used." [R14] Forty-five years later, teams still choose by habit.

What 105 real concurrency bugs actually look like

In 2008, Lu, Park, Seo and Zhou at Illinois pulled 105 randomly selected real-world concurrency bugs out of four open-source codebases — MySQL, Apache, Mozilla and OpenOffice — and characterised them one by one. [R15] [R16] The split is 74 non-deadlock bugs and 31 deadlock bugs. [R17] One finding reframes the problem. Table 1, finding (5): "Many (66%) of the examined non-deadlock concurrency bugs' manifestation involves concurrent accesses to only one variable." [R22] Two thirds of the non-deadlock bugs they examined are one-variable bugs. Not an exotic multi-object interleaving. One field, accessed concurrently. The 34% of examined non-deadlock bugs involving multiple variables are the harder minority. [R23]

That changes what you can do on Monday, because a one-variable problem is enumerable. List the fields where two users can act at once and the decision depends on the current value: remaining capacity, account balance, status transitions, the boolean saying whether a job is claimed. In most products that list is short. (Whether you should have built the feature that way is a different argument, and the answer there doesn't get you out of this one.)

The second finding is about testing, and its denominator is different — this one is across all 105 examined bugs, deadlock and non-deadlock alike. "About 92% of the examined concurrency bugs can be reliably triggered by enforcing certain orders among no more than 4 memory accesses." [R18] [R19] About 96% of all 105 examined bugs manifest under an enforced partial order between just 2 threads. [R25] So testing "can target at exploring possible orders among every small groups of memory accesses, instead of among all memory accesses." [R18] The search space is small enough to attack deliberately. Nobody does, even though the test you'd write runs two requests at once and asserts on the outcome.

Keep the third finding in view before you reach for a mutex. Its denominator is the 74 non-deadlock bugs, not all 105: "About 73% of the examined non-deadlock concurrency bugs were not fixed by simply adding or changing locks, and many of the fixes were not correct at the first try." [R20] [R21] Reaching for a lock is the reflex. In about three quarters of the examined non-deadlock bugs, it wasn't the fix.

Why? Of the examined non-deadlock bugs, about a third (32%) are order violations. [R24] An order violation isn't a missing mutex. It's two operations happening in a sequence the programmer never considered, and you can't lock your way out of an assumption you didn't know you'd made.

The authors' own caveat: "we do not intend to draw any general conclusions about all concurrent applications." [R26] These are 105 filed bug reports from four C/C++-era applications. They tell you the shape of concurrency defects, not the frequency in your Rails app.

Four places to put the concurrency check

Four layers can enforce "only one of these two writes may win." Most teams end up with one by accident rather than by decision.

1. The database constraint. Push the invariant into the engine and let it arbitrate. Bailis and colleagues measured what teams do instead: in open-source Rails codebases, "feral invariants are the most popular means of ensuring integrity (and, by usage, are over 37 times more popular than transactions)." [R64] [R65] Thirty-seven to one, in open-source Rails code: the rules live in the application, whatever the diagram says.

"Up to 86.9% of Rails validation usage by volume is actually safe under concurrent execution. However, the remainder — which include uniqueness violations under insertion and foreign key constraint violations under deletion — are not." [R66] That residue is the double-booking case: the read-then-write gap shipped as a framework default. ActiveRecord's uniqueness validation issues "a 'SELECT' query in SQL and, if no such record is found, Rails updates the instance state," but "under Read Committed or Repeatable Read isolation, no such mutual exclusion will be performed." [R67]

PostgreSQL makes the opposite move. It "enforces SQL uniqueness constraints using unique indexes, which are indexes that disallow multiple entries with identical keys" [R81], and states the guarantee in MVCC terms, not isolation-level terms: "no MVCC snapshot could include two rows with equal index keys." [R82] Our exact case is handled there: "if a conflicting row has been inserted by an as-yet-uncommitted transaction, the would-be inserter must wait to see if that transaction commits... If it commits... there is a uniqueness violation." [R83] The reason is the sentence to quote in your next design review: "there is no obvious way to avoid race conditions unless the conflict check is an integral part of insertion of the new index entry." [R84]

Rails checks, then inserts. PostgreSQL checks as part of inserting, and only the second is safe. Mind the scope — that's PostgreSQL's b-tree access method describing itself, not a portable guarantee across engines or isolation levels.

2. Serializable isolation plus a retry loop. Let the database detect the anomaly and abort one transaction; you run it again. PostgreSQL's Serializable "monitors for conditions which could make... serializable transactions behave in a manner inconsistent with all possible serial... executions." [R43]

The objection is cost, and the cost was measured. Ports and Grittner report that PostgreSQL's "serializable mode has a performance cost of less than 7% relative to snapshot isolation," achieved without blocking: "Transactions that might violate serializability are simply aborted." [R75] [R76] [R77] That's the best argument that the version column you're about to hand-roll is at the wrong layer.

The catch is that aborts are yours, permanently and by design: "It is important to retry the complete transaction, including all logic that decides which SQL to issue... PostgreSQL does not offer an automatic retry facility, since it cannot do so with any guarantee of correctness." [R44] The whole transaction, business logic included — same at Repeatable Read: "abort the current transaction and retry the whole transaction from the beginning." [R41]

Now hold two things at once, because they don't resolve. Ports and Grittner say the database gives you serializability for under 7%, which argues for turning the dial up rather than sprinkling version columns. ACIDRain says 17 of 22 real vulnerabilities manifest even under the strongest transactional guarantees offered by these databases. [R72] Both are true: serializability is a guarantee about transactions, and it can't reach a decision made outside one. If the read happens in one request and the write in the next, no isolation level covers that gap. So "serializable plus retry" is wrong as a universal answer, and right often enough to try first.

There's a measured argument against optimism itself. Yu, Bezerra, Pavlo, Devadas and Stonebraker benchmarked seven schemes to a thousand cores and found OCC among the worst: it and TIMESTAMP have "significantly worse performance than the other algorithms regardless of the number of cores" because they "waste cycles" copying tuples to perform a read. [R78] [R79] [R86] Call that the empirical shape of the cost Kung and Robinson named — the bridge is my reading, not a finding in the paper. The paper won't let locking off the hook either: at the highest contention tested, "the DBMS's throughput peaks at 16 cores and cannot scale beyond that." [R80] Neither scales through contention.

3. The version column. The ORM's optimistic concurrency control, probably already half-implemented in your codebase. Jakarta Persistence defines it by intent: optimistic locking insures "that updates to the database data... are made only when no intervening transaction has updated that data since the entity state was read," and violations "result in an OptimisticLockException being thrown." [R51] @Version marks the field carrying the lock value, one per class; the exception can surface at an API call, a flush or commit. [R52] [R53]

Rails does the same with lock_version: each update increments the column, and the facilities "ensure that records instantiated twice will let the last one saved raise a StaleObjectError if the first was also updated." [R54] The next part is where most implementations break: "This locking mechanism will function inside a single Ruby process. To make it work across all web requests, the recommended approach is to add lock_version as a hidden field to your form." [R55] If the version doesn't travel to the client and back, you've guarded the wrong race.

EF Core exposes the mechanic plainly: "if a concurrent update occurred, the UPDATE fails to find any matching rows and reports that zero were affected. As a result, EF Core's SaveChanges() throws a DbUpdateConcurrencyException." [R60] Zero rows affected is the entire detection mechanism, which is why a silent zero-row UPDATE is dangerous.

Django tells you to stop read-modify-write where you can. F() avoids the case where "the value that the second thread saves will be based on the original value; the work of the first thread will be lost" [R57], and update() "prevents a race condition wherein something might change in your database in the short period of time between loading the object and calling save()." [R56] When you need the row held, select_for_update() generates "a SELECT ... FOR UPDATE SQL statement" [R58], subject to what PostgreSQL says that lock does. [R39]

4. The HTTP layer. It's been in the spec all along and most internal APIs ignore it. An entity tag is "an opaque validator for differentiating between multiple representations of the same resource" [R61], and RFC 9110 names our problem outright: "If-Match is most often used with state-changing methods... to prevent accidental overwrites when multiple user agents might be acting in parallel on the same resource (i.e., to prevent the 'lost update' problem)." [R62] A server evaluating If-Match "MUST NOT perform the requested method if the condition evaluates to false," and may respond 412 Precondition Failed. [R63] One header, one status code, and the conflict surfaces in front of the user about to overwrite someone else's edit. An API that returns 200 to the second write has told the user nothing, and the user who lost their edit finds out from someone else.

Two notes before you build any of this. A retried write can happen more than once, so if it emits events or calls a payment API, idempotence and at-least-once delivery are the constraint you're under. And pick a layer deliberately: all four on one entity is usually four half-implementations, none load-bearing.

How often does a race condition in production happen? Nobody has measured it

Here's the number you want and cannot have. No rate at which lost-update races actually occur in production has ever been published. Not disputed, not contested. Absent. The three figures recruited to stand in for it each measure something else.

The Feral Concurrency Control team ran a stress test: 64 concurrent inserts of the same key, 100 rounds, on EC2. With no validation, "all concurrent requests succeed, resulting in 6300 duplicate records." With ActiveRecord's uniqueness validation and two Unicorn processes, the processes race and produce "70 duplicate records spread across 70 keys"; with three, "249 duplicate records across all 100 keys." [R68] That's a collision engineered to collide. It proves the mechanism is real and that the framework default doesn't stop it. It is not a rate at which your users double-book anything.

ACIDRain's 22 verified attacks across 12 applications is a count of exploitable defects found by one analysis tool. [R71] It tells you the vulnerability is prevalent, not how often it fires under organic traffic. Lu et al.'s 105 bugs is a sample of filed bug reports, which measure what got noticed and written down, and the authors decline to generalise from it. [R16] [R26]

So you're pricing this risk without a rate, which explains a pattern you've probably lived through. Every mitigation here has a cost you can estimate: under 7% for serializable in PostgreSQL [R76], a migration and a form field for a version column, a header and a status code for If-Match. Each gets weighed against a benefit nobody can size, next to features with revenue attached. The unsized side loses, every quarter, until a customer sizes it for you by showing up at a fully booked appointment.

You can't run the expected-value calculation. What you can do is stop pretending you're running one. Decide on mechanism instead: here are the fields where two users can act at once, here is which layer protects each, here is why. That takes an afternoon and doesn't need a number that doesn't exist.

The race is already in the system you inherited

Nobody on your current team wrote that handler. It shipped in 2021, it has worked every day since, and it will keep working right up until the traffic pattern changes — a promotion, an integration that batches, a mobile client that retries on timeout. The read-then-write gap doesn't announce itself when it's introduced. It announces itself when volume stops masking it. [R70]

That kind of work is what Dev On Demand is built for: single stream $3,495/mo with 1 dedicated AI-augmented engineer, dual stream $6,795/mo with 2 in parallel, a 3-day task cycle, daily async updates, and a task-by-task approval gate. First ship in 5 days, a 5-day replacement guarantee, cancel any time with no notice period, and you own the IP.

Open your schema and write down every field where two users can act at the same time and the decision depends on the current value, then check whether the code that reads each one is in the same transaction as the code that writes it.

Frequently asked questions

What is optimistic concurrency control?

Optimistic concurrency control is a family of concurrency-control schemes that take no locks. A transaction reads freely and performs its writes on local copies, then validates at commit time that nothing it read changed underneath it; a failed validation aborts and restarts the transaction. [R2] [R6] The original 1981 paper describes the methods as "'optimistic' in the sense that they rely mainly on transaction backup as a control mechanism, 'hoping' that conflicts between transactions will not occur." [R2] EF Core puts the same assumption in application terms: optimistic concurrency "assumes that concurrency conflicts are relatively rare" and "arranges for the data modification to fail on save if the data has changed since it was queried." [R59]

Will switching to serializable isolation fix the double-booking bug?

Sometimes, and less often than you'd hope. ACIDRain found that all 22 verified vulnerabilities manifest under default isolation, and 17 of the 22 — "due to incorrect transaction usage" — manifest even under the strongest transactional guarantees the databases offer. [R72] The failure in those cases is transaction scope, not isolation level: if the read the decision was based on isn't inside the transaction, no setting on the database can protect it. Serializable is worth turning on — PostgreSQL's serializable mode has a measured performance cost of less than 7% relative to snapshot isolation [R76] — but budget for the retry logic, because PostgreSQL "does not offer an automatic retry facility, since it cannot do so with any guarantee of correctness." [R44]

What's the difference between the lost update problem and write skew?

The difference is the number of rows involved. Lost update (P4) happens on one row: T1 reads a value, T2 updates it, T1 then writes based on its stale read and commits, so "even if T2 commits, T2's update will be lost." [R29] [R30] Write skew (A5B) spans rows: T1 and T2 each read x and y, then write different items, and "if there were a constraint between x and y, it might be violated." [R31] The canonical example is a bank "where account balances are allowed to go negative as long as the sum of commonly held balances remains non-negative." [R32] Snapshot isolation admits write skew, which is why it "admits history anomalies that REPEATABLE READ does not." [R33] Note that PostgreSQL's documentation uses neither phrase — its term is "serialization anomaly." [R45]

Should I use a version column or push the check into the database?

Push the check into the database when the read and the write happen inside one request; use a version column when they happen in different requests. Both are defensible and they protect different gaps. A version column — JPA's @Version with OptimisticLockException [R51] [R52], Rails' lock_version with StaleObjectError [R54], EF Core's zero-rows-affected DbUpdateConcurrencyException [R60] — is the only one of the four layers that covers a read and a write in different requests, which is the common edit-form case. Rails' caveat is the one people miss: the mechanism "will function inside a single Ruby process," so the version has to travel to the client as a hidden field and back. [R55] Serializable plus retry covers whole transactions more thoroughly, and in PostgreSQL costs less than 7% relative to snapshot isolation [R76], but it cannot help across requests. For APIs, RFC 9110's If-Match plus a 412 response is named in the spec as the fix for "the 'lost update' problem." [R62] [R63]

How often do lost-update race conditions happen in production?

Nobody has published a figure. The rate at which lost-update races actually occur in production has never been published — not disputed, not contested, absent. The three numbers usually recruited to answer this are each something else: a controlled stress test of 64 deliberately colliding concurrent inserts over 100 rounds [R68], a count of 22 exploitable vulnerabilities found by an analysis tool across 12 applications [R71], and a sample of 105 filed bug reports whose authors decline to generalise from it [R16] [R26]. What is documented is why you don't see it before production: "low transaction volumes mask many potential concurrency-related errors under normal operation." [R70]

Sources

  • [R1] Kung & Robinson, "On Optimistic Methods for Concurrency Control", ACM TODS Vol. 6 No. 2, June 1981, pages 213–226; authors at Carnegie-Mellon University. — http://www.eecs.harvard.edu/~htk/publication/1981-tods-kung-robinson.pdf
  • [R2] The paper's own framing: the methods "are 'optimistic' in the sense that they rely mainly on transaction backup as a control mechanism, 'hoping' that conflicts between transactions will not occur." (Abstract, p. 213) — http://www.eecs.harvard.edu/~htk/publication/1981-tods-kung-robinson.pdf
  • [R3] The paper lists five "inherent disadvantages" of locking (Section 1, p. 214), of which the fifth is labelled "Most important for the purposes of this paper". — http://www.eecs.harvard.edu/~htk/publication/1981-tods-kung-robinson.pdf
  • [R4] The load-bearing motivation: "locking may be necessary only in the worst case" — and with n roots and two processes, "locking is really needed (if at all) every n transactions, on the average." (Section 1, disadvantage (5), p. 214) — http://www.eecs.harvard.edu/~htk/publication/1981-tods-kung-robinson.pdf
  • [R5] The authors state the conditions under which that argument holds: (a) nodes in the graph very large compared to nodes involved in running transactions, and (b) probability of modifying a congested node is small. (Section 1, p. 214) — http://www.eecs.harvard.edu/~htk/publication/1981-tods-kung-robinson.pdf
  • [R6] The three phases are named in the paper as a read phase, a validation phase and a possible write phase; during the read phase "all writes take place on local copies of the nodes to be modified". (Section 1, item (2), p. 215, Figure 1) — http://www.eecs.harvard.edu/~htk/publication/1981-tods-kung-robinson.pdf
  • [R7] The paper's own boundary condition, stated in the introduction: "Since locks are not used, it is deadlock-free (however, starvation is a possible problem, a solution for which we discuss)." (Section 1, p. 214) — http://www.eecs.harvard.edu/~htk/publication/1981-tods-kung-robinson.pdf
  • [R8] Their proposed starvation remedy is to fall back to locking: a starving transaction is restarted "without releasing the critical section semaphore. This is equivalent to write-locking the entire database". (Section 3, p. 220) — http://www.eecs.harvard.edu/~htk/publication/1981-tods-kung-robinson.pdf
  • [R9] The paper's symmetry claim: "The major difficulty in locking approaches is deadlock, which can be solved by using backup; in an optimistic approach, the major difficulty is starvation, which can be solved by using locking." (Section 7 Conclusions, item (3), p. 225) — http://www.eecs.harvard.edu/~htk/publication/1981-tods-kung-robinson.pdf
  • [R10] Where the authors say optimistic methods fit: "These methods may well be superior to locking methods for systems where transaction conflict is highly unlikely. Examples include query-dominant systems and very large tree-structured indexes." (Section 7, p. 225) — http://www.eecs.harvard.edu/~htk/publication/1981-tods-kung-robinson.pdf
  • [R12] The paper contains NO experiment, benchmark, simulation or measurement. It has seven sections (Introduction; read/write phases; validation; serial validation; parallel validation; B-tree application; Conclusions) and no evaluation section. Its only numbers come from a derived probability bound under stated assumptions. — http://www.eecs.harvard.edu/~htk/publication/1981-tods-kung-robinson.pdf
  • [R13] The B-tree analysis is explicitly modelled, not measured: "Lacking theoretical results on the distribution of the number of keys in B-tree pages, we make the conservative assumption that the number of keys in any page is uniformly distributed…". Its result: "if d = 3, m = 199, and n = 10^4, we have p_C < 0.0007." (Section 6, pp. 224–225) — http://www.eecs.harvard.edu/~htk/publication/1981-tods-kung-robinson.pdf
  • [R14] The authors concede they had not established when optimism wins: "Some techniques are definitely needed for determining all instances where an optimistic approach is better than a locking approach, and in such cases, which type of optimistic approach should be used." (Section 7, p. 225) — http://www.eecs.harvard.edu/~htk/publication/1981-tods-kung-robinson.pdf
  • [R15] Lu, Park, Seo, Zhou, "Learning from Mistakes: A Comprehensive Study on Real World Concurrency Bug Characteristics", ASPLOS'08, March 1–5 2008, Seattle. Authors at University of Illinois at Urbana-Champaign. — https://pages.cs.wisc.edu/~shanlu/paper/asplos122-lu.pdf
  • [R16] "105 randomly selected real world concurrency bugs from 4 representative server and client open-source applications (MySQL, Apache, Mozilla and OpenOffice)." (Abstract) — https://pages.cs.wisc.edu/~shanlu/paper/asplos122-lu.pdf
  • [R17] Table 3 breakdown: MySQL 14 non-deadlock / 9 deadlock; Apache 13 / 4; Mozilla 41 / 16; OpenOffice 6 / 2. Total 74 non-deadlock, 31 deadlock = 105. — https://pages.cs.wisc.edu/~shanlu/paper/asplos122-lu.pdf
  • [R18] Denominator is all examined bugs: "About 92% of the examined concurrency bugs can be reliably triggered by enforcing certain orders among no more than 4 memory accesses." (Abstract, finding (3)) — https://pages.cs.wisc.edu/~shanlu/paper/asplos122-lu.pdf
  • [R19] The same figure restated in Table 1, finding (8): "Almost all (92%) of the examined concurrency bugs are guaranteed to manifest if certain partial order among no more than 4 memory accesses is enforced." — https://pages.cs.wisc.edu/~shanlu/paper/asplos122-lu.pdf
  • [R20] The paper says: "About 73% of the examined non-deadlock concurrency bugs were not fixed by simply adding or changing locks, and many of the fixes were not correct at the first try, indicating the difficulty of reasoning concurrent execution by programmers." (Abstract, finding (4)) Denominator = 74 non-deadlock bugs, not 105. — https://pages.cs.wisc.edu/~shanlu/paper/asplos122-lu.pdf
  • [R21] Table 1 finding (9) restates it: "Three quarters (73%) of the examined non-deadlock bugs are fixed by techniques other than adding/changing locks." — https://pages.cs.wisc.edu/~shanlu/paper/asplos122-lu.pdf
  • [R22] The most practitioner-useful finding — Table 1 finding (5): "Many (66%) of the examined non-deadlock concurrency bugs' manifestation involves concurrent accesses to only one variable." — https://pages.cs.wisc.edu/~shanlu/paper/asplos122-lu.pdf
  • [R23] Its complement, Table 1 finding (6): "One third (34%) of the examined non-deadlock concurrency bugs' manifestation involves concurrent accesses to multiple variables." — https://pages.cs.wisc.edu/~shanlu/paper/asplos122-lu.pdf
  • [R24] Pattern breakdown, Table 1 findings (1) and (2): "Almost all (97%) of the examined non-deadlock bugs belong to one of the two simple bug patterns: atomicity-violation or order-violation" and "About one third (32%) of the examined non-deadlock bugs are order-violation bugs, which are not well addressed in previous work." — https://pages.cs.wisc.edu/~shanlu/paper/asplos122-lu.pdf
  • [R25] Two-thread finding, Table 1 finding (3): "Almost all (96%) of the examined concurrency bugs are guaranteed to manifest if certain partial order between 2 threads is enforced." — https://pages.cs.wisc.edu/~shanlu/paper/asplos122-lu.pdf
  • [R26] The paper's own scope limit, which must be quoted if the post generalises: "While we believe that the applications and bugs we examined well represent a large body of concurrent applications, we do not intend to draw any general conclusions about all concurrent applications." (Section 1) — https://pages.cs.wisc.edu/~shanlu/paper/asplos122-lu.pdf
  • [R27] Berenson, Bernstein, Gray, Melton, O'Neil, O'Neil, "A Critique of ANSI SQL Isolation Levels", Proc. ACM SIGMOD 95, pp. 1–10, San Jose CA, June 1995; also Microsoft Research Technical Report MSR-TR-95-51. — https://arxiv.org/pdf/cs/0701157
  • [R28] The critique's core charge: "The three ANSI phenomena are ambiguous. Even their broadest interpretations do not exclude anomalous behavior. This leads to some counter-intuitive results. In particular, lock-based isolation levels have different characteristics than their ANSI equivalents." (Section 1) — https://arxiv.org/pdf/cs/0701157
  • [R29] P4 Lost Update, defined: "The lost update anomaly occurs when transaction T1 reads a data item and then T2 updates the data item (possibly based on a previous read), then T1 (based on its earlier read value) updates the data item and commits." History: P4: r1[x]...w2[x]...w1[x]...c1. (Section 4.1) — https://arxiv.org/pdf/cs/0701157
  • [R30] Its worked example, history H4: r1[x=100] r2[x=100] w2[x=120] c2 w1[x=130] c1 — "The problem… is that even if T2 commits, T2's update will be lost." (Section 4.1) — https://arxiv.org/pdf/cs/0701157
  • [R31] A5B Write Skew, defined — this IS the canonical primary definition: "Suppose T1 reads x and y, which are consistent with C(), and then a T2 reads x and y, writes x, and commits. Then T1 writes y. If there were a constraint between x and y, it might be violated." History: A5B: r1[x]...r2[y]...w1[y]...w2[x]...(c1 and c2 occur). (Section 3.3) — https://arxiv.org/pdf/cs/0701157
  • [R32] The paper's own write-skew example is a bank, not doctors: "Write Skew (A5B) could arise from a constraint at a bank, where account balances are allowed to go negative as long as the sum of commonly held balances remains non-negative." (Section 3.3) — https://arxiv.org/pdf/cs/0701157
  • [R33] Snapshot Isolation admits write skew, per the paper: "Write Skew (A5B) obviously can occur in a Snapshot Isolation history (e.g., H5)… Therefore Snapshot Isolation admits history anomalies that REPEATABLE READ does not." (Section 4.2) — https://arxiv.org/pdf/cs/0701157
  • [R35] PostgreSQL's default: "Read Committed is the default isolation level in PostgreSQL." (Ch. 13.2, PostgreSQL 18 documentation) — https://www.postgresql.org/docs/current/transaction-iso.html
  • [R36] What Read Committed does not prevent, in PostgreSQL's own words: "it is possible for an updating command to see an inconsistent snapshot: it can see the effects of concurrent updating commands on the same rows it is trying to update, but it does not see effects of those commands on other rows in the database. This behavior makes Read Committed mode unsuitable for commands that involve complex search conditions". (Ch. 13.2.1) — https://www.postgresql.org/docs/current/transaction-iso.html
  • [R37] The re-evaluation rule that produces the surprise: "The search condition of the command (the WHERE clause) is re-evaluated to see if the updated version of the row still matches the search condition. If so, the second updater proceeds with its operation using the updated version of the row." (Ch. 13.2.1) — https://www.postgresql.org/docs/current/transaction-iso.html
  • [R38] What applications must do at non-serializable levels: "When non-serializable writes are possible, to ensure the current validity of a row and protect it against concurrent updates one must use SELECT FOR UPDATE, SELECT FOR SHARE, or an appropriate LOCK TABLE statement." (Ch. 13.4.2) — https://www.postgresql.org/docs/current/applevel-consistency.html
  • [R39] The trap most engineers get wrong, from PostgreSQL's own docs: "SELECT FOR UPDATE does not ensure that a concurrent transaction will not update or delete a selected row. To do that in PostgreSQL you must actually update the row, even if no values need to be changed." (Ch. 13.4.2) — https://www.postgresql.org/docs/current/applevel-consistency.html
  • [R40] PostgreSQL's Repeatable Read is snapshot isolation and is stronger than the standard requires: it "prevents all of the phenomena described in Table 13.1 except for serialization anomalies", and "is implemented using a technique known in academic database literature and in some other database products as Snapshot Isolation." (Ch. 13.2.2) — https://www.postgresql.org/docs/current/transaction-iso.html
  • [R41] Repeatable Read's failure mode: the transaction "will be rolled back with the message ERROR: could not serialize access due to concurrent update… When an application receives this error message, it should abort the current transaction and retry the whole transaction from the beginning." (Ch. 13.2.2) — https://www.postgresql.org/docs/current/transaction-iso.html
  • [R42] PostgreSQL's explicit warning against business rules at Repeatable Read: "Attempts to enforce business rules by transactions running at this isolation level are not likely to work correctly without careful use of explicit locks to block conflicting transactions." (Ch. 13.2.2) — https://www.postgresql.org/docs/current/transaction-iso.html
  • [R43] Serializable, and its cost: it "works exactly the same as Repeatable Read except that it also monitors for conditions which could make execution of a concurrent set of serializable transactions behave in a manner inconsistent with all possible serial (one at a time) executions"; it "is implemented using a technique known in academic database literature as Serializable Snapshot Isolation". (Ch. 13.2.3) — https://www.postgresql.org/docs/current/transaction-iso.html
  • [R44] Retries are the application's job, permanently: "It is important to retry the complete transaction, including all logic that decides which SQL to issue and/or which values to use. Therefore, PostgreSQL does not offer an automatic retry facility, since it cannot do so with any guarantee of correctness." (Ch. 13.5) — https://www.postgresql.org/docs/current/mvcc-serialization-failure-handling.html
  • [R45] Verified negative: the strings "lost update" and "write skew" do not appear anywhere in PostgreSQL chapters 13.2 or 13.4. PostgreSQL's term of art is "serialization anomaly", defined as "The result of successfully committing a group of transactions is inconsistent with all possible orderings of running those transactions one at a time." — https://www.postgresql.org/docs/current/transaction-iso.html
  • [R46] MySQL/InnoDB's default differs from PostgreSQL's: "The default isolation level for InnoDB is REPEATABLE READ." — https://dev.mysql.com/doc/refman/8.4/en/innodb-transaction-isolation-levels.html
  • [R47] MySQL scopes the snapshot rule to non-locking reads only: "Consistent reads within the same transaction read the snapshot established by the first read", while locking reads (FOR UPDATE/FOR SHARE), UPDATE and DELETE follow a separate index-locking regime. — https://dev.mysql.com/doc/refman/8.4/en/innodb-transaction-isolation-levels.html
  • [R48] Oracle's default: "In the read committed isolation level, every query executed by a transaction sees only data committed before the query—not the transaction—began… This isolation level is the default." — https://docs.oracle.com/en/database/oracle/oracle-database/19/cncpt/data-concurrency-and-consistency.html
  • [R49] Oracle's SERIALIZABLE is first-updater-wins against a transaction-start SCN: it "permits a serializable transaction to modify a row only if changes to the row made by other transactions were already committed when the serializable transaction began", raising ORA-08177: Cannot serialize access for this transaction. Oracle's docs never use the phrase "snapshot isolation" for it. — https://docs.oracle.com/en/database/oracle/oracle-database/19/cncpt/data-concurrency-and-consistency.html
  • [R50] SQL Server vs Azure SQL — the "your default is not what you think" point: READ COMMITTED "is the SQL Server default", with READ_COMMITTED_SNAPSHOT OFF by default on SQL Server; but "READ_COMMITTED_SNAPSHOT ON is the default on Azure SQL Database and SQL database in Microsoft Fabric." Azure Synapse Analytics defaults to READ UNCOMMITTED. — https://learn.microsoft.com/en-us/sql/t-sql/statements/set-transaction-isolation-level-transact-sql?view=sql-server-ver17
  • [R51] JPA defines optimistic locking by intent, not mechanism: "Optimistic locking is a technique that is used to insure that updates to the database data corresponding to the state of an entity are made only when no intervening transaction has updated that data since the entity state was read… Transactions that would cause this constraint to be violated result in an OptimisticLockException being thrown and the transaction marked for rollback." (Jakarta Persistence 3.1 spec §3.4.1) — https://jakarta.ee/specifications/persistence/3.1/jakarta-persistence-spec-3.1.html
  • [R52] @Version javadoc: "Specifies the version field or property of an entity class that serves as its optimistic lock value… Only a single Version property or field should be used per class". Supported types: int, Integer, short, Short, long, Long, java.sql.Timestamp. — https://jakarta.ee/specifications/persistence/3.1/apidocs/jakarta.persistence/jakarta/persistence/version
  • [R53] OptimisticLockException javadoc: "Thrown by the persistence provider when an optimistic locking conflict occurs. This exception may be thrown as part of an API call, a flush or at commit time. The current transaction, if one is active, will be marked for rollback." — https://jakarta.ee/specifications/persistence/3.1/apidocs/jakarta.persistence/jakarta/persistence/optimisticlockexception
  • [R54] ActiveRecord's version: "Active Record supports optimistic locking if the lock_version field is present. Each update to the record increments the integer column lock_version and the locking facilities ensure that records instantiated twice will let the last one saved raise a StaleObjectError if the first was also updated." — https://api.rubyonrails.org/classes/ActiveRecord/Locking/Optimistic.html
  • [R55] Rails puts the conflict back on the developer: "You're then responsible for dealing with the conflict by rescuing the exception and either rolling back, merging, or otherwise apply the business logic needed to resolve the conflict." And crucially: "This locking mechanism will function inside a single Ruby process. To make it work across all web requests, the recommended approach is to add lock_version as a hidden field to your form." — https://api.rubyonrails.org/classes/ActiveRecord/Locking/Optimistic.html
  • [R56] Django's own race-condition warning, in its docs for update(): "Using update() also prevents a race condition wherein something might change in your database in the short period of time between loading the object and calling save()." — https://docs.djangoproject.com/en/5.2/ref/models/querysets/
  • [R57] Django's F() docs describe the lost update in plain English: "If two Python threads execute the code in the first example above, one thread could retrieve, increment, and save a field's value after the other has retrieved it from the database. The value that the second thread saves will be based on the original value; the work of the first thread will be lost." (section "Avoiding race conditions using F()") — https://docs.djangoproject.com/en/5.2/ref/models/expressions/
  • [R58] Django's select_for_update(): "Returns a queryset that will lock rows until the end of the transaction, generating a SELECT ... FOR UPDATE SQL statement on supported databases." — https://docs.djangoproject.com/en/5.2/ref/models/querysets/
  • [R59] EF Core states the assumption out loud: "EF Core implements optimistic concurrency, which assumes that concurrency conflicts are relatively rare… optimistic concurrency takes no locks, but arranges for the data modification to fail on save if the data has changed since it was queried." — https://learn.microsoft.com/en-us/ef/core/saving/concurrency
  • [R60] The exact failure mechanic — zero rows affected: "if a concurrent update occurred, the UPDATE fails to find any matching rows and reports that zero were affected. As a result, EF Core's SaveChanges() throws a DbUpdateConcurrencyException, which the application must catch and handle appropriately." — https://learn.microsoft.com/en-us/ef/core/saving/concurrency
  • [R61] HTTP has had optimistic concurrency built in since forever. RFC 9110 §8.8.3: "An entity tag is an opaque validator for differentiating between multiple representations of the same resource, regardless of whether those multiple representations are due to resource state changes over time, content negotiation resulting in multiple representations being valid at the same time, or both." — https://www.rfc-editor.org/rfc/rfc9110.txt
  • [R62] RFC 9110 §13.1.1 names the problem explicitly: "If-Match is most often used with state-changing methods (e.g., POST, PUT, DELETE) to prevent accidental overwrites when multiple user agents might be acting in parallel on the same resource (i.e., to prevent the 'lost update' problem)." — https://www.rfc-editor.org/rfc/rfc9110.txt
  • [R63] And its enforcement: "An origin server that evaluates an If-Match condition MUST NOT perform the requested method if the condition evaluates to false. Instead, the origin server MAY indicate that the conditional request failed by responding with a 412 (Precondition Failed) status code." (§13.1.1) — https://www.rfc-editor.org/rfc/rfc9110.txt
  • [R64] Bailis, Fekete, Franklin, Ghodsi, Hellerstein, Stoica, "Feral Concurrency Control: An Empirical Investigation of Modern Application Integrity", SIGMOD'15, May 31–June 4 2015, Melbourne. — https://www.bailis.org/papers/feral-sigmod2015.pdf
  • [R65] Its headline measurement of what real applications actually do: across a survey of open-source Rails applications, "feral invariants are the most popular means of ensuring integrity (and, by usage, are over 37 times more popular than transactions)", with "over 9950 uses of application-level validations". — https://www.bailis.org/papers/feral-sigmod2015.pdf
  • [R66] How many of those are actually safe: "up to 86.9% of Rails validation usage by volume is actually safe under concurrent execution. However, the remainder—which include uniqueness violations under insertion and foreign key constraint violations under deletion—are not." — https://www.bailis.org/papers/feral-sigmod2015.pdf
  • [R67] The exact double-booking mechanism, in a mainstream ORM: "ActiveRecord accomplishes this by issuing a 'SELECT' query in SQL and, if no such record is found, Rails updates the instance state in the database… under Read Committed or Repeatable Read isolation, no such mutual exclusion will be performed, leading to potential inconsistency." (§4) — https://www.bailis.org/papers/feral-sigmod2015.pdf
  • [R68] The measured duplicate-record result (a stress test, not a production rate): 64 concurrent inserts of the same key, 100 rounds. "With no validation, all concurrent requests succeed, resulting in 6300 duplicate records… with two processes, Unicorn processes race, resulting in 70 duplicate records spread across 70 keys. With three processes, Unicorn produces 249 duplicate records across all 100 keys." (§5) — https://www.bailis.org/papers/feral-sigmod2015.pdf
  • [R69] Warszawski & Bailis, "ACIDRain: Concurrency-Related Attacks on Database-Backed Web Applications", SIGMOD 2017, Stanford InfoLab. — https://www.bailis.org/papers/acidrain-sigmod2017.pdf
  • [R70] THE "only shows up in production" SENTENCE, from the abstract: "While low transaction volumes mask many potential concurrency-related errors under normal operation, determined adversaries can exploit them programmatically for fun and profit." — https://www.bailis.org/papers/acidrain-sigmod2017.pdf
  • [R71] The scale of the finding: "We apply a prototype 2AD analysis tool to 12 popular self-hosted eCommerce applications written in four languages and deployed on over 2M websites. We identify and verify 22 critical ACIDRain attacks that allow attackers to corrupt store inventory, over-spend gift cards, and steal inventory." — https://www.bailis.org/papers/acidrain-sigmod2017.pdf
  • [R72] "All 22 vulnerabilities manifest under the default isolation guarantees of popular transactional databases including Oracle 12c, and 17 vulnerabilities—due to incorrect transaction usage—manifest even under the strongest transactional guarantees offered by these databases." (§1) — https://www.bailis.org/papers/acidrain-sigmod2017.pdf
  • [R73] Coverage: the 12 applications "covers over 55% of eCommerce sites on the Internet… WooCommerce alone accounts for 39% of all online stores"; the conclusion states 22 vulnerabilities "spread across all but one application we tested, affecting over 50% of eCommerce sites on the Internet today." (§4.2.1, §6) — https://www.bailis.org/papers/acidrain-sigmod2017.pdf
  • [R75] Ports & Grittner, "Serializable Snapshot Isolation in PostgreSQL", PVLDB Vol. 5 No. 12 (2012), pp. 1850–1861 — "the first implementation of SSI in a production database release". — https://arxiv.org/pdf/1208.4179
  • [R76] The measured cost of turning serializable on — the strongest evidence for "serializable is cheap enough now": "our serializable mode has a performance cost of less than 7% relative to snapshot isolation, and outperforms two-phase locking significantly on some workloads." (§1) — https://arxiv.org/pdf/1208.4179
  • [R77] Why PostgreSQL chose SSI over locking: "SSI does not require any additional blocking. Transactions that might violate serializability are simply aborted." (§3) — https://arxiv.org/pdf/1208.4179
  • [R78] Yu, Bezerra, Pavlo, Devadas, Stonebraker, "Staring into the Abyss: An Evaluation of Concurrency Control with One Thousand Cores", PVLDB Vol. 8 No. 3 (2014), pp. 209–220. — https://www.vldb.org/pvldb/vol8/p209-yu.pdf
  • [R79] Measured OCC weakness — the retry/waste argument, quantified: "timestamp allocation becomes the bottleneck with a large core count. OCC hits the bottleneck even earlier since it needs to allocate timestamps twice per transaction… Both OCC and TIMESTAMP have significantly worse performance than the other algorithms regardless of the number of cores. These algorithms waste cycles because they copy tuples to perform a read." (§5.1) — https://www.vldb.org/pvldb/vol8/p209-yu.pdf
  • [R80] The corresponding measured weakness of locking, for balance: "With medium contention (theta=0.6), the throughput peaks at several hundred cores and then decreases due to thrashing. At the highest contention level (theta=0.8), the DBMS's throughput peaks at 16 cores and cannot scale beyond that… lock thrashing is the key bottleneck of lock-based approaches that limits scalability in high-contention scenarios." (§5.1) — https://www.vldb.org/pvldb/vol8/p209-yu.pdf
  • [R81] PostgreSQL enforces uniqueness in the index access method, not in application code: "PostgreSQL enforces SQL uniqueness constraints using unique indexes, which are indexes that disallow multiple entries with identical keys." (Ch. 63.5) — https://www.postgresql.org/docs/current/index-unique-checks.html
  • [R82] The guarantee is stated in MVCC terms, not isolation-level terms: "The behavior we actually want to enforce is that no MVCC snapshot could include two rows with equal index keys." (Ch. 63.5) — https://www.postgresql.org/docs/current/index-unique-checks.html
  • [R83] The concurrent-insert case, the exact double-booking scenario, handled by the database: "If a conflicting row has been inserted by an as-yet-uncommitted transaction, the would-be inserter must wait to see if that transaction commits. If it rolls back then there is no conflict. If it commits without deleting the conflicting row again, there is a uniqueness violation." (Ch. 63.5) — https://www.postgresql.org/docs/current/index-unique-checks.html
  • [R84] PostgreSQL's own statement that the check must live at insertion or the race is unavoidable: "there is no obvious way to avoid race conditions unless the conflict check is an integral part of insertion of the new index entry." (Ch. 63.5) — https://www.postgresql.org/docs/current/index-unique-checks.html
  • [R85] ACIDRain §1 states that some databases do not offer serializability at all: "Some databases, including Oracle's flagship offering and SAP HANA, do not offer serializability as an option at all." Table 2 lists Oracle's maximum isolation as SI. — https://www.bailis.org/papers/acidrain-sigmod2017.pdf
  • [R86] The paper's stated scope: "We implemented seven concurrency control algorithms" — the count behind the "seven schemes" phrasing, benchmarked in a simulated 1000-core environment. — https://www.vldb.org/pvldb/vol8/p209-yu.pdf
back to top

Related Articles

Book 30 min with Albert
Smiling man with short dark hair and glasses wearing a black suit, white shirt, and black tie against blue background.
Tell Albert what you're shipping.
He'll read this before joining the call. Phone number comes next, on the calendar step.
↳ info@you-source.com
↳ 4-hour response
Please wait while we retrieve meeting schedules.
Oops! There's a problem with your request. We're working on fixing it. Please try again later.