Getting ahead of the (elliptic) curve: what's in the next consensus upgrade

The next Algorand consensus upgrade is a large one.  Updates come on several fronts, and in two cases, the changes are early steps on a much longer march.  Falcon accounts add native post-quantum (PQ) resiliency, building PQ cryptography deeply into the protocol.  Meanwhile, a raft of new features land that will ease developers' lives, along with a new model for paying for those features. This is the first upgrade where the fee you owe depends on what your transaction actually contains. Let's discuss that change first, because it affects everything else.

How the new rules charge for things

Until now, an Algorand transaction fee was essentially a constant. Every transaction owed the same minimum fee, no matter what the transaction did, or how much space in the block it consumed. Inner transactions became something of an escape hatch. They can be used to get more done, but even they obeyed the same rule – one inner transaction, one minimum fee.

Under the new rules, the network's effort associated with a transaction is called usage. It is reported as a fixed-point multiplier on the minimum fee, expressed in millionths. An ordinary transaction is said to have a usage of 1,000,000 and therefore costs one minfee. A transaction with a (newly legal) large note costs more because it has more usage. A transaction authorized with a post-quantum signature adds 2,000,000 (two minfees). The group's usage is summed, and the group must pay the minimum fee scaled by that sum.

The promise (for now)

No transaction that is legal today will cost more under these new rules.

In this release, every "free" allowance is at least what it was before the upgrade: 1,024 bytes of note, 2,048 bytes of application arguments, three extra program pages, 1,000 bytes of LogicSig per transaction. Surcharges are computed from the number of bytes beyond those allowances. A transaction that fits under the old limits therefore has a fee factor of exactly 1,000,000, the same as it always did.

That means that if you have code that computes fees by counting transactions and multiplying by the minimum fee, that code keeps working. Nothing about this upgrade forces you to change it. The new features are a carrot, not a stick. If you adopt any of the new features – post-quantum signatures, oversized notes, oversized arguments, oversized programs – you can no longer expect counting transactions to determine network fees. The fee varies with the transaction contents and perhaps with future protocol changes. Apps are strongly discouraged from trying to keep track of those extra fees on their own. They should let algod tell them the usage of potential transactions.

What your fees can buy now

Four variable sized transaction fields gain a new limit above their existing allowance. The old allowance is still free, as it always was; everything above it is charged at one ten-thousandth of a minimum fee per byte, which works out to a tenth of a microAlgo per byte at today's minimum fee. The intention is to reflect a portion of the cost that would have been required to put those same bytes on chain by issuing "dummy" transactions to hold the extra bytes.

A maximum-length note, in other words, takes a transaction from 1,000 microAlgos to 1,308. A 16-kilobyte LogicSig costs 2,434 microAlgos while a 1,000-byte LogicSig is still "free" with a minfee transaction.

LogicSigs have another change along with the higher limit. LogicSig bytes have been pooled across a group for some time: each transaction contributes a 1,000-byte allowance, and the group's programs and arguments must fit in the total. A 16,000-byte program was therefore already legal, but only in a group of sixteen. In practice, pooling meant padding your group with fifteen transactions you did not want in order to donate their allowances. Under the new rules the pooled sum becomes the free amount rather than a hard limit, and bytes beyond it are charged. A single, solitary transaction can now carry a 16,000-byte program by paying for the 15,000 bytes it did not have an allowance for. So, instead of 16 minfees, a single transaction with a 16,000 byte LogicSig costs 2.5 minfees.

Transactions cannot pay for larger LogicSig args. Arguments remain capped at their pooled allowance, but they did get a bump. Now, LogicSig arguments receive an independent 1000 byte allocation per transaction, separate from the LogicSig bytecode.

Pricing a transaction with simulate

Until now, dApps have called /v2/transactions/params once, and used the returned min-fee when calculating fees by multiplying by the total number of transactions executed in a group (top-level and inner). Now, dApps should call simulate for each transaction group they intend to submit. simulate will report group-usage in millionths of a minimum fee, for the submitted group including every descendant inner transaction.

Multiply group-usage by min-fee from /v2/transactions/params:

total = ceil(min-fee * group-usage / 1,000,000)

Spread that total fee among the transactions of the group (fees are still pooled across the group).

You may notice there is no per-transaction usage in the response, only per-transaction fees paid. That omission is deliberate. Because fees (and LogicSig bytes) pool across a group, usage is summed across the group.  It is summed with sufficient precision to avoid visible rounding at transaction boundaries. Rounding (always up!) occurs only once at the end of the group.

Reporting a per-transaction usage would invite developers to round transaction by transaction, which is unnecessary and (very slightly) wrong.

simulate will also accept a placeholder post-quantum signature: an envelope carrying only the scheme, with no public key and no signature bytes. Combined with allow-empty-signatures, this lets dApps price a Falcon-authorized transaction before they see the Falcon signature.

Native post-quantum authorization

The AVM has been able to verify Falcon signatures inside a program for a while, which allowed a LogicSig implementation of a "PQ Account". Now, Falcon-1024 is a first-class way to authorize an account on the same footing as an Ed25519 signature or multisig.  The previous LogicSig implementation is just slightly less powerful, as you'll see below.

A transaction may carry a post-quantum authorization envelope: a scheme identifier, a salt, a public key, and a signature. The reason the public key is in the signature, rather than being implied by the sender address, is simply that it does not fit: a Falcon-1024 public key is 1,793 bytes, and an Algorand address is 32. So a post-quantum address is a hash of the scheme, the salt, and the public key, and the key itself is presented at spend time.

What, then, is the salt? A post-quantum address is required to be a 32-byte value that is not a valid Ed25519 curve point. When you derive an address for a Falcon key, a salt is found by rejection sampling – scan upward from zero until the resulting hash fails to be a curve point – and the lowest such salt is canonical.

The consequence is that a post-quantum address is one that no Ed25519 private key could ever control. If elliptic curve cryptography is someday broken, an attacker who can forge Ed25519 signatures still has nothing to forge against a post-quantum address, because that address is not a legal curve point. The account is protected by a Falcon signature and it is impossible to make it appear to be signed by an Ed25519 key.

Post-quantum authorization is the most expensive thing in this upgrade: a Falcon-1024 signature adds 2,000,000 of usage, therefore it costs three minimum fees instead of one. Extra costs are incurred in the transaction size and verification times. The public key is 1,793 bytes and the signature can reach 1,423 bytes, which together dwarf a 64-byte Ed25519 signature and consume block space accordingly. And unlike Ed25519, Falcon cannot be batch verified: Algorand verifies large sets of Ed25519 signatures far faster than one at a time by aggregating the work, and no such trick is known for Falcon. Every Falcon signature is verified on its own.

The PQ address work is deliberately a framework rather than a single feature.  Adding another scheme requires a scheme tag, a verifier, a fee contribution, and signing support in algokey.  Falcon-512 is already sketched in the source and can be expected in the next consensus upgrade. Address derivation, transaction envelope, fee accounting, and simulate integration are already scheme-agnostic.

Applications can be bigger, and can grow

When apps were introduced, Algorand only allowed 1k programs.  For a long time now, that limit has grown, first by pooling with the ClearState program to get 2k, then extensible by up to three additional “pages” of 2,048 bytes each, for a total of 8,192 bytes of combined program size (most apps have only a vestigial ClearState program). That limit has been a real constraint for sophisticated contracts, and the new rules raise the ceiling: up to seven extra pages, allowing a total of 16,384 bytes.

Of course, this wasn't just about changing a consensus parameter.  The existing limit reflected the very real difference in transaction execution costs. A large program requires proportionally more I/O to read (whenever it is invoked) and write (when global state changes or the program is updated).  Further, transactions that create large programs must themselves be quite large, to carry the program code. We already know how to handle large transactions. In this update, programs in transactions larger than the previous limit pay the same per-byte surcharge that other fields do.

At execution time, Algorand has an entirely different mechanism to ensure transactions don't consume too many I/O resources. An application call is limited in the number of "references" it may contain.  These references allow accessing resource objects – assets, accounts, other apps, or boxes.  Large boxes require extra references. So, when an app uses extra pages of code it is accounted for in the same way. Extra references must be included at call time. Once again, callers should use simulate to prepare transactions. This time, it's because simulate can report the number of extra references required to call or access large apps. These rules prevent single transactions from accessing a huge amount of persistent state.  It's easy to speculate, of course, that future upgrades will allow transactions to pay higher fees to include more references.

If you have been writing Algorand applications for a while and have produced a well-loved dApp, you may be worried. Extra program pages have always been fixed at creation. An application declares how many pages it wants in the transaction that creates it, and that number is permanent. Every application already deployed on MainNet would remain stuck at the size it was born with, and the only escape would be to deploy a fresh application. A fresh application means a new application ID, a new application account, and the migration of every box, every asset holding, and every account that has opted in. For a live application with real users and real state, that is not an appealing option.

So the new rules also make the program size mutable. An application update transaction can now change the number of extra program pages and global state schema. An application that is running out of room can obtain more space, keeping its ID, its address, and its state.

Making that work required deciding who pays. Extra pages and global state both require a minimum balance and that requirement has always been carried by the creator. But the account that wants to grow an application is not necessarily the creator — the creator may be long gone, unwilling, or unfunded, while some other maintainer is willing to put up the Algo.

The new rules introduce the SizeSponsor. When an update changes an application's pages or global schema, the minimum balance obligation for those resources moves to whoever sent the update, and the application records that account as its size sponsor. If the creator performs such an update, the sponsor field is cleared and the obligation stays with (or returns to) the creator. The sponsor is visible on-chain, readable from the AVM with app_params_get AppSizeSponsor and from the REST API as size-sponsor.

Boxes can be shared between applications

Since their inception, boxes have been private to the application that owns them. A transaction declares a box reference for any box it will access to pay for the I/O, but only the owning application could read or write the contents. The existing box opcodes do not even take an app argument since the current app is assumed. The new rules add nine opcodes — app_box_create, app_box_extract, app_box_replace, app_box_del, app_box_len, app_box_get, app_box_put, app_box_splice, and app_box_resize — which are the familiar box operations with one addition: an application ID as the first argument. An application can now reach into another application's boxes if the box owning application has explicitly allowed it.

Why this has to be opt-in

Cross-application box access cannot simply be switched on for everyone. First, boxes have, until now, promised on-chain privacy. So allowing reads from other applications is an important change. Off-chain observers can always observe any on-chain state, but there may be applications that demand payment to observe their current state, on-chain. Oracles that "sell" information were the motivation behind private boxes to begin with.

Unrestricted writes would be far worse. A hostile application could destroy any invariant the owner maintains. Worse still, it could lock the owner's money: the minimum balance for a box is always ascribed to the owning application's account, no matter which application created the box. An open write permission would be a standing invitation to fill a stranger's application with boxes until its entire account balance was locked.

So there are two flags of different strength. ForeignBoxReads allows any application to read an application's boxes. FamilyBoxAccess allows applications with the same creator to read and write them. Shared mutable state is restricted to a family of applications under common control.

How opting in works

An application grants these permissions by using a new opcode app_params_set.  There is no transaction field that grants box access, no way for a creator to flip the flag on an application from outside, and no way for one application to grant itself access to another. The only thing that can open an application's boxes is that application's own code. Permission is expressed in the single place that unambiguously speaks for the application.

The flags are readable with app_params_get, on any application, so a caller can check whether an access will be permitted before attempting it. Of course, a transaction must still declare the boxes it intends to touch, and the access still counts against the read and write budgets.

Shared state brings back an old hazard

The AVM has always refused to re-enter an application across inner transactions. If an application calls an application already on the call stack, the AVM walks the chain of callers and rejects the call, and it rejects self-calls too. This was a deliberate decision made when inner transactions were introduced, and it closed off an entire family of attacks that has repeatedly drained contracts on the EVM: call out to an untrusted party while your own state is halfway through an update, and let them call back in to act on the inconsistency. The DAO was the first famous case and not the last. On the EVM, this is a hazard every developer must understand and defend against individually, with patterns and mutexes and careful ordering. On the AVM it can't happen.

That guarantee was inexpensive to provide because an application's mutable state was private to it. The only way to observe an application's state mid-update was to re-enter that application, and re-entry was banned.

Family box access breaks that premise. Two different applications can now write the same box, so the danger returns. Application A touches shared state, calls foreign application F, and F calls application B — same creator as A, same family, different application ID — and B writes the box that A is in the middle of relying on. A was never re-entered. A per-application re-entrency check passes, but the invariant was broken.

So a new ban was added at the family scope. When an application writes family-shared state, the AVM walks the caller chain and rejects the write if a foreign application separates the writer from a family member that has already touched family-shared state. The rule fires only on writes, so foreign applications remain free to call into family members for read-only queries; and the "has touched" mark propagates up to family callers as each frame returns, so it chains across every contiguous family ancestor rather than only the immediate caller.

Opting into shared state means opting back into a hazard the AVM had otherwise designed away. The protocol re-closes it for you rather than leaving it to your discipline. But beware, the check is only enforced when family access is enabled. If set_app_params is used to toggle permissions during execution the result can be hard to reason about. We strongly recommend activating family access in an isolated transaction, and never turning it off. It remains possible to turn off family access because it enables an interesting use case – deactivating a set of applications simultaneously.  Such users should take care.

New cryptographic opcodes

AVM v13 becomes available with this upgrade, and beyond the box opcodes it adds two hash functions.

sha512 computes a full SHA-512 digest, yielding all 64 bytes rather than the truncated 32 that sha512_256 has always provided. Its cost scales with the length of its input. Programs can also now read the new SHA-512 transaction commitments and block hash from the block header via the block opcode, so the sha512 opcode is available to check these hashes on-chain.

poseidon2 computes a Poseidon2 hash, with an immediate argument selecting the curve and parameter set — BN254 or BLS12-381. Poseidon2 is an arithmetization-friendly hash: it is designed to be cheap to prove inside a zero-knowledge circuit.

Load tracking, and the future of fees

Block headers produced under the new rules now indicate their Load, the degree to which the block is full, measured as bytes in the block against the maximum permitted, expressed in millionths. A completely full block is 1,000,000. It is computed by the block evaluator and validated by every node.

Load feeds a second new header value, CongestionTax, computed in a way that should look familiar to anyone who has read about EIP-1559. The CongestionTax is computed over time, targeting blocks that are half full: when the previous block was fuller than that, the tax rises; when it was emptier, the tax falls. It moves in a way that causes prices to change by at most ten percent per block, allowing off-chain processes to estimate the most it might change over short periods of time.

But for now, nothing charges that tax. No fee calculation anywhere in the protocol reads CongestionTax. It is measured, it is validated, and it is ignored.

Then why is it there?

Enforcing the CongestionTax by scaling transaction fees would break the promise at the top of this post, which was deemed too disruptive to implement in a single release.

Consider what would happen if the tax were live from the moment of the upgrade.  It is pushed upward whenever a block is more than half full. After one such block, a transaction paying exactly the minimum fee — legal in every respect, legal for years, legal the day before the upgrade — is suddenly underpaid and no longer accepted.

Making previously-valid transactions invalid during high congestion is reasonable; that is what congestion pricing is and, in fact, Algorand does have a rarely invoked congestion mechanism that occurs when at least three full blocks are queued at a node. But CongestionTax starts rising with a single half full block.  The rise is slow, and small, so dApps will one day need to adapt, but not yet. The ecosystem needs time to adopt simulate pervasively.

Shipping the measurement costs almost nothing. The tax is floored at zero and decays whenever blocks are emptier than half, so on a network with today's block occupancy it simply sits at zero. And in exchange for nothing, everyone — node operators, SDK authors, application developers — gets to watch the value move during real traffic bursts, in public, before it affects fees.

But be prepared!

The fee calculation is more prepared for this than the description above suggests. The function that turns a transaction's usage into an actual fee takes three arguments: the usage, a multiplier, and a running residue. Every caller in the codebase passes 1,000,000 for the multiplier — that is, 1.0. The multiplier will one day be the congestion tax, but currently has no effect.

The residue is a neat bookkeeping artifact. It carries fractional microAlgos at a precision of one part in a trillion, so that a sequence of nested inner transaction groups rounds up once in aggregate rather than once per group.

What to do about it

Fees are likely to change over time. Not in this upgrade — the promise at the top of this post is a promise — but the machinery is now in the protocol, the measurement is running, and the question of when and how to use it is a matter for the ecosystem rather than an implementation detail. That discussion runs through the Algorand Ecosystem Advisory Council, whose remit includes reviewing sustainability-related protocol proposals before they are presented to the wider community.

Which brings us back to where we started, with one addition. It is not enough to say that new features cost more and dApps that use them should use simulate to learn their cost. In fact, every application should stop treating the fee as a constant, whether or not it adopts anything in this upgrade:

1. Do not hardcode fees. Read min-fee from /v2/transactions/params, and read group-usage from simulate.

2. Today's promise is about today's upgrade. Someday soon, it's likely that congestion pricing or surcharges for existing legal transactions may be introduced. If that happens, an ordinary transaction may cost more than the minimum fee.  At that point, fee = min-fee * number_of_transactions breaks for existing apps, not only for those that adopted post-quantum signatures or 16-kilobyte programs.

*A note for the thorough: node-generated heartbeat transactions are the single exception to the compatibility promise above. The fee discount for an account under challenge, previously inferred from an underpaid heartbeat, is now stated explicitly in the transaction. Nodes set the flag automatically according to the protocol in effect, so no application code is affected.*

Algorand v5.0.0 is available now. You can upgrade your node via NodeKit or FUNC. If you need help, reach out via the #node-runners channel in Algorand’s Discord.

Disclaimer: The content provided in this blog is for informational and educational purposes only. The information is provided by the Algorand Foundation and while we strive to keep it accurate and current, we make no representations or warranties of any kind, express or implied, about the completeness, accuracy, reliability, suitability, or availability of the blog or the information contained in it for any purpose. Nothing in this blog is legal, financial, tax, or investment advice, nor an endorsement, guarantee, or investment recommendation. Technical descriptions are provided as of the date of publication and may change. Any statements about future plans, features, integrations, protocol or consensus upgrades, fees, or timelines are forward-looking and subject to change, including subject to community adoption, and are not commitments. You should conduct your own research and consult a qualified professional before acting on any information here; any reliance you place on it is strictly at your own risk. All third-party names and trademarks are the property of their respective owners, and their mention does not imply affiliation or endorsement.