feat(api): bound transaction field occurrences before parsing by waynercheung · Pull Request #10 · waynercheung/java-tron

waynercheung · 2026-10-10T08:08:32Z

What does this PR do?

Adds an admission check that counts field occurrences in a submitted Transaction before it is parsed, and rejects it once it exceeds 50,000. It applies to the API entry points that accept a binary Transaction:

gRPC: BroadcastTransaction, GetTransactionSignWeight, GetTransactionApprovedList, GetShieldTransactionHash, CreateCommonTransaction (the Wallet methods whose request type is Transaction)
HTTP: /wallet/broadcasthex

JSON transaction endpoints and gRPC methods with other request types are unchanged.

Changes:

TransactionAdmissionGuard scans the wire bytes with CodedInputStream (the decoder the generated parser uses) and counts every tag, known or unknown. It recurses into sub-messages by descriptor; bytes and string fields are leaves, so large bytecode counts once. For each Contract it first resolves the effective type, type_url and value under protobuf merge rules (last occurrence wins, repeated parameter occurrences merge), then recurses into that value with the payload type TransactionCapsule.getOwner would unpack. A value that is never unpacked (overwritten, mismatched type_url, unknown type) stays a leaf. Group encoding and nesting deeper than 32 are rejected. The guard checks structure only; it does not validate field contents such as UTF-8 in strings.
GuardedTransactionMarshaller replaces the request marshaller of those gRPC methods. It reads the request with the same bound as maxInboundMessageSize, runs the guard and passes the same bytes to the original marshaller, so parsing is unchanged for accepted requests.
A rejection is returned as a per-reason sentinel instead of an exception, because grpc-java closes a call whose marshaller throws with UNKNOWN and logs a SEVERE stack trace each time. AdmissionCheckedHandler recognizes the sentinel by identity and closes the call with INVALID_ARGUMENT; the service method is never invoked.
RpcService.guardTransactionMethods rebuilds each service definition with the guarded marshaller and handler for methods whose proto input type is Transaction. All services in RpcApiService, RpcApiServiceOnSolidity and RpcApiServiceOnPBFT are registered through it (on the SOLIDITY and PBFT ports, around the existing cursor interceptor), so a Transaction method added later is covered too. Only the five FullNode Wallet methods match today.
BroadcastHexServlet runs the guard before Transaction.parseFrom.
Admission rejections use one client message, transaction rejected by admission check, and one DEBUG line per rejected request with the reason and no stack trace (the gRPC line also names the method and remote address).

Block, database and P2P parsing do not use the guard: a valid block must never fail a local, non-consensus limit.

Why are these changes required?

protobuf-java materializes objects for repeated sub-message and unknown-field occurrences. A submitted Transaction containing many empty ret entries (2 bytes on the wire each) can therefore expand to tens of times its wire size on the heap during parseFrom (a 4 MiB request retained about 265 MiB in a local measurement). Empty raw.contract and raw.auths entries and unknown fields with distinct numbers behave the same way. Existing transaction-level checks, such as the transaction size check and signature verification, run after this allocation.

The outer parse keeps the contract payload (Any.value) as bytes, but getOwner unpacks it before signature verification, exposing its nested messages to the same amplification. Checking only the outer message is not enough.

Counting occurrences limits the structural complexity of the scanned schema, and with it the related object allocation, without field-specific rules. It is not an exact object count or heap size; the existing byte limits on requests stay in place.

Behavior changes (release note)

Transport and admission behavior was compared on the same inputs using a local differential harness with stubbed business handlers (gRPC) and a copy of the base servlet with a mocked Wallet (HTTP). Production business-response mappings were additionally checked against source.

Requests exceeding 50,000 occurrences, using group encoding, nesting deeper than 32, or containing wire-malformed effective contract payloads detected by the guard (including in a second contract) are rejected before generated protobuf materialization:
- gRPC: INVALID_ARGUMENT with description transaction rejected by admission check. Previously, inputs that failed to decode returned UNKNOWN ("Application error processing RPC") with a SEVERE log per request; inputs that decoded reached the service method and returned its result (OK with an error code in the body for most methods, mostly INTERNAL for GetShieldTransactionHash).
- /wallet/broadcasthex: {"Error":"transaction rejected by admission check"} with HTTP 200. Previously a decodable request returned result/code/txid, and a malformed one returned internal server error.
For requests newly rejected by the guard that would already have failed downstream broadcast validation, only the error response changes; broadcasthex no longer includes a txid.
This intentionally tightens API admission relative to the existing broadcast path: a contract deployment whose ABI exceeds the budget is rejected at these API entry points even if existing validation would accept it. With ordinary entries (a name, two inputs, one output) this starts at about 4,170 entries; 3,500 entries (42,019 occurrences, 191 KB) pass the admission guard. These are synthetic encoding and guard compatibility samples, not end-to-end validated deployments. The occurrence limit applies only to these API entry points; P2P admission and consensus validation rules are unchanged.
Logging: the per-request SEVERE with stack trace (gRPC) and the per-request DEBUG with stack trace (HTTP request failed, broadcasthex) are replaced by one DEBUG line per rejected request without a stack trace. Exceptions raised inside service methods are unchanged.
Metrics: tron:grpc_service_latency_seconds has only an endpoint label. Rejected calls are now closed through the intercepted call and are counted in this histogram; previously decode failures were closed by grpc directly and were not.
Unchanged:
- Content errors that the guard does not check, such as invalid UTF-8 in a string field of the contract payload, still fail later at unpack as before.
- On broadcasthex, invalid hex and invalid UTF-8 in string fields of the outer Transaction (for example Any.type_url) still return internal server error.
- gzip decompression over maxInboundMessageSize still returns UNKNOWN. A corrupt gzip header still fails inside grpc before the marshaller runs.
- Uncompressed oversize requests still return RESOURCE_EXHAUSTED.
- A mismatched type_url does not itself cause an admission rejection; normal business validation is unchanged.

Budget

Historical coverage: a full scan of a stopped-node copy of a mainnet block database (blocks 0 to 87,034,934, 15.79 billion transactions, no missing blocks). Each transaction was counted with TransactionAdmissionGuard.countTransaction without a limit (the counting code is identical to this PR; only a comment changed since the scan), both as stored and in a submission-shaped representation with ret removed. With ret removed: p50 15, p99.99 43, maximum 3,926 (a contract deployment; 3,928 as stored), about 12.7x below the budget. 523 transactions exceed 1,000, all contract deployments; none exceed 5,000. The largest count for any other contract type is 198 (AccountPermissionUpdateContract). No transaction would be rejected by the guard, for budget or for structure. This shows the headroom over observed usage, not the largest valid transaction.
Allocation: across the tested payload shapes (20 shapes on x86_64/JDK 8 CMS under Rosetta and ARM64/JDK 17 G1/ZGC), the maximum observed cumulative allocation per request was approximately 41 MiB for guarded gRPC marshalling and 133 MiB for broadcasthex processing. These are not peak-live-heap bounds or whole-node availability guarantees, and the JDK 8 timings are not native x86_64 latency evidence.
HTTP body reading, JSON parsing and hex decoding occur before the guard and still incur allocation for rejected requests (about 70 MiB for the largest rejected broadcasthex request in the same measurements).

Design notes

Upstream limits checked, as of grpc-java 1.83.1 and protobuf-java 3.25.8: byte limits (maxInboundMessageSize, CodedInputStream.setSizeLimit) and the recursion limit (setRecursionLimit, default 100). None of them bounds the number of field occurrences materialized from a message, which is what this guard counts. Worth revisiting if protobuf-java adds an element-count or allocation limit to parsing.
The budget limits scanned structural complexity and the associated object amplification; it is not an exact object-count or peak-heap bound.

This PR has been tested by:

Unit Tests
Manual Testing

New tests (59):

TransactionAdmissionGuardTest (28): counts and exact budget boundaries for each amplification shape; merge semantics (last type, type_url and value win, explicit empty overrides); payload recursion including shielded and multiple contracts; first-pass budget; malformed input; depth; concurrency; a corpus of valid transactions; the pinned budget value; no packed repeated scalar reachable from Transaction or any contract type.
AdmissionSupportTest (10): reason mapping, sentinel identity, bounded read edge cases.
RpcServiceAdmissionTest (12): real loopback Netty server. Valid requests pass (plain and gzip); over-budget, malformed, invalid UTF-8 and corrupt gzip bodies that reach the marshaller get INVALID_ARGUMENT without reaching the service method and without a grpc SEVERE log; interceptor permits are released; cancellation; disabledApi still applies; message size limits unchanged.
RpcServiceRegistrationAdmissionTest (4): the definitions actually registered by each RpcService subclass; exactly five methods guarded on FullNode, none elsewhere; each guarded marshaller uses the production budget and maxMessageSize.
BroadcastHexServletAdmissionTest (5): fixed response without calling Wallet, one DEBUG line per rejection without a stack trace.

The new tests together with the existing suites they touch, run locally on JDK 17 (aarch64): 167 classes, 957 tests, all passing:

./gradlew :framework:test --tests 'org.tron.core.admission.*' \
  --tests 'org.tron.common.application.*' \
  --tests 'org.tron.core.services.*' \
  --tests 'org.tron.core.net.*'

This includes the cursor interceptor wiring tests for the SOLIDITY and PBFT ports, which still pass with the guard applied outside the interceptor. checkstyleMain and checkstyleTest pass.

Follow up

P2P inbound messages are not covered and need a separate design: an over-budget transaction there must not disconnect the peer, since honest nodes may relay large transactions that exceed this budget.
gzip decompression over the size limit still logs a SEVERE stack trace per request on every gRPC method. This predates this PR and will be handled together with request byte limits, mapping it to RESOURCE_EXHAUSTED.
Performance: hand the original marshaller a KnownLength stream to restore grpc's array fast path, and simplify the bounded read; to be measured separately.

Extra details

Design choices that reviewers may ask about:

Not configurable: 50,000 is about 12.7 times the largest value in the full mainnet history, and allocation was measured at this value. Changing it should come with new measurements.
TransactionAdmissionGuard.countTransaction is public so that sampling tools use exactly the production counting logic.
TOO_LARGE is a fallback. grpc enforces the same inbound limit first, but the bound keeps the marshaller's allocation independent of transport settings.
The original marshaller still parses the request (instead of Transaction.parser()) so that its configuration and behavior are kept.
The outer isDebugEnabled() check in the handler avoids evaluating the method name, the remote address and the argument array when DEBUG is off.

…ronprotocol#6975)

…l#7014)

…nprotocol#6978)

* fix(jsonrpc): handle null parameters consistently Validate nullable JSON-RPC arguments before they are dereferenced. Return stable protocol errors instead of jsonrpc4j's unhandled -32001 fallback, which exposes Java exception types. Validate eth_call's required transaction argument before its block selector. The double-null case now returns -32602 instead of -32600. Align eth_uninstallFilter lookup misses with geth: null, unknown and already removed IDs return false. Empty and non-hexadecimal strings also return false when no filter matches, without new format checks. Non-null lookup misses now return false instead of -32000. Use the value returned by remove to report whether a filter was removed and avoid the race between checking and removing the same entry. Remove the obsolete uninstall ItemNotFoundException declaration and error mapping. Java binaries remain compatible, but source callers catching that checked exception, or implementations still declaring it, may need adjustment when recompiled. Keep getFilterChanges and getFilterLogs on their existing lookup-miss errors, and preserve request-source precedence and filter isolation. Treat null block-detail flags as false and retain DTO defaults for explicitly null optional fields. Preserve filter wildcard semantics. Add binding, service, wire and VM execution regressions for null arguments, error contracts, filter removal, source isolation and concurrent uninstall results. * fix(jsonrpc): reject null required positional parameters Return invalid params for an explicit null filter ID in eth_uninstallFilter, eth_getFilterChanges and eth_getFilterLogs, and for a null fullTransactionObjects flag in eth_getBlockByHash and eth_getBlockByNumber. The block hash or selector is still validated first, and the flag is checked before any block lookup, so the result does not depend on whether the block exists. Unknown non-null filter IDs keep their existing results. This replaces the earlier null-as-false handling of the flag and the filter ID. go-ethereum v1.17.6 rejects null for these required arguments; earlier releases decoded it as a zero value. * refactor(jsonrpc): share the null check for required parameters Route the null checks for the ten required JSON-RPC parameter positions through one requireParam helper instead of repeating the same if/throw block at each site. The "invalid params" and "invalid filter request" messages are unchanged and are now both private constants in TronJsonRpcImpl. Remove the null check from the LogFilter constructor. In production code, LogFilter is only built through LogFilterWrapper from eth_getLogs and eth_newFilter, and both reject a null filter request before reaching it, so the constructor check and its unit test are dropped. Null filter requests remain covered at the service and wire level. Extract getBlockByHash(byte[]) and use it from ethGetBlockByHash and getBlockByJsonHash, so ethGetBlockByHash no longer inlines the block lookup. The validation order is unchanged. No behavior change. * test(jsonrpc): use the three-argument TronJsonRpcImpl constructor Adapt the tests added for null parameter handling to the constructor that now takes the Manager, and drop the removed setManager call. Tests built only from mocks pass a mocked Manager, and tests backed by the shared database pass dbManager, matching the existing tests.

…col#6985) Replace jsonrpc4j's unhandled -32001 fallback with a fixed -32603 response so unmapped exception types and raw messages no longer reach clients. Keep the documented -32001 contract for net_version and eth_chainId through explicit mappings with fixed message and data. Propagate VirtualMachineError, ThreadDeath, LinkageError, and TronError through the JSON-RPC boundary. Scan complete cause chains without allocating a visited set, detecting cycles without missing fatal causes. Classify Throwables escaping dispatch before logging or mapping them. Before rethrowing an Error, make a best-effort attempt to commit an empty HTTP 500 so the container does not render its default error page. Preserve the original error if cleanup fails, and leave an already committed response untouched. Route single requests through handleRequest because the servlet handle API catches Throwable. Catch RuntimeException, IOException, and Error at both dispatch boundaries, then propagate a classified fatal cause or return -32603 when a response is appropriate. This also covers non-fatal Errors from interceptor hooks. Discard partial dispatch output on failure. Recover each failed batch element without discarding earlier results or skipping later requests. Share internal-error construction across serialization, dispatch, and response-parsing failures. Count only the replacement when parsing fails, retaining existing overflow rules and the current single/batch handling of requests without an id. Log unmapped exceptions, chain-identity failures, and asynchronous log-query failures and interruptions at DEBUG with the Throwable, classifying fatal causes before logging the failures. Log batch request-serialization and response-parsing recoveries at DEBUG with the element index and the Throwable. Log the first escaped dispatch failure in each batch at ERROR with the Throwable; repeats use DEBUG with only the index and exception type. Restore interrupted status for asynchronous log queries. Reject Boolean, object, and array request IDs before dispatch with -32600 and id:null. Reject non-null scalar params with -32600, preserve valid IDs, and isolate invalid batch elements so their siblings run. Missing, null, array, and object params keep their existing semantics. Remove the obsolete request replay wrapper and HTTP status provider. Add resolver, servlet, embedded-Jetty, chain-identity, asynchronous failure, dispatch-contract, and request-envelope regression coverage. Exercise the real HTTP interceptor chain and observe commitment without performing it on the servlet's behalf. Test interceptor Errors with a real server and preserve the existing notification response-suppression rules with characterization tests. Add a package-private server injection seam for servlet tests without changing production initialization.

…ocol#7019)

…col#6946)

…ts (tronprotocol#7021)

…protocol#7025)

The node.rpc.maxRstStream and secondsPerWindow options defaulted to 0, which left the HTTP/2 RST_STREAM frame limit unset, and the limit was configured only when both values were positive. Give them sensible defaults, validate them at load time, and apply the limit unconditionally so the configuration behaves consistently. - NodeConfig: default maxRstStream/secondsPerWindow to 1000/5. In postProcess, reject negative values for either option and reject Integer.MAX_VALUE specifically for maxRstStream, which grpc-java treats as disabling the limit. Independently replace each zero with its default and log a warning. - RpcService: always configure maxRstFramesPerWindow from the config values; validation stays solely in NodeConfig, consistent with the neighboring maxConcurrentCallsPerConnection option. - reference.conf, docs/configuration.md and config/README.md: document the new defaults and the 0 fallback for compatibility. - Tests: pin the Java and reference defaults, preserve positive overrides, and verify PING at the RST limit followed by GOAWAY after one additional reset using a prebuilt HTTP/2 frame burst. Note: an explicit 0 now falls back to the default instead of leaving the limit unset, and maxRstStream = 2147483647 is rejected at startup.

…tocol#7020)

…l#6871)

protobuf-java materializes objects for repeated sub-message and unknown-field occurrences. A submitted Transaction containing many empty ret entries (2 bytes on the wire each) can therefore expand to tens of times its wire size on the heap during parseFrom. Existing transaction-level checks run after this allocation. The contract payload (Any.value) is also unpacked by TransactionCapsule.getOwner before signature verification, exposing its nested messages to the same amplification. Scan the wire bytes before parsing on the API entry points that accept a binary Transaction, and reject the request once it exceeds 50,000 field occurrences: - TransactionAdmissionGuard counts every tag with CodedInputStream, recurses into known sub-messages by descriptor, and recurses into the effective contract payload with the type getOwner would unpack, following protobuf merge semantics. Group encoding and nesting deeper than 32 are rejected. Block, database and P2P parsing do not use it. - For gRPC, every method whose request is a Transaction gets a request marshaller that reads the request with the inbound size bound, runs the guard and hands the same bytes to the original marshaller. A rejection returns a sentinel instead of throwing, and a handler wrapper closes the call with INVALID_ARGUMENT before the service method runs. This affects BroadcastTransaction, GetTransactionSignWeight, GetTransactionApprovedList, GetShieldTransactionHash and CreateCommonTransaction. - /wallet/broadcasthex runs the guard before Transaction.parseFrom. - Admission rejections use one fixed client message and one DEBUG line per rejected request, without a stack trace. Across the full mainnet history (15.79 billion transactions), the maximum observed count was 3,926 field occurrences. This is an API admission policy, not a consensus-validity limit: previously tolerated encodings and unusually complex transactions may now be rejected at these entry points. Consensus, execution and block validation are unchanged.

waynercheung force-pushed the feat/tx-admission-guard branch 2 times, most recently from 9481e6c to 6b4019f Compare October 10, 2026 08:28

Repository owner deleted a comment from github-actions Bot Oct 10, 2026

0xbigapple and others added 4 commits October 10, 2026 17:10

refactor(api): remove dead WalletExtension gRPC service and config (t…

8bb6150

…ronprotocol#6975)

feat(plugins): backfill section bloom (tronprotocol#6973)

91371cf

refactor(crypto): remove unused SM2 and SM3 (tronprotocol#6999)

f62b25f

fix(config): close resource streams (tronprotocol#7002)

1575c4f

waynercheung force-pushed the feat/tx-admission-guard branch 2 times, most recently from b1677c0 to 1d0c6fe Compare October 11, 2026 15:17

317787106 and others added 7 commits October 12, 2026 11:57

refactor(proto): remove unused proto discover definition (tronprotoco…

e56ddd9

…l#7014)

feat(framework): remove --keystore-factory support from FullNode (tro…

00661e8

…nprotocol#6978)

refactor(trie): remove unused RLP utilities (tronprotocol#6991)

1ff2aae

fix(config): fail startup on invalid DNS publish parameters (tronprot…

7ebc2eb

…ocol#7019)

feat(crypto): harden ECKey validation (tronprotocol#6996)

4c9b5f4

waynercheung force-pushed the feat/tx-admission-guard branch from 1d0c6fe to 0aa4020 Compare October 13, 2026 04:46

halibobo1205 and others added 9 commits October 13, 2026 14:35

fix(toolkit): keep source DB intact when DbMove copy fails (tronproto…

c9bb7d8

…col#6946)

refactor: remove a deprecated KhaosDatabase overload and unused impor…

9ff3596

…ts (tronprotocol#7021)

test(framework): await reward calculation before mutating store (tron…

e1ffb1c

…protocol#7025)

feat(API): refactor merge http servlets (tronprotocol#6976)

c78ef93

feat(API): refactor merge rpc (tronprotocol#6977)

a975a3d

fix(http): time out idle connections at the connection limit (tronpro…

cfcd63e

…tocol#7020)

docs: add AGENTS.md and reorganize project documentation (tronprotoco…

a901f85

…l#6871)

waynercheung force-pushed the feat/tx-admission-guard branch from 0aa4020 to 6284602 Compare October 13, 2026 13:54

waynercheung closed this Oct 14, 2026

Navigation Menu

Repository navigation

feat(api): bound transaction field occurrences before parsing - #10

feat(api): bound transaction field occurrences before parsing#10
waynercheung wants to merge 20 commits into
release_v4.8.3from
feat/tx-admission-guard

waynercheung commented Oct 10, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

waynercheung commented Oct 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

waynercheung commented Oct 10, 2026 •

edited

Loading