Skip to content

[Java] Size DTO var-data byte arrays from a bounds-checked wrap - #1151

Open
slachiewicz wants to merge 2 commits into
aeron-io:masterfrom
slachiewicz:java-dto-vardata-length-check
Open

slachiewicz wants to merge 2 commits into
aeron-io:masterfrom
slachiewicz:java-dto-vardata-length-check

Conversation

@slachiewicz

Copy link
Copy Markdown
Contributor

Generated DTOs decoded raw var data with new byte[decoder.xLength()], taking the array size straight from the length prefix. A frame whose prefix is larger than the buffer therefore allocated (or threw OutOfMemoryError or NegativeArraySizeException) before any bounds check. The DTO now wraps the field first and sizes the array from the wrapped view, so an out-of-range length fails with IndexOutOfBoundsException from the buffer's bounds check. String var data goes through the generated String accessor, which the var-data string length PR covers.

Part of #969.

Verified: DtoVarDataLengthTest fails on master (OutOfMemoryError, NegativeArraySizeException) and passes here; sbe-tool checkstyle, tests and verifyJavaIrCodecsInSync pass.

Comment on lines +811 to +813
decoderValueExtraction.append(blockIndent).append("UnsafeBuffer ")
.append(dataVar).append("View = new UnsafeBuffer();\n")
.append(blockIndent).append("decoder.wrap").append(toUpperFirstChar(formattedPropertyName))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This approach only works if Agrona's bounds checks are not disabled (i.e. agrona.disable.bounds.checks is false). If they are disabled no checks are done and the tests fail with OOME and segfault JVM instead.

We should have an explicit length check similar to #1150.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks. Replaced the bounds-checked-wrap approach with an explicit length check against decoder.buffer().capacity() - decoder.limit() - <prop>HeaderLength() before allocating (plus a negative check for uint32 length fields), mirroring #1150. Generated DTOs now fail with IndexOutOfBoundsException independently of Agrona's bounds-check setting; verified DtoVarDataLengthTest passes both normally and with -Dagrona.disable.bounds.checks=true.

The DTO decoder must take the array size from a bounds-checked wrap of
the field, not from the length prefix, so a length past the end of the
buffer fails before anything is allocated.
Generated DTOs sized byte[] var data from a bounds-checked wrap. That
relies on Agrona bounds checks being enabled; with
agrona.disable.bounds.checks=true the wrap does not validate the length
and an oversized/negative prefix leads to OOME or a native crash.

Generate an explicit length check against the buffer capacity, current
limit and header length (as in PR aeron-io#1150) before allocating, including a
negative check for uint32 length fields.
@slachiewicz
slachiewicz force-pushed the java-dto-vardata-length-check branch from 7024d1d to c90f2df Compare November 19, 2026 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants