Repository navigation
Conversation
For Bitvector[N], when N is not a multiple of 8, the spec serializer does not write the top 8 - N%8 bits of the last byte. These bits are always 0 (consensus-specs ssz/simple-serialize.md, EthLib paddingBits error). The generated decoder did not check these bits. It accepted 0x10 for Bitvector4. A formal-verification counterexample found this. - ssz.ValidateBitvector(buf, bits): checks that the length is (bits+7)/8 and that the padding bits are 0. - Decoder: adds a ValidateBitvector call for each go-bitfield BitvectorN with N%8 != 0 (at this time, only Bitvector4). When N%8 == 0, the generated code does not change. - The encoder also rejects the same values. The EthLib value layer has no padding bits. Thus a Go value with a padding bit set does not represent a spec value. If the encoder emitted such a value, it would make bytes that the decoder of the same type rejects. This is the same kind of bug as the Bitlist length check in the marshaler. - The hash tree root (HTR) does not change. It still uses hh.PutBytes, which has no error path. Tests: ssz TestValidateBitvector (N = 4, 9, 16, padding 0 or 1, wrong length). TestBitvectorPadding in a new compiled fixture, sszgen/testutil/genfixture. The generated file is committed, and sszgen.TestGenFixture pins it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Bitlist encoder used tmplGenerateMarshalValueList (`len(field) > MaxSize`). field is a byte array, but MaxSize is a limit in bits. Thus the encoder encoded bitlists of 258 to 2048 bytes without an error, and the decoder of the same type (ssz.ValidateBitlist) rejected that output. The encoder also accepted an empty array and a last byte of 0. A formal-verification counterexample found this. The spec serializer (consensus-specs ssz/simple-serialize.md BitList[N]) emits e//8+1 bytes and puts the delimiter bit at bit e. Thus the encoding is not empty, the last byte is not 0, and e <= N. This rule is the same as ssz.ValidateBitlist in the decoder. - Non-progressive bitlist: marshal calls ssz.ValidateBitlist(field, limit). - Progressive bitlist: marshal calls ssz.ValidateProgressiveBitlist(field). Like other progressive lists, it does not check a limit. The spec has no limit. - A bitlist in a list or vector element position uses the same path. Tests: genfixture TestBitlistLimit (empty array, last byte 0, 2048 bits, 2049 bits, 258 bytes, 0 bits). The test checks that the decoder and the encoder give the same result. It also checks the round trip of the accepted values. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For a Go array vector field such as [N]uint64, UnmarshalSSZ emitted
`c.A = make([]uint64, N)`. This code did not compile (unmarshalVectorTmpl
in sszgen/render/unmarshal.go). An array destination already has N
elements. Thus the decoder now emits no make and writes the elements in
place. The vector template for variable-size elements gets the same
condition.
Tests: genfixture gets Limbs{A [4]uint64}. The generated code compiles.
TestUint64Array checks the spec serialization (4 little-endian uint64
values), the round trip and the root. The value is one chunk, so the
root is equal to the serialization.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The container decoder checked the first variable offset with
`off != fixedSize → ErrInvalidVariableOffset` and then with
`off > size → ErrOffset`. The size check at the start of UnmarshalSSZ
already returns ErrSize when size < fixedSize. Thus, at the second
check, off == fixedSize <= size. No input reaches the ErrOffset branch.
An example is the dead ErrOffset in ListU64. No meaningful input can
make this branch reachable, so this change removes it. The behavior
does not change.
Tests: genfixture gets ListU64{A uint64; L []uint64 ssz-max:"4"}.
TestListU64FirstOffset checks that the decoder accepts offset 12. It
also checks that the decoder rejects 0, 11, 13, 20 and 0xffffffff with
ErrInvalidVariableOffset (EthLib/remerkleable: first offset = size of
the fixed part). This change regenerates the generated golden files.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The generated MarshalSSZTo writes the offset table and the earlier fields before it checks the lengths of later fields (ErrBytesLength, ErrListTooBig and others). Examples are Scalars, ListU64 and FixedVec. On error, partial output stays in the array past len(dst). This change documents the behavior (option b). It does not move the checks before the writes (option a). Reasons: - All writes are appends, so only the array past len(dst) changes. On error, the function returns nil. Thus dst[:len(dst)], which the caller sees, does not change. - Option a needs a separate pass that checks the full value before any write. The MarshalSSZTo of nested containers and of delegated types writes before its own checks. Thus, if only the top-level checks move to the start, partial writes still occur. Option a is not the shortest change. The ssz.Marshaler documentation now states the promise (nil result, dst[:len(dst)] does not change) and the unspecified range (past len(dst)). The test TestMarshalSSZToErrorKeepsPrefix pins only the promised part. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
merkleizeChunks in ssz/htr.go calls gohashtree.Hash(elements, elements). The input and the output use the same array. The formal-verification work did not prove that this aliasing is safe. It was a premise. To check it, I read the source of gohashtree v0.0.5-beta (the version in go.mod). The results are: - generic: reads chunks 2k and 2k+1, then writes digest k, in k order. - amd64 avx1 x4/x1, avx2 x8, avx512 x16, shani x2/x1: each loop completes all DATA_PTR loads of the batch before it stores to OUTPUT_PTR. - arm64 neon x4/x1, sha2 x1: the same. In x4, the WORD-encoded instructions are LD4 (base R1 = DATA_PTR, in rounds 0-15) and ST4 (base R0 = OUTPUT_PTR, at the end). A batch k0..k0+L-1 writes only below k0+L. The next batch reads from 2(k0+L). Thus the in-place call is safe. A comment on that line records this result. Note: an arm64 host does not run only the generic path. hash_arm64.go sets supportedCPU = true. The test host (Apple M5 Pro) has cpuid SHA2 = true, so it uses the arm64 sha2 path. With GOARCH=amd64 (Rosetta, no AVX2 or SHA), it uses the generic path. Test TestMerkleizeInPlaceHash: for 1..40 pairs, the in-place call, a call with a separate output, and crypto/sha256 (the spec definition) give the same result. The test passed on both paths (arm64 sha2 and generic). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When bits >= MaxUint64-6, the uint64 addition in `(bits+7)/8` overflows and the result is 0..6. Then an empty buf passes the length check, and the next line reads buf[-1] and panics. A review found this. This change computes the byte count from bits/8 and bits%8. Tests: TestValidateBitvector gets new cases. It checks the error return for N = MaxUint64, MaxUint64-6 (empty buf) and MaxUint64-3 (1 byte). It also checks that N = 0 with an empty buf is accepted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After a successful decode, checkBits encoded the input `in` again.
Thus the test passed even when the decoder succeeded but damaged the
value. A review found this: the test did not detect the mutation
`c.V4[0] = 0`. Now the test compares the fields of the decoded `dec`
with the input. It also encodes `dec` again and compares the result
with `enc`. I put in the same mutation and checked that the test now
fails.
This change also adds a compile and run test for arrays with
variable-size elements. genfixture gets Inner{B []byte ssz-max:"8"}
and InnerPair{P [2]*Inner}. TestVariableElementArray compares the
encoding with bytes made by hand from the spec serialization (outer
offset 4, element offsets 8 and 14). It also checks the decoded values
and the rejection of a wrong first element offset.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A review asked for a narrower supported scope in the comments. This change narrows it. - ssz.Marshaler: the comment now says that the "dst[:len(dst)] does not change" promise of the generated code holds only when each delegated MarshalSSZTo that it calls keeps the same contract. - render marshalOp.Overlay: the check applies only to go-bitfield types. Delegated types and HTR are out of scope. core.Dispatch delegates before Overlay. HTR still hashes a BitvectorN with padding bits set. - Progressive bitlist: the encoder applies no limit. The decoder applies a local limit when the field has ssz-max. The comment now records this existing asymmetry. The generated code does not change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Member
Author
|
Superseded by #27 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.