Skip to content

feat: deterministic derivation of L2 item's parent bundle id & block height#894

Open
charmful0x wants to merge 2 commits intoedgefrom
expr/dataitem-parent-of
Open

feat: deterministic derivation of L2 item's parent bundle id & block height#894
charmful0x wants to merge 2 commits intoedgefrom
expr/dataitem-parent-of

Conversation

@charmful0x
Copy link
Copy Markdown

about

this PR introduces ~arweave@2.9/parent-of=<L2_DATAITEM_ID> to resolve an indexed ANS-104 dataitem to the L1 Arweave transaction that contains it (and the containing block)

e2e flow

  • read DATAITEM_ID from the local copycat's arweave offset index
  • use the child start-offset to find the arweave block containing that byte position
  • read that block’s L1 tx list
  • for each L1 tx in the block, read its indexed offset/length
  • teturn the tx with byte range contains the child offset

res shape

  #{
    <<"id">> => ParentTxID,
    <<"offset">> => ParentTxStartOffset,
    <<"length">> => ParentTxLength,
    <<"child">> => ChildDataItemID,
    <<"child-offset">> => ChildStartOffset,
    <<"block-height">> => BlockHeight,
    <<"block-hash">> => BlockIndepHash
  }

block lookup & perf

block lookup uses binary search over arweave block heights. each block has:

end = weave_size
start = weave_size - block_size

so we determine thechild belonging to the block iff start <= child_offset < end

so each binary-search step fetch or read one block, compares the child offset to that block byte range, and search lower/higher heights -- ~ log2(height)

performance tldr:

  • children items and parent tx offsets come from the local copycat/arweave offset index
  • block metadata read go through the existing dev_arweave_block_cache
  • a cold lookup would fetch midpoint block metadata remotely (chain nodes) while the binary search warms the cache (a few seconds, ~< 10s for a warmup)
  • after warmup, repeated lookups for the same block or nearby offsets are local and very fast ( thats because the midpoint blocks and final target block are cached)
  • GQL-less & deterministic

N.B. if a bundler retries settling the same L2 dataitem / bundle, it can exist multiple times in the weave. arweave.net strategy is reporting the latest indexed settlement retry for budndled in -- but parent-of= return the parent for the locally indexed occurrence in local-node copycat. so bundle-id/block-height can differ across indexers

@charmful0x
Copy link
Copy Markdown
Author

for testing it:

application:ensure_all_started(hb).
NodeMsg = hb_opts:default_message().

Opts = maps:merge(NodeMsg, #{
    <<"arweave-mempool-progress">> => true,
    <<"arweave-index-workers">> => 1,
    <<"arweave-pending-chunk-poll-attempts">> => 20,
    <<"arweave-pending-chunk-poll-ms">> => 500
}).


  hb_ao:resolve(
      <<"~copycat@1.0/arweave&mode=write&from=1908215&to=1908215">>,
      Opts
  ).

cold start:

time curl -I http://localhost:8734/~arweave@2.9/parent-of=-f0U9LgZLdD5kd-piF6sINZNCFizS7s_7O8hgagHpKw
HTTP/1.1 200 OK
access-control-allow-methods: GET, POST, PUT, DELETE, OPTIONS
access-control-allow-origin: *
access-control-expose-headers: *
ao-types: block-height="integer", child-offset="integer", length="integer", offset="integer", status="integer"
block-hash: AS7e1yzz8ISobLSH0ZA-BU61ZPuLY02K_T-j2HgsQY5K2Cxtq7tASegIZLzaXqpL
block-height: 1908215
child: -f0U9LgZLdD5kd-piF6sINZNCFizS7s_7O8hgagHpKw
child-offset: 386957230789816
content-type: text/html
date: Thu, 30 Apr 2026 12:20:13 GMT
id: FzPg1dcc-9Lqb18G9zuw-b7XdXOiKz5ESL7STw8ZIlY
length: 50285
offset: 386957230776566
server: Cowboy
signature: comm-qjdumdpwmau-odxzl24z-_jcjg6tf2n2trxowwjtyk0=:HcvS9St6mGfB5jImW+AeQ2ULr6qMlmmZPo5oakbaNnFWfBbRw/2dOnBIuQTwwvP6Pxo/pc0nOy8x/IvuUsf8/0V3IqdqhNAiYkRJEW6wjF6BgoZYnPWFytmtgLrevAAiivoBOg3lIikGZ4PdZYX4E12pS8FcpvgfKHl4lPPgNMGLsqYqTpIoPNan0nQxnKxaL0y+u4dO9DcL+XvHE2FVi5uEzP4NLvZC6R2MDHhZVn9dylL4PTfWWS5vveAlAZIxIWISHSkdwQ1eswsVe/eVREF/K+U6kr6WJQWNHqqlPZ7gvfh1/gCJZt0Y+qwmzDne2qmsYaEznoLn4B/gl+Luqj0nazMIj1NbOZfoC1gosFeNwEHT7N7JY1WDDH8dE3yz3Q7Y0gJR+qWrHxbTpITU4MW8bkpXzeFegIWL7yicJpk7t+9QoP2E1awtd8B6M7guKLBMoVRCY80U4sBoTDfcaE6ULZp3pTv479EZ1NB/FAnxAkF+113LQTp5Oa+zGIXDdTivneKWIVtClKOWTrDcsV7b+jVGCyzEGi05xYXuxH2W8BVkfqTAKzWMIQCQCjQv+nI71+hXFVRhoW/YavvLFJaJKjsyV62nzlKqVcGw2wSVEc4mBuIYyFqdHIWlvKMN7462Gz2rjqYS4akwptSfnG2T44WwkJvnQJ2Xercqnho=:, comm-uh4zw57hpeaogqa5f6xvixfh5pyajnncqiqth1-vite=:D8zXzEb2QrJtP6WsUYlvy/35QQq4S/1jIDdrK+BCOJM=:
signature-input: comm-qjdumdpwmau-odxzl24z-_jcjg6tf2n2trxowwjtyk0=("ao-types" "block-hash" "block-height" "child" "child-offset" "id" "length" "offset" "status");alg="rsa-pss-sha512";keyid="publickey:ubjxPbinZNEMfe5KhqMfxb6OJbHBAZADtv+b9ZTso+FeWTETveBLQg+e0mDMajmabEdIGbNN4E788NNRHEfnOTrLfeRGbVxBYr3OruLYnoSaMzRhGthOCIM0SewqCfTh5jTHPBIC3fX0Q2846B0+CjxIJ4oavCDwfxJh/PU99n4yNLVpBajaLtanSkfxMsfH9dmFOnr1yZGUxddsbH1ed44I0Q4IJgE0yVt9e0D9mfEOlTSieWpiZax42BKON9/a6o4nj+GFFoIiIBpV0a7W5gA2bdshC6r60Q5S4LQha+IKolo0vXBf3IBci3drh3c2kjVaJzqRHp15O6d6aW5VQ5FgFyfWPzQHKEB+4QlZDO7IW7B4VEEMHGXP85hc3f3fPBlryjF3zhLKTMN4KLrDo6RFOtJecQancvhBv3400SA1uJ+DHxrnNTNBcw1qW1lQ4//0u6pn1W/ne3M4BwomgJWfcKSiYeCDhiaIX3GZLvGdQH3vzWveYIQQU7ACLH/vYq38c1nmveJSy9N9GRRFm7Ufp2ayLUEI1lVvQUdk+9CNfBn6NX1X2do7THie5nll+LabkkJuotIC4bJ+crXutyW9FbdMdicP91h+PrOmVjm1tXv4DqnDfM6Z8RYvJjL60my6twyRp8Q4xnFAc122O8RaWT1K7m4cktbIM8UkU9s=";tag="vZypt2S1L3_7hqX6xp2V-HCA79Gi6jt6bY0_RzaaKuA/sepy-569hzfYYkBGWn8Ud6dShcy8Gho0ID3GH7zyI5Q", comm-uh4zw57hpeaogqa5f6xvixfh5pyajnncqiqth1-vite=("ao-types" "block-hash" "block-height" "child" "child-offset" "id" "length" "offset" "status");alg="hmac-sha256";keyid="constant:ao"
status: 200
transfer-encoding: chunked


real	0m1.896s
user	0m0.002s
sys	0m0.006s

warmed up:

time curl -I http://localhost:8734/~arweave@2.9/parent-of=-f0U9LgZLdD5kd-piF6sINZNCFizS7s_7O8hgagHpKw
HTTP/1.1 200 OK
access-control-allow-methods: GET, POST, PUT, DELETE, OPTIONS
access-control-allow-origin: *
access-control-expose-headers: *
ao-types: block-height="integer", child-offset="integer", length="integer", offset="integer", status="integer"
block-hash: AS7e1yzz8ISobLSH0ZA-BU61ZPuLY02K_T-j2HgsQY5K2Cxtq7tASegIZLzaXqpL
block-height: 1908215
child: -f0U9LgZLdD5kd-piF6sINZNCFizS7s_7O8hgagHpKw
child-offset: 386957230789816
content-type: text/html
date: Thu, 30 Apr 2026 12:21:10 GMT
id: FzPg1dcc-9Lqb18G9zuw-b7XdXOiKz5ESL7STw8ZIlY
length: 50285
offset: 386957230776566
server: Cowboy
signature: comm-bgwro8g-zq6d-ai1oocfvy42natqgq6uqjsw8r1pkqy=:PQDCvYEgdFkF663j9FCZDdrvUR4Q0mP52Jcj+KJ0TVxv8PLASU+6tmZ2L0jxxqd4FYQCOuJZzP1F/f07UE8xazcOH/xB3ZNQDeHqopBSkw1lfA2EECNXdnmWgORoQB6tEpsoSvjL7Z3/dAPtl5e29JZAFC7dBlstFyBPGEq/eosJcI0Zn/hZobqdx5u+ja0+kjsPypzXoQNFLir2jt9sPbdK4wj52DMttlZcZ/RHW11gAaanO8tPleOB7NdQz6tfSFnR5iANOE7WN72GHiFKeCsiSHXE6QrkQrwLLP+PNyGh503EKZUiTt+6eDex07ffT1sRGyuFK7X8DcQEGJ6p+PnCk3frK2j6Hz/gQYSHgXoNDbaxodJMhvumR0XdzGQFtCL2ELYGEVZgD1mEzYfm7d2mGyf23wJBLM6q2Khc152DabF+7y3vMGuMb8HbjhvCtm1hZW5I9TBE1aS+50itEfQEnbcKETcx6igPRu4q8+vLz5yOUQ5c/7trI6dIY7SsyFyxU/9AI7YPLYzK4GdIe/KBsxKSt8mt5/ZswaweANcHAdyp2ruk0KLbAmDq/e7+z+nzATuUWsTOCKCSsQ1dYDZQW5UnqHQHRWZbBP3lEAV6fiSxk02IAHsgIAwXWbye0ZS57wQr9U0IuIMTNLE7Fkhda48pn7eMkJfMd70CdqQ=:, comm-uh4zw57hpeaogqa5f6xvixfh5pyajnncqiqth1-vite=:D8zXzEb2QrJtP6WsUYlvy/35QQq4S/1jIDdrK+BCOJM=:
signature-input: comm-bgwro8g-zq6d-ai1oocfvy42natqgq6uqjsw8r1pkqy=("ao-types" "block-hash" "block-height" "child" "child-offset" "id" "length" "offset" "status");alg="rsa-pss-sha512";keyid="publickey:ubjxPbinZNEMfe5KhqMfxb6OJbHBAZADtv+b9ZTso+FeWTETveBLQg+e0mDMajmabEdIGbNN4E788NNRHEfnOTrLfeRGbVxBYr3OruLYnoSaMzRhGthOCIM0SewqCfTh5jTHPBIC3fX0Q2846B0+CjxIJ4oavCDwfxJh/PU99n4yNLVpBajaLtanSkfxMsfH9dmFOnr1yZGUxddsbH1ed44I0Q4IJgE0yVt9e0D9mfEOlTSieWpiZax42BKON9/a6o4nj+GFFoIiIBpV0a7W5gA2bdshC6r60Q5S4LQha+IKolo0vXBf3IBci3drh3c2kjVaJzqRHp15O6d6aW5VQ5FgFyfWPzQHKEB+4QlZDO7IW7B4VEEMHGXP85hc3f3fPBlryjF3zhLKTMN4KLrDo6RFOtJecQancvhBv3400SA1uJ+DHxrnNTNBcw1qW1lQ4//0u6pn1W/ne3M4BwomgJWfcKSiYeCDhiaIX3GZLvGdQH3vzWveYIQQU7ACLH/vYq38c1nmveJSy9N9GRRFm7Ufp2ayLUEI1lVvQUdk+9CNfBn6NX1X2do7THie5nll+LabkkJuotIC4bJ+crXutyW9FbdMdicP91h+PrOmVjm1tXv4DqnDfM6Z8RYvJjL60my6twyRp8Q4xnFAc122O8RaWT1K7m4cktbIM8UkU9s=";tag="vZypt2S1L3_7hqX6xp2V-HCA79Gi6jt6bY0_RzaaKuA/sepy-569hzfYYkBGWn8Ud6dShcy8Gho0ID3GH7zyI5Q", comm-uh4zw57hpeaogqa5f6xvixfh5pyajnncqiqth1-vite=("ao-types" "block-hash" "block-height" "child" "child-offset" "id" "length" "offset" "status");alg="hmac-sha256";keyid="constant:ao"
status: 200
transfer-encoding: chunked


real	0m0.052s
user	0m0.004s
sys	0m0.006s

i also tested it with running the copycat as:

curl -s "http://localhost:8734/~cron@1.0/every?interval=1-second&cron-path=~copycat@1.0/arweave&from=-1&to=-19"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant