Skip to content

Commit 019fb82

Browse files
tac0turtletac0turtle
authored
chore: upstream reth custom payload (#1)
# Summary This PR introduces Lumen, a specialized integration layer that enables Reth to work seamlessly with Rollkit by implementing custom Engine API transaction submission capabilities. This allows Reth to function as an execution client for Rollkit-based rollups. ## Key Changes ### 🏗️ Core Implementation - Custom Payload Builder (crates/node/): Implements RollkitPayloadBuilder that supports transaction submission via Engine API attributes - Rollkit Types (crates/rollkit/): Defines custom Engine API types including RollkitEngineTypes and RollkitPayloadAttributes for transaction encoding/decoding - Main Binary (bin/lumen/): Complete node implementation with CLI, configuration, and custom Engine API validator ### 🧪 Testing Infrastructure - E2E Tests (crates/e2e-tests/): Comprehensive test suite covering: - Engine API transaction submission - Payload building and validation - Integration with standard Reth components - Transaction pool bypass functionality ### 🔧 Development & CI/CD - GitHub Workflows: Added complete CI pipeline including: - Unit tests, integration tests, and E2E tests - Linting (Rust, TOML, Markdown, GitHub Actions) - Dependency management and security scanning - Docker image building and release automation - Docker Support: Multi-stage Dockerfiles for optimized production builds - Documentation: Comprehensive README with architecture overview, usage instructions, and development guidelines ### 📋 Project Setup - Cargo Workspace: Organized monorepo structure with shared dependencies - Configuration: Added development tools configuration (rustfmt, clippy, deny) - Licensing: Dual-licensed under MIT and Apache 2.0 - Issue Templates: Bug report and feature request templates for better collaboration #### Technical Highlights 1. Engine API Extensions: Modified Engine API to accept transactions directly in payload attributes, bypassing the traditional mempool 2. Custom Validation: Implemented RollkitEngineValidator that relaxes certain validation rules for rollup compatibility 3. Gas Limit Flexibility: Support for custom gas limits per payload request 4. State Execution: Full transaction execution with proper state transitions and receipt generation ## Testing - ✅ Unit tests for core components - ✅ Integration tests for node functionality - ✅ E2E tests for complete transaction flow - ✅ Linting and formatting checks
1 parent f446dfd commit 019fb82

48 files changed

Lines changed: 16143 additions & 1 deletion

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/CODEOWNERS‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# CODEOWNERS file for automatic PR assignment
2+
# https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
3+
4+
* @rollkit/binary-builders

‎.github/ISSUE_TEMPLATE/bug_report.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
---
2+
name: Bug report
3+
about: Create a report to help us improve
4+
title: '[BUG] '
5+
labels: 'bug'
6+
assignees: ''
7+
---
8+
9+
## Description
10+
11+
A clear and concise description of what the bug is.
12+
13+
## To Reproduce
14+
15+
Steps to reproduce the behavior:
16+
1. Go to '...'
17+
2. Click on '....'
18+
3. Scroll down to '....'
19+
4. See error
20+
21+
## Expected behavior
22+
23+
A clear and concise description of what you expected to happen.
24+
25+
## Screenshots
26+
27+
If applicable, add screenshots to help explain your problem.
28+
29+
## System information
30+
31+
- OS: [e.g. Ubuntu 22.04]
32+
- Lumen version: [e.g. v0.1.0]
33+
- Rust version: [e.g. 1.81.0]
34+
35+
## Logs
36+
37+
```
38+
Please paste any relevant log output here
39+
```
40+
41+
## Additional context
42+
43+
Add any other context about the problem here.

‎.github/ISSUE_TEMPLATE/feature_request.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
---
2+
name: Feature request
3+
about: Suggest an idea for this project
4+
title: '[FEATURE] '
5+
labels: 'enhancement'
6+
assignees: ''
7+
---
8+
9+
## Is your feature request related to a problem? Please describe.
10+
11+
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
12+
13+
## Describe the solution you'd like
14+
15+
A clear and concise description of what you want to happen.
16+
17+
## Describe alternatives you've considered
18+
19+
A clear and concise description of any alternative solutions or features you've considered.
20+
21+
## Additional context
22+
23+
Add any other context or screenshots about the feature request here.

‎.github/assets/Dockerfile‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# syntax=docker/dockerfile:1
2+
# CI-specific Dockerfile for E2E tests
3+
4+
FROM ubuntu:24.04
5+
6+
# Copy the pre-built binary
7+
COPY dist/lumen /usr/local/bin/lumen
8+
9+
# Make it executable
10+
RUN chmod +x /usr/local/bin/lumen
11+
12+
# Create a non-root user
13+
RUN useradd -m -s /bin/bash lumen
14+
15+
# Switch to non-root user
16+
USER lumen
17+
WORKDIR /home/lumen
18+
19+
# Expose default ports
20+
EXPOSE 8545 8546 30303 6060 9001
21+
22+
# Set the entrypoint
23+
ENTRYPOINT ["/usr/local/bin/lumen"]

‎.github/branch-protection.md‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# Branch Protection Rules
2+
3+
This document outlines the recommended branch protection rules for the `main` branch.
4+
5+
## Required Status Checks
6+
7+
The following GitHub Actions workflows should pass before merging:
8+
9+
### Critical Checks
10+
- `lint / lint success`
11+
- `unit / unit success`
12+
- `integration / integration success`
13+
- `e2e / e2e`
14+
- `lint-actions / actionlint`
15+
16+
### Optional but Recommended
17+
- `docker / build` (for releases)
18+
19+
## Settings
20+
21+
1. **Require a pull request before merging**
22+
- Require approvals: 1
23+
- Dismiss stale pull request approvals when new commits are pushed
24+
- Require review from CODEOWNERS
25+
26+
2. **Require status checks to pass before merging**
27+
- Require branches to be up to date before merging
28+
- Status checks listed above
29+
30+
3. **Require conversation resolution before merging**
31+
32+
4. **Require signed commits** (optional but recommended)
33+
34+
5. **Include administrators** (optional, depends on team preference)
35+
36+
6. **Restrict who can push to matching branches**
37+
- Add team/users who can push directly (for emergency fixes)
38+
39+
## Setting Up
40+
41+
To apply these rules:
42+
43+
1. Go to Settings → Branches in your GitHub repository
44+
2. Add a rule for the `main` branch
45+
3. Configure the settings as described above
46+
4. Save changes
47+
48+
## Merge Queue (Optional)
49+
50+
Consider enabling GitHub's merge queue for better handling of multiple PRs:
51+
- Maximum PRs to merge: 5
52+
- Minimum PRs to merge: 1
53+
- Wait time: 5 minutes

‎.github/dependabot.yml‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
version: 2
2+
updates:
3+
# Enable version updates for Rust dependencies
4+
- package-ecosystem: "cargo"
5+
directory: "/"
6+
schedule:
7+
interval: "weekly"
8+
day: "monday"
9+
open-pull-requests-limit: 10
10+
reviewers:
11+
- "lumen-maintainers"
12+
labels:
13+
- "dependencies"
14+
- "rust"
15+
commit-message:
16+
prefix: "chore"
17+
include: "scope"
18+
19+
# Enable version updates for GitHub Actions
20+
- package-ecosystem: "github-actions"
21+
directory: "/"
22+
schedule:
23+
interval: "weekly"
24+
day: "monday"
25+
open-pull-requests-limit: 5
26+
reviewers:
27+
- "lumen-devops-team"
28+
labels:
29+
- "dependencies"
30+
- "github-actions"
31+
commit-message:
32+
prefix: "ci"
33+
include: "scope"

‎.github/pull_request_template.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
## Description
2+
3+
<!-- Please include a summary of the changes and the related issue. -->
4+
5+
## Type of Change
6+
7+
- [ ] Bug fix (non-breaking change which fixes an issue)
8+
- [ ] New feature (non-breaking change which adds functionality)
9+
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
10+
- [ ] Documentation update
11+
- [ ] Performance improvement
12+
- [ ] Refactoring
13+
14+
## Related Issues
15+
16+
<!-- Link to any related issues -->
17+
Fixes #(issue)
18+
19+
## Checklist
20+
21+
- [ ] I have performed a self-review of my code
22+
- [ ] I have commented my code, particularly in hard-to-understand areas
23+
- [ ] I have made corresponding changes to the documentation
24+
- [ ] My changes generate no new warnings
25+
- [ ] I have added tests that prove my fix is effective or that my feature works
26+
- [ ] New and existing unit tests pass locally with my changes
27+
- [ ] Any dependent changes have been merged and published
28+
29+
## Testing
30+
31+
<!-- Describe the tests you ran to verify your changes -->
32+
33+
## Additional Notes
34+
35+
<!-- Add any additional notes or context about the PR here -->

‎.github/workflows/dependencies.yml‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# Runs `cargo update` periodically.
2+
3+
name: Update Dependencies
4+
5+
on:
6+
schedule:
7+
# Run weekly
8+
- cron: "0 0 * * SUN"
9+
workflow_dispatch:
10+
# Needed so we can run it manually
11+
12+
env:
13+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
14+
BRANCH: cargo-update
15+
TITLE: "chore(deps): weekly `cargo update`"
16+
BODY: |
17+
Automation to keep dependencies in `Cargo.lock` current.
18+
19+
<details><summary><strong>cargo update log</strong></summary>
20+
<p>
21+
22+
```log
23+
$cargo_update_log
24+
```
25+
26+
</p>
27+
</details>
28+
29+
jobs:
30+
update:
31+
name: Update
32+
runs-on: ubuntu-latest
33+
steps:
34+
- uses: actions/checkout@v4
35+
- uses: dtolnay/rust-toolchain@nightly
36+
37+
- name: cargo update
38+
# Remove first line that always just says "Updating crates.io index"
39+
run: cargo update --color never 2>&1 | sed '/crates.io index/d' | tee -a cargo_update.log
40+
41+
- name: craft commit message and PR body
42+
id: msg
43+
run: |
44+
export cargo_update_log="$(cat cargo_update.log)"
45+
46+
echo "commit_message<<EOF" >> $GITHUB_OUTPUT
47+
printf "%s\n\n%s\n" $TITLE $cargo_update_log >> $GITHUB_OUTPUT
48+
echo "EOF" >> $GITHUB_OUTPUT
49+
50+
echo "body<<EOF" >> $GITHUB_OUTPUT
51+
echo "$BODY" | envsubst >> $GITHUB_OUTPUT
52+
echo "EOF" >> $GITHUB_OUTPUT
53+
54+
- name: Create Pull Request
55+
uses: peter-evans/create-pull-request@v7
56+
with:
57+
add-paths: ./Cargo.lock
58+
commit-message: ${{ steps.msg.outputs.commit_message }}
59+
title: ${{ env.TITLE }}
60+
body: ${{ steps.msg.outputs.body }}
61+
branch: ${{ env.BRANCH }}

‎.github/workflows/docker.yml‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Publishes the Docker image.
2+
3+
name: docker
4+
5+
on:
6+
workflow_dispatch: {}
7+
push:
8+
tags:
9+
- v*
10+
11+
env:
12+
REPO_NAME: ${{ github.repository_owner }}/lumen
13+
IMAGE_NAME: ${{ github.repository_owner }}/lumen
14+
CARGO_TERM_COLOR: always
15+
DOCKER_IMAGE_NAME: ghcr.io/${{ github.repository_owner }}/lumen
16+
DOCKER_USERNAME: ${{ github.actor }}
17+
18+
jobs:
19+
build:
20+
name: build and push
21+
runs-on: ubuntu-24.04
22+
permissions:
23+
packages: write
24+
contents: read
25+
steps:
26+
- uses: actions/checkout@v4
27+
- uses: dtolnay/rust-toolchain@stable
28+
- uses: Swatinem/rust-cache@v2
29+
with:
30+
cache-on-failure: true
31+
- uses: taiki-e/install-action@cross
32+
- name: Log in to Docker
33+
run: |
34+
echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io --username ${DOCKER_USERNAME} --password-stdin
35+
- name: Set up Docker builder
36+
run: |
37+
docker run --privileged --rm tonistiigi/binfmt --install arm64,amd64
38+
docker buildx create --use --name cross-builder
39+
- name: Build and push image
40+
run: make docker-build-push

‎.github/workflows/e2e.yml‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Runs E2E tests.
2+
3+
name: e2e
4+
5+
on:
6+
pull_request:
7+
merge_group:
8+
push:
9+
branches: [main]
10+
workflow_dispatch:
11+
12+
env:
13+
CARGO_TERM_COLOR: always
14+
15+
concurrency:
16+
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
17+
cancel-in-progress: true
18+
19+
jobs:
20+
e2e:
21+
runs-on: ubuntu-24.04
22+
timeout-minutes: 60
23+
steps:
24+
- uses: actions/checkout@v4
25+
- uses: dtolnay/rust-toolchain@stable
26+
- uses: Swatinem/rust-cache@v2
27+
with:
28+
cache-on-failure: true
29+
- uses: taiki-e/install-action@nextest
30+
- name: Install Foundry
31+
uses: foundry-rs/foundry-toolchain@v1
32+
- name: Build Lumen
33+
run: |
34+
cargo build --profile release --locked --bin lumen
35+
36+
- name: Run E2E tests
37+
run: |
38+
cargo nextest run \
39+
--locked \
40+
--workspace \
41+
-E "package(lumen-tests)" \
42+
--no-capture
43+
env:
44+
RUST_LOG: debug

0 commit comments

Comments
 (0)