Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support legacy zkevm-node and new zkevm sequence senders and aggregators #234

Draft
wants to merge 98 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
df15b3b
feat: introduce new flag `aggregator_sequence_sender_type`
leovct Aug 21, 2024
7de0137
chore: validate `aggregator_sequence_sender_type`
leovct Aug 21, 2024
64fa201
chore: add constants
leovct Aug 21, 2024
c264574
feat: enable to deploy legacy aggregator and sequence sender components
leovct Aug 21, 2024
22f578c
ci: test legacy agglayer and sequence sender
leovct Aug 21, 2024
09458b5
fix: zkevm node config
leovct Aug 21, 2024
8784076
chore: more generic variable names
leovct Aug 21, 2024
be012e0
fix: typos
leovct Aug 21, 2024
e6a58bf
fix: duplication of zkevm-node config
leovct Aug 21, 2024
b5e63aa
chore: update second param file
leovct Aug 21, 2024
34fb10e
fix: typo
leovct Aug 21, 2024
0af260a
fix: typo
leovct Aug 21, 2024
67a1bfb
Merge branch 'main' into feat/support-legacy-aggregator-and-sequence-…
leovct Aug 21, 2024
02e6f01
Merge branch 'main' into feat/support-legacy-aggregator-and-sequence-…
leovct Aug 21, 2024
9b7e3b6
ci: test matrix
leovct Aug 22, 2024
9aa16b4
chore: clean up sequencer implementation
leovct Aug 22, 2024
70c3064
chore: clean up
leovct Aug 22, 2024
3714e9a
fix: issues
leovct Aug 22, 2024
fec853f
chore: nit
leovct Aug 22, 2024
be4cae5
chore: nit
leovct Aug 22, 2024
19d0da6
chore: clean up
leovct Aug 22, 2024
1e643cc
chore: get rid of `deploy_cdk_erigon_node`
leovct Aug 22, 2024
ab01dec
chore: clean up main.star
leovct Aug 22, 2024
995d8e7
ci: fix
leovct Aug 22, 2024
5eff249
fix: pless
leovct Aug 22, 2024
b612e31
docs: update cmd to deploy legacy stack
leovct Aug 22, 2024
bc4f0ca
chore: nit
leovct Aug 22, 2024
215ee4b
chore: nit
leovct Aug 22, 2024
df8de32
fix: typo
leovct Aug 22, 2024
fb4c9d9
fix: blutgang
leovct Aug 22, 2024
de5784e
fix: blutgang 2
leovct Aug 22, 2024
fad3ec3
Merge remote-tracking branch 'upstream/main' into feat/support-legacy…
leovct Aug 22, 2024
d53a33b
fix: dbs
leovct Aug 22, 2024
6a6c104
fix: run synchronizer w/ aggregator and sequence sender
leovct Aug 22, 2024
b90d4fd
chore: clean up
leovct Aug 22, 2024
55ad02e
fix: application protocols
leovct Aug 22, 2024
0fc3e42
chore: inverse aggregator and sequence sender
leovct Aug 22, 2024
ff804b3
fix: avoid zkevm synchronizer duplication
leovct Aug 22, 2024
3b49876
chore: rename `trusted-node/node-config.toml`
leovct Aug 22, 2024
c9549ea
chore: rename `trusted-node/prover-config.json`
leovct Aug 22, 2024
ea31d45
chore: rename `trusted-node/dac-config.toml`
leovct Aug 22, 2024
db6b62f
chore: rename `trusted-node/sequence_sender-config.toml`
leovct Aug 22, 2024
bdb0cd4
test: missing param in zkevm sequence sender config
leovct Aug 22, 2024
41649f6
chore: clean up
leovct Aug 22, 2024
5aceae2
test: do not deploy trusted node when using erigon sequencer
leovct Aug 22, 2024
7399ebb
fix: legacy stack
leovct Aug 22, 2024
2b83b18
fix: typo
leovct Aug 22, 2024
8c310f2
Merge branch 'main' into feat/support-legacy-aggregator-and-sequence-…
leovct Aug 26, 2024
631fada
fix: use the correct zkevm-sequence-sender
leovct Aug 26, 2024
f92d1d4
fix: sync
leovct Aug 26, 2024
36a2d03
docs: quick fix
leovct Aug 26, 2024
884b924
chore: rename service
leovct Aug 26, 2024
1e1b953
chore: use zkevm-aggregator instead of legacy node
leovct Aug 26, 2024
e771605
fix
leovct Aug 27, 2024
08b05ce
ci: use test matrix
leovct Aug 27, 2024
7440785
Merge branch 'main' into feat/support-legacy-aggregator-and-sequence-…
leovct Aug 27, 2024
7c7b39a
fix: typo
leovct Aug 27, 2024
9075137
fix: args
leovct Aug 27, 2024
816037f
chore: lint
leovct Aug 27, 2024
653f0b5
fix
leovct Aug 27, 2024
e43667e
fix: default params
leovct Aug 27, 2024
cc75d49
chore: bump kurtosis version
leovct Aug 27, 2024
b4018cf
ci: update tests
leovct Aug 27, 2024
734900c
fix: dbs
leovct Aug 27, 2024
1b3c452
ci: fix script path
leovct Aug 27, 2024
b82e284
fix: configless test
leovct Aug 27, 2024
fd21bb9
ci: fix attempt
leovct Aug 27, 2024
d38db96
fix: rollup
leovct Aug 27, 2024
cef7564
fix: monitoring
leovct Aug 27, 2024
4c0964b
ci: fix monitor again
leovct Aug 27, 2024
f2c7b0f
fix: deploy without specifying sequencer
leovct Aug 27, 2024
7ad86bf
fix: gas token
leovct Aug 27, 2024
bbebc30
chore: nit
leovct Aug 27, 2024
d984ae1
fix
leovct Aug 27, 2024
0fbfaf8
ci: nit
leovct Aug 27, 2024
5aa6671
fix: verification of blutgang and panoptichain metrics
leovct Aug 27, 2024
3216ce6
Merge branch 'main' into feat/support-legacy-aggregator-and-sequence-…
leovct Aug 28, 2024
f21d154
fix: zkevm legacy
leovct Aug 28, 2024
b3e0070
ci: make the workflow fail if one of the jobs fail
leovct Aug 28, 2024
014c213
fix: typos
leovct Aug 28, 2024
6f73900
fix: zkevm sequencer
leovct Aug 28, 2024
9a850da
fix: typos
leovct Aug 28, 2024
0fe10da
ci: add `continue-on-error: true`
leovct Aug 28, 2024
c069eb7
fix: keystore path
leovct Aug 28, 2024
b00e285
ci: use `fail-fast: false`
leovct Aug 28, 2024
85317b1
ci: update timeout
leovct Aug 28, 2024
5d9c842
ci: list dbs
leovct Aug 28, 2024
a0cbdb7
chore: nit
leovct Aug 28, 2024
f11ab75
chore: bump imgs
leovct Aug 28, 2024
dc9249b
fix: issue
leovct Aug 28, 2024
68c6d31
chore: nit
leovct Aug 28, 2024
a9a1ee5
fix: typo
leovct Aug 28, 2024
a104309
test: make bridge calls
leovct Aug 28, 2024
992b9ca
chore: defaults
leovct Aug 28, 2024
039ec4d
Revert "test: make bridge calls"
leovct Aug 28, 2024
8a70e1f
test: force update ger
leovct Aug 28, 2024
51068b1
Merge remote-tracking branch 'upstream/main' into feat/support-legacy…
leovct Sep 2, 2024
892815e
chore: bump cdk-node
leovct Sep 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ jobs:
# Additionally, deploy cdk-erigon and zkevm-node as permissionless nodes or RPCs.
- name: Deploy L1 chain and a first CDK L2 chain (erigon)
run: |
yq -Y --in-place '.deploy_cdk_erigon_node = true' params.yml
yq -Y --in-place '.args.sequencer_type = "erigon"' params.yml
yq -Y --in-place '.args.aggregator_sequence_sender_type = "cdk"' params.yml
yq -Y --in-place '.args.additional_services = ["pless_zkevm_node", "tx_spammer"]' params.yml
kurtosis run --enclave=${{ env.ENCLAVE_NAME }} --show-enclave-inspect=false --args-file=params.yml .

Expand Down Expand Up @@ -105,6 +108,7 @@ jobs:
run: |
yq -Y --in-place '.deploy_cdk_erigon_node = false' params.yml
yq -Y --in-place '.args.sequencer_type = "zkevm"' params.yml
yq -Y --in-place '.args.aggregator_sequence_sender_type = "zkevm"' params.yml
yq -Y --in-place '.args.additional_services = ["pless_zkevm_node", "tx_spammer"]' params.yml
kurtosis run --enclave=${{ env.ENCLAVE_NAME }} --show-enclave-inspect=false --args-file=params.yml .

Expand Down
6 changes: 6 additions & 0 deletions agglayer-attach-cdk-params.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,12 @@ args:
# - 'zkevm': Use the legacy sequencer (https://github.com/0xPolygonHermez/zkevm-node).
sequencer_type: erigon

# The type of aggregator and sequence sender to deploy.
# Options:
# - 'cdk': Use the new cdk-node (https://github.com/0xPolygon/cdk).
# - 'zkevm': Use the legacy aggregator and sequence sender (https://github.com/0xPolygonHermez/zkevm-node).
aggregator_sequence_sender_type: cdk

# Deploy agglayer.
deploy_agglayer: false

Expand Down
63 changes: 50 additions & 13 deletions cdk_central_environment.star
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
constants = import_module("./src/package_io/constants.star")
data_availability_package = import_module("./lib/data_availability.star")
service_package = import_module("./lib/service.star")
zkevm_dac_package = import_module("./lib/zkevm_dac.star")
Expand Down Expand Up @@ -41,15 +42,19 @@ def run(plan, args):
)

keystore_artifacts = get_keystores_artifacts(plan, args)
if args["sequencer_type"] == "zkevm":
# Create the zkevm node config.
node_config_template = read_file(

# Create the zkevm-node configuration if needed.
if (args["sequencer_type"] == "zkevm") or (
args["aggregator_sequence_sender_type"]
== constants.AGGREGATOR_SEQUENCE_SENDER_TYPE.zkevm
):
zkevm_node_config_template = read_file(
src="./templates/trusted-node/node-config.toml"
)
node_config_artifact = plan.render_templates(
zkevm_node_config_artifact = plan.render_templates(
config={
"node-config.toml": struct(
template=node_config_template,
template=zkevm_node_config_template,
data=args
| {
"is_cdk_validium": data_availability_package.is_cdk_validium(
Expand All @@ -62,15 +67,16 @@ def run(plan, args):
name="trusted-node-config",
)

if args["sequencer_type"] == "zkevm":
# Start the synchronizer.
zkevm_node_package.start_synchronizer(
plan, args, node_config_artifact, genesis_artifact
plan, args, zkevm_node_config_artifact, genesis_artifact
)

# Start the rest of the zkevm node components.
zkevm_node_components_configs = (
zkevm_node_package.create_zkevm_node_components_config(
args, node_config_artifact, genesis_artifact, keystore_artifacts
args, zkevm_node_config_artifact, genesis_artifact, keystore_artifacts
)
)

Expand All @@ -79,19 +85,23 @@ def run(plan, args):
description="Starting the rest of the zkevm node components",
)

if args["sequencer_type"] == "erigon":
# Deploy aggregator and sequence sender.
if (
args["aggregator_sequence_sender_type"]
== constants.AGGREGATOR_SEQUENCE_SENDER_TYPE.cdk
):
# Create the cdk node config.
node_config_template = read_file(
cdk_node_config_template = read_file(
src="./templates/trusted-node/cdk-node-config.toml"
)
contract_setup_addresses = service_package.get_contract_setup_addresses(
plan, args
)
node_config_artifact = plan.render_templates(
cdk_node_config_artifact = plan.render_templates(
name="cdk-node-config-artifact",
config={
"cdk-node-config.toml": struct(
template=node_config_template,
template=cdk_node_config_template,
data=args
| {
"is_cdk_validium": data_availability_package.is_cdk_validium(
Expand All @@ -106,12 +116,39 @@ def run(plan, args):

# Start the cdk components.
cdk_node_configs = cdk_node_package.create_cdk_node_service_config(
args, node_config_artifact, genesis_artifact, keystore_artifacts
args, cdk_node_config_artifact, genesis_artifact, keystore_artifacts
)

plan.add_services(
configs=cdk_node_configs,
description="Starting the cdk node components",
description="Starting the cdk-node components",
)
elif (
args["aggregator_sequence_sender_type"]
== constants.AGGREGATOR_SEQUENCE_SENDER_TYPE.zkevm
):
zkevm_aggregator_service_config = (
zkevm_node_package.create_aggregator_service_config(
args,
zkevm_node_config_artifact,
genesis_artifact,
keystore_artifacts.sequencer,
keystore_artifacts.aggregator,
keystore_artifacts.proofsigner,
)
)
zkevm_sequence_sender_service_config = (
zkevm_node_package.create_sequence_sender_service_config(
args,
zkevm_node_config_artifact,
genesis_artifact,
keystore_artifacts.sequencer,
)
)
plan.add_services(
configs=zkevm_aggregator_service_config
| zkevm_sequence_sender_service_config,
description="Starting zkevm aggregator and sequence sender",
)

# Start the DAC if in validium mode.
Expand Down
16 changes: 8 additions & 8 deletions cdk_erigon.star
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,15 @@ def run_rpc(plan, args):
plan, args, stateless_executor_config_artifact
)

zkevm_sequencer_service = plan.get_service(
sequencer_service = plan.get_service(
name=args["sequencer_name"] + args["deployment_suffix"]
)
zkevm_sequence_url = "http://{}:{}".format(
zkevm_sequencer_service.ip_address, zkevm_sequencer_service.ports["rpc"].number
sequencer_url = "http://{}:{}".format(
sequencer_service.ip_address, sequencer_service.ports["rpc"].number
)
zkevm_datastreamer_url = "{}:{}".format(
zkevm_sequencer_service.ip_address,
zkevm_sequencer_service.ports["data-streamer"].number,
datastreamer_url = "{}:{}".format(
sequencer_service.ip_address,
sequencer_service.ports["data-streamer"].number,
)

cdk_erigon_node_config_template = read_file(src="./templates/cdk-erigon/config.yml")
Expand All @@ -42,8 +42,8 @@ def run_rpc(plan, args):
"config.yaml": struct(
template=cdk_erigon_node_config_template,
data={
"zkevm_sequencer_url": zkevm_sequence_url,
"zkevm_datastreamer_url": zkevm_datastreamer_url,
"sequencer_url": sequencer_url,
"datastreamer_url": datastreamer_url,
"is_sequencer": False,
}
| args
Expand Down
16 changes: 16 additions & 0 deletions input_parser.star
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ DEFAULT_ARGS = {
"deployment_suffix": "-001",
"global_log_level": "info",
"sequencer_type": "erigon",
"aggregator_sequence_sender_type": "cdk",
"deploy_agglayer": True,
"data_availability_mode": "cdk-validium",
"additional_services": [],
Expand Down Expand Up @@ -77,6 +78,7 @@ DEFAULT_ARGS = {

def parse_args(args):
validate_global_log_level(args["global_log_level"])
validate_aggregator_sequence_sender_type(args["aggregator_sequence_sender_type"])
return DEFAULT_ARGS | args


Expand All @@ -98,3 +100,17 @@ def validate_global_log_level(global_log_level):
constants.GLOBAL_LOG_LEVEL.trace,
)
)


def validate_aggregator_sequence_sender_type(aggregator_sequence_sender_type):
if aggregator_sequence_sender_type not in (
constants.AGGREGATOR_SEQUENCE_SENDER_TYPE.cdk,
constants.AGGREGATOR_SEQUENCE_SENDER_TYPE.zkevm,
):
fail(
"Unsupported aggregator and sequence sender type: '{}', please use '{}' or '{}'".format(
aggregator_sequence_sender_type,
constants.AGGREGATOR_SEQUENCE_SENDER_TYPE.cdk,
constants.AGGREGATOR_SEQUENCE_SENDER_TYPE.zkevm,
)
)
6 changes: 6 additions & 0 deletions kurtosis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,12 @@ description: |-

sequencer_type: erigon

# The type of aggregator and sequence sender to deploy.
# Options:
# - 'cdk': Use the new cdk-node (https://github.com/0xPolygon/cdk).
# - 'zkevm': Use the legacy aggregator and sequence sender (https://github.com/0xPolygonHermez/zkevm-node).
aggregator_sequence_sender_type: cdk

deploy_agglayer: true

# The type of data availability to use.
Expand Down
22 changes: 2 additions & 20 deletions lib/zkevm_node.star
Original file line number Diff line number Diff line change
Expand Up @@ -203,14 +203,6 @@ def create_zkevm_node_components_config(
genesis_artifact,
keystore_artifacts,
):
aggregator_config = create_aggregator_service_config(
args,
config_artifact,
genesis_artifact,
keystore_artifacts.sequencer,
keystore_artifacts.aggregator,
keystore_artifacts.proofsigner,
)
rpc_config = create_rpc_service_config(args, config_artifact, genesis_artifact)
eth_tx_manager_config = create_eth_tx_manager_service_config(
args,
Expand All @@ -222,22 +214,12 @@ def create_zkevm_node_components_config(
l2_gas_pricer_config = create_l2_gas_pricer_service_config(
args, config_artifact, genesis_artifact
)
configs = (
aggregator_config | rpc_config | eth_tx_manager_config | l2_gas_pricer_config
)
configs = rpc_config | eth_tx_manager_config | l2_gas_pricer_config

if args["sequencer_type"] == "zkevm":
sequencer_config = create_sequencer_service_config(
args, config_artifact, genesis_artifact
)

sequence_sender_config = create_sequence_sender_service_config(
args,
config_artifact,
genesis_artifact,
keystore_artifacts.sequencer,
)

return configs | sequencer_config | sequence_sender_config
return configs | sequencer_config
else:
return configs
6 changes: 6 additions & 0 deletions params.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,12 @@ args:
# - 'zkevm': Use the legacy sequencer (https://github.com/0xPolygonHermez/zkevm-node).
sequencer_type: erigon

# The type of aggregator and sequence sender to deploy.
# Options:
# - 'cdk': Use the new cdk-node (https://github.com/0xPolygon/cdk).
# - 'zkevm': Use the legacy aggregator and sequence sender (https://github.com/0xPolygonHermez/zkevm-node).
aggregator_sequence_sender_type: cdk

# Deploy agglayer.
deploy_agglayer: true

Expand Down
5 changes: 5 additions & 0 deletions src/package_io/constants.star
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,8 @@ GLOBAL_LOG_LEVEL = struct(
debug="debug",
trace="trace",
)

AGGREGATOR_SEQUENCE_SENDER_TYPE = struct(
cdk="cdk",
zkevm="zkevm",
)
4 changes: 2 additions & 2 deletions templates/cdk-erigon/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,6 @@ zkevm.data-stream-host: "0.0.0.0"
zkevm.data-stream-port: {{.zkevm_data_stream_port}}
# rpc node {{else}}
txpool.disable: true
zkevm.l2-sequencer-rpc-url: {{.zkevm_sequencer_url}}
zkevm.l2-datastreamer-url: {{.zkevm_datastreamer_url}}
zkevm.l2-sequencer-rpc-url: {{.sequencer_url}}
zkevm.l2-datastreamer-url: {{.datastreamer_url}}
# end {{end}}