feat: implement modulesToCount selection from module limits TOML config

PLUGINS.md

@@ -1,106 +1,43 @@
 # Linea plugins
 
-## Shared components
-### Profitability calculator
-The profitability calculator is a shared component, that is used to check if a tx is profitable.
-It is applied, with different configuration to:
-1. `linea_estimateGas` endpoint
-2. Tx validation for the txpool
-3. Tx selection during block creation
+### Line Counts - LineCountsEndpointServicePlugin
 
-#### CLI Options
+#### `linea_getBlockTracesCountersV2`
 
-| Option Name | Default Value | Command Line Argument |
-|--------------------------|---------------|-------------------------------------------|
-| L1_VERIFICATION_GAS_COST | 1_200_000 | `--plugin-linea-verification-gas-cost` |
-| L1_VERIFICATION_CAPACITY | 90_000 | `--plugin-linea-verification-capacity` |
-| L1_L2_GAS_PRICE_RATIO | 15 | `--plugin-linea-gas-price-ratio` |
-| L2_GAS_PRICE_ADJUSTMENT | 0 wei | `--plugin-linea-gas-price-adjustment` |
-| MIN_MARGIN | 1.0 | `--plugin-linea-min-margin` |
-| ESTIMATE_GAS_MIN_MARGIN | 1.0 | `--plugin-linea-estimate-gas-min-margin` |
-| TX_POOL_MIN_MARGIN | 0.5 | `--plugin-linea-tx-pool-min-margin` |
-| UNPROFITABLE_CACHE_SIZE | 100_000 | `--plugin-linea-unprofitable-cache-size` |
-| UNPROFITABLE_RETRY_LIMIT | 10 | `--plugin-linea-unprofitable-retry-limit` |
-| TX_POOL_ENABLE_CHECK_API | true | `--plugin-linea-tx-pool-profitability-check-api-enabled` |
-| TX_POOL_ENABLE_CHECK_P2P | false | `--plugin-linea-tx-pool-profitability-check-p2p-enabled` |
+The `LineCountsEndpointServicePlugin` registers an RPC endpoint named `getBlockTracesCountersV2`
+under the `linea` namespace. When this endpoint is called, returns trace counters based on the provided request
+parameters.
 
-### L1 L2 Bridge
+#### Plugin Parameters
 
-#### CLI Options
+- Introduces the `plugin-linea-line-counts-modules-to-count-config-file-path` plugin CLI parameter to set the path of a
+ module limits TOML file from which in can read all the included module keys and calculate line counts only for those modules.
+- When `plugin-linea-line-counts-modules-to-count-config-file-path` is not provided it returns line counts for all
+ modules that are intended for counting.
 
-| Option Name | Default Value | Command Line Argument |
-|------------------------------|---------------|---------------------------------------------|
-| L1L2_BRIDGE_CONTRACT_ADDRESS | | `--plugin-linea-l1l2-bridge-contract` |
-| L1L2_BRIDGE_LOG_TOPIC | | `--plugin-linea-l1l2-bridge-topic` |
+#### RPC Request Parameters
 
-## Sequencer
-### Transaction Selection - LineaTransactionSelectorPlugin
-
-This plugin extends the standard transaction selection protocols employed by Besu for block creation.
-It leverages the TransactionSelectionService to manage and customize the process of transaction selection.
-This includes setting limits such as `TraceLineLimit`, `maxBlockGas`, and `maxCallData`, and check the profitability
-of a transaction.
-
-
-#### CLI Options
-
-| Option Name | Default Value | Command Line Argument |
-|----------------------------------|----------------------|---------------------------------------------------|
-| MAX_BLOCK_CALLDATA_SIZE | 70000 | `--plugin-linea-max-block-calldata-size` |
-| MODULE_LIMIT_FILE_PATH | moduleLimitFile.toml | `--plugin-linea-module-limit-file-path` |
-| OVER_LINE_COUNT_LIMIT_CACHE_SIZE | 10_000 | `--plugin-linea-over-line-count-limit-cache-size` |
-| MAX_GAS_PER_BLOCK | 30_000_000L | `--plugin-linea-max-block-gas` |
-
-
-### Transaction validation - LineaTransactionValidatorPlugin
-
-This plugin extends the default transaction validation rules for adding transactions to the
-transaction pool. It leverages the PluginTransactionValidatorService to manage and customize the
-process of transaction validation. This includes, for example, setting a deny list of addresses
-that are not allowed to add transactions to the pool.
-
-#### CLI Options
-
-| Option Name | Default Value | Command Line Argument |
-|-------------------------|-------------------|---------------------------------------|
-| DENY_LIST_PATH | lineaDenyList.txt | `--plugin-linea-deny-list-path` |
-| MAX_TX_GAS_LIMIT_OPTION | 30_000_000 | `--plugin-linea-max-tx-gas-limit` |
-| MAX_TX_CALLDATA_SIZE | 60_000 | `--plugin-linea-max-tx-calldata-size` |
-
-## RPC
-
-### Linea Estimate Gas
-#### `linea_estimateGas`
-
-This endpoint simulates a transaction and returns the estimated gas used ( as the standard `eth_estimateGas`) plus the estimated gas price to be used when submitting the tx.
-
-#### Parameters
-
-same as `eth_estimateGas`
-
-### Counters - CountersEndpointServicePlugin
-#### `rollup_getTracesCountersByBlockNumberV0`
-
-The CountersEndpointServicePlugin registers an RPC endpoint named `getTracesCountersByBlockNumberV0`
-under the `rollup` namespace. When this endpoint is called, returns trace counters based on the provided request parameters.
+- `blockNumber`: _string_ - The block number
+- `expectedTracesEngineVersion`: _string_ - The tracer version. It will return an error if the
+ requested version is different from the tracer runtime
 
-#### Parameters
+### Trace generation - TracesEndpointServicePlugin
 
- - `blockNumber`: _string_ - The block number
+#### `linea_generateConflatedTracesToFileV2`
 
- - `tracerVersion`: _string_ - The tracer version. It will return an error if the
-requested version is different from the tracer runtime
+This plugin registers an RPC endpoint named `generateConflatedTracesToFileV2` under the `linea` namespace.
+The endpoint generates conflated file traces.
 
-### Trace generation - TracesEndpointServicePlugin
+#### Plugin Parameters
 
-This plugin registers an RPC endpoint named `generateConflatedTracesToFileV0` under the `rollup` namespace.
-The endpoint generates conflated file traces.
+- Introduces `plugin-linea-continuous-tracing-traces-output-path` plugin CLI parameter to determine the output
+ directory of the generated trace `.lt` files.
 
-#### Parameters
+#### RPC Request Parameters
 
-- `fromBlock`: _string_ - the fromBlock number
-- `toBlock`: _string_ - The toBlock number
-- `tracerVersion`: _string_ - The tracer version. It will return an error if the
+- `startBlockNumber`: _string_ - the fromBlock number
+- `endBlockNumber`: _string_ - The toBlock number
+- `expectedTracesEngineVersion`: _string_ - The tracer version. It will return an error if the
  requested version is different from the tracer runtime
 
 ## Continuous Tracing

arithmetization/src/main/java/net/consensys/linea/plugins/AbstractLineaPrivateOptionsPlugin.java

@@ -31,11 +31,19 @@ public abstract class AbstractLineaPrivateOptionsPlugin extends AbstractLineaSha
 
  @Override
  public Map<String, LineaOptionsPluginConfiguration> getLineaPluginConfigMap() {
- final var configMap = new HashMap<>(super.getLineaPluginConfigMap());
+ Map<String, LineaOptionsPluginConfiguration> configMap =
+ new HashMap<>(super.getLineaPluginConfigMap());
 
  final RpcCliOptions rpcCliOptions = RpcCliOptions.create();
  configMap.put(RpcCliOptions.CONFIG_KEY, rpcCliOptions.asPluginConfig());
 
+ configMap = getPrivateLineaPluginConfigMap(configMap);
+
+ return configMap;
+ }
+
+ public Map<String, LineaOptionsPluginConfiguration> getPrivateLineaPluginConfigMap(
+ Map<String, LineaOptionsPluginConfiguration> configMap) {
  return configMap;
  }
 

arithmetization/src/main/java/net/consensys/linea/plugins/rpc/linecounts/GenerateLineCountsV2.java

@@ -15,6 +15,8 @@
 
 package net.consensys.linea.plugins.rpc.linecounts;
 
+import java.nio.file.Path;
+import java.nio.file.Paths;
 import java.util.Map;
 import java.util.Optional;
 
@@ -42,11 +44,20 @@ public class GenerateLineCountsV2 {
  private final RequestLimiter requestLimiter;
 
  private final BesuContext besuContext;
+ private Path modulesToCountConfigFilePath;
  private TraceService traceService;
 
- public GenerateLineCountsV2(final BesuContext context, final RequestLimiter requestLimiter) {
+ public GenerateLineCountsV2(
+ final BesuContext context,
+ final RequestLimiter requestLimiter,
+ final LineCountsEndpointConfiguration endpointConfiguration) {
  this.besuContext = context;
  this.requestLimiter = requestLimiter;
+
+ if (endpointConfiguration.modulesToCountConfigFilePath() != null) {
+ this.modulesToCountConfigFilePath =
+ Paths.get(endpointConfiguration.modulesToCountConfigFilePath());
+ }
  }
 
  public String getNamespace() {
@@ -97,7 +108,7 @@ private LineCounts getLineCounts(PluginRpcRequest request) {
  .computeIfAbsent(
  requestedBlockNumber,
  blockNumber -> {
- final ZkTracer tracer = new ZkTracer();
+ final ZkTracer tracer = new ZkTracer(modulesToCountConfigFilePath);
  traceService.trace(
  blockNumber,
  blockNumber,

arithmetization/src/main/java/net/consensys/linea/plugins/rpc/linecounts/LineCountsEndpointCliOptions.java

@@ -0,0 +1,77 @@
+/*
+ * Copyright Consensys Software Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations under the License.
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+package net.consensys.linea.plugins.rpc.linecounts;
+
+import com.google.common.base.MoreObjects;
+import net.consensys.linea.plugins.LineaCliOptions;
+import picocli.CommandLine;
+
+class LineCountsEndpointCliOptions implements LineaCliOptions {
+
+ static final String CONFIG_KEY = "line-counts-endpoint-config";
+
+ static final String MODULES_TO_COUNT_CONFIG_FILE_PATH =
+ "--plugin-linea-line-counts-modules-to-count-config-file-path";
+
+ @CommandLine.Option(
+ names = {MODULES_TO_COUNT_CONFIG_FILE_PATH},
+ hidden = true,
+ paramLabel = "<PATH>",
+ description = "TOML config file path with a list of modules to count")
+ private String modulesToCountConfigFilePath = null;
+
+ private LineCountsEndpointCliOptions() {}
+
+ /**
+ * Create Linea cli options.
+ *
+ * @return the Linea cli options
+ */
+ static LineCountsEndpointCliOptions create() {
+ return new LineCountsEndpointCliOptions();
+ }
+
+ /**
+ * Linea cli options from config.
+ *
+ * @param config the config
+ * @return the Linea cli options
+ */
+ static LineCountsEndpointCliOptions fromConfig(final LineCountsEndpointConfiguration config) {
+ final LineCountsEndpointCliOptions options = create();
+ options.modulesToCountConfigFilePath = config.modulesToCountConfigFilePath();
+ return options;
+ }
+
+ /**
+ * To domain object Linea factory configuration.
+ *
+ * @return the Linea factory configuration
+ */
+ @Override
+ public LineCountsEndpointConfiguration toDomainObject() {
+ return LineCountsEndpointConfiguration.builder()
+ .modulesToCountConfigFilePath(modulesToCountConfigFilePath)
+ .build();
+ }
+
+ @Override
+ public String toString() {
+ return MoreObjects.toStringHelper(this)
+ .add(MODULES_TO_COUNT_CONFIG_FILE_PATH, modulesToCountConfigFilePath)
+ .toString();
+ }
+}

arithmetization/src/main/java/net/consensys/linea/plugins/rpc/linecounts/LineCountsEndpointConfiguration.java

@@ -0,0 +1,24 @@
+/*
+ * Copyright Consensys Software Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations under the License.
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+package net.consensys.linea.plugins.rpc.linecounts;
+
+import lombok.Builder;
+import net.consensys.linea.plugins.LineaOptionsConfiguration;
+
+/** The Linea tracer configuration private to this repo. */
+@Builder(toBuilder = true)
+public record LineCountsEndpointConfiguration(String modulesToCountConfigFilePath)
+ implements LineaOptionsConfiguration {}

arithmetization/src/main/java/net/consensys/linea/plugins/rpc/linecounts/LineCountsEndpointServicePlugin.java

@@ -15,9 +15,12 @@
 
 package net.consensys.linea.plugins.rpc.linecounts;
 
+import java.util.Map;
+
 import com.google.auto.service.AutoService;
 import net.consensys.linea.plugins.AbstractLineaPrivateOptionsPlugin;
 import net.consensys.linea.plugins.BesuServiceProvider;
+import net.consensys.linea.plugins.LineaOptionsPluginConfiguration;
 import net.consensys.linea.plugins.rpc.RequestLimiter;
 import net.consensys.linea.plugins.rpc.RequestLimiterDispatcher;
 import org.hyperledger.besu.plugin.BesuContext;
@@ -36,6 +39,17 @@ public class LineCountsEndpointServicePlugin extends AbstractLineaPrivateOptions
  private BesuContext besuContext;
  private RpcEndpointService rpcEndpointService;
 
+ @Override
+ public Map<String, LineaOptionsPluginConfiguration> getPrivateLineaPluginConfigMap(
+ Map<String, LineaOptionsPluginConfiguration> configMap) {
+ final LineCountsEndpointCliOptions lineCountsEndpointCliOptions =
+ LineCountsEndpointCliOptions.create();
+ configMap.put(
+ LineCountsEndpointCliOptions.CONFIG_KEY, lineCountsEndpointCliOptions.asPluginConfig());
+
+ return configMap;
+ }
+
  /**
  * Register the RPC service.
  *
@@ -52,14 +66,19 @@ public void register(final BesuContext context) {
  public void beforeExternalServices() {
  super.beforeExternalServices();
 
+ final LineCountsEndpointConfiguration endpointConfiguration =
+ (LineCountsEndpointConfiguration)
+ getConfigurationByKey(LineCountsEndpointCliOptions.CONFIG_KEY).optionsConfig();
+
  RequestLimiterDispatcher.setLimiterIfMissing(
  RequestLimiterDispatcher.SINGLE_INSTANCE_REQUEST_LIMITER_KEY,
  rpcConfiguration().concurrentRequestsLimit());
  final RequestLimiter reqLimiter =
  RequestLimiterDispatcher.getLimiter(
  RequestLimiterDispatcher.SINGLE_INSTANCE_REQUEST_LIMITER_KEY);
 
- final GenerateLineCountsV2 method = new GenerateLineCountsV2(besuContext, reqLimiter);
+ final GenerateLineCountsV2 method =
+ new GenerateLineCountsV2(besuContext, reqLimiter, endpointConfiguration);
  createAndRegister(method, rpcEndpointService);
  }
 

arithmetization/src/main/java/net/consensys/linea/plugins/rpc/tracegeneration/TracesEndpointCliOptions.java

@@ -26,9 +26,6 @@ class TracesEndpointCliOptions implements LineaCliOptions {
  static final String CONFLATED_TRACE_GENERATION_TRACES_OUTPUT_PATH =
  "--plugin-linea-conflated-trace-generation-traces-output-path";
 
- static final String CONFLATED_TRACE_GENERATION_CONCURRENT_REQUESTS_LIMIT =
- "--plugin-linea-conflated-trace-generation-concurrent-requests-limit";
-
  @CommandLine.Option(
  required = true,
  names = {CONFLATED_TRACE_GENERATION_TRACES_OUTPUT_PATH},

arithmetization/src/main/java/net/consensys/linea/plugins/rpc/tracegeneration/TracesEndpointServicePlugin.java

@@ -47,10 +47,12 @@ public class TracesEndpointServicePlugin extends AbstractLineaPrivateOptionsPlug
  private RpcEndpointService rpcEndpointService;
 
  @Override
- public Map<String, LineaOptionsPluginConfiguration> getLineaPluginConfigMap() {
+ public Map<String, LineaOptionsPluginConfiguration> getPrivateLineaPluginConfigMap(
+ Map<String, LineaOptionsPluginConfiguration> configMap) {
  final TracesEndpointCliOptions tracesEndpointCliOptions = TracesEndpointCliOptions.create();
+ configMap.put(TracesEndpointCliOptions.CONFIG_KEY, tracesEndpointCliOptions.asPluginConfig());
 
- return Map.of(TracesEndpointCliOptions.CONFIG_KEY, tracesEndpointCliOptions.asPluginConfig());
+ return configMap;
  }
 
  /**