Add gRPC block service (cometbft#1142)

* initial logic for the gRPC block service (cometbft#1094) * initial logic for gRPC block client and test (cometbft#1094) * block response and request (cometbft#1094) * add configuration for block service (cometbft#1094) * use pointer for request parameter (cometbft#1094) * change block service GetBlock response (cometbft#1094) * return block information (cometbft#1094) * convert core type to proto type (cometbft#1094) * hookup server (cometbft#1094) * hooking directly to the BlockStore instead of Environment (cometbft#1094) * changing client return type, use core type (cometbft#1094) * convert from proto to core type (cometbft#1094) * implemented proper logic to test block service (cometbft#1094) * return the latest height if height is 0 (cometbft#1094) * adding proper grpc error handling and return (cometbft#1094) * rename rpc and message in proto to match ADR-101 spec (cometbft#1094) * renaming service and client methods structs to match ADR (cometbft#1094) * additional error handling (cometbft#1094) * rename get block request and response (cometbft#1094) * update method name in the client (cometbft#1094) * proto entries for GetLatestHeight logic (cometbft#1094) * added logic for streaming new blocks as part of GetLatestHeight (cometbft#1094) * ensure subcribers have their own unique id (cometbft#1094) * client logic to use a channel parameter (cometbft#1094) * added test for GetLatestHeight (cometbft#1094) * better streaming test to prevent error (cometbft#1094) * remove empty line Co-authored-by: Thane Thomson <connect@thanethomson.com> * removing UNARY_RPC to prevent lint on server streaming (cometbft#1094) * remove server reflection, doesn't work well with gogoproto (cometbft#1094) * handle negative height parameter (cometbft#1094) * remove `Block` from `GetBlockLatestHeight` Co-authored-by: Thane Thomson <connect@thanethomson.com> * changed method name and fixes to test (cometbft#1094) * fixes from PR feedback (cometbft#1094) * remove vars Co-authored-by: Thane Thomson <connect@thanethomson.com> * fix declaration Co-authored-by: Thane Thomson <connect@thanethomson.com> * simplify error return Co-authored-by: Thane Thomson <connect@thanethomson.com> * simplest error return Co-authored-by: Thane Thomson <connect@thanethomson.com> * remove line Co-authored-by: Thane Thomson <connect@thanethomson.com> * remove var declaration, not needed Co-authored-by: Thane Thomson <connect@thanethomson.com> * use simplest error return Co-authored-by: Thane Thomson <connect@thanethomson.com> * simplest error return Co-authored-by: Thane Thomson <connect@thanethomson.com> * simplest error return Co-authored-by: Thane Thomson <connect@thanethomson.com> * simplify error handling Co-authored-by: Thane Thomson <connect@thanethomson.com> * simplified the logic for subscriber id (cometbft#1094) * more fixs based on PR feedback (cometbft#1094) * generated new protos (cometbft#1094) * remove else logic to allow compilation (cometbft#1094) * added logging capabilities to the block service and added log messages (cometbft#1094) * ensure node is a valid testing node (cometbft#1094) * added more conditions to handle subscription cancel (cometbft#1094) * use break in the for loop (cometbft#1094) * renamed ResultBlock to Block (cometbft#1094) * refactored the client logic use a channel for errors (cometbft#1094) * simplify subscriber name Co-authored-by: Thane Thomson <connect@thanethomson.com> * adding error to log Co-authored-by: Thane Thomson <connect@thanethomson.com> * update logging Co-authored-by: Thane Thomson <connect@thanethomson.com> * Simplified and improved logging (cometbft#1094) * added logic to drop the height publish if channel full (cometbft#194) * improved test to ensure is not light or seed node (cometbft#1094) * refactored block service to use one channel with result type (cometbft#1094) * blockservice: Apply service name to all logs Signed-off-by: Thane Thomson <connect@thanethomson.com> * blockservice: Capitalize start of all log messages Signed-off-by: Thane Thomson <connect@thanethomson.com> * blockservice: Remove unnecessary logs Signed-off-by: Thane Thomson <connect@thanethomson.com> * blockservice: Add trivial RPC error response tracing mechanism for easier debugging Signed-off-by: Thane Thomson <connect@thanethomson.com> * blockservice: Simplify constructor Signed-off-by: Thane Thomson <connect@thanethomson.com> * blockservice: Extract type assertion Signed-off-by: Thane Thomson <connect@thanethomson.com> * Add changelog entries Signed-off-by: Thane Thomson <connect@thanethomson.com> * docs: Update configuration-related content Signed-off-by: Thane Thomson <connect@thanethomson.com> * grpc/client: Format Signed-off-by: Thane Thomson <connect@thanethomson.com> * grpc: Extract function to validate or update block height from request Signed-off-by: Thane Thomson <connect@thanethomson.com> --------- Signed-off-by: Thane Thomson <connect@thanethomson.com> Co-authored-by: Thane Thomson <connect@thanethomson.com>
osmosis-labs · Jan 21, 2024 · e93beda · e93beda
1 parent 30d63e8
commit e93beda
Show file tree

Hide file tree

Showing 17 changed files with 1,555 additions and 8 deletions.
diff --git a/.changelog/unreleased/features/1094-grpc-block-service-cfg.md b/.changelog/unreleased/features/1094-grpc-block-service-cfg.md
@@ -0,0 +1,2 @@
+- `[config]` Add `[grpc.block_service]` section to configure gRPC `BlockService`
+  ([\#1094](https://github.com/cometbft/cometbft/issues/1094))
diff --git a/.changelog/unreleased/features/1094-grpc-block-service.md b/.changelog/unreleased/features/1094-grpc-block-service.md
@@ -0,0 +1,3 @@
+- `[grpc]` Add `BlockService` with client to facilitate fetching of blocks and
+  streaming of the latest committed block height
+  ([\#1094](https://github.com/cometbft/cometbft/issues/1094))
diff --git a/config/config.go b/config/config.go
@@ -561,19 +561,24 @@ type GRPCConfig struct {
 	// The gRPC version service provides version information about the node and
 	// the protocols it uses.
 	VersionService *GRPCVersionServiceConfig `mapstructure:"version_service"`
+
+	// The gRPC block service provides block information
+	BlockService *GRPCBlockServiceConfig `mapstructure:"block_service"`
 }
 
 func DefaultGRPCConfig() *GRPCConfig {
 	return &GRPCConfig{
 		ListenAddress:  "",
 		VersionService: DefaultGRPCVersionServiceConfig(),
+		BlockService:   DefaultGRPCBlockServiceConfig(),
 	}
 }
 
 func TestGRPCConfig() *GRPCConfig {
 	return &GRPCConfig{
 		ListenAddress:  "tcp://127.0.0.1:36670",
 		VersionService: TestGRPCVersionServiceConfig(),
+		BlockService:   TestGRPCBlockServiceConfig(),
 	}
 }
 
@@ -606,6 +611,22 @@ func TestGRPCVersionServiceConfig() *GRPCVersionServiceConfig {
 	}
 }
 
+type GRPCBlockServiceConfig struct {
+	Enabled bool `mapstructure:"enabled"`
+}
+
+func DefaultGRPCBlockServiceConfig() *GRPCBlockServiceConfig {
+	return &GRPCBlockServiceConfig{
+		Enabled: true,
+	}
+}
+
+func TestGRPCBlockServiceConfig() *GRPCBlockServiceConfig {
+	return &GRPCBlockServiceConfig{
+		Enabled: true,
+	}
+}
+
 //-----------------------------------------------------------------------------
 // P2PConfig
 

diff --git a/config/toml.go b/config/toml.go
@@ -296,6 +296,10 @@ laddr = "{{ .GRPC.ListenAddress }}"
 [grpc.version_service]
 enabled = {{ .GRPC.VersionService.Enabled }}
 
+# The gRPC block service returns block information
+[grpc.block_service]
+enabled = {{ .GRPC.BlockService.Enabled }}
+
 #######################################################
 ###           P2P Configuration Options             ###
 #######################################################

diff --git a/docs/core/configuration.md b/docs/core/configuration.md
@@ -34,13 +34,13 @@ like the file below, however, double check by inspecting the
 proxy_app = "tcp://127.0.0.1:26658"
 
 # A custom human readable name for this node
-moniker = "anonymous"
+moniker = "thinkpad"
 
 # If this node is many blocks behind the tip of the chain, BlockSync
 # allows them to catchup quickly by downloading blocks in parallel
 # and verifying their commits
 #
-# Deprecated: this key will be removed and BlockSync will be enabled 
+# Deprecated: this key will be removed and BlockSync will be enabled
 # unconditionally in the next major release.
 block_sync = true
 
@@ -148,14 +148,14 @@ unsafe = false
 # 1024 - 40 - 10 - 50 = 924 = ~900
 max_open_connections = 900
 
-# Maximum number of unique clientIDs that can /subscribe
+# Maximum number of unique clientIDs that can /subscribe.
 # If you're using /broadcast_tx_commit, set to the estimated maximum number
 # of broadcast_tx_commit calls per block.
 max_subscription_clients = 100
 
-# Maximum number of unique queries a given client can /subscribe to
-# If you're using GRPC (or Local RPC client) and /broadcast_tx_commit, set to
-# the estimated # maximum number of broadcast_tx_commit calls per block.
+# Maximum number of unique queries a given client can /subscribe to.
+# If you're using /broadcast_tx_commit, set to the estimated maximum number
+# of broadcast_tx_commit calls per block.
 max_subscriptions_per_client = 5
 
 # Experimental parameter to specify the maximum number of events a node will
@@ -215,6 +215,40 @@ tls_key_file = ""
 # pprof listen address (https://golang.org/pkg/net/http/pprof)
 pprof_laddr = ""
 
+#######################################################
+###       gRPC Server Configuration Options         ###
+#######################################################
+
+#
+# Note that the gRPC server is exposed unauthenticated. It is critical that
+# this server not be exposed directly to the public internet. If this service
+# must be accessed via the public internet, please ensure that appropriate
+# precautions are taken (e.g. fronting with a reverse proxy like nginx with TLS
+# termination and authentication, using DDoS protection services like
+# CloudFlare, etc.).
+#
+
+[grpc]
+
+# TCP or UNIX socket address for the RPC server to listen on. If not specified,
+# the gRPC server will be disabled.
+laddr = ""
+
+#
+# Each gRPC service can be turned on/off, and in some cases configured,
+# individually. If the gRPC server is not enabled, all individual services'
+# configurations are ignored.
+#
+
+# The gRPC version service provides version information about the node and the
+# protocols it uses.
+[grpc.version_service]
+enabled = true
+
+# The gRPC block service returns block information
+[grpc.block_service]
+enabled = true
+
 #######################################################
 ###           P2P Configuration Options             ###
 #######################################################
@@ -386,7 +420,7 @@ chunk_fetchers = "4"
 [blocksync]
 
 # Block Sync version to use:
-# 
+#
 # In v0.37, v1 and v2 of the block sync protocols were deprecated.
 # Please use v0 instead.
 #
@@ -432,6 +466,7 @@ create_empty_blocks_interval = "0s"
 
 # Reactor sleep duration parameters
 peer_gossip_sleep_duration = "100ms"
+peer_gossip_intraloop_sleep_duration = "0s"
 peer_query_maj23_sleep_duration = "2s"
 
 #######################################################

diff --git a/internal/rpctrace/rpctrace.go b/internal/rpctrace/rpctrace.go
@@ -0,0 +1,13 @@
+package rpctrace
+
+import "github.com/gofrs/uuid"
+
+// New returns a randomly generated string which can be used to assist in
+// tracing RPC errors.
+func New() (string, error) {
+	id, err := uuid.NewV4()
+	if err != nil {
+		return "", err
+	}
+	return id.String(), nil
+}
diff --git a/node/node.go b/node/node.go
@@ -1194,6 +1194,9 @@ func (n *Node) startRPC() ([]net.Listener, error) {
 		if n.config.GRPC.VersionService.Enabled {
 			opts = append(opts, grpcserver.WithVersionService())
 		}
+		if n.config.GRPC.BlockService.Enabled {
+			opts = append(opts, grpcserver.WithBlockService(n.blockStore, n.eventBus, n.Logger))
+		}
 		go func() {
 			if err := grpcserver.Serve(listener, opts...); err != nil {
 				n.Logger.Error("Error starting gRPC server", "err", err)

diff --git a/proto/buf.yaml b/proto/buf.yaml
@@ -8,4 +8,3 @@ lint:
   use:
     - BASIC
     - FILE_LOWER_SNAKE_CASE
-    - UNARY_RPC