-
Notifications
You must be signed in to change notification settings - Fork 78
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Failed to execute method NodeOps.repair #1370
Comments
We have also encountered the same issue when enabling
From the Reaper docs we understand that for an Incremental repair the only allowed value for
This is checked by the management-api here which indeed throws the error that I'm seeing. Run exec into a reaper pod to check it's configuration, we see that its
We can further check this by looking at the reaper tables inside cassandra:
which confirms that the default parallelism was set to My confusion comes from where this variable is set. From my limited research is not specified in the reaper deployment, it is not inside the Dockerfile, nor can it be configured from CRD. For possible workarounds, I see the following:
@adejanovski what do you think? |
Hello @adejanovski Can you tell us if there is enough information? |
What happened?
Cassandra container shows the following error in the logs:
Did you expect to see something different?
/api/v2/repairs status=500 Internal Server Error
should return with a 200.How to reproduce it (as minimally and precisely as possible):
Visible in the cassandra logs
Environment
this error is visible with:
and
Image hash
Kubernetes version information:
`kubectl version````kubectl version
Alias tip: k version
Client Version: v1.30.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.2-eks-db838b0
kubectl version
Alias tip: k version
Client Version: v1.30.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.4-eks-036c24b
apiVersion: cassandra.datastax.com/v1beta1
kind: CassandraDatacenter
metadata:
annotations:
eks.amazonaws.com/skip-containers: cassandra,server-system-logger,server-config-init
finalizers:
generation: 1
labels:
app.kubernetes.io/component: cassandra
app.kubernetes.io/name: k8ssandra-operator
app.kubernetes.io/part-of: k8ssandra
k8ssandra.io/cleaned-up-by: k8ssandracluster-controller
k8ssandra.io/cluster-name: cassandra
k8ssandra.io/cluster-namespace: k8ssandra-operator
name: us-east
namespace: k8ssandra-operator
spec:
additionalServiceConfig:
additionalSeedService: {}
allpodsService: {}
dcService: {}
nodePortService: {}
seedService: {}
clusterName: cassandra
config:
cassandra-env-sh:
additional-jvm-opts:
cassandra-yaml:
authenticator: PasswordAuthenticator
authorizer: CassandraAuthorizer
auto_bootstrap: true
auto_snapshot: true
batch_size_fail_threshold: 1500KiB
batch_size_warn_threshold: 10KiB
client_encryption_options:
enabled: true
keystore: /mnt/client-keystore/keystore
keystore_password: READACTED
optional: false
require_client_auth: false
truststore: /mnt/client-truststore/truststore
truststore_password: READACTED
concurrent_counter_writes: 64
concurrent_materialized_view_writes: 64
concurrent_reads: 64
concurrent_writes: 64
counter_cache_size: 50MiB
materialized_views_enabled: true
native_transport_port: 9042
num_tokens: 256
range_request_timeout: 10000ms
read_request_timeout: 15000ms
request_timeout: 20000ms
role_manager: CassandraRoleManager
server_encryption_options:
internode_encryption: all
keystore: /mnt/server-keystore/keystore
keystore_password: READACTED
require_client_auth: false
truststore: /mnt/server-truststore/truststore
truststore_password: READACTED
write_request_timeout: 2000ms
jvm-server-options:
initial_heap_size: 4294967296
jmx-connection-type: local-no-auth
jmx-port: 7199
jmx-remote-ssl: true
max_heap_size: 4294967296
jvm11-server-options:
garbage_collector: G1GC
configBuilderResources: {}
managementApiAuth: {}
networking: {}
podTemplateSpec:
metadata: {}
spec:
containers:
value: "no"
value: "128000000"
value: "true"
livenessProbe:
failureThreshold: 3
httpGet:
path: /api/v0/probes/liveness
port: 8080
scheme: HTTP
initialDelaySeconds: 230
periodSeconds: 15
successThreshold: 1
timeoutSeconds: 10
name: cassandra
readinessProbe:
failureThreshold: 3
httpGet:
path: /api/v0/probes/readiness
port: 8080
scheme: HTTP
initialDelaySeconds: 270
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
resources: {}
volumeMounts:
name: certs
name: cqlsh-config
subPath: cqlshrc
name: nodetool-config
subPath: nodetool-ssl.properties
name: client-keystore
name: client-truststore
name: server-keystore
name: server-truststore
resources: {}
value: GRPC
value: /var/lib/cassandra
valueFrom:
fieldRef:
fieldPath: metadata.name
valueFrom:
secretKeyRef:
key: username
name: cassandra-medusa
valueFrom:
secretKeyRef:
key: password
name: cassandra-medusa
image: docker.io/k8ssandra/medusa:0.21.0
imagePullPolicy: IfNotPresent
livenessProbe:
exec:
command:
failureThreshold: 10
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: medusa
ports:
name: grpc
protocol: TCP
readinessProbe:
exec:
command:
failureThreshold: 10
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
limits:
memory: 512Mi
requests:
cpu: 10m
memory: 116Mi
volumeMounts:
name: server-config
name: server-data
name: cassandra-medusa
name: podinfo
name: certificates
initContainers:
image: busybox:1.28
name: sysctl
resources: {}
securityContext:
privileged: true
resources: {}
value: RESTORE
value: /var/lib/cassandra
valueFrom:
fieldRef:
fieldPath: metadata.name
valueFrom:
secretKeyRef:
key: username
name: cassandra-medusa
valueFrom:
secretKeyRef:
key: password
name: cassandra-medusa
image: docker.io/k8ssandra/medusa:0.21.0
imagePullPolicy: IfNotPresent
name: medusa-restore
resources:
limits:
memory: 8Gi
requests:
cpu: 100m
memory: 100Mi
volumeMounts:
name: server-config
name: server-data
name: cassandra-medusa
name: podinfo
name: certificates
volumes:
secret:
secretName: cassandra-jks-keystore
name: cqlsh-config
name: cqlsh-config
name: nodetool-config
name: nodetool-config
secret:
items:
path: keystore
secretName: cassandra-jks-keystore
secret:
items:
path: truststore
secretName: cassandra-jks-keystore
secret:
items:
path: keystore
secretName: cassandra-jks-keystore
secret:
items:
path: truststore
secretName: cassandra-jks-keystore
name: cassandra-medusa
name: cassandra-medusa
items:
fieldPath: metadata.labels
path: labels
name: podinfo
secret:
secretName: medusa-certificates
racks:
nodeAffinityLabels:
topology.kubernetes.io/zone: us-east-1a
nodeAffinityLabels:
topology.kubernetes.io/zone: us-east-1b
nodeAffinityLabels:
topology.kubernetes.io/zone: us-east-1c
resources:
limits:
memory: 9Gi
requests:
cpu: "1"
memory: 9Gi
serverType: cassandra
serverVersion: 4.1.4
size: 3
storageConfig:
additionalVolumes:
name: vector-config
volumeSource:
configMap:
name: cassandra-us-east-cass-vector
name: metrics-agent-config
volumeSource:
configMap:
items:
path: metrics-collector.yaml
name: cassandra-us-east-metrics-agent-config
cassandraDataVolumeClaimSpec:
accessModes:
resources:
requests:
storage: 300Gi
storageClassName: ebs-xfs-sc
superuserSecretName: cassandra-superuser
systemLoggerResources:
limits:
memory: 512Mi
requests:
cpu: 100m
memory: 128Mi
users:
superuser: true
superuser: true
apiVersion: k8ssandra.io/v1alpha1
kind: K8ssandraCluster
metadata:
annotations:
config.kubernetes.io/origin: |
path: ../../base/k8ssandra-encrypted.yaml
k8ssandra.io/initial-system-replication: '{"us-east":3}'
finalizers:
generation: 5
name: cassandra
namespace: k8ssandra-operator
spec:
auth: true
cassandra:
clientEncryptionStores:
keystorePasswordSecretRef:
name: jks-password
keystoreSecretRef:
key: keystore.jks
name: cassandra-jks-keystore
truststorePasswordSecretRef:
name: jks-password
truststoreSecretRef:
key: truststore.jks
name: cassandra-jks-keystore
config:
cassandraYaml:
authenticator: PasswordAuthenticator
authorizer: CassandraAuthorizer
auto_bootstrap: true
auto_snapshot: true
batch_size_fail_threshold: 1500KiB
batch_size_warn_threshold: 10KiB
client_encryption_options:
enabled: true
optional: false
require_client_auth: false
concurrent_counter_writes: 64
concurrent_materialized_view_writes: 64
concurrent_reads: 64
concurrent_writes: 64
counter_cache_size: 50MiB
materialized_views_enabled: true
native_transport_port: 9042
num_tokens: 256
range_request_timeout: 10000ms
read_request_timeout: 15000ms
request_timeout: 20000ms
server_encryption_options:
internode_encryption: all
require_client_auth: false
write_request_timeout: 2000ms
jvmOptions:
additionalOptions:
gc: G1GC
heap_initial_size: 4Gi
heap_max_size: 4Gi
jmx_connection_type: local-no-auth
jmx_port: 7199
jmx_remote_ssl: true
containers:
failureThreshold: 3
httpGet:
path: /api/v0/probes/liveness
port: 8080
scheme: HTTP
initialDelaySeconds: 230
periodSeconds: 15
successThreshold: 1
timeoutSeconds: 10
name: cassandra
readinessProbe:
failureThreshold: 3
httpGet:
path: /api/v0/probes/readiness
port: 8080
scheme: HTTP
initialDelaySeconds: 270
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
volumeMounts:
name: certs
name: cqlsh-config
subPath: cqlshrc
name: nodetool-config
subPath: nodetool-ssl.properties
datacenters:
image: busybox:1.28
name: sysctl
securityContext:
privileged: true
metadata:
name: us-east
perNodeConfigInitContainerImage: mikefarah/yq:4
racks:
nodeAffinityLabels:
topology.kubernetes.io/zone: us-east-1a
nodeAffinityLabels:
topology.kubernetes.io/zone: us-east-1b
nodeAffinityLabels:
topology.kubernetes.io/zone: us-east-1c
resources:
limits:
memory: 9Gi
requests:
cpu: 1
memory: 9Gi
size: 3
stopped: false
extraVolumes:
volumes:
secret:
secretName: cassandra-jks-keystore
name: cqlsh-config
name: cqlsh-config
name: nodetool-config
name: nodetool-config
metadata:
annotations:
eks.amazonaws.com/skip-containers: cassandra,server-system-logger,server-config-init
mgmtAPIHeap: 128M
networking:
hostNetwork: false
perNodeConfigInitContainerImage: mikefarah/yq:4
serverEncryptionStores:
keystorePasswordSecretRef:
name: jks-password
keystoreSecretRef:
key: keystore.jks
name: cassandra-jks-keystore
truststorePasswordSecretRef:
name: jks-password
truststoreSecretRef:
key: truststore.jks
name: cassandra-jks-keystore
serverType: cassandra
serverVersion: 4.1.4
softPodAntiAffinity: false
storageConfig:
cassandraDataVolumeClaimSpec:
accessModes:
resources:
requests:
storage: 300Gi
storageClassName: ebs-xfs-sc
telemetry:
mcac:
enabled: false
prometheus:
enabled: true
vector:
components:
sinks:
target = "stdout"
[sinks.console_output.encoding]
codec = "json"
inputs:
name: console_output
type: console
enabled: true
resources:
limits:
memory: 512Mi
requests:
cpu: 100m
memory: 128Mi
scrapeInterval: 30s
medusa:
certificatesSecretRef:
name: medusa-certificates
containerImage:
name: medusa
registry: docker.io
repository: k8ssandra
tag: 0.21.0
containerResources:
limits:
memory: 512Mi
requests:
cpu: 10m
memory: 116Mi
storageProperties:
bucketName: dow-backups
concurrentTransfers: 10
credentialsType: role-based
maxBackupAge: 0
maxBackupCount: 0
multiPartUploadThreshold: 104857600
prefix: cassandra-tests
region: us-east-1
secure: true
storageProvider: s3
storageSecretRef:
name: ""
transferMaxBandwidth: 90MB/s
reaper:
ServiceAccountName: default
autoScheduling:
enabled: true
initialDelayPeriod: PT15S
percentUnrepairedThreshold: 10
periodBetweenPolls: PT10M
repairType: AUTO
scheduleSpreadPeriod: PT6H
timeBeforeFirstSchedule: PT5M
containerImage:
name: cassandra-reaper
repository: thelastpickle
tag: 3.6.0
deploymentMode: SINGLE
heapSize: 2Gi
httpManagement:
enabled: true
keyspace: reaper_db
secretsProvider: internal
telemetry:
cassandra:
endpoint:
address: 0.0.0.0
mcac:
enabled: false
prometheus:
enabled: true
vector:
enabled: true
resources:
limits:
cpu: 100m
memory: 512Mi
requests:
cpu: 100m
memory: 128Mi
secretsProvider: internal
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:31:35,347 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v2/repairs status=500 Internal Server Error
INFO [nioEventLoopGroup-2-1] 2024-07-16 12:31:38,541 Cli.java:663 - address=/10.210.20.219:56784 url=/api/v0/probes/readiness status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:31:43,538 Cli.java:663 - address=/10.210.20.219:51656 url=/api/v0/probes/liveness status=200 OK
INFO [nioEventLoopGroup-2-1] 2024-07-16 12:31:48,540 Cli.java:663 - address=/10.210.20.219:51666 url=/api/v0/probes/readiness status=200 OK
INFO [nioEventLoopGroup-2-1] 2024-07-16 12:31:58,539 Cli.java:663 - address=/10.210.20.219:48066 url=/api/v0/probes/liveness status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:31:58,540 Cli.java:663 - address=/10.210.20.219:48068 url=/api/v0/probes/readiness status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:32:02,818 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v0/metadata/endpoints status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:32:02,820 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v0/metadata/endpoints status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:32:02,909 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v1/ops/tables/compactions status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:32:05,371 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v0/metadata/endpoints status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:32:05,373 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v0/metadata/endpoints status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:32:05,466 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v1/ops/tables/compactions status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:32:08,541 Cli.java:663 - address=/10.210.20.219:55514 url=/api/v0/probes/readiness status=200 OK
INFO [nioEventLoopGroup-2-1] 2024-07-16 12:32:13,538 Cli.java:663 - address=/10.210.20.219:58392 url=/api/v0/probes/liveness status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:32:18,540 Cli.java:663 - address=/10.210.20.219:58402 url=/api/v0/probes/readiness status=200 OK
INFO [nioEventLoopGroup-2-1] 2024-07-16 12:32:28,539 Cli.java:663 - address=/10.210.20.219:52776 url=/api/v0/probes/liveness status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:32:28,540 Cli.java:663 - address=/10.210.20.219:52790 url=/api/v0/probes/readiness status=200 OK
INFO [nioEventLoopGroup-2-1] 2024-07-16 12:32:38,541 Cli.java:663 - address=/10.210.20.219:39932 url=/api/v0/probes/readiness status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:32:40,870 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v0/metadata/endpoints status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:32:40,873 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v0/metadata/endpoints status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:32:40,989 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v1/ops/tables/compactions status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:32:41,561 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v0/metadata/endpoints status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:32:41,564 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v0/metadata/endpoints status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:32:41,657 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v1/ops/tables/compactions status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:32:43,539 Cli.java:663 - address=/10.210.20.219:44100 url=/api/v0/probes/liveness status=200 OK
INFO [nioEventLoopGroup-2-1] 2024-07-16 12:32:48,540 Cli.java:663 - address=/10.210.20.219:44112 url=/api/v0/probes/readiness status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:32:58,538 Cli.java:663 - address=/10.210.20.219:36508 url=/api/v0/probes/liveness status=200 OK
INFO [nioEventLoopGroup-2-1] 2024-07-16 12:32:58,541 Cli.java:663 - address=/10.210.20.219:36520 url=/api/v0/probes/readiness status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:33:08,541 Cli.java:663 - address=/10.210.20.219:52446 url=/api/v0/probes/readiness status=200 OK
INFO [nioEventLoopGroup-2-1] 2024-07-16 12:33:13,538 Cli.java:663 - address=/10.210.20.219:52002 url=/api/v0/probes/liveness status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:33:17,148 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v0/metadata/endpoints status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:33:17,150 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v0/metadata/endpoints status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:33:17,152 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v0/metadata/endpoints status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:33:17,161 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v0/metadata/endpoints status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:33:17,162 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v0/metadata/endpoints status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:33:17,254 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v1/ops/tables/compactions status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:33:18,537 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v0/metadata/endpoints status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:33:18,539 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v0/metadata/endpoints status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:33:18,540 Cli.java:663 - address=/10.210.20.219:52018 url=/api/v0/probes/readiness status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:33:18,643 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v1/ops/tables/compactions status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:33:26,184 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v0/metadata/endpoints status=200 OK
INFO [nioEventLoopGroup-2-2] 2024-07-16 12:33:26,186 Cli.java:663 - address=/10.210.18.172:49500 url=/api/v0/metadata/endpoints status=200 OK
com.datastax.oss.driver.api.core.servererrors.ServerError: Failed to execute method NodeOps.repair
at com.datastax.oss.driver.api.core.servererrors.ServerError.copy(ServerError.java:54)
at com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures.getUninterruptibly(CompletableFutures.java:149)
at com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:53)
at com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:30)
at com.datastax.oss.driver.internal.core.session.DefaultSession.execute(DefaultSession.java:230)
at com.datastax.oss.driver.api.core.cql.SyncCqlSession.execute(SyncCqlSession.java:54)
at com.datastax.mgmtapi.CqlService.executePreparedStatement(CqlService.java:57)
at com.datastax.mgmtapi.resources.v2.RepairResourcesV2.lambda$repair$0(RepairResourcesV2.java:80)
at com.datastax.mgmtapi.resources.common.BaseResources.handle(BaseResources.java:67)
at com.datastax.mgmtapi.resources.v2.RepairResourcesV2.repair(RepairResourcesV2.java:71)
at jdk.internal.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:170)
at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:130)
at org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:643)
at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:507)
at org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$2(ResourceMethodInvoker.java:457)
at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:364)
at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:459)
at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:419)
at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:393)
at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:68)
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:492)
at org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:261)
at org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:161)
at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:364)
at org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:164)
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:247)
at org.jboss.resteasy.plugins.server.netty.RequestDispatcher.service(RequestDispatcher.java:86)
at org.jboss.resteasy.plugins.server.netty.RequestHandler.channelRead0(RequestHandler.java:51)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61)
at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Unknown Source)
The text was updated successfully, but these errors were encountered: