ISSUE-70 Send all the failed message to the error reporter topic #69
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
maniskum
force-pushed
the
PLAT-188185
branch
2 times, most recently
from
9e5b5b4
to
8d3014c
Compare
vivetiwa
previously approved these changes
May 28, 2024
| if (Objects.nonNull(errorReporter)) { | ||
| final int responseCode = messageResponse.get(STATUS_KEY).asInt(); | ||
| errorReporter.report(failedMessage.getRight(), | ||
| new HttpException(String.format("error response= %s", failedMessage), responseCode)); |
There was a problem hiding this comment.
failedMessage is already provided to errorReported. In place of failedMessage inlet failure message should be added.
Suggested change
| new HttpException(String.format("error response= %s", failedMessage), responseCode)); | |
| new HttpException(String.format("error response= %s", <message>), responseCode)); |
|
|
||
| Assertions.assertEquals(1, consumerRecords.count()); | ||
|
|
||
| ConsumerRecord<byte[], byte[]> consumerRecord = consumerRecords.iterator().next(); |
There was a problem hiding this comment.
Can you please add following properties to connector config and test if headers have all error information -
"errors.deadletterqueue.context.headers.enable": "true",
"errors.log.enable": "true"
vivetiwa
changed the title
vivetiwa
reviewed
May 31, 2024
| | "errors.tolerance": "all","errors.log.enable": "true or false","errors.deadletterqueue.topic.name": "topic-name"|Connector will continue on error and error occurrence will be logged as per the option and failed message will be send to dead letter topic | | ||
| | "errors.tolerance": "all","errors.log.enable": "true or false","errors.deadletterqueue.topic.name": "topic-name","errors.deadletterqueue.context.headers.enable": "true" | Connector will continue on error and error occurrence will be logged as per the option and failed message will be send to dead letter topic and failure reason will be logged in message header. | | ||
| Please follow documentation for more information : `https://www.confluent.io/blog/kafka-connect-deep-dive-error-handling-dead-letter-queues` | ||
|
|
There was a problem hiding this comment.
Let's add notes in the end that in case of Authentication error such as 401 and 403 connector will stop irrespective of what error tolerance value is.
…nd throw exception and Updated the documentation
vivetiwa
approved these changes
May 31, 2024
vivetiwa
changed the title
This was referenced Jun 10, 2024
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Send all the failed message to the error reporter topic and throw exception
Summary
1-2 line summary of changes
Related Issue
Changes
Please enter each change as a new bullet point
Relevant Documentation
Please enter the links of any docs updated to reflect this change
How Has This Been Tested?
Types of changes
Checklist