-
Notifications
You must be signed in to change notification settings - Fork 20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ISSUE-70 Send all the failed message to the error reporter topic #69
Conversation
9e5b5b4
to
8d3014c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall LGTM! Few minor comments.
if (Objects.nonNull(errorReporter)) { | ||
final int responseCode = messageResponse.get(STATUS_KEY).asInt(); | ||
errorReporter.report(failedMessage.getRight(), | ||
new HttpException(String.format("error response= %s", failedMessage), responseCode)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
failedMessage is already provided to errorReported. In place of failedMessage inlet failure message should be added.
new HttpException(String.format("error response= %s", failedMessage), responseCode)); | |
new HttpException(String.format("error response= %s", <message>), responseCode)); |
|
||
Assertions.assertEquals(1, consumerRecords.count()); | ||
|
||
ConsumerRecord<byte[], byte[]> consumerRecord = consumerRecords.iterator().next(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please add following properties to connector config and test if headers have all error information -
"errors.deadletterqueue.context.headers.enable": "true",
"errors.log.enable": "true"
| "errors.tolerance": "all","errors.log.enable": "true or false","errors.deadletterqueue.topic.name": "topic-name"|Connector will continue on error and error occurrence will be logged as per the option and failed message will be send to dead letter topic | | ||
| "errors.tolerance": "all","errors.log.enable": "true or false","errors.deadletterqueue.topic.name": "topic-name","errors.deadletterqueue.context.headers.enable": "true" | Connector will continue on error and error occurrence will be logged as per the option and failed message will be send to dead letter topic and failure reason will be logged in message header. | | ||
Please follow documentation for more information : `https://www.confluent.io/blog/kafka-connect-deep-dive-error-handling-dead-letter-queues` | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's add notes in the end that in case of Authentication error such as 401 and 403 connector will stop irrespective of what error tolerance value is.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
DOne
…nd throw exception and Updated the documentation
Send all the failed message to the error reporter topic and throw exception
Summary
1-2 line summary of changes
Related Issue
Changes
Please enter each change as a new bullet point
Relevant Documentation
Please enter the links of any docs updated to reflect this change
How Has This Been Tested?
Types of changes
Checklist