The problem was not your service crash — it was Kafka coordinator downtime during MSK maintenance, which blocked offset commits and caused lag to explode...
Restart the consumer after MSK maintenance and reduce max-poll-records / increase default.api.timeout.ms so offset commits don’t time out again.
Yes but we could either add retries without increasing timeouts or use asyncCommit if duplicates are fine to circumvent transient errors so that app doesn’t end up being doing nothing when readers can’t commit in configured timeouts
The problem was not your service crash — it was Kafka coordinator downtime during MSK maintenance, which blocked offset commits and caused lag to explode...
Restart the consumer after MSK maintenance and reduce max-poll-records / increase default.api.timeout.ms so offset commits don’t time out again.
Yes but we could either add retries without increasing timeouts or use asyncCommit if duplicates are fine to circumvent transient errors so that app doesn’t end up being doing nothing when readers can’t commit in configured timeouts