Spring Boot Interview Question - Kafka Consumer Became a Zombie

The Consumer Was Alive. The Business Was Not.

Scenario

You’re on-call for a Spring Boot service called TransactionIndexerService.

The service consumes transaction events from Kafka and indexes them into Elasticsearch.

The service runs in Kubernetes and uses Spring Kafka.

The application has been stable for months.

Incident

At 2:17 AM:

Consumer Lag: 0 → 8 Million

You open Grafana:

CPU Usage      : 3%
Memory Usage   : Stable
Pod Restarts   : 0
Error Rate     : Low
Consumer Lag   : Growing rapidly

The service appears healthy.

You find logs:

02:17:03 ERROR

org.apache.kafka.common.errors.TimeoutException:
Timeout of 60000ms expired before successfully committing offsets

A few minutes later the infrastructure team reports:

AWS MSK performed a maintenance operation.

During maintenance the consumer group coordinator became unavailable for approximately 2 minutes.

---

📢 Get actionable Java and Spring Boot insights every week, including practical code tips and real-world, use-case-based interview questions, to help you level up your backend skills—join 7500+ subscribers for hand-crafted, no-fluff content.

First 100 paid subscribers will get the annual membership at $50/year forever that is ~ $4/mo ( 92 already converted to paid, 11 remaining)

So far we have covered 62 real world based interview questions and will add up to 100 by end of this year.

Testimonials

---

Question 1

The application never crashed. Pods remained healthy. However:

records-consumed-rate = 0

for more than 30 minutes.

How is this possible?

Explain what might have happened inside:

• Spring Kafka

• Kafka consumer group

• Offset commit mechanism

Reasoning

The pod was healthy, but the Kafka consumer was not.

From the logs shared by infrastructure team, during the MSK maintenance, the consumer group coordinator became unavailable. Since offset commits are sent to the coordinator, commit operations started timing out.

A likely sequence is:

Coordinator unavailable
↓
Offset commits fail/timeout
↓
Consumer stops polling regularly
↓
Heartbeats stop
↓
Consumer removed from group
↓
Partitions revoked
↓
No records consumed

Even after the coordinator recovered, the consumer may not have successfully rejoined the group or resumed polling, leaving the application in a "zombie" state.

From Spring Kafka’s perspective:

• Listener container alive

• JVM alive

• Pod alive

but:

• No partition assignments

• No successful polls

• No message processing

so records-consumed-rate remains zero.

The key lesson is that process health != consumer health. Kubernetes only sees a running Spring Boot application, while the Kafka consumer group has stopped making progress.

Under what conditions do Spring Kafka will recover automatically?

Spring Kafka will recover automatically only when the underlying consumer loop keeps functioning and Kafka can re-establish group coordination on its own.

1. Poll loop is continuously running

Spring Kafka recovery depends on this core rule:

poll() must keep being called

As long as:

• listener returns quickly

• no blocking in user code

• container thread is alive

then Kafka client can:

send heartbeats
refresh metadata
retry coordinator lookup
rejoin group

2. Temporary coordinator or broker failure

If failure is short-lived:

Coordinator down (MSK maintenance)
↓
commit/poll fails temporarily
↓
Kafka client retries internally
↓
Coordinator comes back
↓
Consumer resumes

No app restart needed.

3. Heartbeats are not interrupted

Recovery works only if:

heartbeat.interval.ms keeps firing

That happens only when:

• poll is happening regularly

• thread is not blocked

Then Kafka can detect:

group instability → trigger rebalance → recover

4. Consumer stays in a valid group state

If consumer is only temporarily disconnected:

• still has valid session

• not fully evicted from group

Then it can rejoin automatically when coordinator returns.

5. Rebalance completes successfully

Spring Kafka recovers when:

rebalance → partitions reassigned → listener resumes

This is fully automatic if the container is healthy.

When Spring Kafka will not recover automatically ?

1. Blocked poll thread (most common production issue)

Example:

commitSync(); // blocks

Then:

poll() stops
↓
heartbeat stops
↓
consumer removed from group
↓
no recovery path

2. Session timeout exceeded

If no heartbeat happens within:

session.timeout.ms

Kafka forcibly removes consumer.

Recovery becomes unreliable because:

• consumer must fully rejoin group

• may get stuck if thread is blocked

3. Listener thread stuck in long processing

If business logic blocks:

process() takes too long

then:

• poll not called

• heartbeat stops

• consumer dies in group

4. Silent exception swallowing + no poll loop recovery

If:

catch (Exception e) {
    log.warn(e);
}

but loop doesn’t recover state:

• consumer remains in broken state

• never reinitializes metadata

Explain the difference between commitSync vs commitAsync, which one to use and when?

commitSync()

• Sends offset commit request

• Blocks until Kafka acknowledges it

Flow:

process records
↓
commitSync()
↓
wait for coordinator ACK
↓
success or exception

Characteristics:

• Strong guarantee (commit is confirmed before moving on)

• Higher latency

• Can block consumer thread

Risk:

If Kafka coordinator is slow/unavailable:

commitSync() blocks
↓
poll() stops
↓
heartbeats stop
↓
consumer can die / lag increases

commitAsync()

• Sends commit request

• Does not wait for response

Flow:

process records
↓
commitAsync()
↓
continue processing immediately

Characteristics:

• High throughput

• Non-blocking

• Better for hot loops

Risk:

• Commit failures may be missed

• No strong guarantee unless handled carefully

Example issue:

commit 100 fails
commit 200 succeeds
→ offset 100 retry later can cause inconsistency

Production best practice

Most real systems use both together:

while (running) {
    process(records);

    consumer.commitAsync(); // fast path
}

// on shutdown
consumer.commitSync(); // safe final commit

But important point to note here is when crash or forced termination happens

like:

OOM kill
SIGKILL
node crash
network partition

Then:

NO shutdown hook
NO commitSync
NO final flush

hybrid pattern gives no guarantee here.

---

Thats all for this week friends! Thanks for reading this far. If you liked it please share with your network.

Before you leave, don’t forget to unlock all the real world use case based interview questions , just at $4/mo (with annual subscription).

Happy Coding 🚀
Suraj

Subscribe | Sponsor us | LinkedIn | Twitter