Skip to content

Unexpected drift reported for message.timestamp.before.max.ms, message.timestamp.after.max.ms settings #191

Description

@nickdella

When running topicctl check, we're getting drift-related failures for two settings on a particular topic:

[2024-04-17 09:47:27]  INFO Check failed for topic <topic> (cluster=<cluster>, env=staging):
-----------------------------+----+----------------------------------------------------------------------------------------------------------------------------------
             NAME            | OK |                                                             DETAILS
-----------------------------+----+----------------------------------------------------------------------------------------------------------------------------------
  config correct             | ✓  |
  configs consistent         | ✓  |
  topic exists               | ✓  |
  config settings correct    | ✗  | 2 keys have different values between cluster and topic config: [message.timestamp.after.max.ms message.timestamp.before.max.ms]
  replication factor correct | ✓  |
  partition count correct    | ✓  |
  throttles clear            | ✓  |
  replicas in-sync           | ✓  |

Our topic configuration YAML does not explicitly configure these settings. From what I can tell, these happen to be fairly new settings introduced in Kafka 3.6 (Oct 2023). I noticed a similar issue in the Confluent Terraform provider. Perhaps these settings need to be added to topicctl?

Additional details:

confluent kafka topic describe reports the following values for these settings (these are the defaults):

message.timestamp.after.max.ms 9223372036854775807
message.timestamp.before.max.ms 9223372036854775807

When running apply, I see the following output:

[2024-04-17 10:32:52]  WARN Found 2 key(s) set in cluster but missing from config:
----------------------------------+----------------------
                KEY               |    CLUSTER VALUE
----------------------------------+----------------------
  message.timestamp.before.max.ms | 9223372036854775807
  message.timestamp.after.max.ms  | 9223372036854775807
----------------------------------+----------------------
These will be left as-is.

Running check after the apply results in the same output as above.

  • topicctl version: v1.16.1
  • Kafka version: 3.6 (Confluent Cloud)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions