Revision note (2026-09-15). The earlier version of this article asked for "Kafka 2.0 or higher (with support for incremental config updates)" and "Java 8 or newer", linked the Kafka 2.5 javadoc, pinned no client version, wrote a SASL username and password inline in the sample while telling readers never to do that, called a createTopics wrapper "idempotent" because it swallowed TopicExistsException, and listed failure modes without a single observed error. incrementalAlterConfigs exists since Kafka 2.3, the 4.x clients need Java 11 and brokers 2.1 or newer, and the old alterConfigs method is not deprecated any more: it is gone. This version is rebuilt from a lab: one Kafka 4.3.1 broker in Docker, kafka-clients 4.3.1, eight tests, and every output pasted as printed.
What was tested, and with what
- Broker: the
apache/kafka:4.3.1image, a single KRaft node, plaintext listener,auto.create.topics.enable=false,StandardAuthorizerenabled withallow.everyone.if.no.acl.found=true. - Client:
org.apache.kafka:kafka-clients:4.3.1(the current release on Maven Central as of 2026-09-16), Java 17, JUnit 5.13.4, Gradle 8.8. - For one comparison:
kafka-clients:3.9.1, the last 3.x line, run in a child JVM against the same broker.
Not tested: SASL or TLS connections, clusters with more than one broker, replication factor above 1, brokers older than 4.3.1, broker-level dynamic configs. Where the text below states something about those, it says so and names the source.
Which API, and which versions
The Admin interface (returned by Admin.create(...); AdminClient is the older abstract class with the same methods) has had two ways to change configs:
alterConfigs(Map<ConfigResource, Config>): replaces the whole config set of the resource with exactly the entries you pass. Deprecated in 2.3 by KIP-339. Removed in 4.0.incrementalAlterConfigs(Map<ConfigResource, Collection<AlterConfigOp>>): applies individualSET,DELETE,APPEND,SUBTRACToperations and leaves every other config alone. Available since 2.3.
That is not a stylistic point. The lab compiled a five-line program calling alterConfigs against both client versions:
== javac -Xlint:deprecation against kafka-clients 3.9.1
LegacyAlterConfigs.java:24: warning: [deprecation] alterConfigs(Map<ConfigResource,Config>) in Admin has been deprecated
admin.alterConfigs(Map.of(resource, replacement)).all().get();
^
1 warning
exit=0
== javac against kafka-clients 4.3.1
LegacyAlterConfigs.java:24: error: cannot find symbol
admin.alterConfigs(Map.of(resource, replacement)).all().get();
^
symbol: method alterConfigs(Map<ConfigResource,Config>)
location: variable admin of type Admin
1 error
exit=1The Kafka 4.0.0 upgrade notes state the two other version facts this article depends on: "The minimum Java version required by clients and Kafka Streams applications has been increased from Java 8 to Java 11 while brokers, connect and tools now require Java 17", and "Users should ensure brokers are version 2.1 or higher before upgrading Java clients … to 4.0". The Admin.class in the 4.3.1 jar has class-file major version 55, which is Java 11.
Dependency used throughout:
dependencies {
implementation 'org.apache.kafka:kafka-clients:4.3.1'
runtimeOnly 'org.slf4j:slf4j-simple:2.0.17'
}One long-lived Admin, bounded waits
Admin is thread-safe and holds a network thread and connections; create one per process and close it at shutdown. Every Admin call returns a result object whose futures complete asynchronously, so automation code normally waits with a bound. The lab wrapper looks like this:
public final class TopicConfigAdmin implements AutoCloseable {
private static final Duration TIMEOUT = Duration.ofSeconds(30);
private final Admin admin;
public TopicConfigAdmin(String bootstrapServers) {
this.admin = Admin.create(Map.of(
AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers,
AdminClientConfig.REQUEST_TIMEOUT_MS_CONFIG, "15000",
AdminClientConfig.DEFAULT_API_TIMEOUT_MS_CONFIG, "30000"));
}
public void alterTopic(String topic, AlterConfigOp... ops)
throws InterruptedException, ExecutionException, TimeoutException {
ConfigResource resource = new ConfigResource(ConfigResource.Type.TOPIC, topic);
admin.incrementalAlterConfigs(Map.of(resource, Arrays.asList(ops)))
.all().get(TIMEOUT.toMillis(), TimeUnit.MILLISECONDS);
}
public static AlterConfigOp set(String name, String value) {
return new AlterConfigOp(new ConfigEntry(name, value), AlterConfigOp.OpType.SET);
}
public static AlterConfigOp delete(String name) {
return new AlterConfigOp(new ConfigEntry(name, null), AlterConfigOp.OpType.DELETE);
}
@Override
public void close() {
admin.close(TIMEOUT);
}
}Connection settings such as security.protocol and sasl.jaas.config belong in the same map, loaded from the environment or a secrets store at startup. The lab ran plaintext, so nothing about SASL was exercised here; the point of not writing a password into a .java file does not need a test.
Creating a topic: what "already exists" does and does not tell you
createTopics fails with TopicExistsException when the name is taken. A common wrapper catches it and reports success. The lab asked for a topic that already existed with 1 partition and default configs, this time requesting 3 partitions and retention.ms=1000:
[lab] createTopics(exists-41e9fbae, 3 partitions, retention.ms=1000) on an existing 1-partition topic -> TopicExistsException; topic still has 1 partition(s), retention.ms=604800000 DEFAULT_CONFIGThe exception is about the name only. If your automation must guarantee a shape, treat "exists" as the start of a second step: describeTopics for the partition count, describeConfigs for the settings, then createPartitions and incrementalAlterConfigs to converge. Partition counts only go up:
[lab] increaseTo(3) -> 3 partitions
[lab] increaseTo(2) on 3 partitions -> InvalidPartitionsException: The topic parts-beae24e6 currently has 3 partition(s); 2 would not be an increase.Reading configs: value, source, and where a default comes from
describeConfigs with includeSynonyms(true) returns, for every config, the effective value, its ConfigSource, and the chain of settings behind it. The lab prints each entry as name=value SOURCE [synonyms]. A fresh topic:
[lab] fresh retention.ms=604800000 DEFAULT_CONFIG
[lab] fresh max.message.bytes=1048588 DEFAULT_CONFIG [message.max.bytes=1048588 DEFAULT_CONFIG]
[lab] fresh cleanup.policy=delete DEFAULT_CONFIG [log.cleanup.policy=delete DEFAULT_CONFIG]DEFAULT_CONFIG means nobody set it anywhere; the synonym shows the broker property it derives from (max.message.bytes on a topic mirrors message.max.bytes on the broker). After an override the source becomes DYNAMIC_TOPIC_CONFIG, and the synonym list keeps the default underneath it, which is the value a DELETE would fall back to.
public Map<String, ConfigEntry> describeTopicConfigs(String topic) throws Exception {
ConfigResource resource = new ConfigResource(ConfigResource.Type.TOPIC, topic);
Config config = admin.describeConfigs(List.of(resource), new DescribeConfigsOptions().includeSynonyms(true))
.all().get(30, TimeUnit.SECONDS).get(resource);
Map<String, ConfigEntry> byName = new TreeMap<>();
config.entries().forEach(entry -> byName.put(entry.name(), entry));
return byName;
}The four op types, observed
SET changes only the key you name. Two overrides applied one after the other:
[lab] SET#1 retention.ms=3600000 DYNAMIC_TOPIC_CONFIG [retention.ms=3600000 DYNAMIC_TOPIC_CONFIG]
[lab] SET#1 max.message.bytes=1048588 DEFAULT_CONFIG [message.max.bytes=1048588 DEFAULT_CONFIG]
[lab] SET#2 retention.ms=3600000 DYNAMIC_TOPIC_CONFIG [retention.ms=3600000 DYNAMIC_TOPIC_CONFIG]
[lab] SET#2 max.message.bytes=2000000 DYNAMIC_TOPIC_CONFIG [max.message.bytes=2000000 DYNAMIC_TOPIC_CONFIG, message.max.bytes=1048588 DEFAULT_CONFIG]The second call named only max.message.bytes; retention.ms kept its override.
DELETE removes the override and the default returns. Deleting a key that has no override is not an error:
[lab] created retention.ms=3600000 DYNAMIC_TOPIC_CONFIG [retention.ms=3600000 DYNAMIC_TOPIC_CONFIG]
[lab] DELETE retention.ms=604800000 DEFAULT_CONFIG
[lab] DELETE of a key that is not overridden -> completed without errorAPPEND and SUBTRACT work on LIST-typed configs only. cleanup.policy is one:
[lab] SET cleanup.policy=compact -> compact
[lab] APPEND cleanup.policy+=delete -> compact,delete
[lab] SUBTRACT cleanup.policy-=compact -> delete
[lab] APPEND retention.ms -> InvalidConfigurationException: Can't APPEND to key retention.ms because its type is not LIST.Re-applying the same ops is idempotent. The same SET pair sent twice produced identical describeConfigs output both times (asserted in the lab), so a reconciliation loop can send its desired state on every run without a diff step. Whether that is a good idea at scale is a broker-load question the lab did not measure.
What the removed API did to configs you did not mention
This is the reason alterConfigs was deprecated and then removed, and the earlier article never said it. The lab set two overrides with the 4.3.1 client, then ran a 3.9.1 client that called alterConfigs with only retention.ms:
[lab] before retention.ms=3600000 DYNAMIC_TOPIC_CONFIG [retention.ms=3600000 DYNAMIC_TOPIC_CONFIG]
[lab] before max.message.bytes=2000000 DYNAMIC_TOPIC_CONFIG [max.message.bytes=2000000 DYNAMIC_TOPIC_CONFIG, message.max.bytes=1048588 DEFAULT_CONFIG]
[lab] legacy process: legacy alterConfigs applied retention.ms=1800000 to legacy-856df94e
[lab] after retention.ms=1800000 DYNAMIC_TOPIC_CONFIG [retention.ms=1800000 DYNAMIC_TOPIC_CONFIG]
[lab] after max.message.bytes=1048588 DEFAULT_CONFIG [message.max.bytes=1048588 DEFAULT_CONFIG]max.message.bytes went back to the default because it was not in the replacement set. The 4.3.1 broker still accepts the old request from old clients, so a mixed fleet of tools can still produce this; a 4.x client simply cannot send it.
The errors you actually get
Every failure below is the getCause() of the ExecutionException from .get().
- Wrong value type:
InvalidConfigurationException: Invalid value one-day for configuration retention.ms: Not a number of type LONG - Unknown key:
InvalidConfigurationException: Unknown topic config name: retention.minutes - APPEND on a scalar:
InvalidConfigurationException: Can't APPEND to key retention.ms because its type is not LIST. - Shrinking partitions:
InvalidPartitionsException: The topic ... currently has 3 partition(s); 2 would not be an increase. - Name taken:
TopicExistsException(see above; says nothing about the spec) - Not authorized:
TopicAuthorizationException: Topic authorization failed.(below)
The first three share one class, so code that branches on exception type cannot tell a typo in the key from a bad value; the message can.
Authorization failure, reproduced without SASL
On a plaintext listener every client is User:ANONYMOUS, and with allow.everyone.if.no.acl.found=true the authorizer allows anything on a resource that has no ACL. The lab created one ACL on one topic, DENY User:ANONYMOUS ALTER_CONFIGS, and tried again:
[lab] createAcls: (pattern=ResourcePattern(resourceType=TOPIC, name=acl-1587e022, patternType=LITERAL), entry=(principal=User:ANONYMOUS, host=*, operation=ALTER_CONFIGS, permissionType=DENY))
[lab] incrementalAlterConfigs with DENY ALTER_CONFIGS -> TopicAuthorizationException: Topic authorization failed.
[lab] describeConfigs on the same topic (only ACL on it: DENY ALTER_CONFIGS; allow.everyone.if.no.acl.found=true) -> TopicAuthorizationException: Topic authorization failed.
[lab] after deleteAcls, describeConfigs: retention.ms=604800000 DEFAULT_CONFIG
[lab] after deleteAcls, SET retention.ms=1000: retention.ms=1000 DYNAMIC_TOPIC_CONFIG [retention.ms=1000 DYNAMIC_TOPIC_CONFIG]The first denial is what was asked for. The second was not: no ACL mentions DESCRIBE_CONFIGS, yet describeConfigs is denied too. "No ACL found" is evaluated per resource, so the moment a topic has any ACL, everything not explicitly allowed on it is denied. On a real cluster with named principals the same rule applies: an automation user needs explicit ALTER_CONFIGS and DESCRIBE_CONFIGS on the topics it manages (or a prefixed pattern), and the error message will not tell you which of the two is missing.
Deleting a topic
deleteTopics(...).all().get() returned, and within the lab's 5 second poll the topic was gone from listTopics():
[lab] deleteTopics -> topic absent from listTopics: trueThe broker property delete.topic.enable defaults to true. Whether producers or consumers still use the topic is not something the API checks; that remains the caller's problem.
Sources
- Kafka 4.3.1 javadoc,
Admin: https://kafka.apache.org/43/javadoc/org/apache/kafka/clients/admin/Admin.html - Kafka 4.3.1 javadoc,
AlterConfigOp.OpType: https://kafka.apache.org/43/javadoc/org/apache/kafka/clients/admin/AlterConfigOp.OpType.html - Kafka 4.3.1 javadoc,
ConfigEntry.ConfigSource: https://kafka.apache.org/43/javadoc/org/apache/kafka/clients/admin/ConfigEntry.ConfigSource.html - Kafka 4.3 upgrade notes, "Notable changes in 4.0.0" (Java 11 for clients,
alterConfigsremoved, brokers 2.1+): https://kafka.apache.org/43/getting-started/upgrade/ - Kafka 4.3 authorization and ACLs: https://kafka.apache.org/43/security/authorization-and-acls/
- KIP-339, Create a new IncrementalAlterConfigs API: https://cwiki.apache.org/confluence/display/KAFKA/KIP-339%3A+Create+a+new+IncrementalAlterConfigs+API
kafka-clients4.3.1 on Maven Central: https://central.sonatype.com/artifact/org.apache.kafka/kafka-clients/4.3.1apache/kafkaDocker image: https://hub.docker.com/r/apache/kafka