Kafka
desktop only The browser sandbox doesn’t allow raw TCP connections, so Kafka — which requires a long-lived TCP session — is available only in the desktop app.
What works
Section titled “What works”-
Produce — pick a topic, set text headers, target an optional partition, and send a key + value as UTF-8 text, validated JSON, or Base64-encoded arbitrary bytes. Base64 is decoded only at the Electron boundary, so invalid UTF-8 bytes are preserved exactly. Record headers use a key/value table with bulk edit (one
key: valueper line,//to disable a row) and an optional description per row; they’re produced exactly as typed. -
Consume — read a topic and see messages live with partition / offset / timestamp, either in read-only Browse mode (the default) or as a Consumer group member. See Consume modes.
-
Live consumer controls — pause and resume the whole consumer, or individual partitions assigned to it from the Partitions table under the consume controls (other partitions keep flowing). Seek one assigned partition of the running consumer to an exact offset or to the first record at or after a timestamp, without resubscribing. Seek targets are clamped to the partition’s current earliest/latest offsets; a timestamp newer than every record lands at the end. While this consumer keeps the partition, a paused partition resumes from the first record it withheld (or from where you seeked it), and withheld records are never auto-committed. If a rebalance moves a paused partition to another member, its pause is dropped and the new owner starts from the group’s committed offset.
-
Smooth high-throughput consumption — records are batched into the message view about ten times a second, and the desktop main process pauses fetching while more than 500 decoded records are waiting to be delivered (resuming at 250), so a fast topic can’t swamp the UI or memory. Your own pause always wins: a drained backlog never resumes a consumer you paused.
-
Message log — Freeze holds the view while records keep arriving (a counter shows how many are new). The log keeps the newest 1,000 records per connection; once older ones are dropped a notice shows how many. Scrolling up stops autoscroll and shows a Jump to latest pill.
-
Export the log — download the current filtered messages as JSON or NDJSON (one message per line).
-
Saved produces — the producer keeps named produce templates per connection (key, value, encodings, tombstone, headers, partition), like MQTT’s saved publishes. Save the composer, rename, reorder and delete them; click to load into the composer, or produce directly with the send button / Shift+Enter. They are stored unresolved (
{{variables}}resolve at send) and stay with the local connection — they aren’t part of a collection preset. -
Shortcuts and errors — Ctrl/Cmd+Enter in the value editor produces. A failed connect shows its error inline above the panel; it clears when you try again. Deleting a connection asks first, since it disconnects and removes its settings and message log.
-
Topic inspector — per-partition earliest/latest watermarks (and the approximate message count) plus the topic’s broker config.
-
Consumer-group inspector — members and state, committed offset per partition, and consumer lag (topic log-end minus the group’s committed offset) — computed on demand for any group, no active subscription needed.
-
Reset consumer-group offsets — for one committed topic or all committed topics of the group (committed topics that have since been deleted are skipped and listed), to:
- earliest / latest — the partition’s log start / end;
- specific — explicit per-partition offsets (single topic only);
- date/time — each partition’s first record at or after the chosen local time (Kafka
ListOffsetsby timestamp). A partition with no record at or after that time resets to latest; - shift by ±N — move each committed offset forward or back, clamped to the partition’s earliest/latest range. Partitions without a committed offset are skipped and listed (they keep following the consumer’s
auto.offset.reset).
Preview runs a dry run and shows current → new offsets per partition (with clamp/fallback notes) without committing anything; Reset asks for confirmation and then shows the applied offsets.
-
Delete committed offsets — remove the group’s committed offsets for one topic or all committed topics (KIP-496
OffsetDelete), confirm-gated. Consumers then fall back toauto.offset.reset.Resetting and deleting offsets both require an inactive group (no members, not rebalancing): Restura checks this before writing and the broker enforces it too.
-
Delete consumer group — for empty/inactive groups.
-
Multiple brokers — bootstrap list, broker discovery happens through the cluster.
-
SASL over
SASL_PLAINTEXTorSASL_SSL:PLAIN,SCRAM-SHA-256,SCRAM-SHA-512— username and password.OAUTHBEARERwith a static token (no automatic refresh), or with the OAuth 2.0 client-credentials flow — token endpoint, client ID, client secret, optional scope. The desktop app fetches the token and refreshes it once less than a quarter of its lifetime (at least 30 seconds) remains, so broker re-authentication always gets a fresh token. The token endpoint must usehttps(plainhttponly forlocalhostor a private IP) and follows the same SSRF policy as HTTP requests.AWS_MSK_IAM— access keys (optional session token) or a named AWS profile; signed with SigV4 and sent as MSK’s OAUTHBEARER token. RequiresSASL_SSL, and the bootstrap brokers must be MSK endpoints in the chosen region. For PrivateLink or custom DNS, turn on Allow non-AWS broker hostnames — only for brokers you control, because the signed token is sent to every broker in the list.GSSAPI(Kerberos) — listed but not available in this build: it needs the nativekerberosmodule, which the desktop app doesn’t bundle yet.
The OAuth client secret and AWS keys are stored as OS-keychain handles scoped to Kafka; the UI never reads them back. SASL passwords, static OAuth tokens, TLS key passphrases and Schema Registry passwords / tokens are stored the same way — as main-process secret handles bound to the connection — and the connect request carries only the handle; the desktop main process resolves it just before dialling. If a stored credential is gone, connect fails with Stored Kafka credential is missing — re-enter it rather than sending an empty value. Values saved by older versions move into a handle the first time the connection is used.
-
TLS — pick
SSLfor TLS without SASL (encryption plus optional client-certificate mTLS), or layer it underSASL_SSL; server certificate validation either way. The CA, certificate and key files you pick are read by the desktop main process at connect (only from your home, documents or app-data folders, and never from credential directories such as~/.ssh), so key material never passes through the UI. -
Compression — choose
none, gzip, snappy, lz4 or zstd in the producer; it isn’t negotiated automatically. -
Save to a collection — Save to Collection on a Kafka tab stores a shareable connection preset (brokers, auth mode, TLS paths, client id, produce topic / key / payload template / headers, consumer group and topics). Opening it from the sidebar opens this panel prefilled. Passwords, OAuth tokens and key passphrases stay in the encrypted secret-handle store — copied between the connection and the preset inside the main process — and are never exported. See Kafka and MQTT connection presets.
-
{{variables}}— bootstrap brokers (at connect), consumer group and topics (at subscribe), and the produced topic, key, value and header names/values (at send) are resolved with the same scopes as HTTP. In JSON mode a value substituted inside a string literal is JSON-escaped. Unresolved names are sent literally and flagged in the message log; secret values are masked in the logged copy. -
Confluent Schema Registry — point a connection at a registry URL (HTTP Basic or Bearer auth, HTTPS with your custom CA / mTLS); produce schema-encoded keys and values by subject + version (or
latest) or by raw schema ID, browse subjects and view schemas, and consume with automatic Avro / Protobuf / JSON decode of both. Follows the Confluent wire format (verified against the official Confluent serializers). See Schema Registry.
Producing, consuming and administering
Section titled “Producing, consuming and administering”Producer (produce tab)
- Modes — Single record, Batch (a JSON array of typed records sent together), Stream (open a stream, write batches to it, close it) and Transaction.
- Transactions — set a Transactional ID (this also turns on the idempotent producer; reconnect to apply), then Begin transaction, Send batch in transaction as often as you like, and Commit or Abort.
- Acks — pick
0,1or-1(all in-sync replicas). Idempotent producer turns on exactly-once-per-partition de-duplication and locks acks to-1; reconnect to apply it. - Compression — per produce, as above.
- Tombstone — in Single mode, send the record with a Kafka
nullvalue (compacted topics treat it as a delete marker).
Consumer (consume tab)
- Commit — automatic, or Manual per message (a commit button on each delivered record).
- Isolation — read uncommitted or Read committed.
- Group protocol — Classic, or Consumer (the KIP-848 protocol; needs a broker that supports it, and adds an optional Remote assignor).
- Missing commit fallback — when starting from the committed offset: Latest, Earliest or Fail.
- Performance tuning (collapsed) — static member ID, session / rebalance / heartbeat timeouts, auto-commit interval, fetch min / max bytes (total and per partition), max wait and stream high-water mark. Blank fields use the client library’s defaults.
- Pause / resume and seek — see What works. Lag and rebalance events are shown for consumer-group mode.
Admin (desktop only, per connection)
- Topics — create (name, partitions, replication factor) and delete topics (confirm-gated), plus the topic inspector’s partition watermarks and broker config.
- Advanced admin (collapsed sections under the admin panel):
- Cluster metadata — describe the cluster (brokers, controller, ID).
- Advanced topic operations — increase a topic’s partition count and alter a topic config (both with a Validate dry run before applying), and delete records before an offset on one partition.
- ACLs — describe, create and delete ACLs from typed ACL/filter JSON.
- Client quotas — describe, validate and apply the producer byte-rate quota (
producer_byte_rate) for a user entity.
Writes in the advanced sections need a typed confirmation phrase — ALTER <topic>, DELETE RECORDS <topic>, CREATE ACL, DELETE ACLS or ALTER QUOTAS — which the desktop main process checks again before calling the broker. Admin actions run with the connection’s own credentials, so the broker’s ACLs decide what succeeds.
Schema Registry
Section titled “Schema Registry”Enable Schema Registry on the connection form and enter the registry URL.
- Auth — choose HTTP Basic (username + password, stored encrypted) or Bearer token (for OAuth-fronted registries such as Confluent Cloud with OAuth, Apicurio or Karapace behind a gateway). The bearer token field supports inline entry, a keychain-stored secret handle, or an external secret reference; handles and external references are resolved only in the desktop main process and never reach the renderer. Stored handles must be created from the Kafka token field (they are scoped to Kafka); a handle stored for another feature is refused. Credentials are resolved when you connect, so reconnect after rotating a stored token. Credentials are only sent over plain
http://to a loopback or private-network registry; public registries needhttps://. - TLS — HTTPS registries use the same TLS selection as the brokers’ policy: the host-matched (or global) custom CA and client certificate (mTLS) from Settings → Certificates, plus the Verify SSL certificates toggle and TLS version / cipher settings. An active global proxy is not supported for the registry connection and fails the connect rather than bypassing the proxy.
- Network policy — the same as the brokers: the URL as written is checked against the Settings → Security localhost / private-network toggles (so a literal private IP needs Allow private / internal IPs), and cloud-metadata endpoints are always refused. A registry hostname on a Docker or VPC network that resolves to a private address works with the default settings, just like a broker hostname. Every new registry connection re-checks the DNS answers, refusing cloud-metadata addresses always and loopback addresses when Allow localhost is off.
Producing with a subject. In the producer, set the key or value schema to Registry subject. The subject list is filtered to <topic>-value / <topic>-key by default (and pre-selected when it exists); pick a version or latest. The desktop app resolves the subject and version to a schema ID when you publish, so latest always follows the newest registered version. The selected schema is shown read-only next to the editor, and Generate example fills the payload with a JSON skeleton for Avro and JSON Schema subjects (Protobuf schemas are viewable but not yet generated). Avro bytes, fixed and decimal values are written — and shown when consumed — as strings, one character (code point 0-255) per byte, following the Avro JSON convention. The numeric Schema ID mode is still available.
Wire compatibility. Records use the Confluent wire format, so they interoperate with the official Confluent serializers in both directions (verified byte-for-byte against the official Python serializers). Integers beyond JavaScript’s safe range can be typed as plain numbers: 64-bit Avro long and Protobuf int64/uint64 keep every digit, and are shown as strings when consumed so nothing is rounded. JSON Schema records keep every digit when produced, inspected and exported. Protobuf payloads use the standard proto3 JSON mapping (the same as the gRPC client): field names from the .proto file (camelCase names are also accepted), enum names or numbers, Base64 for bytes, RFC 3339 strings for google.protobuf.Timestamp, and google.protobuf.Any with @type; the file’s first message type is produced. Protobuf message types are located using the descriptor the registry itself serves (Confluent Schema Registry and Redpanda), so Restura agrees with the registry’s own deserializers. Two edge cases remain for a .proto that declares a map field before a nested message: registries that cannot serve descriptors, and Redpanda, which reorders stored schemas — a producer built from the original .proto may then disagree about which nested message a record holds. Avro unions can be written either unwrapped or in the Avro JSON wrapped form ({"string": "x"}); unions that mix long with another number or string type are shown wrapped ({"long": 5}), because a bare JSON value would be ambiguous. JSON Schema drafts 04, 06, 07, 2019-09 and 2020-12 are supported, and format keywords are informational and never block a publish.
Consuming. Registry-decoded keys and values show which schema decoded them — Schema 12 (orders-value v3) — when the registry can map the ID back to a subject and version (looked up once per ID and cached).
Consume modes
Section titled “Consume modes”The Consume tab has a mode toggle. New Kafka tabs start in Browse.
| Browse (read-only) | Consumer group | |
|---|---|---|
| Joins or creates a consumer group | No | Yes, using the Consumer group ID |
| Commits offsets | Never | Automatically, or per message with Manual per message |
Needs GROUP ACLs | No, only READ on the topic | Yes |
| Start positions | Latest, earliest, last N per partition, specific offset, timestamp | Committed offset (with a fallback), latest, earliest, specific offset, timestamp |
| Stop condition | Optional Stop after N records | Runs until you unsubscribe |
| Lag, rebalance and member events | Not available because there’s no group | Shown |
Browse is for looking at data. Restura assigns the partitions directly and reads them with the Kafka ListOffsets and Fetch APIs. It never sends FindCoordinator, JoinGroup or OffsetCommit. Peeking at a shared or production cluster leaves nothing behind: no restura-… group appears in kafka-consumer-groups --list, and nothing is committed. The panel shows a Read-only — no group badge, and the group, commit and rebalance settings are hidden.
- Last N per partition starts each partition at
max(earliest, latest − N)and then keeps streaming new records. An empty partition starts at its end. - Timestamp starts each partition at its first record at or after that time. A partition with nothing that new starts at its end and waits for new records.
- Specific offset reads only the partitions you name. Offsets outside the retained range are clamped to the earliest or latest offset.
- Read committed isolation is honoured, so
latestis the last stable offset. - The set of partitions is fixed when you subscribe. Partitions added to the topic afterwards aren’t picked up until you subscribe again.
- If retention deletes records you haven’t read yet, Browse moves that partition forward to its new earliest offset and carries on.
- Browse connects to whichever partition leaders the cluster advertises, the same as Consumer group mode. The broker guard checks the bootstrap addresses, not the advertised leaders, so the residual risk in ADR 0006 applies to both modes.
Consumer group joins the group named in Consumer group ID and behaves like any other member. Use it when you want to resume from committed offsets, test a group’s behaviour, or commit offsets on purpose. The consumer-group inspector and the reset and delete actions work on any group in either mode.
Saved connections from before Browse mode existed keep Consumer group mode if you changed the group ID from the generated restura-… default. Connections still using the default group ID switch to Browse. If they started from the committed offset, they now start from the last 50 records per partition.
Broker guard
Section titled “Broker guard”Every broker address and Schema Registry URL runs through a pre-flight SSRF check (electron/main/security/kafka-broker-guard.ts) before the connection opens, using the Settings → Security localhost / private-network policy. Cloud-metadata endpoints (169.254.169.254 and friends, metadata.google.internal, …) and malformed addresses are always rejected, so a malicious or misconfigured bootstrap.servers can’t aim traffic at your machine’s metadata endpoint.
With the default settings (Allow localhost on, Allow private / internal IPs off), a broker on localhost / 127.0.0.1 works, but a LAN, VPC or Docker-network broker — a private IP literal, or a hostname that resolves to one — is refused until you turn on Allow private / internal IPs in Settings → Security.
Broker hostnames are then resolved and every A/AAAA answer of every bootstrap broker is checked against the same policy (electron/main/security/broker-dns-guard.ts): an answer in loopback space needs Allow localhost, an answer in private / link-local space (RFC 1918, CGNAT, …) needs Allow private / internal IPs, and cloud-metadata answers are always refused. A broker name that only resolves on a Docker or VPC network (for example kafka:9092 in Docker Compose) therefore needs Allow private / internal IPs turned on. The Kafka client resolves the name again when it dials, so this narrows but does not close a DNS-rebind window. Advertised brokers that the cluster returns in its metadata (the addresses the client actually talks to after bootstrap) are not re-checked — the residual gap in ADR 0006. Schema Registry connections additionally re-check every DNS answer per connection for cloud-metadata and (when localhost is disabled) loopback addresses (see above).
The global connection policy is applied to every bootstrap broker, not just the first: if the configured proxy applies to any broker the connect fails (the Kafka client can’t tunnel through a proxy), and because the client uses one TLS context for all brokers, brokers whose host-matched Settings → Certificates material differs (for example a per-host client certificate that matches only one of them) are rejected — split them into separate connections.
- Set the client ID (defaults to
restura-<first 8 characters of the connection id>) to make your activity easy to spot in broker logs. - For schema-encoded payloads, set the connection’s Schema Registry URL, then pick a value and/or key subject (or schema ID) when producing — the field is parsed as JSON, Confluent-encoded, and decoded again automatically on consume. Schema encoding and raw Base64 are intentionally mutually exclusive for the same field.
- Partition pause and live seek are emulated on top of the client library, which has no native per-partition pause or mid-stream seek: a paused partition is still fetched, but its records are discarded before decoding, and a seek or partition resume briefly swaps the consumer’s fetch stream (it stays in its group — no rebalance). After a seek, partitions newly assigned by a later rebalance start from the group’s committed offset (or the missing-commit fallback).
- Automatic commit commits only records that were actually delivered to the message view — every second by default, or at the configured auto-commit interval, and on unsubscribe/disconnect. Records withheld from a paused partition, or still waiting to be decoded, are never committed.
- Commits are last-write-wins at the broker: after seeking a partition backwards, the next automatic commit moves the group’s committed offset back too, so other group members (and a later resubscribe) re-read from there.
- Use JSON mode to validate a JSON document before publishing its UTF-8 text. Use Base64 bytes for arbitrary binary payloads; Restura accepts canonical Base64 only and displays consumed non-UTF-8 fields as Base64 so they can be copied without byte loss.
Related
Section titled “Related”- Self-hosting — Kafka stays desktop-only even when self-hosting the SPA; the consume/produce path needs Node.
- Capability matrix
- Local test stack — run a real Kafka broker (Redpanda) locally to test against instead of a mock.