Skip to content

feat(connectors): add MQTT sink connector - #4584

Open
ygndotgg wants to merge 16 commits into
apache:masterfrom
ygndotgg:feat/connectors-mqtt-sinkk
Open

ygndotgg wants to merge 16 commits into
apache:masterfrom
ygndotgg:feat/connectors-mqtt-sinkk

Conversation

@ygndotgg

Copy link
Copy Markdown
Contributor

Which issue does this PR address?

Closes #3385

Rationale

Iggy lacked a native MQTT sink connector. Users could not publish Iggy stream data to MQTT brokers such as EMQX without a separate adapter service.

What changed?

This PR adds the iggy_connector_mqtt_sink plugin. It reads Iggy messages and publishes them to MQTT brokers through rumqttc and the iggy_connector_sdk C-FFI boundary.

The connector supports MQTT 3.1.1 and MQTT 5. It supports QoS 0, QoS 1, and QoS 2. It supports static, template, and dynamic MQTT topic routing.

The connector tracks MQTT acknowledgments by packet identifier. It bounds in-flight publications, reconnects after uncertain failures, and republishes only unconfirmed messages. It also supports broker authentication, TLS, mutual TLS, MQTT 5 properties, and controlled shutdown.

This PR adds sink integration tests with EMQX. The tests cover protocols, QoS levels, routing modes, MQTT 5 envelopes, broker rejection, pipelining, restart, and reconnect behavior.

Local Execution

Done

AI Usage

  1. Which tools? Codex
  2. Scope of usage? Generated and refined connector functions, test fixtures, integration tests, and documentation.
  3. How did you verify the generated code works correctly? I reviewed the MQTT sink behavior and test coverage end to end
  4. Can you explain every line of the code if asked? Yes

@github-actions

Copy link
Copy Markdown

Thanks for the PR. It is labeled S-waiting-on-review and queued for review.

Slash commands (own line, regular comment) move it around the queue:

  • /ready - back to S-waiting-on-review after addressing feedback
  • /author - flip to S-waiting-on-author while you finish changes
  • /request-review @user-or-team - request a reviewer
  • /pin - exempt the PR from the stale bot, /unpin to undo

See CONTRIBUTING.md for details.

@github-actions github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Oct 19, 2026
Comment thread core/connectors/sinks/mqtt_sink/src/lib.rs Outdated
Comment thread core/connectors/sinks/mqtt_sink/src/lib.rs
Comment thread core/connectors/sinks/mqtt_sink/src/lib.rs Outdated
Comment thread core/connectors/sinks/mqtt_sink/src/driver.rs
Comment thread core/connectors/sinks/mqtt_sink/src/driver.rs
Comment thread core/integration/tests/connectors/mqtt/mqtt_sink.rs Outdated
Comment thread core/connectors/sinks/mqtt_sink/src/lib.rs
Comment thread core/connectors/sinks/mqtt_sink/src/driver.rs Outdated
Comment thread core/integration/tests/connectors/fixtures/mqtt/container.rs Outdated
Comment thread core/connectors/sinks/mqtt_sink/Cargo.toml Outdated
@ygndotgg
ygndotgg force-pushed the feat/connectors-mqtt-sinkk branch from 747fdc1 to 1ea6e94 Compare October 23, 2026 14:02
@ygndotgg
ygndotgg requested a review from numinnex October 23, 2026 15:08
@codecov

codecov Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.83193% with 481 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.71%. Comparing base (6ce2f92) to head (3af7263).

Files with missing lines Patch % Lines
core/connectors/sinks/mqtt_sink/src/driver.rs 75.47% 257 Missing and 55 partials ⚠️
core/connectors/sinks/mqtt_sink/src/lib.rs 88.69% 98 Missing and 14 partials ⚠️
core/connectors/sinks/mqtt_sink/src/routing.rs 91.99% 40 Missing and 17 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #4584      +/-   ##
============================================
- Coverage     90.77%   90.71%   -0.06%     
  Complexity     2989     2989              
============================================
  Files          1459     1462       +3     
  Lines        348165   351137    +2972     
  Branches     292747   295721    +2974     
============================================
+ Hits         316042   318533    +2491     
- Misses        25485    25878     +393     
- Partials       6638     6726      +88     
Components Coverage Δ
Rust Core 92.00% <83.83%> (-0.09%) ⬇️
Java SDK 77.77% <ø> (-0.03%) ⬇️
C# SDK 83.56% <ø> (-0.19%) ⬇️
Python SDK 85.69% <ø> (ø)
PHP SDK 90.67% <ø> (ø)
Node SDK 96.44% <ø> (ø)
Go SDK 70.17% <ø> (+0.20%) ⬆️
C++ SDK 85.51% <ø> (ø)
Files with missing lines Coverage Δ
core/connectors/sinks/mqtt_sink/src/routing.rs 91.99% <91.99%> (ø)
core/connectors/sinks/mqtt_sink/src/lib.rs 88.69% <88.69%> (ø)
core/connectors/sinks/mqtt_sink/src/driver.rs 75.47% <75.47%> (ø)

... and 41 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ygndotgg
ygndotgg force-pushed the feat/connectors-mqtt-sinkk branch from 1ea6e94 to 3af7263 Compare October 26, 2026 15:38

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-review PR is waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(connectors): Implement MQTT source & sink

2 participants