Conversation
✅ Deploy Preview for suse-obs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
l4zy0n3
force-pushed
the
longhorn-ha-storage-docs
branch
from
August 20, 2026 10:41
c6823e8 to
b007ebf
Compare
l4zy0n3
changed the title
l4zy0n3
force-pushed
the
longhorn-ha-storage-docs
branch
from
August 20, 2026 10:46
b007ebf to
d9eeb3a
Compare
l4zy0n3
changed the title
l4zy0n3
force-pushed
the
longhorn-ha-storage-docs
branch
2 times, most recently
from
August 20, 2026 10:53
cfbcf45 to
1912e1e
Compare
akashraj4261
approved these changes
Aug 20, 2026
rb3ckers
requested changes
Aug 21, 2026
|
|
||
| == Using Longhorn with self-replicating components | ||
|
|
||
| Kafka, ZooKeeper, ClickHouse, and HDFS DataNodes already replicate their data across pods/nodes at the application layer. If you run https://longhorn.io/[Longhorn] as your storage backend, its default StorageClass also replicates each volume (3 copies by default), multiplying storage usage on top of the application's own copies. |
Contributor
There was a problem hiding this comment.
For elasticsearch we need a helm chart update I believe, right?
Contributor
There was a problem hiding this comment.
We should also include that this applies to an HA deployment. For non-ha deployments we do not recommend this strict-local setup but instead we might want to recommend a storage class with only the default replica count set to 2 instead of 3.
We could actually suggest that also for all other PVCs in HA: use another custom storage class with replicaCount: "2" (instead of 3) by setting it as the global.storageClass. WDTY?
Contributor
There was a problem hiding this comment.
And we can include a link to Longhorn best practices:
See also Longhorn [IO Performance best practices](https://longhorn.io/docs/1.12.1/best-practices/#io-performance)
|
|
||
| [NOTE] | ||
| ==== | ||
| `dataLocality: strict-local` is incompatible with `ReadWriteMany` volumes. Also, on Longhorn with `numberOfReplicas: 1`, draining that node hangs indefinitely by default -- set the https://longhorn.io/docs/latest/references/settings/#node-drain-policy[Node Drain Policy] to `always-allow` beforehand. |
Contributor
There was a problem hiding this comment.
We should include a little bit more I think:
Suggested change
| `dataLocality: strict-local` is incompatible with `ReadWriteMany` volumes. Also, on Longhorn with `numberOfReplicas: 1`, draining that node hangs indefinitely by default -- set the https://longhorn.io/docs/latest/references/settings/#node-drain-policy[Node Drain Policy] to `always-allow` beforehand. | |
| `dataLocality: strict-local` is incompatible with `ReadWriteMany` volumes. Also, on Longhorn with `numberOfReplicas: 1`, draining that node hangs indefinitely by default -- set the https://longhorn.io/docs/latest/references/settings/#node-drain-policy[Node Drain Policy] to `always-allow` beforehand. | |
| When removing or replacing a node follow the [Longhorn documentation](https://longhorn.io/docs/1.12.1/nodes-and-volumes/nodes/graceful-node-removal/) to do this 1 node at a time only, and wait for the data replication on the database level to have completed. Without waiting for replication to finish data is likely to be lost. |
Contributor
There was a problem hiding this comment.
They should also make sure to know what to do when a node fails (and the disk is lost). Also documented by Longhorn: https://longhorn.io/docs/1.12.1/high-availability/node-failure/
Document how HA components with application-level replication can avoid storage-level double replication. Include the Longhorn strict-local StorageClass, global routing with HDFS metadata-node overrides, the ClickHouse profile restriction, PVC immutability during upgrades, and one-node-at-a-time maintenance guidance.
l4zy0n3
force-pushed
the
longhorn-ha-storage-docs
branch
from
August 26, 2026 10:05
1912e1e to
90f7ad3
Compare
l4zy0n3
changed the title
|
|
||
| Some `HA` sizing-profile components run with both application-level replication (2-3 copies of their own data across pods) and storage-level replication (your StorageClass keeping another 2-3 copies of each volume). Combined, this multiplies storage usage far beyond what's needed for redundancy -- for example, 3 application copies on a StorageClass that also keeps 3 copies results in 9 copies of the same data. | ||
|
|
||
| For components where the application already replicates, you can use a StorageClass that keeps only one copy per volume, relying entirely on the application's own replication for redundancy instead. This applies to any storage backend that replicates by default -- consult your backend's own documentation for the equivalent setting. https://longhorn.io/[Longhorn] is documented in detail below since it's the backend we've tested this against. |
Contributor
There was a problem hiding this comment.
Suggested change
| For components where the application already replicates, you can use a StorageClass that keeps only one copy per volume, relying entirely on the application's own replication for redundancy instead. This applies to any storage backend that replicates by default -- consult your backend's own documentation for the equivalent setting. https://longhorn.io/[Longhorn] is documented in detail below since it's the backend we've tested this against. | |
| For components where the application already replicates, you can use a StorageClass that keeps only one copy per volume, relying entirely on the application's own replication for redundancy instead. This applies to any storage backend that replicates by default -- consult your backend's own documentation for the equivalent setting. link:https://longhorn.io/[Longhorn] is documented in detail below since it's the backend we've tested this against. |
|
|
||
| === Longhorn | ||
|
|
||
| Longhorn's own https://longhorn.io/docs/latest/best-practices/#io-performance[best practices] recommend `strict-local` data locality with a single replica for exactly this case ("applications that support data replication, e.g. a distributed database"), or reducing `numberOfReplicas` to `2` as a smaller middle-ground step. |
Contributor
There was a problem hiding this comment.
Suggested change
| Longhorn's own https://longhorn.io/docs/latest/best-practices/#io-performance[best practices] recommend `strict-local` data locality with a single replica for exactly this case ("applications that support data replication, e.g. a distributed database"), or reducing `numberOfReplicas` to `2` as a smaller middle-ground step. | |
| Longhorn's own link:https://longhorn.io/docs/latest/best-practices/#io-performance[best practices] recommend `strict-local` data locality with a single replica for exactly this case ("applications that support data replication, e.g. a distributed database"), or reducing `numberOfReplicas` to `2` as a smaller middle-ground step. |
akashraj4261
approved these changes
Aug 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Document Longhorn single-replica, strict-local storage for SUSE Observability components that already replicate their data at the application layer. Also gives a backend-neutral recommendation to avoid redundant storage-level replication.
Verified behavior
Documentation scope