Why then not just use ZFS or BTRFS? Way less overhead.
Ceph's main advantage is the distribution of storage over multiple nodes, which you're not planning on doing?
this post was submitted on 07 Mar 2024
17 points (100.0% liked)
Selfhosted
45508 readers
603 users here now
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
founded 2 years ago
MODERATORS
I mean, yeah, I'd prefer ZFS but, unless I am missing something, it is a massive pain to add disks to an existing pool. You have to buy a new set of disks and create a new pool to transition from RAID z1 to z2. That's basically the only reason it fails the criteria I have. I think I'd also prefer erasure encoding instead of z2, but it seems like regular scrub operations could keep it reliable.
BTRFS sounds like it has too many footguns for me, and its raid5/6 equivalents are "not for production at this time."
LVM, mdraid, dm-crypt? LVM will let you make volumes and pools of basically any shape or size.
Found an interesting read regarding the matter here:
https://old.reddit.com/r/ceph/comments/mppwas/single_node_ceph_vs_zfsbtrfs/
Most seem to recommend going for ZFS instead if using a single machine but there is a person discussing his first hand experience with single node Ceph.
This was really neat, kinda boils down to "you don't want to deal with the complexity and it's horrifically slow."
Neat! Thank you
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:
| Fewer Letters | More Letters |
|---|---|
| LVM | (Linux) Logical Volume Manager for filesystem mapping |
| RAID | Redundant Array of Independent Disks for mass storage |
| ZFS | Solaris/Linux filesystem focusing on data integrity |
3 acronyms in this thread; the most compressed thread commented on today has 9 acronyms.
[Thread #581 for this sub, first seen 7th Mar 2024, 17:15] [FAQ] [Full list] [Contact] [Source code]
Create 3 VM's and pass-through disks to each VM. Boom ceph cluster on a single computer.
ZFS/BRTFS might still be better, but if you really want Ceph this should work and provide expansion and redundancy at a block device level, though you wont have any hardware redundancy regarding power/nodes.
Since you are talking mismatched disks, I have gone to unraid after running a ceph cluster. I found it easy to keep adding and upgrading disks in unraid where it made more sense than maintaining or adding nodes. While I like the concept of being able to add nodes for very large storage arrays. My current unraid server is 180tb.
It is super simple to add/upgrade the storage one disk at a time.
Oh, neat, I'll have to look into that more. It's able to have some redundancy and does some sort of rebalancing on disk failures?
It has parity disks, which always need to be the largest disks in your array. You can run with either a single one double parity disk.
It seems to work well, as that's how I've had to replace a dozen disks in the last year upgrading from 8tb disks to 18 or 22tb disks.
Ceph is a huge amount of overhead, both engineering and compute resources, for this usecase.
I did it for a while and last everything. Go for it if your have adequate backups.