Sysadmin

10254 readers

2 users here now

A community dedicated to the profession of IT Systems Administration

No generic Lemmy issue posts please! Posts about Lemmy belong in one of these communities:
[email protected]
[email protected]
[email protected]
[email protected]

founded 2 years ago

MODERATORS

[email protected]

What to do when a MySQL/MariaDB database gets too large for a single host? (programming.dev)

submitted 2 months ago by [email protected] to c/[email protected]

7 comments fedilink hide all child comments

What are your strategies when a MySQL/MariaDB database server grows to have too much traffic for a single host to handle, i.e. scaling CPU/RAM is not an option anymore? Do you deploy ProxySQL to start splitting the traffic according to some rule to two different hosts? What would the rule be, and how would you split the data? Has anyone migrated to TiDB? In that case, what was the strategy to detect if the SQL your app uses is fully compatible with TiDB?

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 7 points 2 months ago (1 children)

I work for a company that handles this In a few ways. We set up read replicas to handle large read queries. To offload the reads from the primary server. Data is replicated to the read replicas so reporting can be run from that server. And not add load to the primary server.

The second approach is sharding. Sharding breaks a large table into smaller, more manageable chunks, distributing them across systems. This reduces the burden on any one server, improves performance, and enables scaling out as data or traffic increases.

[–] [email protected] 1 points 2 months ago

These are the most common approaches I'm aware of.

The only other real alternative, without changing the underlying DB or architecture, is you could break up the database or archive data.

Many times you don't need old data and can ship it off to S3 or something similar. Additionally, you may have tables/data that are disconnected and could be broken off into two databases. Both of these tend to just push off the inevitable, but could buy time for a larger architectural change.