New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
Unlimited qps is not going to be cheap.
Honestly i think you have some architecture issues that you need to evaluate.
Are you queuing? Are you caching? Is your code isolating read queries and sending them to a secondary?
Traditional relational databases just arent designed to scale quickly or in a dynamic fashion.
Aurora isn't cheap because its still a bandaid (albeit a very good one) to a model where performance for writes is mosrly constrained to vertical scaling?
Can't you just throw RAM at the db server and solve like 99% of all scaling problems?
Pretty much.
It scales like this:
AWS is using PC4-19200 (DDR4-2400Mhz) mostly for Aurora (based on my read speed tests i did at Grindr some year ago), capping at ~19GB/s per channel, also limited by the Infiniband interconnect.
We tested also PC4-24000 (DDR4-3000), with around 23GB/s. That was the highest commercial viable i have seen, with an EPYC 7702 in 8 channel config we managed to get reads of ~80GB/s (of theoretical max. 160GB/s before overhead). A copy in RAM could give you Nx100GBs up to (DB size/[Channels*Speed]) but this is only of use for local operations. In reality the CPU capped out by it's own core clock and the InfinityFabric at this point (as it takes like 16 cycles to read from 4 memory locations and you need to factor in how many CCX the chip has, plus the InfinityFabric layout, Intel is WAY out of league here with a maximum of 4 QPI links at 120GB/s total - InfinityFabric is around 4Tbit/500GB/s in an 8 CCX config).
As even Infiniband x4 caps out at 15GB/s post-overhead, due to the PCIe 3.0 x16 link speed of 16GB/s before overhead we've assumed that as the absolute maximum possible (PCIe 4.0 was not a thing commercially yet).
We invested a good amount of time and money into this experiments.
While I agree with @William I'm almost certain that @TerensM 's problems are in a lower range, but due to lack of information provided I can't offer much more than pointing at @lanefu.
Absolutely sure they are. I just wanted to provide an insight what happens at the very high end and that there is no real solution for that either.
More read replicas will solve every issue anyway, this is what we went with also then. If you need more write capacity than that your code/design is flawed and needs redesign (like a queue system, the enduser in his app does not care or notice a 2 second delay for his action anyway).