New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
I don't need redis to hold the data forever. I think I just need to hold the data until they finished the test, which is usually 90 minutes. Also the answers is both stored on the local storage and the server. The only way the answers gone forever is when both the server the users device fucked up at the same time.
Fun Fact : I have a few occasions when my client reset the database without retrieveng the test result. I just tell them to ask their students to login and submit the answers (because the answers is still on the client device).
@yokowasis
let me know when you want the AX41 go, i will take it
By the time I outgrow AX41, I would probably don't remember having this conversation.
Why ? Let's assume the test consist of 40 questions. 1 student will at least send 40 (Write) Queries to the server. When there area 20k students login at the same time, it's safe to assume in 1 second there will be more than 1, even thousands of write query sent to the server in the first few minutes.
Yeah, you were right, I don't really know about it.
That's why I am asking, so I am not taking wrong turns.
They are not read / evaluated right away. They will be read when the students / the teacher, want to retrieve the score. Usually after the students finished their test, they want to check their scores.
The relevant factor (besides number of students) is the minimum and average time students need to answer a question. Sure, there probably are simple questions, which are answered on average within 2 - 3 seconds but there highly likely are more complicated questions too which require more time.
So, your solution must be able to handle a peak rate but highly likely the average rate is very considerably lower.
Another potentially very important factor is the kind of answers. Whether the answers are multiple choice out of say max 8 options -or- whether the answers are numbers (e.g. math tests) or even free text makes a very major difference. But you can smartly use that because multiple choice answers will come in quicker (higher rate) but also can be handled more efficiently, while textual answers are much larger but also come in more slowly (lower rate).
Every halfway decent DB already does caching anyway, so be sure to understand your application (wrt the DB view) and the DB (how to config caching).
But there are other tricks too. Example: Indices make searches much faster but they also slow down insertions considerably, so when a use case is like yours that is, many insertions and next to no searches during the test, it can be helpful to simply dump the data into the DB (no indexing) and to only later (after the test) creating indices.
See above.
TL;DR
Why not go with Galera multi-master MySQL cluster so you can split your reads and writes across multiple MySQL servers? It should be the easiest way for the implementation in your app too (no changes needed to the application code).
Thanks.
I write it using nodejs.
I prefer not increasing the hardware, and more to what can I do to reduce the load.
Where should I start?
I don't come cheap if you want to hire me
@letrocks for clarity- did you write the system or work for the person who did the architecture?
Before jumping in to offer help I just want to clarify a thing or two.
It seems like you are dead set on using the existing hardware and redis (or another cache) and any suggestions or guidance to the contrary will be ignored or refuted with incorrect information.
Are you willing to accept that your idea is not the right one for the situation and that others advice from experience is the solution? Or is this thread going to be pointless until someone tells you that redis is a panacea and helps you destroy your application?
Edit: Sorry, just saw the other thread on slow logging. This is going to be a pointless thread.
It was an existing startup product where choices for the components already made.
I joined the team and worked on the system for almost past 4 years and improved the performance significantly. Including the database. I was hired to solve the data scale issue in the first place as I have prior experience in that area.
Well, I got the idea of using redis from other member on LET. And upon further search even to the point people call it swiss army knives. So naturally, I goes for redis solution.
Also if mysql really cache query effectively, what's even the point of having cache plugin, nginx cache, cdn, redis , or other cache solution. Wherever I go, it's just using a cache is recommended, this is the only time people telling me to ditch cache solution and using the built in mysql cache.
Yes, I am a deadset of using existing hardware. Because it's LET, you do more with less. I alrady have AX41 Standing By. But I want to find out what can I do with this limited resources. After all, you can only scale the hardware so much until it's a dead end, alrady did this route with Cloudjiffy and Failed. Improving the logic of the apps whenever possible is always a better choice.
Pointless ? No. If it's pointless I already create a queue system to write into the database. People telling me it's a bad idea is what stopping from doing it.
The other thread have nothing to do this. I just want to know why query that looks normal, is getting logged as slow query. It's not even slow, it has got index, and yet mysql / mariadb is complaining that it's a slow query. It's more like finding the logic behind it.