New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
With a professional service, a good schema, and a much better DB than MySQL.
What @WSS said...and hadoop and MariaDB are my preferences at this time.
A dedicated server with lots of bandwidth should do the job....whatever math problem that is
You need good servers, network, and your professional team ))
We can offer you servers and network.
Thanks but i don't need servers or network to do this
@tarek4web I would recommend MariaDb in version 10.2, regarding the size of the DB, since it is abstract to the user, besides not consulting all the data at the same time, as comments @wss only needs a good database schema, there are many programs modeling and analysis of the use of memory and the speed of BD in the network, basically is what you need to know, another good recommendation is the language to use, since depending on it, many have tools that facilitate their programming and access, I would recommend if you do web MVC, many frameworks like lavavel and yii2.
If you like learning and want a higher quality job that will require more work, I think postgresql will be another good option better than MariaDB.
Another thing as I calculate the size of the BD, unless you save images, 2TB sounds like something exaggerated, if you use images there are many ways to save them in the BD, without saving the image that takes up so much space.
For a reasonable answer you will have to tell us much more about the database and your needs, e.g. kind of records, about what you do and how you want to access those records, and so on.
Right on. What is the actual data being stored?
If this is data generated by a program, follow best practices for that program. It's hard to say how to store 2TB of data: no-sql,sql, lucene index. What's the purpose of the data?
Okay, what you call huge specify in size please?
Not to beat a dead horse, but "it really depends" is the answer here. RDBMS are strong at structured, normalized, relational data. If you are looking to store opaque blobs of data, consider not even using a database.
Tell us more about your data storage and access patterns. What is the shape of the data you are storing? How will you search for data and retrieve the data?
Can you split out 95% of the content into blobs on disk addressed by key, and index 5% of the data in a data structure that allows access to the information you want?
2TB isn't that much data, but it does require some pre-planning to maintain it long term and allow it to continue to grow.
Please, don't forget that your schema needs a well planned backup schedule.
We have several databases 50-100+TB on MS SQL Server. And 2TB is not huge.
I just wonder is it a live OLTP DB with 100+ TB of hot data or just a sum of weekly backups? Could you share your daily average and peak logical reads/sec on that monster?