Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Subscribe to our newsletter

Advertise on LowEndTalk.com

Latest LowEndBox Offers

    Relational database - no longer a thingy?
    New on LowEndTalk? Please read our 'Community Rules' by clicking on it in the right menu!

    Relational database - no longer a thingy?

    spammyspammy Member

    Was chatting with some friends who are semi-close to the high tech circle today and I was told that all the big tech companies like Amazon and Google, even some fintech companies, are no longer using relational database to store information. Things like MS-SQL or MySQL or Oracle are somewhat outdated and replaced by some new technology...

    Is that true or my friend was being misinformed? I am not too familiar with that side of the world so I am semi-suspicious. If he is right, what is the popular types of DB those companies use these days?

    Comments

    • ITLabsITLabs Member
      edited November 7

      "A one size fits all database doesn't fit anyone", from Werner Vogels (CTO Amazon) may answer your question.

      TL;DR; use the DB that will best serve your customer.

      Ref: https://www.allthingsdistributed.com/2018/06/purpose-built-databases-in-aws.html

      Edit: reference updated, sorry for the wrong link :-(

    • servarica_haniservarica_hani Member, Provider
      edited November 7

      Actually I would say that is not true
      relational DBs are still used and they will remain in use for a long time

      the no-sql DBs are still hot in the industry but the problem people starting using them for everything even when the use case does not benefit from no-sql db

      I have witness the power and the issue of nosql dbs myself

      for me when i need no schema restrictions I will go with mongoDB but for any large project with many developers who tend to do mistakes i will stick with relational DB and having transactions is good plus .

      as anyone who is working in big project based on mongoDB or any nosql DB and they will tell you how hard it is to maintain the data in the DB after a while

      search google on many projects that switched back to relational DB after a while on nosql

      VPS and Cloud hosting at
      http://servarica.com

    • nemnem Member

      Some things never die...

      Strike an optimal balance between your use case and abilities. Sometimes a nail is just as good as a screw and you can get the work done in a third of the time with equal results. Choosing a technology that you have little familiarity with that serves as a linchpin to your organization... that'll have disastrous consequences once shit hits the fan.

      It's all worth learning. At the very least it can help you can make a more informed decision.

      Thanked by 1Trav
    • jsgjsg Member

      I think that's BS from the silicon valley hype mill.

      Funnily that same group of "semi-close to the high tech circle" people also preach that one should use the right tool for the job.

      The truth (well, my version of it) is largely not seen at all: The best DB is the one that really is tailored to the job - which also means that SQL DBs (which most mean when they say "relational DB") are indeed often not the best choice - but for an entirely different reason: The versatility offered by SQL is simply often not needed, so why pay the price for it. Often a relational (but not SQL) DB is the best choice.

    • Hani said: the no-sql DBs

      Just did some Googling and wow this is completely new to me! Closest I got with them is probably using Redis (which I have no idea how it works other than knowing it is a PHP accelerator of some sort).

      Pardon me for the ignorance, so noSQL conceptually means you store things in files with no schemas? so if using school class as an example, basically I store the information about each student as a file (JSON/CSV/something else) and then the name of the student as the file name and the DBMS is smart enough to, say, go to file for Joe and search for the key-value pair age when I do "select age from students where name = Joe".

      Is that understanding correct? Obviously I was simplifying things I lot.

    • jsg said: Often a relational (but not SQL) DB is the best choice.

      Relational but not using SQL? Do you mind give an example of such? You mean like JSON file?

    • FAT32FAT32 Member, Moderator, Deal Compiler Extraordinaire

      Just to drop this here

      Perhaps, some dreams can only be dreams.

    • servarica_haniservarica_hani Member, Provider

      @spammy said:

      Hani said: the no-sql DBs

      Just did some Googling and wow this is completely new to me! Closest I got with them is probably using Redis (which I have no idea how it works other than knowing it is a PHP accelerator of some sort).

      Pardon me for the ignorance, so noSQL conceptually means you store things in files with no schemas? so if using school class as an example, basically I store the information about each student as a file (JSON/CSV/something else) and then the name of the student as the file name and the DBMS is smart enough to, say, go to file for Joe and search for the key-value pair age when I do "select age from students where name = Joe".

      Is that understanding correct? Obviously I was simplifying things I lot.

      it is not a file based DB

      they call sometimes document based DB (document here means json usually) .

      you still have schema if you want so the school class will have lets say location , grade etc

      but the schema is not enforced so you could have first 10 records in the table strictly following your schema and then you decided I forgot to add number of students
      So in next save you simply add the extra json field and save it and it will go just fine !!

      the weird part will happen when you will need to read from it , yo will have to be prepare that the documents (jsons orr records) are not all following the same schema exactly

      this idea is really powerful when you need to save unstructured data that are related to entity , you can save them all with zero processing and while retrieving you can do some checks

      good example of this if you want to save logs to db
      all logs have common attributes like date time but every log format has different fields so you can save them all to same table then process them later when reading

      Thanked by 1spammy

      VPS and Cloud hosting at
      http://servarica.com

    • ksugksug Member

      Also, I have found that instead of relying on 1 single huge relational db, it's sometimes better to utilize multiple smaller relational db's (and non relational too). That way, outage in the db handling comments (not critical) does not affect the ability to process sales for example.

      Unlike say 15 years ago, spinning up multiple small db's is a no brainer and cheap. 3 x 2-4 GB mem per db is enough to kickstart most projects. And if you host at Hetzner, you pay less than an hour's wage each month for your db needs.

      The downside of this approach is that you can't JOIN some data together. So you have to do multiple round trips for complex queries. Most queries though, are not that complex.

      Thanked by 1Hetzner_OL
    • jsgjsg Member

      @spammy said:

      jsg said: Often a relational (but not SQL) DB is the best choice.

      Relational but not using SQL? Do you mind give an example of such? You mean like JSON file?

      Well, beneath every SQL DB lurks a relational DB. Keep in mind that SQL is an interface, not a DB.

      But there are also non-SQL relational DBs available. dBase and Raima are relatively well known examples.

      Thanked by 3spammy vimalware uptime
    • Both have their places, though honestly there's little you can do in a NoSQL database that you can't do in a SQL database, since SQL databases support large indexable documents like JSON, XML, etc. But I'm getting ahead of myself.

      The classic NoSQL case is storing json documents which are similar but not identical. In a lot of use cases, some things in every document will be universal (an owner name, a unique Id, etc.) but any given document will have some fields but not others, and the app can sort all that when it gets the document. If you did it in SQL, you'd have to define columns for every possible field and have lots of nulls, while in NoSQL you can just store the documents and then index whatever retrieval index you want.

      The nice thing about NoSQL is you don't have to design your schema ahead of time and can change it whenever you want without changing DB structure. Makes dev life easy. But there are tradeoffs.

      Think of a non-relational document DB like a coat check. You check in your coat, you get a ticket. Later you can take that ticket (i.e., key number, serial number, whatever) and get your coat back very quickly. However, if you were asked to count up how many coats have pockets or how many are brown, it would take forever because you'd have to pull out each coat. So retrieval by a key is very fast and you get the whole coat (i.e., document).

      OTOH a SQL database excels at these kinds of summarizing functions.

      The NoSQL popularity is more about relaxing consistency requirements where they're not needed. Social media is an often-used example. If you're managing a credit card database, you need it to always return completely consistent answers (two clients won't get different answers). OTOH, if the question is just how many likes a post has, maybe it's OK if two clients get slightly (eventually consistent) answers...particularly if relaxing that factor allows for higher performance. All depends on the need.

      So it depends what you need. If you want to stash documents away, don't want to worry about setting up what each field will be in advance, don't need a lot of slicing/dicing, want the freedom to quickly iterate a design, and have relaxed consistency constraints with high throughput requirements, then NoSQL might be a good fit. OTOH if you need a lot of analytical needs, have a fixed design, need many different access paths, may need future analysis you can't predict, etc. then you may want SQL.

      It's a bit blurred these days because you can insert something like JSON or XML into a SQL database (Oracle, Pg, etc.) and it will index all the json fields, so you get the best of both worlds...and to be honest, people were doing "noSQL"-ish things 30 years ago by stashing text blobs/clobs in databases (i.e., one table for metadata and one table for blobs/clobs). But hey, fashion...

      As to relational DBs dying...no, they're not. It's hard to think of a more stubbornly persistent technology than SQL. Tons and tons of application spaces (accounting, inventory, scientific data, marketing research, scientific data, the list goes on and on) need strongly consistent data that is easily sliced and diced in many ways and SQL still rules there.

      For LET support, please visit the interim support desk.

    • FranciscoFrancisco Top Provider

      It isnt webscale.

      Francisco

      Thanked by 1EdgeWeb
      BuyVM - Dedicated KVM Slices / Anycast Support! / Stallion Control Panel / Windows 2008, 2012, & 2016! / Unmetered Bandwidth!
      BuyShared - Shared & Reseller Hosting / cPanel + Softaculous + CloudLinux / Pure SSD! / Free Dedicated IP Address
    • somiksomik Member
      edited November 7

      Honestly speaking, it's all depending on your use case.

      I normally code in PHP. However, depending on my needs, I might not use MariaDB, instead opt for Sqlite3 for read heavy sites or JSON or even flat file DBs directly on directory level. This is done to reduce server overheads in cases where a relational DB is not required.

      There is no case where a relational DB will be obsolete soon.

      Same goes for PHP. I've been hearing it'll be obsolete soon since 2005 when I first picked up PHP, but PHP is widely used on the web.

      Also nothing goes out like a bulb. They are always slowly faded out, so you'll know when relational DBs are not in use.

      As far as I know, major software development companies are still using relational DBs, just not mysql or ms sql but their own variety. Appian relational DB is one example. You can search for more.

      Don't be so serious. It's just a forum. No one cares what you think anyway.

    • Completely off topic:

      As with most other communities, it's what the members make of it. I've read the topics created by the LET owners/managers and all the (justified) criticism by LET members.

      In spite of that, thanks to the people who still take the time and make the effort to answer, this has been, for me at least, one of the most educational web-hosting related places on the Net.

      Hope that in a few years time, I too will be able to contribute (more than just posting cat and pigeon pictures).

      This, on top of some good humour and high quality trolling of any trolls. :)

      That's it, sorry for the off topic (and I'm not even drunk).

      Mostly harmless™

      I/O Gremlin

    • spammy said: Things like MS-SQL or MySQL or Oracle are somewhat outdated and replaced by some new technology...

      Yes. This technology is MariaDB (it's MySQL fork which retains compatibility, many big companies replaced other SQL solutions with it).

      On a more serious note, big companies replace complex database systems (RDBMS) with even more complex database systems (e.g. Cassandra).

      It's important to note that it's not like they're switching from SQL to key-value storage like redis (this is what people usually mean when they say NoSQL).

    • Look at Oracle and say it's not a thingy...

      Wordpress Hosting - Home made!

    • bdlbdl Member

      "friends who are semi-close to the high tech circle" = sales monkeys :smile:

    • @intelpentium said:

      spammy said: Things like MS-SQL or MySQL or Oracle are somewhat outdated and replaced by some new technology...

      Yes. This technology is MariaDB (it's MySQL fork which retains compatibility, many big companies replaced other SQL solutions with it).

      From what I could gather (correct me if I'm wrong), MySQL 8.0 is "a lot better performing", but not compatible with MariaDB.
      Also, cPanel still doesn't support MySQL 8.0, but DirectAdmin does - for those using and needing control panels.

      Mostly harmless™

      I/O Gremlin

    • tgltgl Member
      edited November 7

      not true, and I will give you a simple example: how many websites are running wordpress?

      https://www.whoishostingthis.com/compare/wordpress/stats/

      NoSQL is good for big projects (usually IoT, big data), but for small stuff there is no need to use such a thing, unless you want to re-invent the wheel or waste a lot of resources on acting like an evangelist

      if you want a well paid job, yes, you should learn these new technologies, but thats another story

    • If you're starting out today with a flexible and reliable OSS relational DB, I think investing time into Postgresql is the best bang for the time spent.
      A craftsman chooses her tools carefully.

      Or you can half-ass exploring 10 'db-of-the-week' github repos.

      I should have listened to this exact advice when an experienced 30-something software principal gave me the same advice when I was 22. Hubris of youth is always consistent, unlike their Databases.

      Down with the sic_K_ness...
      The doctor might be going K-razy with the low end spirit. 😉

    • jsgjsg Member

      @tgl said:
      not true, and I will give you a simple example: how many websites are running wordpress?

      https://www.whoishostingthis.com/compare/wordpress/stats/

      NoSQL is good for big projects (usually IoT, big data), but for small stuff there is no need to use such a thing, unless you want to re-invent the wheel or waste a lot of resources on acting like an evangelist

      if you want a well paid job, yes, you should learn these new technologies, but thats another story

      Not quite. The difference isn't about size but about the need for structure and about the abilities to interface the data.

    • tgltgl Member

      in truth you can adapt relational databases (MySQL) to anything, but its gonna be slow (e.g. full text search) or you need to use an indexer (e.g. sphinx); the new solutions were developed to provide faster access mainly to resources (e.g. documents), not because it was not possible to perform the operations with the old ones.

      anyway, here i was talking about overdoing it (e.g. modify wordpress to use mongo just to show people you can), the point was if traditional dbs are out or not, and they arent, they work fine on most websites, but for special projects where you need fast performance, replication, bla bla bla, you need the new solutions.

      @jsg said:

      @tgl said:
      not true, and I will give you a simple example: how many websites are running wordpress?

      https://www.whoishostingthis.com/compare/wordpress/stats/

      NoSQL is good for big projects (usually IoT, big data), but for small stuff there is no need to use such a thing, unless you want to re-invent the wheel or waste a lot of resources on acting like an evangelist

      if you want a well paid job, yes, you should learn these new technologies, but thats another story

      Not quite. The difference isn't about size but about the need for structure and about the abilities to interface the data.

    • somiksomik Member

      @jsg said:
      Not quite. The difference isn't about size but about the need for structure and about the abilities to interface the data.

      @tgl, Mysql or MariaDB is great. You can define your tables and save data to them. Done.

      Now think of a specific use case where you need to save a 3D or 4D array with multiple types of data to a table. You can work around this with Mysql, but isn't it better to create a database with the specific structure you need?

      Thats a OODB comes in.

      An object oriented database is a database management system in which information is represented in the form of objects as used in object-oriented programming. Object oriented databases are different from relational databases which are table-oriented. Object-relational databases are a hybrid of both approaches.

      Many big companies use this instead of relational databases when needed. But the key word is still "when needed". Relational DBs are not obsolite.

      Don't be so serious. It's just a forum. No one cares what you think anyway.

    • edfoxedfox Member

      Nosql databases are for zoomers

    • NeoonNeoon Member

      No, it depends on the use case.
      Stop with that bullshit.

      Their are some things that have multiple use case but that's it.

      Thanked by 1flatland_spider
    • For my use case, a small website, even SQLite is perfectly enough.

      Thanked by 1vimalware
    • edfoxedfox Member

      @Neoon said:
      No, it depends on the use case.

      Most applications will be a lot of structured relational data, and a smaller part of schema-less stuff. For that, just use k,v tables, or jsonb your shit up.
      I have to admit that nosql however is neat for temporary random stuff (session data especially), but I would never use it as a database because I'm not scared of SQL

    • NeoonNeoon Member

      @edfox said:

      @Neoon said:
      No, it depends on the use case.

      Most applications will be a lot of structured relational data, and a smaller part of schema-less stuff. For that, just use k,v tables, or jsonb your shit up.
      I have to admit that nosql however is neat for temporary random stuff (session data especially), but I would never use it as a database because I'm not scared of SQL

      I have to admit, that I prefer wooden dildos.

    • also might find certain applications (for example, social networks and so-called "semantic web") that might make good use of specialized graph databases such as neo4j.

      (Note that features for processing "many-to-many" relations in highly connected data - that is, graphs - have also been included as extensions to various SQL and NoSQL database systems as well.)

      the Amitz.party lives on!

    • @raindog308 said:
      Both have their places, though honestly there's little you can do in a NoSQL database that you can't do in a SQL database, since SQL databases support large indexable documents like JSON, XML, etc. But I'm getting ahead of myself.

      The classic NoSQL case is storing json documents which are similar but not identical. In a lot of use cases, some things in every document will be universal (an owner name, a unique Id, etc.) but any given document will have some fields but not others, and the app can sort all that when it gets the document. If you did it in SQL, you'd have to define columns for every possible field and have lots of nulls, while in NoSQL you can just store the documents and then index whatever retrieval index you want.

      The nice thing about NoSQL is you don't have to design your schema ahead of time and can change it whenever you want without changing DB structure. Makes dev life easy. But there are tradeoffs.

      Think of a non-relational document DB like a coat check. You check in your coat, you get a ticket. Later you can take that ticket (i.e., key number, serial number, whatever) and get your coat back very quickly. However, if you were asked to count up how many coats have pockets or how many are brown, it would take forever because you'd have to pull out each coat. So retrieval by a key is very fast and you get the whole coat (i.e., document).

      OTOH a SQL database excels at these kinds of summarizing functions.

      The NoSQL popularity is more about relaxing consistency requirements where they're not needed. Social media is an often-used example. If you're managing a credit card database, you need it to always return completely consistent answers (two clients won't get different answers). OTOH, if the question is just how many likes a post has, maybe it's OK if two clients get slightly (eventually consistent) answers...particularly if relaxing that factor allows for higher performance. All depends on the need.

      So it depends what you need. If you want to stash documents away, don't want to worry about setting up what each field will be in advance, don't need a lot of slicing/dicing, want the freedom to quickly iterate a design, and have relaxed consistency constraints with high throughput requirements, then NoSQL might be a good fit. OTOH if you need a lot of analytical needs, have a fixed design, need many different access paths, may need future analysis you can't predict, etc. then you may want SQL.

      It's a bit blurred these days because you can insert something like JSON or XML into a SQL database (Oracle, Pg, etc.) and it will index all the json fields, so you get the best of both worlds...and to be honest, people were doing "noSQL"-ish things 30 years ago by stashing text blobs/clobs in databases (i.e., one table for metadata and one table for blobs/clobs). But hey, fashion...

      As to relational DBs dying...no, they're not. It's hard to think of a more stubbornly persistent technology than SQL. Tons and tons of application spaces (accounting, inventory, scientific data, marketing research, scientific data, the list goes on and on) need strongly consistent data that is easily sliced and diced in many ways and SQL still rules there.

      Thanks, very informative!

    • evnixevnix Member

      Story of every modern project ever,

      • Start with MongoDB, Serverless Computing, Kafka, Kubernetes/Docker, Terraform, Consul, Ansible etc... spend 6 months on architecture design meetings, learning, installing and configuring and then realize you have no running app yet.

      • Then come to a conclusion that you could do the same in PHP+MySQL(or RoR or Django) in under a week and host it for $1/Month hosting.

      If you haven't realized the above, you likely have a lot to learn.

      I recommend iwstack cloud | MxRoute for Mail | LightTable for Coding | Noisli for Noise

    • BlaZeBlaZe Member, Provider

      I learned more here than I learned in my college.

    • nottynotty Member

      vimalware said: Hubris of youth is always consistent, unlike their Databases.

      Crazy talk, you must be on ACID.

      Thanked by 2raindog308 vimalware
    • @vimalware said:
      If you're starting out today with a flexible and reliable OSS relational DB, I think investing time into Postgresql is the best bang for the time spent.
      A craftsman chooses her tools carefully.

      Indeed. Postgres is really nice, and the community adds features as needed. Adding JSONB was especially nice. Instead of two different DBs I only have to deal with one.

      Granted, I wasn't doing anything super complex, and I had a mixture of consistent data and really random data.

      @spammy said:
      Just did some Googling and wow this is completely new to me! Closest I got with them is probably using Redis (which I have no idea how it works other than knowing it is a PHP accelerator of some sort).

      True, redis is mainly known for being used as a caching layer, but it's more then that though. It has PubSub features, and it stores several different data structures, which JSON is not one of then weirdly enough. Redis, the company, calls it a data structure database.

      Pardon me for the ignorance, so noSQL conceptually means you store things in files with no schemas? so if using school class as an example, basically I store the information about each student as a file (JSON/CSV/something else) and then the name of the student as the file name and the DBMS is smart enough to, say, go to file for Joe and search for the key-value pair age when I do "select age from students where name = Joe".

      NoSQL generally means HTTP is used to interact with the database, or it uses some query language (QL) which isn't SQL. Postgres wasn't a SQL DB until 1995. It was a relational database, but it had it's own QL. Elastic has added SQL to Elasticsearch, so now Elasticsearch is an SQL database too. :smile:

      It's mostly coincidental that most of the NoSQL databases also aren't relational. Columner databases like Cassandra have nothing in common with document databases like MongoDB, CouchDB, or Elasticsearch, or time series databases like InfluxDB or OpenTSDB.

      NoSQL is also mostly a catchy marketing term, and it's generally pretty useless outside of marketing since it's such a catchall. There have been lots of different databases for ages. BerkleyDB, LMDB (which incidentally powers Redis), and others have been around for years before the NoSQL term was coined.

      The search part varies. Elasticsearch is built for searching data inside documents. MongoDB is not.

    • flatland_spider said: Postgres wasn't a SQL DB until 1995. It was a relational database, but it had it's own QL.

      I knew this fact but had forgotten it. You are quite right.

      For LET support, please visit the interim support desk.

    • notty said: Crazy talk, you must be on ACID.

      Got this shirt at Oracle OpenWorld a few years ago. It was for a company offering "transactional hadoop".

      image

      For LET support, please visit the interim support desk.

    Sign In or Register to comment.