Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Looking for Hosting Large Database Site
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Looking for Hosting Large Database Site

I have a site right now on a reseller hosting account that is about 8GB the database size alone. It's basically one database with several large tables. So far the reseller hosting company hasn't complained but I'm not liking the loading times much to retrieve stuff on the front end on the website. The database is growing and I could see it being 20 or more within a year and continually growing.

I'd also like storage for downloadable files, which would probably be another 30GB, maybe 40GB, in time. Right now the downloadable files are hosted on mega but I'd rather it all be integrated into the same hosting account for better control.

The databases are called to the front end to show results and are filterable on the front end. This is where it's slow right now. Right now there are few users because it's not open to the public yet. I'm concerned with as more users start joining and searching, the database will become much slower.

I'd also like a control panel and of course some type of support if I'm in a jam. Right now the site is on a CPanel server and I do most of the database updates through PHPMyAdmin.

So I think at least 60GB diskspace, preferably more. What else do I need in terms of specs?

I was looking at BuyVM and their slabs for more storage but it seems like they have been sold out for a while, so I'll probably have to look elsewhere.

Any recommendations for my situation?

Comments

  • RAINMANRAINMAN Member
    edited November 2019

    Can't offer anything but you want lots of ram for the database. Ideally put the whole thing in ram or at a minimum a fast sad.

    Also make sure your database it's optimized, indexes etc.

    Thanked by 1spun
  • spun said: I was looking at BuyVM and their slabs for more storage but it seems like they have been sold out for a while, so I'll probably have to look elsewhere.

    I would definitely recommend @Francisco but since they are out. Go for hetzner then. They have additional storage as an addon.

    Thanked by 2spun Hetzner_OL
  • JordJord Moderator, Host Rep

    Dedi or Hetzner with their additional storage. Probably dedi if you want more power for that large DB. Processing times should be quicker than too. Maybe look out for NVMe as that will help a lot too.

    Thanked by 2spun Hetzner_OL
  • HostDocHostDoc Member
    edited November 2019
    4vCores @ 2.4 GHz
    Unmetered @ 1Gbps
    8GB RAM
    1 IPv4 + /64v6
    50GB SSD
    50GB HDD
    Kansas City
    £4.99/m
    

    KC3

    We provide free direct admin licenses but the plan needs to be above £5 per month otherwise the license is an additional £5 per month.

    Thanked by 2spun kp93
  • FranciscoFrancisco Top Host, Host Rep, Veteran

    spun said: I was looking at BuyVM and their slabs for more storage but it seems like they have been sold out for a while, so I'll probably have to look elsewhere.

    Cancellations run tomorrow so we'll have stock then :)

    Francisco

    Thanked by 1spun
  • raindog308raindog308 Administrator, Veteran

    spun said: I'm concerned with as more users start joining and searching, the database will become much slower.

    In addition to relocation, you might consider:

    • running mysqltuner.pl - that's the name of the script and its url
    • looking at your top queries and running explain on them to see if you need to add or removing some indexes, etc.

    Good database performance is something of an art but those are the 80% before you worry about the 20%.

  • At that scale, I'd consider a VPS dedicated to mariadb (with just csf as an extra), with the front-end hosted on a neighbouring VPS. Quite a few providers offer internal IPs.

    Thanked by 1spun
  • I appreciate all of the comments.

    @HostDoc good prices. I'll pay you £5.01 per month. :smiley: Those are actually similar specs to the plan I was looking at. The problem is it says unmanaged. I would like at least a basic level of management, doesn't have to be proactive management.

    @Francisco I'll keep an eye out. Still considering my options.

    @AlwaysSkint I was considering hosting the DB separate but I'm not sure I'm at that level where it makes sense yet.

  • jsgjsg Member, Resident Benchmarker

    @spun

    Frankly, I think you are doomed. Not because of your parameters (an 8 GB database is everything but "large") but because the clear lack of relevant knowledge.

    My advice is

    • limit the queries your visitors can do. Even better create some web interface that only allows certain queries plus only x queries per y time.
    • Be sure to have lots of memory
    • Read @raindog308 's advice again.
    Thanked by 1spun
  • servarica_haniservarica_hani Member, Patron Provider

    if you need to host the whole db in ram go with our NVMe cheetah 2

    it comes with 18GB of ram and 240GB of NVMe space

    https://servarica.com/clients/cart.php?a=add&pid=409

    another option if you want to keep having cpanel for cheap go with our shared storage offer but in that case the database will run fom disk not from ram

    https://servarica.com/webhosting/

    Thanked by 3spun timelapse kp93
  • raindog308raindog308 Administrator, Veteran

    AlwaysSkint said: At that scale

    We actually don't know the scale. An 8GB database with 2 users is not a problem at all (really, an 8GB DB isn't that big these days). Heck, I've run a 100GB DB on a laptop and since I was the only user, it was quite speedy.

    OTOH, a 10M database with 1,000 users can be a huge problem if they're all fighting to update the same rows. Even an 8GB database with 1,000 users may not be a big deal if 999 of them are doing reads and things are well setup.

    All depends on the nature of the queries and degree of contention.

  • SNACTeamSNACTeam Member
    edited November 2019

    I would recommend DigitalOcean or Upcloud, both scalable and reliable.

    Thanked by 1spun
  • Appreciate the answers.

    @raindog308 No one is updating any rows. It's read only access with ability to search the DB on the frontend.

    @Hani another good offer for me to consider.

    @jsg I might be doomed over the long haul and have to learn to optimize everything better than what I'm doing now but I don't expect to have a large user base very quickly. I'm trying to future proof me at least for the next few months and then see where to go from there. The queries that they can do are limited already.

  • @spun said:
    Appreciate the answers.

    It's read only access with ability to search the DB on the frontend.

    So try to find long running queries and, if applicable, optimize your code to improve the database performance. Since you are (mainly) dealing with read operations, perhaps a caching mechanism may help you in the future (take a look at redis and/or proxysql).

  • Buyvm + Offloaded sql
    https://wiki.buyvm.net/doku.php/sharedsql

    I can see that fran's been here already but something for you to discuss with him about

  • @raindog308 I guess I better shutup then 'cos I obviously know SFA. :-(

  • loeloe Member
    edited November 2019

    If some requests will be made several time caching can help.

    Vanish makes it easy to cache whole pages. Redis for object caching can do wonder. But this would make sense once you have a lot of traffic, until then @raindog308 advice + a decent VPS with SSD storage (check iops!) should be more than enough.

    Tweaking your code could also be an important step; we don't know how it's done.

  • 1) Go with @LiteServer ask them for semimanaged plans with DA licence, they can help to tune MySQL very well

    If you need backup
    you can purchase additional ftp backup from them or from @KuJoe securedragon
    they are reliable, I am using both

    If its not in your budget request for one time optimization with a small fee.

    2) @Hani Servarica is reliable also provider , love their chat support . When online response time is less than 2 mins

    Their snapshot feature is awesome .

Sign In or Register to comment.