Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


apnscp 3.1 released!
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

apnscp 3.1 released!

nemnem Member, Host Rep

Panel demoRelease notesPrevious 3.0 announcement

apnscp 3.1 (a/k/a "ApisCP") is out the door after 7 months of development! 3.0 focused on achieving widescale adoption whereas 3.1 is business as usual with innovation. Among the 1650+ commits rolled into this release,

  • PHP-FPM, runs off socket activation to mitigate a thundering herd problem on large servers. Each worker pool spins up jailed to the account synthetic filesystem as part of BoxFS. For those used to the single-user behavior of cPanel, it supports running the worker pool as the account owner but from a security standpoint strongly discouraged.
  • TimescaleDB, converts the panel database into an efficient time-series storage system with minimal overhead. It's stupid fast, bandwidth overage queries dropped from 20 seconds to ~150 ms on a hot view. As part of 3.1, TimescaleDB will provide continuous aggregation of resource monitoring to allow apnscp to react quickly to threshold surges. CPU, IO in particular will get 24-hour enforcement windows.
  • Expanded resource throttling to I/O bandwidth + IOPS. apnscp now covers throttling memory, PIDs, CPU, and IO all without requiring third-party licensing.
  • SSO into subordinate domains, domains that are parented to a domain may now be transitioned via SSO to the subordinate domain. It's a compromise on reseller support and opportunity for third-party vendors to integrate billing more readily into apnscp.
  • IPv6 support + NAT/hairpinning auto-detection, apnscp will automatically configure your external IP on install.
  • Delegated whitelisting grants site administrators the option of protecting one or more IP addresses from brute-force deterrence built into Rampart. It solves a problem of 1 user in an office updating their password and getting the entire SOHO blocked. Users still get notified on panel login, but it won't deny access to the affected service.
  • Heightened protection on key URIs, apnscp throttles POST requests on xmlrpc.php and wp-login.php thus improving deterrence to common vectors of abuse.
  • ACMEv2 support. Includes wildcard DNS provided you've connected apnscp to one of 6 supported DNS providers.
  • PowerDNS integrated into mainline, as part of some excellent work by Lithium Hosting. apnscp can piggyback off your cPanel PowerDNS cluster without interference to facilitate migrations.
  • cPanel migrations, introduced in 3.0 but expanded in 3.1. See Migrations.md for more info!
  • FLARE helps get the word out when a critical update comes our way. FLARE checks every 30 minutes for a signal and when found, runs upcp obeying your update policy. It's an excellent solution to ensure you remain protected 24x7x365.

And many more. Be sure to check out the release announcement for all the tasty details. Next on the list with 3.1 is logical replication in rspamd to extend per-user preferences to its before-queue milter, y'know the part that rejects a message before it goes into the mail system and chews up CPU only to be spam.

Thank you everyone for feedback, grit, blood, sweat, and hopefully not too many tears whilst testing. Any other questions feel free to shoot me a message or hop on Discord.

Another 50 lifetime redemption codes have been added. Enjoy!

«1345

Comments

  • Doing a 'fresh' install now..

  • Any plans to accept PayPal payment?

  • Will I still be able to upgrade with the already active license @nem

  • nemnem Member, Host Rep

    @plumberg said:
    Will I still be able to upgrade with the already active license @nem

    Those licenses aren't restricted in any way. It'd be irresponsible to give people a product, then stop supporting particular versions for myriad reasons. Bitrot happens and ultimately the business' reputation suffers because of it.

    The panel will process the upgrade manually overnight unless you manually invoke an upgrade using upcp or your upgrade policy prohibits minor version updates.

    @bula said:

    Any plans to accept PayPal payment?

    I can process those manually with lifetime licenses. Email me at [email protected] if you're interested. I don't have any immediate plans to shoehorn PayPal/IPN as a payment source though. There's too much that's opened up to development in 3.1 via time-series data aggregation. TimescaleDB :love:.

    On the lifetime note, before geeking out with metrics, I'm circling back to licensing. Lifetime licenses will swell to $299 once monthly licenses are out. For now, monthly is slated to cost $15 per license with plans to increase costs for new subscribers down the road.

    Thanked by 2plumberg Chuck
  • Will do thanks

    @bula said:

    Any plans to accept PayPal payment?


    I can process those manually with lifetime licenses. Email me at [email protected] if you're interested. I don't have any immediate plans to shoehorn PayPal/IPN as a payment source though. There's too much that's opened up to development in 3.1 via time-series data aggregation. TimescaleDB :love:.

    On the lifetime note, before geeking out with metrics, I'm circling back to licensing. Lifetime licenses will swell to $299 once monthly licenses are out. For now, monthly is slated to cost $15 per license with plans to increase costs for new subscribers down the road.

  • Cool, a bunch of updates, google docs, seems produced by someone who knows what he is doing.
    I don't know whether need it but bought it, would try it later.

  • I am giving this a try using the lifetime redemption code on an idling server to see how well it works. Since this is a trial server for me, I will shut it down when I am done mucking around with it and if I need to put it to production in future (sounds useful for me to teach an introduction to self-hosting online), I suppose I can reinstall it on another box or dedicated with the lifetime license LET license, right @nem? Just not sure about the details on reinstalling on another box and the lifetime licence.

  • AlwaysSkintAlwaysSkint Member
    edited October 2019

    14 hours later it's still "installing" - /root/apnscp-bootstrapper.log is 23MB and growing.
    I knew I should've assigned 10 CPU threads to this installation. :-o

    EDIT: A VM reassignment (1.5GB RAM, 10 threads) seems to indicate a single core and 1GB RAM are the bottlenecks. "12% of 10 vCPUs : 1.2 GB of 1.4GB RAM" is the current Load.

    It definitely looks like a control panel not for lowend boxes.

    Thanked by 1TimboJones
  • nemnem Member, Host Rep
    edited October 2019

    @hiphiphip0 said:
    Cool, a bunch of updates, google docs, seems produced by someone who knows what he is doing.
    I don't know whether need it but bought it, would try it later.

    Thanks for supporting development!

    @poisson said:
    I am giving this a try using the lifetime redemption code on an idling server to see how well it works. Since this is a trial server for me, I will shut it down when I am done mucking around with it and if I need to put it to production in future (sounds useful for me to teach an introduction to self-hosting online), I suppose I can reinstall it on another box or dedicated with the lifetime license LET license, right @nem? Just not sure about the details on reinstalling on another box and the lifetime licence.

    Licenses may be backed up and reinstalled at a later date. See LICENSE.md for details.

    @AlwaysSkint said:
    14 hours later it's still "installing" - /root/apnscp-bootstrapper.log is 23MB and growing.
    I knew I should've assigned 10 CPU threads to this installation. :-o

    EDIT: A VM reassignment (1.5GB RAM, 10 threads) seems to indicate a single core and 1GB RAM are the bottlenecks. "12% of 10 vCPUs : 1.2 GB of 1.4GB RAM" is the current Load.

    It definitely looks like a control panel not for lowend boxes.

    You've got an issue with the environment. PM me what's in grep -B10 -m1 /root/apnscp-bootstrapper.log. Installer will cycle, pulling down an updated panel core on each attempt, until it is installed assuming transient issues such as networking. There's a quirk in systemd garbage collection that tosses out restart cycle counts outside a narrow window, so while I'd like to stop it at 3 attempts in 24 hours it won't hit that threshold. :neutral:

    Edit: 2 GB is the official minimum. It's still possible to install on a 1 GB machine if has_low_memory is set, but YMMV. RAM's cheap and with compaction/deduplication/NVMe-backed paging it becomes even cheaper to oversubscribe. RHEL8/CentOS8 official minimum is 2 GB as well.

    Cores aren't that important; what matters is how quickly a core can address a parcel of work. apnscp will spin up n+1 builds when compiling from source otherwise it's in serial, bound by the CPU performance. Mitogen speeds up Ansible drastically, but there's an outstanding bug with sudo open since August.

  • Grepping as I type.. it may be some time.

  • AlwaysSkintAlwaysSkint Member
    edited October 2019

    @nem said:
    Edit: 2 GB is the official minimum. It's still possible to install on a 1 GB machine if has_low_memory is set, but YMMV. RAM's cheap and with compaction/deduplication/NVMe-backed paging it becomes even cheaper to oversubscribe. RHEL8/CentOS8 official minimum is 2 GB as well.

    See how many offers on here don't start with 2GB? ;-) RAM is cheap on a desktop/laptop but not (relatively speaking) on a server/VPS, IMHumbleO.

    EDIT:

    curl https://raw.githubusercontent.com/apisnetworks/apnscp-bootstrapper/master/bootstrap.sh | bash -s - -s use_robust_dns='true' -s dns_default_provider='builtin' -s whitelist_ip='79.67.xxx.xxx' -s apnscp_admin_user='xxxadminxxx' -s apnscp_admin_email='[email protected]' -s apnscp_update_policy='all' -s system_hostname='apnscp.xxxx.com' -s has_low_memory='true' -s passenger_enabled='false' 'let-license-lt'

             total        used        free      shared  buff/cache   available
    

    Mem: 1393 372 293 112 727 727
    Swap: 511 0 511

  • nemnem Member, Host Rep

    See how many offers on here don't start with 2GB?

    Give it a few years and we'll have a similar conversation with 4 GB instead of 2 GB just like a few years prior it was 1 GB. Technology is always moving, so those that spend a few extra dollars a month for 2 GB are already ahead of the curve... or how 200 MB storage on ProHosting was the bees knees and one paid top dollar for that much storage.

    free -m

    Still have 727 MB available on the box that can be allocated as needed. The rest get opportunistically tied up in caches.

    PM me the log when you have it.

  • AlwaysSkintAlwaysSkint Member
    edited October 2019

    Still no response to the grep and installation appears to be looping. I'll see if I can start again but only willing to 'waste' just a little more time on this.

    TASK [apnscp/bootstrap : Setting dns => recursive_ns = 1.0.0.1,1.1.1.1]

    I didn't ask for that. :-|

  • nemnem Member, Host Rep

    @AlwaysSkint said:
    Still no response to the grep and installation appears to be looping. I'll see if I can start again but only willing to 'waste' just a little more time on this.

    It's not looking for a string, didn't specify:
    grep -B10 -m1 failed= /root/apnscp-bootstrapper.log

    TASK [apnscp/bootstrap : Setting dns => recursive_ns = 1.0.0.1,1.1.1.1]

    I didn't ask for that. :-|

    That's use_robust_dns. Introduced based upon DnsPerf metrics and the overwhelming fact that a lot of mom and pop DNS servers are remarkably unreliable. That unreliability translates into installer errors due to network failures, so concessions are necessary.

  • @AlwaysSkint said:
    Still no response to the grep and installation appears to be looping. I'll see if I can start again but only willing to 'waste' just a little more time on this.

    I am also installing on a 1GB machine and I made the silly mistake of selecting the benchmark flavour and am also mired in some installation issues. I think you should not grep as I didn't have a response with grep. I just went ahead with opening the log file in nano and paged-down all the way to the end to copy the relevant error logs for @nem to look at.

  • nemnem Member, Host Rep

    @poisson said:
    I am also installing on a 1GB machine and I made the silly mistake of selecting the benchmark flavour and am also mired in some installation issues. I think you should not grep as I didn't have a response with grep. I just went ahead with opening the log file in nano and paged-down all the way to the end to copy the relevant error logs for @nem to look at.

    It's not looking for a string, didn't specify:
    grep -B10 -m1 failed= /root/apnscp-bootstrapper.log

    Time to relocate the benchmark option. You're the second person to do this today as well :smiley:

    Thanked by 2vimalware poisson
  • Any flags to not install Timebasedb or whatever it's called - prefer the simplicity of munin, thanks?

  • @nem said:
    Time to relocate the benchmark option. You're the second person to do this today as well :smiley:

    Once is a mistake, twice is a choice.

    Thanked by 1nem
  • nemnem Member, Host Rep

    @AlwaysSkint said:
    Any flags to not install Timebasedb or whatever it's called - prefer the simplicity of munin, thanks?

    Can't use the panel then, sorry! TimescaleDB presently powers bandwidth/storage tracking and will also provide windowed analytics into CPU/IO/memory cgroup usage later on in 3.1. By 3.2 it'll expand to handle dynamic tuning.

  • AlwaysSkintAlwaysSkint Member
    edited October 2019

    Still trying a reinstall (3 hours so far), with 2G RAM, 2 threads:

    2019-10-30 11:42:57,287 p=20565 u=root | TASK [php/build-from-source : Verify clamd stopped (OOM)]

    That's clamd for ya!

    EDIT: updated install time, plus..

    TASK [mysql/install : Remove passwordless users]
    .. ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO) ..

  • nemnem Member, Host Rep

    @AlwaysSkint said:
    Still trying a reinstall (3 hours so far), with 2G RAM, 2 threads:

    2019-10-30 11:42:57,287 p=20565 u=root | TASK [php/build-from-source : Verify clamd stopped (OOM)]

    That's clamd for ya!

    It's disabled on machines with less than ~3.25 GB now for ample clearance when compiling.

    EDIT: updated install time, plus..

    TASK [mysql/install : Remove passwordless users]
    .. ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO) ..

    It's failing and you're looping again (systemd garbage collection quirk noted above). /root/.my.cnf isn't present, correct? When the machine is freshly imaged, does it have MariaDB/Percona/MySQL present? Any prior data won't get replaced when the database is setup if it existed prior to.

  • AlwaysSkintAlwaysSkint Member
    edited October 2019

    I gave up waiting after 4 hours and since stopped that VM in favour of an ispconfig one (they share an IP). It was a minimal Centos 7 install, with only a yum update = no database.

    [Flaky clamd/freshclam/clamscan is wrecking ispconfig too.]

  • nemnem Member, Host Rep

    @AlwaysSkint said:
    I gave up waiting after 4 hours and since stopped that VM in favour of an ispconfig one (they share an IP). It was a minimal Centos 7 install, with only a yum update = no database.

    [Flaky clamd/freshclam/clamscan is wrecking ispconfig too.]

    If you ever want to pick it up again send me a PM with your install log so I can see what was different about your system.

  • nemnem Member, Host Rep

    @AlwaysSkint said:
    I gave up waiting after 4 hours and since stopped that VM in favour of an ispconfig one (they share an IP). It was a minimal Centos 7 install, with only a yum update = no database.

    [Flaky clamd/freshclam/clamscan is wrecking ispconfig too.]

    Found the problem. sudo wrapper was pulled from a 3.1 commit last week, which isn't a problem except the default sudo policy initializes HOME=, which is needed for MySQL to pick up the right .my.cnf file when pruning passwordless users. Unless HOME= is set by running Bootstrapper from a login session, it'll exhibit the problem from your logs above.

    Fixed in v3.1.2.

  • seenuseenu Member
    edited November 2019

    feature list looks impressive

    demo site loads forever

    and i am amazed at how much you customized Laravel...

  • nemnem Member, Host Rep

    @seenu said:
    feature list looks impressive

    demo site loads forever

    and i am amazed at how much you customized Laravel...

    Race condition in mysqlnd library. Backend spawns children, which clone the address space of the master, including its static session database connection. In high concurrency environments, such as the demo, multiple workers may drain from the same inherited socket resulting a deadlock. Older platforms relied on libmysqlclient that handles cloning differently. It's fixed going forward.

    Laravel was bolted onto apnscp only a couple years ago. It provides job/queues, migrations, and templating. Down the road, I'd like to expand it to allow apps flexibility to use its routing subsystem instead of the apnscp's inflexible controller, which isn't as elegant.

  • Is it possible to run it on Debian?

  • nemnem Member, Host Rep

    @saudiqbal said:
    Is it possible to run it on Debian?

    No, apnscp is built for a specific platform with specific capabilities in mind. I've found people that need a platform to host also too need the platform to help manage. If your decision to employ a platform comes down to the OS flavor, then either you don't need a platform to help you manage or the platform you have used to manage before it woefully incomplete. Having to muck in low-level configuration is a deficiency of the hosting platform. Most of the low-level operations are abstracted in Scopes in apnscp. These tie back into Ansible for platform integrity checks, which having a single platform to build against greatly improves the reliability of these checks.

    Besides - having to diagnose bugs, like the one above, on a variety of platforms hurts my ability to engineer something that's efficient and stable. When things break, I want to be able to isolate as many confounded variables as possible and quickly arrive at a resolution. Running a variety of OSes diminishes my ability to control dependent factors (mysqlnd/libmysqlclient) and hurts the overall quality of the product.

  • Well, I finally got this beast (a.k.a. Magento Panel) installed, taking over 5 hours!
    The slimline python/ansible installer gobbled one CPU for most of that time - glad I didn't do this on a commercial shared VPS.
    When looking at the installer logs I couldn't help but wonder if I'd be quicker doing the install by hand. A few changes to text config files (mysql, ssh, postfix etc.) would take well over a minute each. Jeez, a simple bash script would've done the tasks in almost no time.
    I saw mondodb stuff pop up, plus ruby (whatever the heck they are :p ) and it looks highly likely that firewalld is the enforced firewall package, propped up by fail2ban. All in all, it's not just a different control panel to learn but a swathe of ancillaries to be aware of. 'tis a shame CSF can't be used without significant effort.

    (The lovely crafted html at the end of the installation is wasted, when a reboot is required. Looking at it in the tail of the installation log, it doesn't look pretty at all.)

    Now what do I put in the Domain entry at login? (Smells like a Windoze server login.)

  • nemnem Member, Host Rep

    @AlwaysSkint said:
    Well, I finally got this beast (a.k.a. Magento Panel) installed, taking over 5 hours!
    The slimline python/ansible installer gobbled one CPU for most of that time - glad I didn't do this on a commercial shared VPS.

    Depends on the hypervisor performance and how oversubscribed those CPUs are. I build out on Vultr, which completes within 90 minutes worst case (~70 minutes is normal). Hetzner is also good in Europe. Yesterday, I built out on DigitalOcean as a prebuilt image for their Marketplace and it took north of 3 hours to complete the same tasks. Not all VPSes are created equal. This really is next-generation shared hosting that does a better job masking oversubscription numbers.

    Likewise generating a new install from a prebuilt image took ~10 minutes on Vultr and 30 minutes on DO. It's flipping the same bits. Clearly one node is more oversubscribed than the other.

    Hand-edits are quite risky, especially when you botch something. There's around 12,000 lines of yaml that go into printing out a server at present. I expect as system complexity grows so too will the additional lines. Having a consistent, reproducible process is invaluable to ensure no single platform deviates from what is expected. As with the prebuilt image link above, the panel has the ability to unbreak itself if configuration drifts. It also doubles as a deployment mechanism for periodic platform updates distributed with periodic releases.

    'tis a shame CSF can't be used without significant effort.

    No need for CSF. Everything is integrated into the panel through its API (see FIREWALL.md), including delegated whitelisting, which gives your users the ability to protect themselves from accidentally falling under the crosshairs of brute-force protection. If they block themselves, upon login, the panel pops up a modal informing them of the service(s) blocked. It's better to have these services integrated so you can do more with what's available. Everything in the panel is streamlined. There's also a separate bit that ramps up sensitivity on high value URIs.

    Now what do I put in the Domain entry at login? (Smells like a Windoze server login.)

    INSTALL.md has you covered - leave it empty if you're admin.

    Thanked by 1AlwaysSkint
Sign In or Register to comment.