Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Dacentec.com - Page 2
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Dacentec.com

2»

Comments

  • @kris

    Packet loss galore though!

    Also no ipv6 routing.

    Thanked by 1Kris
  • Main site/billing seems to be back up

  • Also some servers still have no routing, I have one with routing, the other is still no go.

    Thanked by 1Kris
  • I'm still unable to mtr (still stopping at Telia in ATL) but zabbix is picking up data again and so are other various sources.

  • KuJoeKuJoe Member, Host Rep

    Their IPs are back online for us so we can get to some of our servers, of course our IPs are not being announced so no chance of me going to sleep early tonight. :(

  • Any RFOs yet?

  • toshosttoshost Member, Host Rep

    hmm IP back online their website also back online

  • Spoke too soon, took 1 monitor as gospel. 2/3 down.

    Routes and prefixes are not routing, including bog standard monthly rentals.

    Maybe they need to wake up level 3 network tech, or rebuilding router config? Something's gone to hell.

  • Outside:
    http://uptime.munroenet.com/index.php

    Things aren't even routing properly internally:
    https://munroenet.com/uptime/

    As you can see, things are still clearly broken.

  • Quite.

    Taking Charter out, that's a first.

     4. dct-ds42-ve33.dacentec.com                                                     0.0%    25    0.4   2.4   0.3  10.5   3.3
     5. 75-130-191-25.static.hlrg.nc.charter.com                                      48.0%    25   20.4  20.8  19.4  22.2   1.0
     6. dtr02hckrnc-tge-0-0-0-2.hckr.nc.charter.com                                   48.0%    25   20.7  20.8  15.6  23.0   1.9
     7. 96.34.64.50                                                                   41.7%    24   21.6  20.8  19.2  22.1   0.8
     8. crr12spbgsc-bue-10.spbg.sc.charter.com                                        39.1%    24   21.3  22.9  21.1  26.4   1.7
     9. 96.34.2.50                                                                    33.3%    24   23.4  24.5  22.7  29.3   1.5
    10. bbr01gnvlsc-bue-1.gnvl.sc.charter.com                                         47.8%    24   27.8  25.5  22.1  30.6   2.3
    11. lag-108.ear1.Atlanta2.Level3.net                                              54.2%    24  475.8  85.5  25.7 475.8 132.0
    12. d.resolvers.level3.net                                                        34.8%    24   25.6  26.4  24.4  30.8   1.5
    

    What I can guess: their major uplinks are down / bit the dust.

    It's all running on one congested Charter uplink (maybe a 1Gbps after-thought for OOB usually) that does not have ACL setup for their main subnets.

    Ergo, packet loss and many prefixes down. Does not answer why Level3, etc. pissed out. F

    or $25/month I'll watch MTR as things get fixed while STFU and letting them do the needful.

  • mike0000mike0000 Member
    edited December 2015

    @Kris said: Does not answer why Level3, etc. pissed out. F

    Tends to happen when your sole Charter backhaul from ATL -> Lenoir goes down...

    Everything will come back up.. as soon as Charter does :)

  • My server apparently just came back up.

  • KuJoeKuJoe Member, Host Rep

    And we're back online.

    Thanked by 1hawkjohn7
  • ExonHostExonHost Member, Host Rep

    Everything is back online now.

  • Back online now too 2hours 39mins downtime.

  • Everything online, total full outage was 49 mins for me but had packet loss until 3am EST.

  • cfgguycfgguy Member, Host Rep

    UPDATE: Our Atlanta based upstream provider has advised us that they are conducting a transport upgrade maintenance that was previously thought not to include the CentriLogic circuit. CentriLogic was not notified of the maintenance window, which was scheduled to last until 06:00 AM EST. We have escalated with the upstream to restore service as soon as practical, but the maintenance is currently underway. They are aware of the unintended impact and will expedite the work as much as possible to restore service.

  • According to uptime robot, my 1 vm with them was:
    Down:5:24 am
    Up:7:59 am

  • dacentecdacentec Member, Host Rep

    We did have some network issues last night. I'll post all of our updates from Facebook last night.

    Update #1: Multiple upstream providers originating in Charlotte, NC have been affected by a region-wide fiber cut. Our alternate provider originating in Atlanta is also experiencing an issue at present. We are currently using a limited capacity local circuit which has restored some connectivity to the Data Center. We are working with the Atlanta provider to restore connectivity.

    UPDATE 2: Our Atlanta based upstream provider has advised us that they are conducting a transport upgrade maintenance that was previously thought not to include the CentriLogic circuit. CentriLogic was not notified of the maintenance window, which was scheduled to last until 06:00 AM EST. We have escalated with the upstream to restore service as soon as practical, but the maintenance is currently underway. They are aware of the unintended impact and will expedite the work as much as possible to restore service.

    After that it looks like the backup link was restored at around 4:00 AM EST. The regional fiber cut was to be repaired by 9:00 AM EST as we were informed.

    Customers may request a PIR via a ticket through Hostbill and we will provide it once it is completed. Anytime you're unable to reach the site you can always submit a ticket via email to [email protected] via your registered email.

    Thanked by 1vpsGOD
  • @dacentec said:

    Any plans to add some type of redundancy to your unprotected network? Fiber cut & maintenance in one night, at around the same time... tough break.

    Any reason all your maintenance windows & downtime coincide with Charter maintenance? Could be worth investing in some additional non-Charter backhaul.

  • teknolaizteknolaiz Member
    edited December 2015

    @mikeyur said:
    Any reason all your maintenance windows & downtime coincide with Charter maintenance? Could be worth investing in some additional non-Charter backhaul.

    I wonder if you read the UPDATE 2? Like how are you expected to know your backup upstream is doing a maintenance when they didn't tell you and didn't even include you into their plans. I say the Atlanta upstream fucked up here. No fault here for @dacentec.

  • What is a PIR?

  • What's a PIR?

    dacentec said: Customers may request a PIR via a ticket through Hostbill and we will provide it once it is completed.

  • @Mun said:
    What is a PIR?

    Pain in (the) Rear.

    Thanked by 1karjaj
  • @Hidden_Refuge said:
    I wonder if you read the UPDATE 2?

    Oh I read it, I just don't buy it.

    @Hidden_Refuge said:
    how are you expected to know your backup upstream is doing a maintenance when they didn't tell you

    What if there is no real backup? There was an outage this summer which Dacentec blamed on Charter dropping a 100Gig bundle between Charlotte & Atlanta - which appears to be exactly what happened last night.

Sign In or Register to comment.