All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
VMPort - NJ1 Service Outage.
Ash_Hawkridge
Member
I would like to take the time to announce an ongoing service outage on our KVM node NJ1.
We have received a rather disturbing email from our New Jersey provider regarding removing our server from the network due to over 10 plus 4Gbit+ attacks outbound and inbound on the node over the last 48 hours. All i can say at this time is that we are disgusted at how the said provider is dealing with this situation, as you may or may not know the KVM side of our business is new as is our relationship with this company...
They are not prepared to give us an ETA at this time.
What i will say is that our pingdom monitoring has showed no sign's of connection issues of which they speak nor have we received any complaints from clients, which is giving me cause for concern. http://stats.pingdom.com/0hb9pb5bu6h3
A 1Gbit port would be easily saturated by attacks this large which would, to the best of my knowledge, show as a "Service Disruption" in pingdom.
We will keep all clients updated within our network blog.
I would like to apologize for any inconvenience this ongoing issue is causing.
All other OpenVZ/Xen VPS and Shared hosting account's are online as normal.
Regards,
Ash @ VMPort
Comments
What provider are you with for NJ?
@Jeffrey
I'm not going to mention any names just yet since we go through an individual that may use these forums.
I don't blame them personally, To block a attack like that you would need Multiple 10 Gbit uplinks and $1000's in DDOS Protection equipment.
If it is affecting the network, Then obviously your going to get dropped.
How were you able to send a 4Gbit flood on a 1Gbit port? o_O
Francisco
@Francisco
Im assuming that was regarding an inbound attack, they didn't give me any numbers for outbound. The message was vague to say the least.
My wording is a bit misleading, i have just been woken up by this :P
Ash
Still brutal for sure
Do they not have any sort of auto nullroutes?
Francisco
Apparently not, would you agree with my comment regarding pingdom picking up these attacks that have supposedly been going on for 48 hours?
Tbh we are just use to our system at Equinix. Email > An incoming DDoS attack has been picked up on xx.xx.xxx, click to blackhole this IP.
Not just "Your server has been unplugged for an unspecified amount of time"
Ash
So they can't null route the IP that was being attacked and they took the whole server offline? If there was really a 4Gbit+ attack going on for 48+ hours, you would have known. I say run from whoever you have that server with.
@VMPort, that thread you made of the new site template you were making had a crappy template and it looks terribad. However, what you have now is smaller and thus looks hella' coo.
@VMPort did you colocate with them? If so, run to their datacenter and grad your dedi/dedis and go somewhere else.
Possibly BurstNET?
Maybe another provider here can talk to their DC and hook you up for a quick move somewhere better.
why did I not get an email, and why do I not see an announcement on your site? that should be the first thing you do, not come here and post about how it's your provider's fault. I JUST moved my git repos over to your KVM... would really like it if I can get that back lol
Cool, but... I don't understand why you don't monitor your nodes :S or why you didn't noticed.
Did you not read all of his post? Read it again.
Where it says that?
No
May be is obvious, but nowhere in the posts say something about something like "noticed the issue, then notified the provider" or "powered off the server" or "my monitor system triggered some alarms".
Don't try to look for something between the words.
Yes, that monitors uptime, doesn't monitor bandwidth, i/o, load, etc. That kind of monitoring system.
Again, I am not saying that.
Nevermind... /out
@VMport I hope you solve your issues with that datacenter.
Packetloss would be shown.
Francisco
Looks like its back up. Looks like my VM was rebooted, is that standard for network outages?
They already stated that their server was shut off. So it have to boot up.
no, but if the host suspended the server for the ddos attacks by unplugging it yes.
@kbar
We did send out an email straight away, as well as a post in our network blog (http://network.vmport.com), do you have accurate contact details on file?
@yomero
Yes we have munin and pingdom monitoring, but there not much use when the server is unplugged
Why the entire server was uplugged? Did they send you the target IPs of the attack? Was the main ip of the server part of the attack?
Do you have remote KVM (keyboard) access to the server? Or can your colocator provide such out of band access so you can still be on server to work togheter with the network engineers to mitigate/solve the issue?
@prometeus
That's kind of what i was getting at, its not on. No they didn't send any info just basically said "Its unplugged, deal with the issue when its back online"
Yes we have KVM access but its no good when the server is completely unplugged.
This is really unusual afaik... you really unplug things as a last resort when you dont know what else to do...
It took them 48 hours to detect 4Gbit incoming attack, and then they had no better idea than shutting down the whole node? Couldn't they just unplug the network cable instead? >_<
@vedran
Dont ask, im quite frankly not prepared to believe that over 10 attacks went on over 48 hours without us or our clients noticing.
The server was certainly shut down and not just unplugged though.
12:34:22 up 3:39, 1 user, load average: 0.48, 0.44, 0.37
if it was really 4 gbps it would be forced to drop 3 out of every 4 packtets so it would defiantly be noticeable.
It was, but I was testing how well it ran on my $26/yr SecureDragon Xen VPS when it went down.