Postmortem: Glo-Net DD_DoS outage server attack incident report (creation)
Report incident date: Monday, November 11th, 2080
This is the report from the incident that occurred at November 11th making our Glo-Net service outage from normal operation. This incident blocked our service for 1:50 hrs preventing nodes from communicating between each other and affecting 100% of our service.
Here we will describe in detail the attack produced to Glo-Net.
Issue Summary
From 17:50 to 19:00 UTC. “Connection timed out” message displayed on Glo-Net browsers. Glo-Net browsers interrupted connection with mirror connectors' failure to complete. Traffic was blocked completely, servers denial of service was a complete outage of Glo-Net service. The root of this outage was a DD_DoS (Double Distributed_Denial of Service) invalidating connectors servers.
Timeline (UTC)
- 17:50 PM: System flooded DD_DoS outage begins
- 17:50:02 PM: Browsers unable to input/output data.
- 17:51 PM: Multiple nodes alert connection broken.
- 17:52 PM: Traffic-dispersion implementation sent.
- 18:37 PM: Barrier restored service in a range of 3 million nodes.
- 18:55 PM: Blackhole node routing activated.
- 18:56 PM: Nodes communication system reconnected.
- 18:57 PM: Glo-Net live.
- 19:00 PM: Communication 100% restored.
Root Cause
17:50 PM UTC, system flooded with a DD_DoS attack that completely blocked connection between nodes. Nodes were not able to send or receive any data. Nodes intern alert was activated once they were not able to connect to the nearest node. The DD_DoS attack disrupts connections and it doesn't allow any traffic, leaving the connection between nodes adrift. The attack covers the node's sphere with a distributed similar Glo-Net node-connection making it impossible to the node to input/output data.
Once the node alert is activated, it takes between 40 to 45 min to reconnect the whole system. The traffic dispersion implementation sends an electric shock that breaks the DD_DoS attack traffic letting the nodes reconnect to their nearest 3 million friendly nodes renewing keys. Then the black hole routing is activated to identify malicious wrong key node traffic. Unidentified keys are sent through the black hole. The attack is stopped.
Corrective and Preventive Measures
The corrective measures were taken once the denial of service was executed. The first measure was to send an intern node alert that notifies core nodes that connections have been blocked. Cores nodes work independently from shells so they can identify external attacks. Shells protect and confirm consensus rules by monitoring input/output key confirmations. Once an attack blocks the keys sent from other shells, shells we're not able to break communications. Following these events we considered the following corrective measures to the Glo-Net system code.
- Once 420 unidentified traffic keys are not confirmed, shells should stop confirming.
- Core nodes were updated with pla-codes that will eventually protect the second layer of the system.
- Blackhole routing code was updated to let traffic flow uninterruptedly and be able to hide as a shell receiving the attack. This will distract the attackers by letting them know that the attack is still being executed.
Glo-Net systems is committed to their clients and know the extreme precautions that need to be taken to keep service active. We will keep updating and monitoring traffic to prevent attacks of this size.
Sincerely,
POT — Prevention Operation Team
