Giant breakdown at Monaco Telecom: we explain the causes and consequences
“L‘one of the most serious incidents in the history of Monaco Telecom.” To qualify the major breakdown suffered this Tuesday afternoon by the Monegasque telecommunications operator, its general manager, Martin Peronnet, does not hide his face and, above all, does not underestimate the consequences of such a crisis. At 2:34 p.m., very precisely, landline, mobile and internet services were severely disrupted in Monaco.
Although part of the technical incident was resolved on time, it was not until 8:45 p.m. that fixed telephony was restored for all, in the meantime obliging the operators of vital importance (OIV) of the Principality organize themselves internally to ensure continuity of emergency services. All steered by a government crisis unit at the Ministry of State. We tell you what happened.
It all starts with maintenance
While Monaco Telecom was carrying out maintenance on one of the main data center from the country, located in the “J zone” of Fontvieille under the shopping center, the generator supposed to take over broke down.
At 2:34 p.m., an hour and a half after the start of the operation. “An equipment inside of it has failed, tripped”says Martin Péronnet.
The impacted equipment, which is particularly energy-intensive, may not have lasted long with its internal batteries. Until 2:50 p.m. “We had to stop maintenance and urgently reconnect the data center to the SMEG network. The power supply was restored at 3:15 p.m..”
The mobile antennas have thus gradually resumed service and the impacted internet connections have been restored. There remained a major problem.
Fixed telephony out
The telephone switch, through which the lines using an analog signal pass, has not restarted. “Most busy telephones have not yet migrated to more modern technologies. This will be the case by December 2024 with the extinction of copper network technology”explains Martin Péronnet.
This has therefore impacted businesses of all sizes: from the pizzeria that can no longer receive telephone calls from its customers to the major hotels in the Principality.
But also, and this is more thorny, the famous OIV, among others the Public Security, the firefighters of Monaco, the Princess Grace Hospital Center or the cardio-thoracic center.
These vital entities, unreachable from the outside, then became the priority of the crisis unit. “We migrated the emergency numbers to functional lines as quickly as possible. Around 5 p.m., the changeover was made for everyone”, he continues.
In the meantime, and in constant contact with the crisis unit, these OIVs had organized themselves internally so as not to be cut off from the world (read below).
In connection with Nokia, the manufacturer of the switch, Monaco Telecom restarted the equipment at 8:45 p.m.
The teachings
“We are never confronted with this kind of breakdown. Everything is done to avoid this. The last energy incident of this magnitude dated from 1997, says Martin Péronnet. There, all the protections jumped one by one. We will think about how to build more effective protection, additional safeguards.”
The move of data center to the “F zone” of Fontvieille and the Grand Ida building, during the restructuring works of the shopping centre, should be the occasion to review its architecture in order to avoid similar incidents. The transition from the copper network to optical fiber is also one of the keys. “It will be much safer. The process has started with the government. We have two years to migrate everyone.”
As part of its open days, Monaco Telecom will today receive a government delegation, led by the Minister of State and two of his government advisers-minister, Patrice Cellario and Céline Caron-Dagioni. There is no doubt that breakdowns will take up most of the conversations and will also be an opportunity to make crisis management processes ever more efficient.
Comment on the CHPG experienced this crisis internally
When the management of the Princess Grace Hospital Center became aware that the outage was going to last, with no visibility on its resolution, an internal crisis unit was activated in close collaboration with the general management of the Department of Social Affairs and Health.
The main hospital managers and their teams first established a precise diagnosis of the situation to ensure that continuity of care was guaranteed. “The care equipment, in particular in the emergency room, in intensive care and in cardiology, was not impacted, details Malik Albert, deputy director at the CHPG. Neither are medical records and internal phone lines.”
Only the fixed telephone communications with the outside no longer worked. Problematic in the event of a life-threatening emergency. “We warned the actors in the Alpes-Maritimes most likely to send us patients, such as Samu 06 and the Nice University Hospital, and those in the Principality that we could be reached on our personal mobile phones, he continues. As the CHPG is multi-site and one of them, Cap Fleuri in Cap-d’Ail, was not affected by the outage, we also switched the CHPG standard to that of Cap Fleuri. We have reinforced the team to take calls.”
The situation gradually returned to normal around 6 p.m. before, fortunately, the start of the night shift. “It is unprecedented what we have experienced. This has generated difficulties for the families of hospitalized patients but no delay in care has been listed, assures the deputy director. This allowed us to ensure the responsiveness of all services.”
A courier sent to the CHPG by the cardiothoracic center
If the Monegasque hospital, an operator of vital importance, is equipped with generators and is protected from cyberattacks, zero risk does not exist. “It requires regular training and solid procedures.”
Same scenario at the Monaco Cardio-Thoracic Center which was forced to switch its emergency number to a functional line. “We warned the actors of Nice and Menton by email and mobile. I even sent a courier to the CHPG, the old fashioned way, so that he could deliver the information live to the emergency department, confides Guy Nervo, deputy president of the structure. The situation was restored in the evening. During this outage, we received two emergencies. We did what was necessary internally to secure our home, our patients, and not interrupt our public service mission.