[ad_1]
Italyonline issued a new statement on the down of email services Libero Mail and Virgilio Mail. The company confirmed what it said yesterday, namely that “the progressive return to normal operation continues of the Libero and Virgilio mailboxes” after the fix activity on the operating system bug “has been completed by the vendor”.
“A part of our users was able to return to the availability of their e-mail, while the remainder will gradually be able to access and use mailbox services in the next few hours. In these hours of return to normal, a slowdown in reopening is physiological due to the traffic load accumulated in recent days.
For all users, however, the two pages are more important “Free Mail FAQ” And “FAQ Virgilio Mail” where the company answers the many questions that people are asking themselves these days. Inviting you to consult them, here is a part of the question and answer:
We have worked in these hours to protect the integrity of our users’ data. For the e-mails received on Libero mail and Virgilio Mail during the time window relating to the disservice, we sent a non-delivery message to the sender, also closing the SMTP from 4pm on 23 January last; as regards the subsequent behavior of the mail providers, who managed the traffic of incoming e-mails on Libero mail and Virgilio Mail once they received our non-delivery message, the scenarios can be different depending on the technical configurations of each provider; as a rule, these configurations should allow automatic re-sending in case of problems (such as the mailbox full). However, not all mail delivery services follow the same procedures.
We have worked in these hours to protect the integrity of our users’ data. For messages sent by other providers received on Libero Mail and Virgilio Mail, the scenarios can be different depending on the configurations of the mail sending services of each; normally, in case of problems (such as the box full), these configurations should allow resubmission, even for automatic services, such as those for sending payments or bills or other information/notification services (so-called “no reply” services). However not all mail delivery services follow the same procedures.
No, it was a bug in the storage operating systemdeveloped by a third-party vendor, which compromised its correct functioning and consequently the e-mail service. The data has always remained in the availability of Italiaonline and have been kept, for the functionality of the service, always in their integrity and in maximum security; there was no fraudulent third party access.
There was no hacking and no data breaches. Our priority is our users, mailboxes and the integrity of the data they contain. We have worked to protect all of our users’ data and we exclude data loss.
Reading the data on the storage affected by the outage was not possible as it was inaccessible. However, using the APP, users were able to see their emails with a certain historical depth, because they are stored on their device in that portion of memory which is technically called “cache”. This mechanism has the benefit of allowing quick access to your recent data on your phone by limiting the use of the network connection. If data with greater historical depth is required, the device must instead access the central data, which was not possible in this period. Webmail does not use these caching mechanisms and reads all content directly from the central data.
In consideration of the size of the database (about 4 PB), the cardinality of the files and the high transactionality of the application workload, intensive functional and load tests preceded the migration of mailboxes from the old to the new storage. Simulations were carried out with the joint monitoring of Italiaonline and the vendor and with the functional validation of the application supplier. The migration began in the last week of November 2022 and at the time of the incident approximately 75% of mailboxes were successfully migrated to new storage without any kind of problem ever occurring in their movement. Notwithstanding that a definitive incident report has yet to be produced by the vendor, the disservice was caused by the reaching of an alarm threshold by a hidden parameter of the storage operating system.
The increase in value does not appear to be directly related to volumes, migration methods, load or type of workload and for this reason it could not be subjected to specific tests or predictions/simulations aimed at verifying its behaviour. Moreover, the parameter in question is known only to the manufacturer and is not documented. Therefore, IOL could not know its presence and meaning in the context of both simulation and test operations and in production. Reopening even temporarily the service on the old storage for all mailboxes was not a viable solution because the users of the emails already migrated to the new storage would not have the availability of the emails received after the migration date of their mailbox. Restoring mailboxes from backup would not have given the expected result: the malfunction of the parameter made the primary storage unavailable while a restore on the old storage would have resulted in a high RTO (recovery time objective) and a non-zero RPO (recovery point objective), with consequent risk of lost operations.
Once the incident is closed, they will be analyzed cases and methods of compensation for the disservice and inconvenience suffered.
.
[ad_2]
Source link
