Resolved -
This incident has been resolved.
Jan 24, 11:01 PST
Monitoring -
Thank you for your patience as we restore your IT Glue data as quickly and safely as possible.
All IT Glue data before 11:30pm PST on January 18th was successfully restored on January 19th, and all passwords and documents stored between 11:30pm PST on January 18th and 9:00am PST on January 19th were successfully restored on January 20th.
For all other IT Glue assets, please continue to use your IT Glue Integrations and open API to automatically sync and update your IT Glue information.
In the rare exception that you still have some IT Glue assets or information that have not been recovered, please contact IT Glue support at support@itglue.com with detailed information of the IT Glue data not yet recovered. Our support team will work with you to safely recover your data with minimal duplications.
We thank you for your patience and understanding.
Jan 24, 11:00 PST
Update -
Thank you for your patience as we restore your IT Glue data as quickly and safely as possible.
All IT Glue data before 11:30pm PST on January 18th was successfully restored on January 19th, and all passwords and documents stored between 11:30pm PST on January 18th and 9:00am PST on January 19th were successfully restored on January 20th.
Jan 23, 10:25 PST
Update -
Thank you for your patience while we restore your IT Glue data as quickly and safely as possible. We wanted to provide you an update on the data restoration progress and the read-only banner.
Data Restoration Progress:
All passwords have now been restored and we are currently working on restoring your documents. Document restoration will happen in 3 phases. Document folders are currently being restored, files will follow shortly, and finally documents. We expect that all documents will be restored by 5pm EST on Friday, January 20th.
Please note, restored documents/files/folders will appear as duplicates in IT Glue as “name.docx RESTORED”. This will allow you to further track down changes and then delete the outdated duplicate.
Read-only Banner:
In addition, to address the issue with the read-only banner that some users have observed sporadically today, we will be restarting the web servers tonight, starting at 6pm PST / 9pm EST. This process should take about 90 minutes and you can continue your IT Glue operations as usual, but some users may be logged out and will have to re-log in to the app. We expect no other customer impact.
As a reminder, you can now continue your operations as usual and can store, view and edit IT Glue information. We thank you for your patience.
Jan 19, 16:26 PST
Update -
Some users may have observed a read-only maintenance banner in the app today. Please be advised that is unintentional and likely due to browser cache and/or cookies. Please clear all cache and cookies from your browser or try a private/incognito window, and you should not see the banner anymore and continue your IT Glue operations as usual. If you still do, please report it to our Support team.
We are still working diligently to restore documents, files and folders. We thank you for your patience and want to assure you that we are committed to restoring the data as quickly and safely as possible.
Jan 19, 14:33 PST
Update -
All passwords have been restored and we are currently in the process of restoring documents.
As a reminder, you can now continue your operations as usual and can store, view and edit IT Glue information. We thank you for your patience and want to assure you that we are committed to restoring the data as quickly and safely as possible.
More updates will be shared as they become available.
Jan 18, 17:35 PST
Update -
New password records are in place now. Modified passwords are currently being restored and will be followed by documents. Our team is working continuously to restore the remaining data from January 18th, 2023.
As a reminder, you can now continue your operations as usual and can store, view and edit IT Glue information. We thank you for your patience and want to assure you that we are committed to restoring the data as quickly and safely as possible.
Jan 18, 15:47 PST
Update -
Our team is working continuously to restore the remaining data from January 18th, 2023. Passwords are currently being restored and will be followed by documents. We will provide another update at 4:00 pm PST.
You can now continue your operations as usual and can store, view and edit IT Glue information.
We thank you for your patience and want to assure you that we are committed to restoring the data as quickly and safely as possible.
Jan 18, 13:15 PST
Update -
IT Glue is no longer in read-only mode. Users will be able to resume normal operations.
All data from yesterday, January 17th 2023, up to January 18th, 2023 2:30 am Eastern, has been restored as we continue to restore the remaining data from January 18th, 2023.
Thank you for your patience as we work diligently to restore all data for affected customers. Updated details will be shared as they become available.
Jan 18, 09:33 PST
Update -
After receiving reports of data loss from some customers due to the database maintenance we performed overnight we have put IT Glue in read-only mode to restore lost data.
We thank you for your patience as we work to get the data to restore done, which is currently in progress. We may have a period of 5- 10 minutes of downtime as we perform this operation and users may see a 502 Bad Gateway error. More updates will be shared once available.
Jan 18, 08:27 PST
Identified -
We are undergoing emergency database maintenance currently to resolve an issue some customers are experiencing. We anticipate the process to take one hour from now. More updates will be shared here as they become available.
To help minimize interruption to business, IT Glue and MyGlue will be available in Read-Only mode most of this time to our North American Datacenter customers.
We may have a period of 5- 10 minutes of downtime as we perform this emergency maintenance
When in this mode, a maintenance banner will be displayed across the browser interface, content modification will be disabled for all users and any API call to create, update or delete resources will return an error code: HTTP 503 - “Read-Only mode”
Region: North America
Applications: Web App, Mobile App, Chrome Extension
Jan 18, 08:05 PST