Failures and timeouts
It happens that things don't go smoothly, and of course DataUnlocker implements best measures to keep systems running even in case of failures.
Timeouts
Automation pipeline steps won't run indefinitely and will eventually time out, issuing
the timeout failure at this step. The default timeout is set to be 2 days,
while many steps have this timeout configurable:
We strongly recommend not to set custom timeouts over 2 days, as you risk to react on these failures slowly.
Exponential backoff
In case when the response status of the webhook is not 2xx, DataUnlocker will keep trying to deliver this webhook using
the exponential backoff strategy, with the maximum of 16 minutes between invocations. Each next delivery will be scheduled
in the following way:
- Deliver the webhook immediately.
- When failed, deliver it after
1 minute. - When failed again, deliver it after
2 minutes. - When failed again, deliver it after
4 minutes. - When failed again, deliver it after
9 minutes. - When failed again, deliver it after
16 minutes. - Each next failure will deliver it after
16 minutes.
This will continue up to 12 hours of the maximum waiting time (be default), after which DataUnlocker will eventually fail
the automation pipeline at this step and send email notifications to the automation pipeline owners.
Notifications
In case when the automation pipeline fails:
- DataUnlocker sends email notifications to all users receiving "dev" notification types (can be configured in the property's user management page).
- You will be able to examine the failed pipeline and all related info in the DataUnlocker's Admin Console UI.
- You won't receive email notifications when you manually abort the pipeline by clicking the "Abort" button.
Failed automatic installation refresh pipeline most likely means that you will need to fix the issue and then start this pipeline manually from the DataUnlocker's Admin Console UI, waiting until it succeeds.