debian-mirror-gitlab/doc/operations/incident_management/alerts.md

238 lines
10 KiB
Markdown
Raw Normal View History

2020-10-24 23:57:45 +05:30
---
stage: Monitor
2022-04-04 11:22:00 +05:30
group: Respond
2022-11-25 23:54:43 +05:30
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
2020-10-24 23:57:45 +05:30
---
2021-03-11 19:13:27 +05:30
# Alerts **(FREE)**
2020-10-24 23:57:45 +05:30
2021-02-22 17:27:13 +05:30
Alerts are a critical entity in your incident management workflow. They represent a notable event that might indicate a service outage or disruption. GitLab provides a list view for triage and detail view for deeper investigation of what happened.
2021-01-03 14:25:43 +05:30
## Alert List
2022-04-04 11:22:00 +05:30
Users with at least the Developer role can
2021-09-04 01:27:46 +05:30
access the Alert list at **Monitor > Alerts** in your project's
2021-01-03 14:25:43 +05:30
sidebar. The Alert list displays alerts sorted by start time, but
2022-11-25 23:54:43 +05:30
you can change the sort order by selecting the headers in the Alert list.
2020-10-24 23:57:45 +05:30
The alert list displays the following information:
2021-01-03 14:25:43 +05:30
![Alert List](img/alert_list_v13_1.png)
2020-10-24 23:57:45 +05:30
2021-01-03 14:25:43 +05:30
- **Search**: The alert list supports a simple free text search on the title,
2020-10-24 23:57:45 +05:30
description, monitoring tool, and service fields.
([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213884) in GitLab 13.1.)
2021-01-03 14:25:43 +05:30
- **Severity**: The current importance of a alert and how much attention it
should receive. For a listing of all statuses, read [Alert Management severity](#alert-severity).
- **Start time**: How long ago the alert fired. This field uses the standard
GitLab pattern of `X time ago`, but is supported by a granular date/time
tooltip depending on the user's locale.
- **Alert description**: The description of the alert, which attempts to
capture the most meaningful data.
- **Event count**: The number of times that an alert has fired.
- **Issue**: A link to the incident issue that has been created for the alert.
- **Status**: The current status of the alert:
2021-06-08 01:23:25 +05:30
- **Triggered**: Investigation has not started.
2020-10-24 23:57:45 +05:30
- **Acknowledged**: Someone is actively investigating the problem.
- **Resolved**: No further work is required.
2021-06-08 01:23:25 +05:30
- **Ignored**: No action will be taken on the alert.
2021-01-03 14:25:43 +05:30
2021-02-22 17:27:13 +05:30
NOTE:
2021-01-03 14:25:43 +05:30
Check out a live example available from the
[`tanuki-inc` project page](https://gitlab-examples-ops-incident-setup-everyone-tanuki-inc.34.69.64.147.nip.io/)
2020-11-24 15:15:51 +05:30
in GitLab to examine alerts in action.
2020-10-24 23:57:45 +05:30
2021-01-03 14:25:43 +05:30
## Alert severity
Each level of alert contains a uniquely shaped and color-coded icon to help
you identify the severity of a particular alert. These severity icons help you
immediately identify which alerts you should prioritize investigating:
![Alert Management Severity System](img/alert_management_severity_v13_0.png)
2020-10-24 23:57:45 +05:30
2021-01-03 14:25:43 +05:30
Alerts contain one of the following icons:
2020-10-24 23:57:45 +05:30
2021-11-18 22:05:49 +05:30
<!-- vale gitlab.SubstitutionWarning = NO -->
2021-01-03 14:25:43 +05:30
| Severity | Icon | Color (hexadecimal) |
|----------|-------------------------|---------------------|
| Critical | **{severity-critical}** | `#8b2615` |
| High | **{severity-high}** | `#c0341d` |
| Medium | **{severity-medium}** | `#fca429` |
| Low | **{severity-low}** | `#fdbc60` |
| Info | **{severity-info}** | `#418cd8` |
| Unknown | **{severity-unknown}** | `#bababa` |
2020-10-24 23:57:45 +05:30
2021-11-18 22:05:49 +05:30
<!-- vale gitlab.SubstitutionWarning = YES -->
2021-01-03 14:25:43 +05:30
## Alert details page
2020-10-24 23:57:45 +05:30
2021-01-29 00:20:46 +05:30
Navigate to the Alert details view by visiting the [Alert list](alerts.md)
2022-04-04 11:22:00 +05:30
and selecting an alert from the list. You need at least the Developer role
2021-01-03 14:25:43 +05:30
to access alerts.
2020-10-24 23:57:45 +05:30
2021-02-22 17:27:13 +05:30
NOTE:
2021-01-03 14:25:43 +05:30
To review live examples of GitLab alerts, visit the
[alert list](https://gitlab.com/gitlab-examples/ops/incident-setup/everyone/tanuki-inc/-/alert_management)
for this demo project. Select any alert in the list to examine its alert details
page.
2020-10-24 23:57:45 +05:30
2021-01-03 14:25:43 +05:30
Alerts provide **Overview** and **Alert details** tabs to give you the right
amount of information you need.
2020-10-24 23:57:45 +05:30
2021-01-03 14:25:43 +05:30
### Alert details tab
2020-10-24 23:57:45 +05:30
2021-02-22 17:27:13 +05:30
The **Alert details** tab has two sections. The top section provides a short list of critical details such as the severity, start time, number of events, and originating monitoring tool. The second section displays the full alert payload.
2020-10-24 23:57:45 +05:30
2021-01-03 14:25:43 +05:30
### Metrics tab
2020-10-24 23:57:45 +05:30
2022-06-21 17:19:12 +05:30
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217768) in GitLab 13.2.
> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/340852) in GitLab 14.10. In GitLab 14.9 and earlier, this tab shows a metrics chart for alerts coming from Prometheus.
2020-10-24 23:57:45 +05:30
2022-06-21 17:19:12 +05:30
In many cases, alerts are associated to metrics. You can upload screenshots of metric
charts in the **Metrics** tab.
2020-10-24 23:57:45 +05:30
2022-06-21 17:19:12 +05:30
To do so, either:
2021-10-27 15:23:28 +05:30
2022-06-21 17:19:12 +05:30
- Select **upload** and then select an image from your file browser.
- Drag a file from your file browser and drop it in the drop zone.
2021-10-27 15:23:28 +05:30
2022-06-21 17:19:12 +05:30
When you upload an image, you can add text to the image and link it to the original graph.
2020-10-24 23:57:45 +05:30
2022-06-21 17:19:12 +05:30
![Text link modal](img/incident_metrics_tab_text_link_modal_v14_9.png)
2020-10-24 23:57:45 +05:30
2022-06-21 17:19:12 +05:30
If you add a link, it is shown above the uploaded image.
2020-10-24 23:57:45 +05:30
2021-01-03 14:25:43 +05:30
#### View an alert's logs
2020-10-24 23:57:45 +05:30
2021-03-11 19:13:27 +05:30
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/201846) in GitLab Ultimate 12.8.
> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/217768) in GitLab 13.3.
2021-11-18 22:05:49 +05:30
> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25455) from GitLab Ultimate to GitLab Free in 12.9.
2020-10-24 23:57:45 +05:30
2021-01-03 14:25:43 +05:30
Viewing logs from a metrics panel can be useful if you're triaging an
application incident and need to [explore logs](../metrics/dashboards/index.md#chart-context-menu)
from across your application. These logs help you understand what's affecting
your application's performance and how to resolve any problems.
2020-10-24 23:57:45 +05:30
2021-10-27 15:23:28 +05:30
Prerequisite:
2022-04-04 11:22:00 +05:30
- You must have at least the Developer role.
2021-10-27 15:23:28 +05:30
2021-01-03 14:25:43 +05:30
To view the logs for an alert:
2020-10-24 23:57:45 +05:30
2022-10-11 01:57:18 +05:30
1. On the top bar, select **Main menu > Projects** and find your project.
2021-10-27 15:23:28 +05:30
1. On the left sidebar, select **Monitor > Alerts**.
2021-01-03 14:25:43 +05:30
1. Select the alert you want to view.
1. Below the title of the alert, select the **Metrics** tab.
1. Select the [menu](../metrics/dashboards/index.md#chart-context-menu) of
the metric chart to view options.
1. Select **View logs**.
2020-10-24 23:57:45 +05:30
2021-01-03 14:25:43 +05:30
### Activity feed tab
2020-10-24 23:57:45 +05:30
2021-01-03 14:25:43 +05:30
> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3066) in GitLab 13.1.
2020-10-24 23:57:45 +05:30
2021-01-03 14:25:43 +05:30
The **Activity feed** tab is a log of activity on the alert. When you take action on an alert, this is logged as a system note. This gives you a linear
timeline of the alert's investigation and assignment history.
2021-02-22 17:27:13 +05:30
The following actions result in a system note:
2021-01-03 14:25:43 +05:30
- [Updating the status of an alert](#update-an-alerts-status)
- [Creating an incident based on an alert](#create-an-incident-from-an-alert)
- [Assignment of an alert to a user](#assign-an-alert)
2022-05-07 20:08:51 +05:30
- [Escalation of an alert to on-call responders](paging.md#escalating-an-alert)
2021-01-03 14:25:43 +05:30
![Alert Details Activity Feed](img/alert_detail_activity_feed_v13_5.png)
## Alert actions
2021-02-22 17:27:13 +05:30
There are different actions available in GitLab to help triage and respond to alerts.
2021-01-03 14:25:43 +05:30
### Update an alert's status
2022-05-07 20:08:51 +05:30
**Triggered** is the default status for new alerts. For users with the Developer role or higher, the
alert status can be updated from these locations:
- [Alert list](#alert-list): select the status dropdown corresponding to an alert, then select an
alternate status.
- [Alert details page](#alert-details-page): select **Edit** in the right-hand side bar, then select
an alternate status.
To stop email notifications for alert reoccurrences in projects with [email notifications enabled](paging.md#email-notifications-for-alerts),
[change the alert's status](alerts.md#update-an-alerts-status) away from **Triggered**.
In projects with GitLab Premium, on-call responders can respond to [alert pages](paging.md#escalating-an-alert)
by changing the status. Setting the status to:
- **Resolved** silences all on-call pages for the alert.
- **Acknowledged** limits on-call pages based on the project's [escalation policy](escalation_policies.md).
- **Triggered** from **Resolved** restarts the alert escalating from the beginning.
2022-08-13 15:12:31 +05:30
In GitLab 15.1 and earlier, updating the status of an [alert with an associated incident](alerts.md#create-an-incident-from-an-alert)
also updates the incident status. In [GitLab 15.2 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/356057),
the incident status is independent and does not update when the alert status changes.
2021-01-03 14:25:43 +05:30
### Create an incident from an alert
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217745) in GitLab 13.1.
The Alert detail view enables you to create an issue with a
description populated from an alert. To create the issue,
select the **Create Issue** button. You can then view the issue from the
alert by selecting the **View Issue** button.
2022-05-07 20:08:51 +05:30
You can also [create incidents for alerts automatically](incidents.md#create-incidents-automatically).
Closing a GitLab issue associated with an alert [changes the alert's status](#update-an-alerts-status) to
**Resolved**. See [Alert List](#alert-list) for more details
2021-01-03 14:25:43 +05:30
about alert statuses.
### Assign an alert
> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3066) in GitLab 13.1.
In large teams, where there is shared ownership of an alert, it can be
difficult to track who is investigating and working on it. Assigning alerts eases collaboration and delegation by indicating which user is owning the alert. GitLab supports only a single assignee per alert.
To assign an alert:
2021-10-27 15:23:28 +05:30
1. Display the list of current alerts:
2022-10-11 01:57:18 +05:30
1. On the top bar, select **Main menu > Projects** and find your project.
2021-10-27 15:23:28 +05:30
1. On the left sidebar, select **Monitor > Alerts**.
2021-01-03 14:25:43 +05:30
2021-06-08 01:23:25 +05:30
1. Select your desired alert to display its details.
2021-01-03 14:25:43 +05:30
2022-07-16 23:28:13 +05:30
![Alert Details View Assignees](img/alert_details_assignees_v13_1.png)
2021-01-03 14:25:43 +05:30
1. If the right sidebar is not expanded, select
2022-07-23 23:45:48 +05:30
**Expand sidebar** (**{chevron-double-lg-right}**) to expand it.
2021-06-08 01:23:25 +05:30
2021-10-27 15:23:28 +05:30
1. On the right sidebar, locate the **Assignee**, and then select **Edit**.
From the list, select each user you want to assign to the alert.
2021-01-03 14:25:43 +05:30
GitLab creates a [to-do item](../../user/todos.md) for each user.
After completing their portion of investigating or fixing the alert, users can
unassign themselves from the alert. To remove an assignee, select **Edit** next to the **Assignee** dropdown menu
2022-05-07 20:08:51 +05:30
and clear the user from the list of assignees, or select **Unassigned**.
2021-01-03 14:25:43 +05:30
2021-01-29 00:20:46 +05:30
### Create a to-do item from an alert
2021-01-03 14:25:43 +05:30
> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3066) in GitLab 13.1.
You can manually create [To-Do list items](../../user/todos.md) for yourself
from the Alert details screen, and view them later on your **To-Do List**. To
2021-01-29 00:20:46 +05:30
add a to-do item:
2021-01-03 14:25:43 +05:30
2021-10-27 15:23:28 +05:30
1. Display the list of current alerts:
2022-10-11 01:57:18 +05:30
1. On the top bar, select **Main menu > Projects** and find your project.
2021-10-27 15:23:28 +05:30
1. On the left sidebar, select **Monitor > Alerts**.
2021-01-03 14:25:43 +05:30
1. Select your desired alert to display its **Alert Management Details View**.
2021-10-27 15:23:28 +05:30
1. On the right sidebar, select **Add a to do**:
2021-01-03 14:25:43 +05:30
2021-03-11 19:13:27 +05:30
![Alert Details Add a to do](img/alert_detail_add_todo_v13_9.png)
2021-01-03 14:25:43 +05:30
2021-10-27 15:23:28 +05:30
To view your To-Do List, on the top bar, select **To-Do List** (**{todo-done}**).