Skip to content

Commit 6fa4c08

Browse files
feat(dsm): DSMON-894 Add Messages documentation for Data Streams Monitoring (DataDog#30389)
* feat(dsm): Add Live Messages (v2) documentation for Data Streams Monitoring * feat(dsm): Improve Live Messages documentation formatting and clarity * feat(dsm): Minor text improvements in Live Messages documentation * piotr feedback * feat(dsm): Update Live Messages permissions section and add profile link * feat(dsm): Restructure Live Messages setup with table format and improved navigation * feat(dsm): Improve Live Messages documentation with clearer agent setup instructions * feat(dsm): Fix apostrophe encoding in Live Messages documentation * feat(dsm): Rename Live Messages documentation file to messages.md * feat(dsm): Simplify Kafka consumer configuration section in Live Messages documentation * Update content/en/data_streams/messages.md Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com> * Update content/en/data_streams/messages.md Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com> * Update content/en/data_streams/messages.md Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com> * Update content/en/data_streams/messages.md Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com> * Update content/en/data_streams/messages.md Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com> * Update content/en/data_streams/messages.md Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com> * Update content/en/data_streams/messages.md Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com> * Update content/en/data_streams/messages.md Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com> * Update content/en/data_streams/messages.md Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com> * Update content/en/data_streams/messages.md Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com> --------- Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com>
1 parent 9eaee84 commit 6fa4c08

File tree

1 file changed

+67
-0
lines changed

1 file changed

+67
-0
lines changed
Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
---
2+
title: Messages
3+
---
4+
5+
Use the Messages feature to view Kafka messages directly and troubleshoot your data streams efficiently. You can view messages at specific offsets within Kafka. Inspecting these messages can be crucial when troubleshooting issues related to specific services or data streams.
6+
7+
## Setup
8+
Before diving into detailed configuration steps, here's an overview of what's involved in setting up the Messages feature:
9+
10+
| Step | Description | Setup Guide |
11+
|--------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------|
12+
| Agent setup | An agent needs Kafka consumer integration enabled, be on at least version 7.69.0, and have remote configuration enabled | [Agent Setup][1] |
13+
| Verify permissions | You need a few log access permissions, which usually come standard as well as `Data Streams Monitoring Capture Messages` enabled in Datadog | [Required Permissions][2] |
14+
15+
16+
### Agent setup
17+
18+
#### Selecting an agent
19+
Any Datadog agent having access to Kafka can be used for setup.
20+
21+
If you self-host Kafka, Datadog recommends setting up Messages on one of your Kafka brokers' agents. Otherwise, choose any one agent your producer or consumer services communicate with that has access to your Kafka cluster.
22+
23+
#### Step-by-step guide
24+
25+
Complete the following steps on the same Datadog agent:
26+
27+
##### 1. Verify agent version
28+
29+
Ensure your agent is running version `7.69.0` or later.
30+
31+
To check your agent version, run `datadog-agent version` on your host.
32+
33+
##### 2. Enable [remote configuration][3]
34+
35+
Confirm that Remote Configuration is enabled for your agent (it is typically enabled by default). Verify this setting on the [Fleet Automation page][4] and also confirm it at the organizational level from the [Remote Configuration settings page][5].
36+
37+
##### 3. Configure Kafka consumer integration
38+
39+
Follow the steps detailed at [Kafka consumer integration documentation][6] to configure a Datadog agent for the Kafka consumer integration.
40+
41+
##### 4. Verify setup
42+
43+
* Review agent logs for `kafka_consumer` entries to confirm successful configuration.
44+
* Verify data ingestion by inspecting the `kafka.broker_offset` metric in Datadog's Metrics Explorer, filtering by the relevant Kafka topics.
45+
46+
47+
### Required permissions
48+
49+
You must have the following permissions enabled:
50+
51+
* `Data Streams Monitoring Capture Messages`
52+
* `Logs Read Index Data`
53+
* `Logs Read Data`
54+
* `Logs Live Tail`
55+
56+
You can verify your current permissions on your [Profile page][7].
57+
58+
To enable permissions, edit an existing role or create a new one on the [Roles page][8]. If you don't have adequate access to modify roles, contact your organization's administrator.
59+
60+
[1]: #agent-setup
61+
[2]: #required-permissions
62+
[3]: /agent/remote_config
63+
[4]: https://app.datadoghq.com/fleet
64+
[5]: https://app.datadoghq.com/organization-settings/remote-config
65+
[6]: /integrations/kafka-consumer/?tab=host#setup
66+
[7]: https://app.datadoghq.com/personal-settings/profile
67+
[8]: https://app.datadoghq.com/organization-settings/roles

0 commit comments

Comments
 (0)