This weblog publish is co-authored with Erick Leon, Sr. Technical Alliance Supervisor from Dynatrace.
Amazon Kinesis Information Firehose is the best approach to reliably load streaming knowledge into knowledge lakes, knowledge shops, and analytics companies. With only a few clicks, you may create fully-managed supply streams that auto scale on demand to match the throughput of your knowledge. Clients already use Kinesis Information Firehose to ingest uncooked knowledge from varied knowledge sources, together with logs from AWS companies. Kinesis Information Firehose now helps delivering streaming knowledge to Dynatrace. Dynatrace begins analyzing incoming knowledge inside minutes of Amazon CloudWatch knowledge era.
Beginning at this time, you should use Kinesis Information Firehose to ship CloudWatch Metrics and Logs on to the Dynatrace observability platform to carry out your explorations and evaluation. Dynatrace, an AWS Accomplice Community (APN) has offered full observability into AWS Providers by ingesting CloudWatch metrics which can be revealed by AWS companies. Dynatrace ingests this knowledge to carry out root-cause evaluation utilizing the Dynatrace Davis AI engine.
On this publish, we describe the Kinesis Information Firehose and associated Dynatrace integration.
Conditions
For this walkthrough, you need to have the next stipulations:
- AWS account.
- Entry to the CloudWatch and Kinesis Information Firehose with permissions to handle HTTP endpoints.
- Dynatrace Clever Observability Platform account, or get a free 15 day trial right here.
- Dynatrace model 1.182+.
- An up to date AWS monitoring coverage to incorporate the extra AWS companies.
To replace the AWS Id and Entry Administration (IAM) coverage, use the JSON within the hyperlink above, containing the monitoring coverage (permissions) for all supporting companies. - Dynatrace API token: create token with the next permission and hold available in a notepad.

Determine 1 â Dynatrace API Token
The way it works

Determine 2 â Amazon Kinesis Information Firehose HTTP endpoint supply
Merely create a log stream to your Amazon companies to ship your context wealthy logs to the Amazon CloudWatch Logs service. Subsequent, choose your Dynatrace HTTP endpoint to reinforce your logs streams with the ability of the Dynatrace Intelligence Platform. Lastly, you may as well again up your logs to an Amazon Easy Storage Service (Amazon S3) bucket.
Setup directions
So as to add a service to monitoring, observe these steps:
- Within the Dynatrace menu, go to Settings > Cloud and virtualization, and choose AWS.
- On the AWS overview web page, scroll down and choose the specified AWS occasion. Choose the Edit button.
- Scroll down and choose Add service. Select the service identify from the drop-down, and choose Add service.
- Choose Save adjustments.
To course of and ship AWS CloudWatch Metrics to Dynatrace, observe these steps.
- Log in to the AWS console and kind âKinesisâ within the textual content search bar. Choose Kinesis

Determine 3 â AWS Console
- On the Amazon Kinesis companies web page, choose the radio button for Kinesis Information Firehose and choose the Create supply stream button.

Determine 4 â Amazon Kinesis
- Select the âDirect PUTâ from the drop down, and from Vacation spot drop down, select âDynatraceâ.

Determine 5 â Amazon Kinesis Information Firehose
- Supply stream identify â Give your stream a brand new identify, for instance: â âKFH-StreamToDynatraceâ

Determine 6 â Supply stream identify
- Within the part âVacation spot settingsâ:

Determine 7 â Vacation spot settings
- HTTP endpoint identify â âDynatraceâ.
- HTTP endpoint URL â From the drop down, choose âDynatrace â USâ.
- API token â Enter Dynatrace API TOKEN created within the prerequisite part.
- API URL â enter the Dynatrace URL to your tenant, for instance: https://xxxxx.reside.dynatrace.com
- Again Up Settings â Both choose an current S3 bucket or create a brand new one and add particulars and choose the Create supply stream button.

Determine 8 â Backup settings
As soon as profitable, your AWS Console will appear like the next:

Determine 9 â Amazon Kinesis Information Firehose
The Dynatrace Expertise
As soon as the preliminary setups are accomplished in each Dynatrace and the AWS Console, observe these steps to visualise your new KHF stream knowledge within the Dynatrace console.
- Log in to the Dynatrace Console, and on the left facet menu develop the âinfrastructureâ part, and choose âAWSâ
- From the display screen, choose the AWS account that you just need to add the KFH stream to.
- Subsequent, youâll see a virtualization of your AWS property for the account chosen. Choose the field marked âSupporting Providersâ.
- Subsequent, press the âConfigure companiesâ button.
- Subsequent, choose âAdd serviceâ.
- From the drop down, choose âKinesis Information Firehoseâ.
- Subsequent, choose the âAdd metricâ button, and choose the metrics that you just need to see for this stream. Dynatrace has a complete checklist of metrics that may be chosen from the UI. The checklist might be discovered on this hyperlink.
Troubleshooting
- After configuration, load to the brand new KFH stream no knowledge within the Dynatrace Console.
- Verify the Error Logs tab test to guarantee that the Vacation spot URL is right for the Dynatrace Tenant.

Determine 10 â Vacation spot error logs
- Invalid or misconfigured Dynatrace API token or scope isnât correctly set.

Determine 11 â Vacation spot error logs
Conclusion
On this publish, we exhibit the Kinesis Information Firehose and associated Dynatrace integration. As well as, engineers can use CloudWatch Metrics to discover their manufacturing methods alongside occasions in Dynatrace. This gives a seamless, present view of your system (from logs to occasions and traces) in a single knowledge retailer.
To study extra about CloudWatch Service, see the Amazon CloudWatch residence web page. When you’ve got any questions, publish them on the AWS CloudWatch service discussion board.
For those who havenât but signed up for Dynatrace, then you may check out Kinesis Information Firehose with Dynatrace with a free Dynatrace trial.
Concerning the Authors
Erick Leon is a Technical Alliances Sr. Supervisor at Dynatrace, Observability Follow Architect, and Buyer Advocate. He promotes robust technical integrations with a concentrate on AWS. With over 15 years as a Dynatrace buyer, his real-world experiences and classes realized convey helpful insights into the Dynatrace Clever Observability Platform.
Shashiraj Jeripotula (Raj) is a San Francisco-based Sr. Accomplice Options Architect at AWS. He works with varied impartial software program distributors (ISVs), and companions who focus on cloud administration instruments and DevOps to develop joint options and speed up cloud adoption on AWS.