How to Get AWS Data into Splunk (and Pick the Best Way)
Maciek Stopa with Claude Fable 516 min read#splunk, #aws, #hec, #firehose, #aws-add-on, #data-onboarding, #gdi
Getting AWS data into Splunk looks difficult at first: many services, many ingestion methods, and every choice seems to need research. It comes down to two families: push, where AWS delivers to Splunk's HTTP Event Collector, and pull, where the Splunk Add-on for AWS fetches on a schedule. This guide covers every method and the criteria that pick the right one for your case.
This post is the AWS deep dive of my field guide to every way of getting data into Splunk. In the previous post, I covered syslog ingestion and what can go wrong.
Push ingestion methods
Push means AWS delivers the data and Splunk listens. On the Splunk side there is one main thing to set up: the HTTP Event Collector (HEC), an HTTP/HTTPS endpoint that accepts events from anyone holding a valid token. Everything else happens on the AWS side, and that is where the five push methods differ: what kind of data they can carry, how much AWS-side setup they require, and what they do when Splunk is unreachable.
Push 1: Amazon Data Firehose
AWS service
Data Firehosevia CloudWatch
AWS service
CloudWatch LogsData Firehose- Delivery
- confirmed; failures saved to S3
- Scale
- managed by AWS
- You run
- a stream + a backup bucket
Amazon Data Firehose is a managed delivery pipe: you create a stream, point your sources at it, give it your HEC endpoint and token, and AWS runs everything in between. The stream collects incoming records into a buffer and flushes it to Splunk when the buffer fills or a timer runs out; with the default settings (5 MB, 60 seconds) data arrives within about a minute, and setting the timer to zero brings that down to seconds. Firehose also waits for Splunk to confirm each delivery; when the confirmation does not come, it retries and then saves the failed batch to an S3 bucket, so nothing disappears silently.
One pattern worth knowing: many services cannot write to Firehose directly, but almost everything can write to CloudWatch Logs, AWS's default log store, and a subscription filter forwards those logs into Firehose. That is the second row in the diagram above.
Compatible sources
Plus anything that logs to CloudWatch Logs, for example:
Hover or tap a logo to see what it is.
Push 2: Lambda
S3 / CloudWatch Logs / events
Lambda- Latency
- seconds
- Delivery
- no confirmation unless you code it
- You run
- your own code
Lambda is the do-it-yourself version. A function of yours wakes whenever new data appears (an object lands in S3, a log batch arrives from CloudWatch, an event hits a bus) and your code POSTs the records to HEC. Splunk used to publish ready-made functions for the common cases, but that repo has been quiet since late 2024, so whatever you deploy becomes yours to maintain. And unlike Firehose, Lambda gets no confirmation that Splunk indexed anything unless you write that check yourself.
Compatible sources
Plus dozens of other triggers:
Hover or tap a logo to see what it is.
Push 3: Kinesis Data Streams + Lambda
Producers
Kinesis Data StreamsLambda- Latency
- seconds
- Scale
- grows with shards
- You run
- a stream + your code
At higher volumes, you can put a buffer in front of your Lambda function. Kinesis Data Streams is AWS's durable, ordered stream: producers append records, the stream stores them in shards (parallel lanes, each keeping its records in order), and Lambda reads batches off each lane and forwards them to HEC. The buffer absorbs traffic spikes, lets you replay recent data after a downstream failure, and keeps producers writing happily even while Splunk is down.
Compatible sources
Hover or tap a logo to see what it is.
Push 4: EventBridge
Service events
EventBridgeor forwarded
Service eventsEventBridgeFirehose or Lambda- Latency
- seconds
- Delivery
- retried up to 24 h, then DLQ
- Fits
- alerts and findings
EventBridge carries events, not log streams. Security services in particular (GuardDuty, Security Hub, Inspector, Macie) publish their findings to this bus natively, so the data is already there; you only add a rule that sends matched events to Splunk. The direct route is an API destination: EventBridge POSTs to HEC itself, with your token in an authorization header. Or the rule hands events to Firehose or Lambda and takes the paths above: Firehose when you want its buffering and the S3 backup of failed deliveries, Lambda when events need reshaping before they reach Splunk. The direct route is the least to build and fine for modest volumes. The default limits (300 events per second, 1 MB per event) tell you what it is built for: alerts and findings, not gigabytes of raw logs.
Compatible sources
Plus many more AWS services:
Hover or tap a logo to see what it is.
Push 5: Data Inputs
CloudWatch LogsFirehose, deployed for you- Latency
- same as Firehose underneath
- Delivery
- confirmed (Firehose underneath)
- You run
- one CloudFormation deploy
- Platform
- Splunk Cloud only
If you run Splunk Cloud, you may not need to assemble any of this yourself. Data Inputs (until recently called Data Manager) is a built-in Splunk Cloud app that builds the push chain for you: you answer a few questions in Splunk Web, it generates CloudFormation templates, and running those templates in your AWS account creates the Firehose pipeline fully wired; collection starts as soon as AWS makes the data available. Splunk's admin manual calls it the best practice for the sources it supports.
The trade: someone must be allowed to deploy CloudFormation stacks in every data account.
Compatible sources
Hover or tap a logo to see what it is.
Pull ingestion methods
Pull turns the arrow around: nothing in AWS sends anything; Splunk reaches in and fetches on a schedule. The fetching is done by the Splunk Add-on for AWS, and the add-on needs a Splunk instance to run on. Self-managed deployments give it a heavy forwarder: a Splunk server dedicated to collecting data rather than searching it. Splunk Cloud can provide the instance instead: on Victoria Experience you install the add-on on the search head tier yourself, and on Classic Experience Splunk Support installs it on an Inputs Data Manager, however, a heavy forwarder of your own still works with Splunk Cloud.
Pull 1: API polling
Service APIs- Latency
- your polling interval
- Scope
- one input per service
- Ceiling
- AWS API rate limits
- Runs on
- a heavy forwarder, or inside Splunk Cloud
The oldest inputs call an AWS API on a timer, one service each: the metadata input walks the describe APIs for an inventory of your resources, the Inspector input fetches findings, the Config Rules input reads the compliance state of your rules, the CloudTrail Lake input runs SQL queries against an event data store, and the Billing input collects your Cost and Usage Reports, by default once a day. Setup is a few clicks, which made per-service inputs popular, but they inherit every API's rate limits, and the docs steer the biggest sources to other methods: the CloudTrail page calls SQS-based S3 "a more fault-tolerant and higher-performing alternative", and the CloudWatch Logs page advises against API pulls entirely.
Compatible sources
Hover or tap a logo to see what it is.
Pull 2: SQS-based S3
S3
SQSfan-out variant
S3
SNSSQS- Latency
- minutes; 300 s default interval
- Delivery
- at least once
- Throughput
- ~2 TB/day per input
- Runs on
- heavy forwarders, or inside Splunk Cloud
The workhorse of the pull family, built on a simple pattern. Most AWS services can deliver their logs as files into an S3 bucket, and a bucket can announce every new file as a notification into an SQS queue, either directly or through an SNS topic that fans the same notification out to several queues at once (useful when a second consumer, like another SIEM, needs the same events). The input reads the queue and downloads only the files that were announced, so nobody ever scans the whole bucket.
The design has two properties worth paying for. It is stateless: several inputs can share one queue, and if one dies, the others keep collecting without missing a message. And it is fast: Splunk's own tests moved roughly 2 TB/day through a single input.
Compatible sources
Hover or tap a logo to see what it is.
Pull 3: CloudWatch metrics
CloudWatch metrics- Granularity
- 60 s at best
- Throughput
- slowest input, 240 metrics/s
- AWS cost
- every API call billed
- Runs on
- a heavy forwarder, or inside Splunk Cloud
CloudWatch metrics are time series: numeric measurements such as CPU utilization, queue depth, or request counts, sampled at fixed intervals for every AWS resource. This input polls the CloudWatch API for them, at best one data point per 60 seconds.
Compatible sources
Plus every service that publishes metrics:
Hover or tap a logo to see what it is.
Pull 4: Kinesis Data Streams
Kinesis Data Streams- Latency
- near real time
- Consumers
- exactly one input
- Streams
- provisioned mode only
- Runs on
- a heavy forwarder, or inside Splunk Cloud
The add-on can also read a Kinesis stream directly: the pull twin of Push 3, with the add-on in place of the Lambda. In practice its documented limits make it a niche pick: records must be gzip or plain text, streams in on-demand mode are not supported, and "Multiple inputs collecting data from a single stream cause duplicate events in the Splunk platform", so it is exactly one input per stream, with no failover partner.
Compatible sources
Hover or tap a logo to see what it is.
Pull 5: DB Connect (DBX)
RDS / Redshift- Latency
- your query schedule
- Progress
- remembers the last collected row
- Drivers
- one JDBC add-on per engine
- Runs on
- a heavy forwarder or Victoria search head
One kind of AWS data never appears in any log stream: rows sitting in a database. DB Connect (4.3.0 as of May 2026) covers it by running scheduled SQL over JDBC, from a heavy forwarder or a Victoria search head, with a driver add-on per database engine.
Compatible sources
Plus other supported engines:
Hover or tap a logo to see what it is.
Pull 6: Data Inputs
S3SQS- Latency
- configurable polling interval
- Delivery
- at least once
- You run
- bucket-to-SQS notifications (prerequisite)
- Platform
- Splunk Cloud only
Data Inputs closes the circle: the same Splunk Cloud app also manages pull inputs, for sources that live in S3. Here the AWS-side plumbing is a prerequisite, not something it deploys: you wire the bucket notifications into an SQS queue first, then Splunk Cloud consumes the queue. The one-input-per-account limit from the push section applies here too.
Compatible sources
Hover or tap a logo to see what it is.
How to choose
With the catalog in hand, let's explore the criteria that decide which method to pick.
Best practices, and when to ignore them
Splunk's Validated Architectures (SVA), the company's official deployment blueprints, are unusually opinionated about AWS:
- The add-on is "Not recommended for data volumes greater than 1 TB/day".
- Push ingestion has "Lower latency data ingest than the pull approach", while "Pull approach uses scheduled data retrievals which introduces ingestion latency".
- For Splunk Cloud, Data Manager (now Data Inputs) "uses the best-practice approach for data inputs".
Lantern, Splunk's guidance library, gives the same advice per source:
- Push CloudWatch-based sources to HEC (Data Inputs deploys the chain).
- Pull S3-based sources from their buckets through SQS.
These are defaults, not rules. When your constraints point elsewhere, pick the method that fits them. A no-public-endpoints policy can override the push preference. Cost sensitivity can override latency (numbers below). A requirement to replay history can override both, because it favors S3 storage over a transient pipe. Best practices optimize Splunk's median customer, not your constraints.
Cost
An apples-to-apples comparison across paths is not honestly possible. The S3 path includes durable storage and replay, Firehose includes transport and backup only, and the Kinesis path includes a buffer you may not need. The calculator below prices what each path actually deploys, on these assumptions:
| Assumption | Value | Source |
|---|---|---|
| Firehose ingestion | $0.029 per GB ingested (first 500 TB/month, billed in 5 KB increments) | pricing page |
| Firehose decompression (CloudWatch Logs) | $0.00325 per GB decompressed (CloudWatch Logs) | pricing page |
| CloudWatch Logs ingestion | $0.5 per GB ingested (Standard log class) | pricing page |
| S3 storage | $0.023 per GB-month (S3 Standard, first 50 TB) | pricing page |
| S3 PUT requests | $0.005 per 1,000 PUT requests | pricing page |
| SQS requests* | $0.4 per million requests (standard queue, after the free million) | pricing page |
| Lambda requests | $0.2 per million requests | pricing page |
| Lambda duration | $0.0000166667 per GB-second (x86) | pricing page |
| Kinesis on-demand ingestion | $0.08 per GB in (on-demand) | pricing page |
| Kinesis on-demand stream-hours | $0.04 per stream-hour (on-demand) | pricing page |
| Data transfer to Splunk Cloud | $0.01 per GB, forwarder to a same-region Splunk Cloud endpoint | AWS scenarios post |
| NAT gateway hours | $0.045 per NAT gateway hour | pricing page |
| NAT gateway data processing | $0.045 per GB processed by the NAT gateway | pricing page |
| Heavy forwarder EC2 (m7i.large)* | $74 per month, m7i.large on-demand (heavy forwarder reference box) | pricing page |
| Billing month | 30 days | model assumption |
| S3 objects (pull path) | 8 MB average, 3 SQS requests per object | model assumption |
| S3 retention (pull path) | 30 days | model assumption |
| Kinesis mode | on-demand | model assumption |
| Lambda batches (Kinesis path) | 5 MB per invocation, 512 MB memory, 2 seconds | model assumption |
| NAT gateway uptime | 730 hours per month | model assumption |
| NAT gateway scope (when ticked) | egress leg only; S3 downloads use a free gateway endpoint | model assumption |
All prices us-east-1, checked 2026-08-12. Prices marked * come from JavaScript-rendered pricing pages; verify them in calculator.aws before budgeting.
Not modeled:
- Splunk license, SVC, or workload cost: identical data volume prices differently on the Splunk side per contract, so it stays out.
- Lambda-to-HEC egress on the Kinesis path, billed at EC2 data transfer rates. Firehose delivery carries no data transfer charge at all, so the push paths have no transfer line to model. (AWS scenarios post)
- Cross-region transfer, CloudWatch Logs storage, S3 lifecycle transitions, PrivateLink endpoints, and replay reads.
| Path | AWS cost / month | Breakdown |
|---|---|---|
| Firehose push | $44 | Firehose ingestion: $44Billed in 5 KB increments, so many tiny records inflate the effective rate. No data transfer charge applies to delivery. |
| CloudWatch Logs + Firehose | $798 | CloudWatch Logs ingestion: $750 · Firehose ingestion: $44 · Firehose decompression: $4.88CloudWatch ingestion dominates. If the logs land in CloudWatch anyway, treat it as sunk cost and untick it. A log group mixing sourcetypes or indexes adds a transform Lambda hop, about $5/month at 50 GB/day. |
| S3 + SQS pull (add-on) | $125 | S3 storage (30-day retention): $35 · S3 PUT requests: $0.96 · SQS requests*: $0.23 · Data transfer to Splunk Cloud: $15 · Heavy forwarder EC2*: $74If the logs were landing in S3 anyway, untick the storage line: the bucket is sunk cost. One m7i.large is a floor reference; the published add-on benchmarks used a 16 vCPU box. On Splunk Cloud the forwarder line does not apply: the add-on runs on the search head tier (Victoria) or an IDM (Classic). |
| Kinesis + Lambda push | $154 | Kinesis on-demand ingestion: $120 · Kinesis stream-hours: $29 · Lambda requests + duration: $5.06On-demand stream pricing; provisioned shards are cheaper at steady, predictable volume. |
Illustrative numbers from public us-east-1 prices, checked 2026-08-12, on the assumptions above. Not a quote.
CloudWatch is the expensive middle stop. CloudWatch Logs Standard ingestion costs $0.50 per GB; Firehose Direct PUT ingestion costs $0.029 per GB. Route logs through CloudWatch on the way to Splunk and you pay both prices on the same bytes. Send sources straight to Firehose or S3 when they support it, and treat CloudWatch ingestion as sunk cost only when the logs must land there anyway.
The pull path's real cost is the fleet, not AWS. The S3 plus SQS path's AWS-side cost is storage plus a cent per GB of transfer; what it really costs is the heavy forwarder fleet and its care. That bill lands on self-managed deployments. Splunk Cloud runs the add-on on the search head tier (Victoria) or an Inputs Data Manager (Classic), so untick the forwarder line there.
AWS-only cost rankings flip the ordering. Guidance that ranks paths by AWS charges alone assumes the bucket and the forwarders already exist, and under those assumptions the pull path is the cheapest option of all. The calculator prices the full deployment instead; tick "AWS-only view" to reproduce that ranking.
NAT gateways tax private-subnet pullers. AWS prices the forwarder-to-Splunk-Cloud leg at $0.01 per GB in the same region; a NAT gateway adds $0.045 per processed GB, which takes that leg to 5.5 times the base rate. AWS's own guidance documents no additional data transfer charge on the Firehose path.
Performance and scale
On the pull side, the add-on publishes measured numbers, and the input you choose moves the ceiling more than threefold:
Splunk's own tests on a 16 vCPU heavy forwarder; a rough reference, not a guarantee. CloudWatch metrics polling is not on the chart: Splunk measures it as 240 metrics/s in the same test and publishes no byte rate for it. Source: add-on performance reference.
Scaling past one input is horizontal: "optimal performance throughput is reached when running 4 inputs on a single heavy forwarder instance", and beyond that you add heavy forwarders consuming the same SQS queue. The same page publishes a multi-input table showing what that buys: up to 45,000 KB/s per forwarder, roughly 3.9 TB/day.
Firehose publishes no faster number. What changes is who does the scaling work:
- A Direct PUT stream starts at 5 MiB/s in us-east-1, us-west-2, and eu-west-1 (1 MiB/s elsewhere): about 450 GB/day, below every bar in the chart above. The difference is what happens at the ceiling. Firehose raises its own limit when throttled; a forwarder at its ceiling waits for you to provision another forwarder.
- With a Kinesis stream as the source, the quota does not apply at all: Firehose "scales up and down with no limit".
- Splunk publishes no per-HEC-endpoint throughput figure; the documented scaling answer is HEC on the indexers themselves, and Firehose delivers to Splunk clusters behind a Classic or Application Load Balancer.
Latency
Only documented figures below; the real end-to-end number is the sum of a path's stages.
| Path stage | Documented latency |
|---|---|
| Firehose Splunk destination | Buffers at most 60 seconds |
| CloudWatch Logs subscription | Delivery "usually less than three minutes" |
| Add-on SQS-based S3 input | 300-second default interval between collection runs |
| Add-on generic S3 input | 1,800-second default interval |
| CloudWatch metrics | 60-second minimum granularity; 1-minute EC2 metrics require paid detailed monitoring |
| VPC Flow Logs (source-side) | 10-minute default aggregation, 1-minute optional |
| AWS Config (source-side) | History files delivered to S3 every six hours |
The pattern: push paths are bounded in seconds to minutes, pull paths in minutes to tens of minutes, and the source's own delivery interval can dwarf both.
Delivery guarantees
HEC answers 200 on receipt, but receipt is not indexing. Indexer acknowledgements close that gap, with three catches: every request must carry a channel identifier, the sender must poll for ack status itself, and "Amazon Data Firehose is the only supported sending client for HEC tokens with indexer acknowledgement activated in the Splunk Cloud Platform".
- Firehose is the only push path with end-to-end acknowledgements. It waits for the HEC ack, retries within your configured window, and then backs unacknowledged data up to S3 rather than dropping it. The same page is honest about the flip side: after an ack timeout "it's possible the data was indexed successfully in Splunk", so retries can duplicate. At-least-once is the documented contract.
- CloudWatch Logs subscriptions retry for up to 24 hours on retryable errors, but a non-retryable error disables the filter for up to 10 minutes and "logs are skipped".
- Lambda paths retry by invocation. Async invocations get two retries plus an optional dead letter queue; Kinesis-triggered functions retry until the record expires and stall their shard meanwhile. What Lambda cannot give you without custom code is confirmation that Splunk indexed anything.
- SQS-based S3 pull survives collector death by design. Unprocessed messages reappear after the visibility timeout and another input picks them up; the cost, again, is duplicates.
Plan for duplicates on every path and let the S3 backup or DLQ be the replay source, not a rerun of the pipeline.
Gotchas that rule options out
Each one either rules methods out or adds lead time you must plan for.
- Firehose needs a public HEC endpoint. A Splunk platform in a VPC "must be publicly accessible with a public IP address", with Firehose's published CIDR ranges allowed. PrivateLink does not change this for Firehose: it covers producers calling the Firehose API, not delivery from Firehose to Splunk. A strict no-public-endpoints policy therefore rules out the Firehose family, Data Inputs push included; Data Inputs pull reads from SQS and S3 instead of delivering to HEC, so it stays available. Pull is not the only option left, though. A VPC-attached Lambda still reaches HEC privately: in the same VPC for your own deployment, or through Splunk Cloud private connectivity, which carries forwarder and HEC ingest over AWS PrivateLink. HEC can also terminate on heavy forwarders inside your VPC that then forward to the indexers. So the choice becomes Push 2 or Push 3 with code you own, or pull inputs over private networks or VPC endpoints.
- Splunk Cloud Firehose enablement is a support ticket. Splunk Support has to enable HEC for Firehose use on your stack; plan the lead time.
- Some orgs will not sign off the AWS-side footprint. Data Inputs push needs CloudFormation deploy rights in every data account, and security teams that treat SIEM tooling as read-only will refuse. There is an authentication catch on Splunk Cloud too: on your own EC2 machine the add-on can sign in to AWS without any stored secret, through the IAM role attached to the machine, but Splunk Cloud restricts that option "due to security reasons". Pull inputs there sign in with stored IAM access keys, long-lived credentials that some security teams also refuse.
- API rate limits are a real ceiling on the pull side. The add-on's own sizing doc walks through CloudWatch metric counts that exceed the default API rate limit, and its troubleshooting doc has a dedicated procedure for finding throttling in the internal logs.
All product names, versions, quotas, prices, and links above come from official public documentation and were verified on 12 August 2026. Diagrams use AWS Architecture Icons to refer to AWS products. If you spot drift (docs and prices move fast), tell me and I'll fix it.