The last UI-driven tier in your stack
Most platform engineering teams went GitOps for everything in the 2020s. Application code in a repo. Infrastructure in Terraform. Secrets in Vault with a code-driven access control surface. RBAC in YAML manifests. Even runbooks in markdown alongside the code they describe. By 2026, the question for any non-trivial engineering organization is not whether GitOps applies, but whether anything in the stack is still being clicked instead of declared.
Observability tends to be the last holdout. SIEM dashboards, parser configurations, detection rules, and pipeline routing logic still live inside vendor UIs in most environments, with no audit log, no diff history, no test suite, and no automated rollback. The team that just spent eighteen months migrating their service mesh, their database schemas, their CI runners, and their feature flags to fully declarative tooling will then point an analyst at a Splunk web console and ask them to tweak the parser for the new firewall format by Tuesday.
The Cribl Stream layer changes that, but only if it is implemented as code. The Cribl Terraform provider, combined with Cribl's Git-backed worker group support, lets the entire pipeline tier live in a repository the same way the application stack already does. Worker groups, routes, pipelines, sources, destinations, schemas, and detection enrichment all become resources that PRs review, CI tests, version control tracks, and disaster recovery rebuilds in minutes instead of weeks.
This post walks through what specifically changes when Cribl pipelines move from the UI to Git, why the Terraform provider is the load-bearing piece, how AI-assisted pipeline maintenance becomes possible once the configuration is declarative, and what disaster recovery looks like when the entire pipeline tier is reproducible from code.
Why observability ended up as the UI-driven holdout
Observability ended up as the last UI-driven tier in most stacks for a reason. The earliest SIEMs (Splunk in the early 2000s, QRadar around the same era, ArcSight) were built as monolithic appliances with proprietary configuration formats. Their administration model assumed an operator clicking through a web console. By the time platform engineering culture caught up to demand declarative config for everything, those vendors had decades of UI-first muscle memory baked into their products and their customer enablement materials.
The cost of that legacy shows up in three places.
First, configuration drift. Two environments that started identical diverge as different operators make different small adjustments at different times. Without a Git history, the diff is invisible until production breaks.
Second, audit trail. Compliance frameworks (SOC 2, ISO 27001, PCI-DSS, HIPAA) increasingly expect a defensible record of every configuration change, who made it, when, and what was reviewed. UI-driven configuration produces no such record without third-party SIEM-of-the-SIEM tooling.
Third, change review. A new detection rule pushed straight into production through the UI cannot be peer-reviewed. There is no PR. The engineer who clicked save and the engineer who would have caught the regex bug are not in the same workflow.
The Cribl Stream layer inherits the platform-engineering era rather than the early-SIEM era. Configuration is structured (JSON under the hood), addressable via API, and increasingly managed declaratively. The Terraform provider is the formal expression of that direction. Once it is in place, observability stops being the last UI-driven holdout. The pipeline tier joins the rest of the stack in a single PR cadence, one CI suite, one disaster recovery story, one access control model, one history.
The Terraform provider is the load-bearing piece
The official Cribl Terraform provider, published on the HashiCorp Terraform Registry, exposes the major Cribl Stream resources as first-class Terraform resources. Worker groups, pipelines, routes, sources, destinations, lookups, and parsers all map to resource types. State is stored wherever the rest of the team's Terraform state lives, typically S3, Azure Blob, or HCP Terraform, under the same access control as the rest of the infrastructure.
In practice, the layout we deploy in production for multi-team organizations looks like this. A dedicated cribl/ repository or directory at the root of the existing infrastructure monorepo. Inside that, an environments/ subdirectory with one folder per environment (dev, staging, prod, plus any per-region or per-tenant variants). A modules/ subdirectory that contains reusable Terraform modules for common pipeline patterns: a "syslog ingest with reduction and replay" module, a "Microsoft Graph audit log enrichment" module, a "PII masking via Guard" module. Each environment's root module composes those reusable modules with environment-specific variables.
The Terraform resources cover the structural surface. Worker group definitions including capacity, region, and network configuration. Routes that determine which pipelines a given event traverses based on source, type, and metadata predicates. Pipelines that reference function libraries, parser definitions, and downstream destinations. Sources and destinations with their authentication, throttling, and connection parameters. Lookups referencing CSVs or external services. The configuration that sits inside individual Cribl functions (the regex transformations, the field renames, the conditional drops) is structured enough that it round-trips cleanly through Terraform's HCL.
A representative pipeline-creation block in HCL looks like this in shape. A resource "cribl_pipeline" "firewall_reduction" block with attributes for the pipeline's name, the source filters, the destinations, and a list of function blocks each with their own configuration. The function blocks reference reduction functions, schema-normalization functions, OCSF mapping functions, and Guard functions for PII masking. When the team needs to add a new firewall vendor's source, the change is a PR that adds one new module instantiation. The reviewer sees the diff. The CI runs terraform plan against a non-production worker group. The merge triggers terraform apply against staging, then production, gated on whatever approval cadence the team prefers.
The benefits compound. Configuration drift between environments is no longer possible because every environment renders from the same modules with different variables. Detection logic, parser rules, and routing decisions are all in the diff history, attributable, peer-reviewed, and reversible. New team members onboard by reading the repo, not by being granted UI access and shadowing a senior engineer. The Cribl tier joins the rest of the platform engineering surface area on the same cadence and the same review model.
The Terraform provider also opens up state-as-source-of-truth workflows. terraform import can pull existing UI-managed Cribl configurations into Terraform state in a one-time migration, after which the Terraform definition becomes authoritative. Drift between the live system and the declared state is detected on every plan run. The team that has wanted observability config in Git for years gets a defensible path to it, executed in days rather than the multi-quarter projects that platform engineering migrations usually become.
Git-backed worker groups: the second layer
Alongside the Terraform provider, Cribl supports Git-backed worker group configuration natively. The worker group's configuration directory can be sourced from a Git repository. The Cribl Leader pulls from the configured Git remote on a defined cadence or on push, validates the configuration, and deploys to the worker group without UI involvement. This is distinct from the Terraform provider but complementary.
The two layers cover different scopes. Terraform handles the structural infrastructure: which worker groups exist, where they run, what their network policies are, which routes feed which pipelines, what the destinations look like. Git-backed worker groups handle the internal configuration of an individual worker group: the precise function chain inside a pipeline, the parser regex, the lookup table referenced by an enrichment step. Both layers are version-controlled. Both layers go through PR review. Both layers run through CI before reaching production.
A practical division of ownership: the platform engineering team owns the Terraform layer (worker groups, network, scaling, infrastructure). The detection engineering or SOC team owns the Git-backed worker group repos (pipeline internals, parser logic, detection enrichment). The two teams operate on the same change-review cadence with the same review tooling, but they can move at different velocities for changes that affect only their own scope.
Combined, the two layers eliminate the last reasons a Cribl administrator would need to log into the UI for routine work. The UI remains useful for live debugging, real-time data-flow inspection, and one-off investigations, but configuration changes flow through the repository. The shift in working pattern is the same shift that GitOps introduced to application deployment over the last decade, applied to the observability tier.
Disaster recovery as a deploy, not a rebuild
Disaster recovery for the pipeline tier matters more than most teams budget for, because the pipeline is the primary visibility surface of the entire security and observability stack. If the SIEM is down but Cribl is alive, the pipeline buffers events and replays them. If Cribl is down, the SIEM loses its source. The agents on every endpoint are pointing at Cribl. If Cribl is unrecoverable, every endpoint has to be reconfigured, and the visibility gap during the reconfiguration is exactly when an attacker would want to move.
Under a UI-managed configuration, disaster recovery is a manual rebuild. The runbook is "restore from backup" if the backup is current, or "rebuild from screenshots and tribal knowledge" if it is not. RTO is measured in days for an organization of any complexity. The team does not get to test disaster recovery routinely because the rebuild process is manual and time-consuming, and untested DR is no DR at all.
Under a code-managed configuration, disaster recovery is a deploy. terraform apply against a fresh region with the same module set produces an identical Cribl Stream stack. Worker groups stand up. Routes activate. Pipelines load. Sources and destinations reauthenticate against their respective endpoints. RTO is measured in minutes, gated mostly by Cribl Leader provisioning and DNS propagation. The team can test DR weekly because a DR drill is a CI job, not a war room. Recovery confidence becomes empirical rather than aspirational.
The 2026 angle on this is that DR drills are increasingly a regulatory requirement, not just a best practice. SEC cybersecurity disclosure rules, NIS2 in Europe, the updated PCI-DSS 4.0 framework, and most cyber insurance policies all expect documented, tested recovery procedures with measured RTO and RPO numbers. Producing those numbers requires actually running the procedures. Code-managed DR turns a quarterly compliance burden into a CI job that runs on a schedule and reports the results.
The same architecture covers a less dramatic but more frequent case. Spinning up a new environment for a new client, a new region, or a new compliance boundary is the same operation as a DR rebuild. Each of those cases historically takes weeks because the configuration has to be manually replicated. With the pipeline tier as code, each one takes a terraform apply against a new variable set.
AI-assisted pipeline maintenance
The strongest argument for pipeline-as-code in 2026 is that AI coding assistants can read it, modify it, test it, and propose changes against it. The vendor UI, by contrast, is opaque to every AI tool on the market. Copilot cannot click. Cursor cannot navigate a Cribl admin console. Claude Code cannot adjust a regex inside a UI text field. Anything that requires a human to manually interact with a vendor's web interface excludes itself from the AI-augmented engineering workflows that have become standard for everything else.
Once the Cribl configuration is in HCL and Git-backed YAML, every AI-assisted workflow that the team uses for application code becomes available for the pipeline tier. A new firewall vendor's onboarding becomes "Copilot, generate a Cribl pipeline that parses Fortinet syslog format following the same pattern as the Palo Alto pipeline in modules/firewalls/palo-alto/". A request to add PII masking on a specific field becomes a PR that the AI assistant drafts and the engineer reviews. An incident that requires an urgent routing change becomes a chat-driven Terraform diff that the on-call engineer reviews and applies, with the entire change history preserved in Git rather than in the chat log.
The asymmetric advantage is that the pipeline tier is the most repetitive configuration surface in the entire security and observability stack. Onboarding new sources, adjusting parsers for vendor format changes, updating PII rules for new compliance requirements, tweaking routing decisions for new destinations. All of these are pattern-matched work that AI assistants are exceptionally good at. A team with pipeline-as-code can reasonably expect significant multipliers on engineering capacity in this area, comparable to what the same teams have already seen on application code.
The team that left the pipeline tier in the UI loses access to all of that. They are still doing 2018-era observability administration in a 2026 platform engineering culture. The skill expectation for a senior platform engineer in 2026 includes AI-augmented workflows. UI-only work is a self-imposed handicap.
Testing pipelines: what CI looks like
Pipelines as code unlock the same testing apparatus as application code. The minimum baseline is terraform validate and terraform plan running on every PR. Validate catches HCL syntax errors and type mismatches before they reach the live system. Plan shows the exact resource diff a merge would apply, which is a precondition for meaningful review.
One layer up, a dedicated CI pipeline runs the Terraform plan against a non-production Cribl Leader, applies it to a sandbox worker group, and runs synthetic events through the pipeline using a corpus of representative source data. The synthetic event corpus lives in the same repository as the pipeline definitions. Tests assert that the pipeline produces the expected output: correct field names, correct enrichment, correct routing decisions, correct PII masking. Failures block the merge.
A second layer handles regression testing. When a pipeline change is proposed, CI runs the new pipeline against a corpus of historical events from production (anonymized or sampled appropriately for the development environment), measures the divergence between old and new outputs, and flags any non-trivial behavioral change. If a routing change unexpectedly drops events that previously routed to the SIEM, the test fails before merge. The detection engineering team catches the regression in CI rather than during a real incident.
A third layer applies for security-critical pipelines. Schema validation enforces that every event produced by the pipeline conforms to the OCSF, ECS, or ASIM schema defined for the destination. Detection-rule contract tests confirm that the field names and types the SIEM detection rules depend on are still emitted by the pipeline. If a pipeline change would break a downstream detection rule, the test fails and the rule's owner is notified before the change is applied.
None of this testing apparatus is available for UI-managed Cribl configurations. A pipeline change in the UI takes effect immediately, with no automated regression check, no schema validation, no detection-rule contract test. The first signal of a regression is a missed alert in production. Testing in production has never been a defensible position for application code. It is no more defensible for the pipeline tier that feeds detection.
How to start
The fastest path from a UI-managed Cribl deployment to a code-managed one is incremental. Pick a single worker group with a stable configuration. Run terraform import against each of its resources to capture the existing state in Terraform. Commit the resulting HCL to a new repository. Add a CI workflow that runs terraform plan on every PR and terraform apply on merges to the main branch.
For the first few weeks, the team operates in a mixed mode. New configuration changes go through PRs. Existing UI-driven workflows continue for changes that have not been migrated yet. Drift between the live system and the Terraform state is reviewed on every plan run and reconciled either by codifying the drift (the UI change becomes part of the Terraform definition) or rolling back the drift (the live system is brought back in line with the declared state). After two to four weeks of this, most teams choose to disable UI write access for non-emergency changes, which forces all routine work through the PR cadence.
From there, expand the scope. Migrate the next worker group. Add the AI-assisted workflows. Add the test layers. The architecture is the same as the team's existing GitOps cadence for application code, applied to the pipeline tier.
Related reading
Pipeline-as-code is one slice of the broader 2026 observability architecture shift. If you are working through adjacent ground, these blogs cover neighboring pieces in more detail.
- Why Companies Are Buying Observability Pipelines Without a Cost Problem covers the strategic case for adopting a pipeline layer in the first place, focused on the buying motive that does not lead with cost reduction.
- Anatomy of a Two-Week SIEM Migration covers the license-overlap math that the Terraform-managed pipeline architecture changes during a parallel migration. Once the pipeline is in code, dual-writes across SIEMs are a routing-rule diff in a PR.
- Cribl Reduction Is Not Data Loss: A Replay Strategy Your SIEM Cannot Offer explains the open-format archive mechanics that make pipeline-led DR viable, with detailed cost ratios between the SIEM analytics tier and object storage.
On the platform side, the Telemetry Modernization solution page covers the Cribl Edge plus Stream architecture from a platform engineering perspective. The Vendor Lock-In solution page explains how the same code-driven pipeline architecture decouples telemetry from any single SIEM contract. The Cost Optimization solution page lays out the routing model across SIEM, APM, and object storage tiers in full.
The discovery call
If your platform engineering team has already gone GitOps for everything else and the pipeline tier is the last UI-driven holdout, the right time to model the transition is before the next vendor consolidation or audit forces it. A thirty-minute call is usually enough to scope a single-worker-group Terraform import as the proof point and identify the modules that would compose into a multi-environment layout.
Schedule a discovery call and we will walk through the Terraform-provider layout against your existing pipeline footprint, scope the CI test layers that match your detection content, and identify the first DR drill the team should run once the configuration is in code.
The pipeline tier deserves the same engineering rigor as the rest of the stack. The tools to give it that rigor have been available for a while. The decision to use them is what separates teams that ship observability changes through PRs from teams that still click through UIs.