> ## Documentation Index > Fetch the complete documentation index at: https://docs.visiqlabs.com/llms.txt > Use this file to discover all available pages before exploring further. # Stream logs to Elasticsearch > Continuously ship your governance decision and audit logs to your own Elastic deployment with a least-privilege API key. VisIQ can forward your governance **decision** and **audit** logs into your own Elasticsearch deployment as they happen, indexed as ordinary log documents you can search, alert on and retain under your existing policy. Set it up at **Connectors → Log Streaming → Elasticsearch**. It takes about three minutes. *** ## Before you start * An Elasticsearch deployment reachable over **https from the public internet**, and Kibana access to create an API key. * A VisIQ account with the **`settings:update`** permission — the log-destination endpoints are guarded by it. Streaming starts from the moment you save. There is **no historical backfill** — logs recorded before the destination existed stay in VisIQ's audit trail and are not replayed into Elastic. *** ## Step 1 — Create a least-privilege API key In Kibana, go to **Stack Management → Security → API keys → Create API key**. Turn on **Control security privileges** and paste this role descriptor: ```json Elastic role descriptor theme={null} { "visiq_ingest": { "cluster": [ "monitor" ], "indices": [ { "names": [ "logs-*-*" ], "privileges": [ "auto_configure", "create_doc" ] } ] } } ``` Then copy the **Base64 encoded** value — Kibana shows it only once. This is the whole grant, and it is narrow on purpose: | Privilege | What it allows | What it cannot do | | ------------------------------ | ---------------------------------------------------------- | -------------------------------------------------------------------- | | `create_doc` on `logs-*-*` | Add new log documents to your `logs-*` data streams. | Update, overwrite, read, or delete anything already in your cluster. | | `auto_configure` on `logs-*-*` | Let the data stream create itself on first write. | Touch indices outside the pattern. | | `cluster: monitor` | Let the connection check confirm the cluster is reachable. | Any write or admin operation. | If your organisation prefers to provision the key another way, the only requirement is that it can create documents under `logs-*-*`. A key with more privilege than this works but is not needed. *** ## Step 2 — Point VisIQ at your deployment Enter your **Elasticsearch endpoint URL** — the deployment's base URL, for example `https://my-deployment.es.us-central1.gcp.cloud.es.io`. Then paste the encoded API key. The endpoint has to clear two checks: * **In the browser**, a shape check: it must be `https`, and it must not carry embedded credentials (`https://user:pass@host` is refused, because such a URL would be stored in plain text and breaks the HTTP client anyway). * **On the server**, the authoritative one: the host is DNS-resolved and rejected if it lands in a private, reserved or link-local range. This runs at the route boundary **and again immediately before every single send**, so an endpoint that later resolves inward cannot become an egress path. A self-hosted cluster therefore has to be reachable at a public address; an endpoint on a private network cannot be used. **Changing the endpoint host requires re-entering your API key.** VisIQ refuses to ship a stored credential to a host it was not issued for, so editing the endpoint to a different host locks Save until you paste a fresh key and it verifies against the new host. Changing the path or port of the same host does not trigger this. *** ## Step 3 — Choose what to stream Pick at least one stream. Both are on by default. | Stream | Contents | | ------------- | -------------------------------------------------------------------- | | **Decisions** | Action, retrieval, and Human-in-the-Loop governance decision events. | | **Audit** | Platform configuration and access audit log. | You can also set an optional **dataset override** (for example `logs-visiq.decisions-default`) if you want the documents to land somewhere other than the default target. *** ## Step 4 — Verify and save There is no **Test connection** button. Once the endpoint and key are in, VisIQ writes a test event automatically and **Save** unlocks when it is verified. If it fails, correct the endpoint or the key and VisIQ re-verifies on its own. Once saved, the connector card shows delivered and failed counts. *** ## How delivery behaves VisIQ writes through Elasticsearch's `_bulk` API with `create` operations and a **deterministic document id** per event. That is what makes retries safe: a redelivered document comes back as a `409` version conflict, which VisIQ counts as already-delivered rather than inserting a duplicate. The response handling is deliberate, because `_bulk` can return `200` while individual documents failed: | Outcome | What VisIQ does | | --------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Document accepted, or `409` conflict | Counted delivered. | | Per-document `429` | Retried — the batch is not advanced past. | | Per-document `400` (mapping or parse error) | Counted as a permanent drop, so one poison document cannot wedge the stream forever. | | Whole-request `401` / `403` / `413` | Buffered with back-off, never dropped — a fixable credential or sizing problem must not cost you logs. | | Whole-request `429` / `5xx` / network failure | Retried. | | `200` with a body VisIQ cannot parse | **Retried, not counted delivered.** An unparseable `200` usually means a proxy answered instead of Elastic, and VisIQ will not attest delivery it cannot confirm. | *** ## The API behind the card `GET` needs `settings:view`; `POST` needs `settings:update`. ```json POST /api/log-destinations theme={null} { "name": "Elastic — production", "type": "elasticsearch", "endpoint_url": "https://my-deployment.es.us-central1.gcp.cloud.es.io", "streams": ["decisions", "audit"], "auth_config": { "api_key": "…" }, "index_name": "logs-visiq.decisions-default", "enabled": true } ``` Note that the persisted type is `elasticsearch` even though the connector card reads "Elasticsearch". `streams` must be a non-empty subset of `decisions` and `audit`. On a later update, omitting `auth_config` keeps the stored key unchanged. *** ## Troubleshooting The API key is wrong, or it lacks `create_doc` on `logs-*-*`. Re-create it with the role descriptor above and make sure you copied the **encoded** value, not the key id. The host resolved to a private, reserved or link-local address. VisIQ only streams to publicly-resolvable endpoints, and it re-checks before every send rather than trusting the value that was validated at save time. You changed the host, so a fresh API key is required. This is intentional — a key issued for one cluster is never shipped to another. Check the recorded error on the destination. Documents rejected with a mapping or parse error are counted as dropped rather than retried forever; everything else is buffered and retried, so the shortfall is usually temporary. *** ## Related The same streams into Datadog, with the intake host derived from your site. InsightIDR Custom Logs, where the webhook URL is the whole credential.