> ## Documentation Index > Fetch the complete documentation index at: https://docs.getbifrost.ai/llms.txt > Use this file to discover all available pages before exploring further. # Log Exports > Offload Bifrost request and response payloads to S3 or GCS object storage while keeping searchable metadata in the logs database. Bifrost's log store can be paired with an **object storage** backend (S3 or GCS) so that large request/response payloads are streamed to durable object storage while the logs database (SQLite or Postgres) keeps only searchable metadata, indexes, and pointers. This keeps the database small and fast, makes payloads cheap to retain for long periods, and lets you query archived traffic from your own data lake. ## How it works `object_storage` is **not** a standalone feature. It is a sub-config of `logs_store`: ``` logs_store ├── enabled ├── type (sqlite | postgres) ├── config (SQLite path or Postgres connection) ├── object_storage ← S3 or GCS payload offload (optional) └── object_storage_exclude_fields ``` Retention is configured separately, at `client_config.log_retention_days` - see [Retention policy](#retention-policy) below. When `logs_store.object_storage` is set: 1. Bifrost writes per-request metadata (timestamps, provider, model, latency, token counts, cost, status, IDs) to the logs database. 2. Large payload fields (request body, response body, streamed chunks, tool call arguments, etc.) are uploaded to the configured bucket under `prefix/`. 3. The database row stores the object key, so the UI and API can fetch the payload on demand. 4. `object_storage_exclude_fields` lets you skip specific fields from offload (for example, when you do not want to retain raw user prompts at all). Only **S3** and **GCS** are supported today. Azure Blob, local filesystem, and data warehouse destinations are not implemented. ## Configuration via `config.json` ### Amazon S3 #### Required IAM Permissions The IAM user or role needs the following permissions on your bucket: ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Sid": "BucketAccess", "Effect": "Allow", "Action": ["s3:ListBucket"], "Resource": "arn:aws:s3:::bifrost-logs" }, { "Sid": "ObjectAccess", "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:PutObjectTagging", "s3:GetObjectTagging" ], "Resource": "arn:aws:s3:::bifrost-logs/*" } ] } ``` ```json theme={null} { "logs_store": { "enabled": true, "type": "sqlite", "config": { "path": "/app/data/logs.db" }, "object_storage": { "type": "s3", "bucket": "env.AWS_S3_BUCKET", "region": "env.AWS_REGION", "access_key_id": "env.AWS_ACCESS_KEY_ID", "secret_access_key": "env.AWS_SECRET_ACCESS_KEY", "prefix": "bifrost/logs", "compress": true }, "object_storage_exclude_fields": [] } } ``` S3 fields (all values support the `env.` indirection to read from environment variables): | Field | Required | Notes | | ------------------- | ----------- | -------------------------------------------------------------------------------------------------------------------------- | | `type` | yes | Must be `"s3"`. | | `bucket` | yes | Target bucket name. | | `region` | yes | AWS region, e.g. `us-west-2`. | | `access_key_id` | conditional | Required for static credentials. Omit when using `role_arn` or the default credential chain (IRSA, instance profile, env). | | `secret_access_key` | conditional | Pairs with `access_key_id`. | | `session_token` | optional | For temporary STS credentials. | | `role_arn` | optional | Assume this role instead of using static keys. | | `endpoint` | optional | Override for S3-compatible stores (MinIO, Cloudflare R2, Wasabi). | | `force_path_style` | optional | Set `true` for most S3-compatible endpoints. | | `prefix` | optional | Object key prefix. Defaults to `bifrost`. | | `compress` | optional | Gzip payloads before upload. Defaults to `false`. | A live example is checked in at `examples/configs/withobjectstorages3/config.json`. ### Google Cloud Storage ```json theme={null} { "logs_store": { "enabled": true, "type": "sqlite", "config": { "path": "/app/data/logs.db" }, "object_storage": { "type": "gcs", "bucket": "env.GCS_BUCKET", "credentials_json": "env.GCS_KEY", "project_id": "env.GCP_PROJECT_ID", "prefix": "bifrost/logs", "compress": true }, "object_storage_exclude_fields": [] } } ``` GCS fields: | Field | Required | Notes | | ------------------ | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `type` | yes | Must be `"gcs"`. | | `bucket` | yes | Target bucket name. | | `credentials_json` | conditional | Service-account JSON (or `env.GCS_KEY` pointing at it). Omit when running on GKE with Workload Identity or any environment where Application Default Credentials work. | | `credentials` | deprecated | Legacy alias for `credentials_json`. Use `credentials_json` in new configs. | | `project_id` | optional | Useful when ADC cannot infer the project. | | `prefix` | optional | Object key prefix. Defaults to `bifrost`. | | `compress` | optional | Gzip payloads before upload. Defaults to `false`. | A live example is checked in at `examples/configs/withobjectstoragegcs/config.json`. ### Choosing what stays in the database vs. what is offloaded By default, every offloadable payload field is uploaded to object storage. `object_storage_exclude_fields` lets you pin specific fields to the **database only**, so they are never written to the bucket. Listed fields are **not dropped** - they continue to live in the logs DB row as before. Everything not listed is offloaded. Values must be **database column names** (not JSON paths). Common choices: | Column | Contents | | ---------------- | ---------------------------------------- | | `raw_request` | The verbatim provider request body. | | `raw_response` | The verbatim provider response body. | | `input_history` | The full conversation sent to the model. | | `output_message` | The model's primary output message. | ```json theme={null} { "logs_store": { "object_storage": { "...": "..." }, "object_storage_exclude_fields": [ "raw_request", "raw_response" ] } } ``` Unknown column names are silently ignored, so a typo will not error - it will just leave that field on the default (offloaded) path. Reference tests covering this behaviour live at `framework/logstore/hybrid_test.go`. Typical use cases: * **Data residency**: keep raw prompts and responses inside the DB (which may itself sit inside a controlled VPC) while still benefiting from offload for less sensitive fields. * **Operational queries**: keep `input_history` in the DB so SQL queries and the UI's search can run against the full conversation without paying an object-fetch round trip. ### Guardrail redaction When Enterprise guardrail redaction is enabled, object storage receives the same persisted log payload fields that Bifrost logs use. That means redacted fields are offloaded as redacted fields. For reversible modes such as `logs_only` and `runtime_reversible`, the offloaded payload contains placeholders like `[EMAIL-1]`. The reveal mapping stays with the Bifrost log row and is not written into the offloaded object. If `disable_content_logging` is enabled, Bifrost skips content logging and does not persist reveal data. For the full mode matrix, see [Guardrail Redaction](/enterprise/guardrails/redaction). ## Retention policy Bifrost ships a background log cleaner that deletes old logs on a fixed schedule. ### Configuration Retention is configured on the **client config**, not on `logs_store`. The active key is `client_config.log_retention_days`: ```json theme={null} { "client_config": { "log_retention_days": 30 } } ``` | Aspect | Value | | ----------------- | ----------------------------------------------------- | | Default | `365` days | | Minimum | `1` day (enforced by validator) | | Disable | Set to `0` (cleanup is skipped entirely) | | Cleanup cadence | Every 24 hours, plus a random jitter of 15-30 minutes | | Startup behaviour | A cleanup pass runs immediately when Bifrost starts | | Delete batch size | 100 rows per query | | Per-pass timeout | 30 minutes | Source: `framework/logstore/cleaner.go` and `transports/bifrost-http/server/server.go`. ### What gets deleted The cleaner deletes **database rows** older than `now - retention_days` (UTC). It deletes from the main `Log` table as well as the MCP tool logs table. ### What does NOT get deleted automatically > **Important**: the retention cleaner does **not** delete the corresponding payloads in S3 or GCS. The hybrid log store explicitly delegates object cleanup to the bucket's own lifecycle policy. If you have object storage enabled, configure a lifecycle / object-lifecycle-management rule on the bucket to expire objects under your `prefix/`. Pick a duration that matches (or exceeds) `log_retention_days`. **Amazon S3 lifecycle rule (example)** ```json theme={null} { "Rules": [ { "ID": "bifrost-logs-expire-30d", "Status": "Enabled", "Filter": { "Prefix": "bifrost/logs/" }, "Expiration": { "Days": 30 } } ] } ``` Apply with `aws s3api put-bucket-lifecycle-configuration --bucket --lifecycle-configuration file://lifecycle.json`. See the AWS docs on [object lifecycle management](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html). **Google Cloud Storage lifecycle rule (example)** ```json theme={null} { "lifecycle": { "rule": [ { "action": { "type": "Delete" }, "condition": { "age": 30, "matchesPrefix": ["bifrost/logs/"] } } ] } } ``` Apply with `gcloud storage buckets update gs:// --lifecycle-file=lifecycle.json`. See the GCS docs on [Object Lifecycle Management](https://cloud.google.com/storage/docs/lifecycle). ### Manual deletes still cascade Single and batch deletes triggered explicitly (via the UI's **Delete log** action or the underlying `DeleteLog` / `DeleteLogs` APIs) **do** remove the associated objects from S3 or GCS at the same time. Only the time-based retention sweep is bucket-blind. ## Configuration via Helm The Bifrost Helm chart at `helm-charts/bifrost/` does not expose dedicated values for `logs_store.object_storage`. It renders the same `config.json` schema documented above, so you supply the block under the chart's `bifrost.*` configuration tree (or mount your own `config.json` via an existing ConfigMap/Secret). ### Inline values ```yaml theme={null} bifrost: logsStore: enabled: true type: sqlite config: path: /app/data/logs.db objectStorage: type: s3 bucket: env.AWS_S3_BUCKET region: env.AWS_REGION accessKeyId: env.AWS_ACCESS_KEY_ID secretAccessKey: env.AWS_SECRET_ACCESS_KEY prefix: bifrost/logs compress: true ``` For GCS, substitute the `objectStorage` block: ```yaml theme={null} bifrost: logsStore: enabled: true type: sqlite config: path: /app/data/logs.db objectStorage: type: gcs bucket: env.GCS_BUCKET credentialsJson: env.GCS_KEY projectId: env.GCP_PROJECT_ID prefix: bifrost/logs compress: true ``` The chart converts camelCase keys to the `snake_case` form Bifrost expects when it writes the runtime `config.json`. Provide the referenced env vars (`AWS_*`, `GCS_*`) through `extraEnv`, `envFrom`, or an existing Kubernetes Secret. ### BYO config.json If you prefer to manage the full `config.json` yourself, mount it as a Secret and point the chart at it. Use the same `logs_store.object_storage` blocks shown in the [config.json](#configuration-via-config-json) section verbatim. ## Configuration via the UI The web UI does **not** currently expose a form for `logs_store.object_storage`. To enable payload offload: 1. Edit `config.json` (or the Helm values) using the snippets above. 2. Restart Bifrost so the new log store wiring takes effect. 3. Confirm new requests are landing in the bucket by browsing the configured `prefix/` path. Once configured, the existing **Logs** screen in the UI transparently fetches payloads from object storage when you open a log entry. No UI changes are needed on the read path. ## Verifying the setup 1. Send a request through Bifrost. 2. Confirm a row appears in the logs DB (visible via the UI's **Logs** screen). 3. List the bucket under your `prefix/`. You should see one or more objects per request. 4. Open the log entry in the UI. The payload pane should render the content fetched from object storage. If the bucket stays empty, check the Bifrost logs for `objectstore` errors. The most common causes are missing credentials, a region/endpoint mismatch, or a bucket policy that blocks the credentials' principal.