Runway ML

Runway ML API conversion guide - text-to-video, image-to-video, and video-to-video generation

Overview

Runway ML provides video generation via an asynchronous task-based API. Bifrost maps the unified video schema to Runway’s task API and polls until completion.

Supported Operations

Operation	Supported	Endpoint
Video Generation	✅	/v1/text_to_video, /v1/image_to_video, /v1/video_to_video
Video Retrieve	✅	/v1/tasks/{task_id}
Video Download	✅	via Retrieve + URL download
Video Delete	✅	/v1/tasks/{task_id} (cancel)
Video List	❌	-
Video Remix	❌	-

---

1. Video Generation

Generate (POST /v1/videos)

Request Parameters

Parameter	Type	Required	Notes
model	string	✅	Runway model
prompt	string	✅	Text description of the video
input_reference	string	❌	Input image for image-to-video
seconds	string	❌	Duration in seconds (default: "2")
size	string	❌	Resolution as WxH (e.g., 1280x720; default: 1280x720) - converted to W:H ratio
seed	int	❌	Gen models only
audio	bool	❌	Enable audio generation. Veo models only
video_uri	string	❌	Source video URL for video-to-video. gen4_aleph only

Extra Params

Key	Type	Notes
references	array	Video reference objects [{"uri": "...", "tag": "..."}] for video-to-video
content_moderation	object	Content moderation config
reference_images	array	Reference image objects for style/asset guidance

Generation Modes (auto-detected from inputs)

• Text-to-video: prompt only
• Image-to-video: prompt + input_reference
• Video-to-video: prompt + video_uri - gen4_aleph only

Response: BifrostVideoGenerationResponse with id, status, videos[]

Bifrost statuses (normalized): queued → in_progress → completed / failed

These values are the normalized view returned by Bifrost’s API. Runway’s native statuses are: PENDING, THROTTLED, RUNNING, SUCCEEDED, FAILED, CANCELLED.

Retrieve / Download / Delete

Operation	Endpoint	Notes
Get status	GET /v1/videos/{id}	Poll until status: completed
Download content	GET /v1/videos/{id}/content	Returns raw video bytes (MP4)
Cancel/Delete	DELETE /v1/videos/{id}	Cancels the running task

---

Setup & Configuration

Configure Runway as a provider.

Web UI

1. Navigate to Models > Model Providers. Look for Runway ML under Configured Providers. If it is missing, click on Add New Provider and select Runway ML.
2. Click Add Key or edit an existing key.
3. Set a name for your key.
4. Paste your API key directly or use an environment variable (for example, env.RUNWAY_API_KEY).
5. Set Allowed Models to All Models (default) or the specific model allowlist you want this key to serve.
6. Save the provider configuration.

config.json

{
  "providers": {
    "runway": {
      "keys": [
        {
          "name": "runway-key-1",
          "value": "env.RUNWAY_API_KEY",
          "models": [
            "*"
          ],
          "weight": 1.0
        }
      ]
    }
  }
}

API

curl --location 'http://localhost:8080/api/providers' \
--header 'Content-Type: application/json' \
--data '{
  "provider": "runway",
  "keys": [
    {
      "name": "runway-key-1",
      "value": "env.RUNWAY_API_KEY",
      "models": [
        "*"
      ],
      "weight": 1.0
    }
  ]
}'

Go SDK

case schemas.Runway:
    return []schemas.Key{{
        Name:   "runway-key-1",
        Value:  *schemas.NewEnvVar("env.RUNWAY_API_KEY"),
        Models: []string{"*"},
        Weight: 1.0,
    }}, nil

---

Reference Links

• Runway ML API Documentation
• Runway ML Models