Fireworks

Use Fireworks for chat, vision, embeddings, and image generation with advanced grammar and JSON modes through Portkey.

Quick Start

Get started with Fireworks in under 2 minutes:

Python

from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @fireworks-ai provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@fireworks-ai/accounts/fireworks/models/llama-v3-70b-instruct",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @fireworks-ai provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@fireworks-ai/accounts/fireworks/models/llama-v3-70b-instruct",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @fireworks-ai provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@fireworks-ai/accounts/fireworks/models/llama-v3-70b-instruct",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @fireworks-ai provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@fireworks-ai/accounts/fireworks/models/llama-v3-70b-instruct",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)

# 1. Add @fireworks-ai provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@fireworks-ai/accounts/fireworks/models/llama-v3-70b-instruct",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Add Provider in Model Catalog

Before making requests, add Fireworks to your Model Catalog:

1. Go to Model Catalog → Add Provider
2. Select Fireworks
3. Enter your Fireworks API key
4. Name your provider (e.g., fireworks-ai)

Complete Setup Guide

See all setup options and detailed configuration instructions

---

Fireworks Capabilities

Embeddings

Generate embeddings using Fireworks models:

Python

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

embeddings = portkey.embeddings.create(
    model="thenlper/gte-large",
    input="create vector representation on this sentence"
)

print(embeddings.data[0].embedding)

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@fireworks-ai'
});

const embeddings = await portkey.embeddings.create({
    model: "thenlper/gte-large",
    input: "create vector representation on this sentence"
});

console.log(embeddings.data[0].embedding);

Vision Models

Process images with vision models:

Python

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

response = portkey.chat.completions.create(
    model="accounts/fireworks/models/firellava-13b",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Can you describe this image?"},
            {
                "type": "image_url",
                "image_url": {
                    "url": "https://images.unsplash.com/photo-1582538885592-e70a5d7ab3d3"
                }
            }
        ]
    }]
)

print(response.choices[0].message.content)

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@fireworks-ai'
});

const response = await portkey.chat.completions.create({
    model: "accounts/fireworks/models/firellava-13b",
    messages: [{
        role: "user",
        content: [
            { type: "text", text: "Can you describe this image?" },
            {
                type: "image_url",
                image_url: {
                    url: "https://images.unsplash.com/photo-1582538885592-e70a5d7ab3d3"
                }
            }
        ]
    }]
});

console.log(response.choices[0].message.content);

Image Generation

Generate images with Stable Diffusion:

Python

from portkey_ai import Portkey
import base64
from io import BytesIO
from PIL import Image

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

image = portkey.images.generate(
    model="accounts/fireworks/models/stable-diffusion-xl-1024-v1-0",
    prompt="An orange elephant in a purple pond"
)

Image.open(BytesIO(base64.b64decode(image.data[0].b64_json))).save("generated.png")

import Portkey from 'portkey-ai';
import fs from 'fs';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@fireworks-ai'
});

const image = await portkey.images.generate({
    model: "accounts/fireworks/models/stable-diffusion-xl-1024-v1-0",
    prompt: "An orange elephant in a purple pond"
});

const imageData = image.data[0].b64_json;
fs.writeFileSync("generated.png", Buffer.from(imageData, 'base64'));

Function Calling

Use function calling with Fireworks models:

Function Calling Guide

Explore function calling examples and best practices

Grammar Mode

Fireworks lets you define formal grammars to constrain model outputs. You can use it to force the model to generate valid JSON, speak only in emojis, or anything else. (Originally created by GGML)

Grammar mode is set with the response_format param. Just pass your grammar definition with {"type": "grammar", "grammar": grammar_definition}

Let’s say you want to classify patient requests into 3 pre-defined classes:

Python

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

patient_classification = """
root      ::= diagnosis
diagnosis ::= "flu" | "dengue" | "malaria"
"""

response = portkey.chat.completions.create(
    model="accounts/fireworks/models/llama-v3-70b-instruct",
    messages=[{"role": "user", "content": "Patient has fever and chills"}],
    response_format={"type": "grammar", "grammar": patient_classification}
)

print(response.choices[0].message.content)

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@fireworks-ai'
});

const patient_classification = `
root      ::= diagnosis
diagnosis ::= "flu" | "dengue" | "malaria"
`;

const response = await portkey.chat.completions.create({
    model: "accounts/fireworks/models/llama-v3-70b-instruct",
    messages: [{ role: "user", content: "Patient has fever and chills" }],
    response_format: { type: "grammar", grammar: patient_classification }
});

console.log(response.choices[0].message.content);

Fireworks Grammar Guide

Learn more about grammar mode and examples

JSON Mode

Force JSON output with optional schema validation:

Python

from portkey_ai import Portkey
from pydantic import BaseModel
from typing import List

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

class Recipe(BaseModel):
    title: str
    description: str
    steps: List[str]

response = portkey.chat.completions.create(
    model="accounts/fireworks/models/llama-v3-70b-instruct",
    messages=[{"role": "user", "content": "Give me a recipe for making Ramen"}],
    response_format={
        "type": "json_object",
        "schema": Recipe.schema_json()
    }
)

print(response.choices[0].message.content)

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@fireworks-ai'
});

const response = await portkey.chat.completions.create({
    model: "accounts/fireworks/models/llama-v3-70b-instruct",
    messages: [{ role: "user", content: "Give me a recipe for making Ramen" }],
    response_format: {
        type: "json_object",
        schema: {
            type: "object",
            properties: {
                title: { type: "string" },
                description: { type: "string" },
                steps: { type: "array" }
            }
        }
    }
});

console.log(response.choices[0].message.content);

Fireworks JSON Mode Guide

Learn more about JSON mode

---

Supported Models

Fireworks Models

View the complete list of models available on Fireworks

---

Next Steps

Gateway Configs

Add fallbacks, load balancing, and more

Observability

Monitor and trace your Fireworks requests

Prompt Library

Manage and version your prompts

Metadata

Add custom metadata to requests

For complete SDK documentation:

SDK Reference

Complete Portkey SDK documentation