Feedback

Portkey’s Feedback APIs provide a simple way to get weighted feedback from customers on any request you served, at any stage in your app.

This feature is available on all Portkey plans.

You can capture this feedback on a request or conversation level and analyze it by adding meta data to the relevant request.

Adding Feedback to Requests

1. Find the `trace-id`

Portkey adds trace ids to all incoming requests. You can find this in the x-portkey-trace-id response header.

To use your own trace IDs, send them as part of the request headers - Adding a trace ID to your requests

2. Add feedback

You can append feedback to a request through the SDKs or the REST API.

NodeJS

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
});

// Add the feedback
portkey.feedback.create({
    traceID: "your trace id",
    value: 5, // Integer between -10 and 10
    weight: 1, // Optional
    metadata: {
        ... // Pass any additional context here like comments, _user and more
    }
})

Python

portkey = Portkey(
    api_key="PORTKEY_API_KEY"
)

feedback = portkey.feedback.create(
    trace_id="TRACE_ID",
    value=5,  # Integer between -10 and 10
    weight=1,  # Optional
    metadata={
        # Pass any additional context here like comments, _user and more
    }
)
print(feedback)

cURL

curl --location 'https://api.portkey.ai/v1/feedback' \
    --header 'x-portkey-api-key: YOUR_API_KEY' \
    --header 'Content-Type: application/json' \
    --data '{
        "trace_id": "YOUR_TRACE_ID",
        "value": -10,
        "weight": 0.5,
        "metadata": {
            "text": "title was irrelevant",
            "_user": "fef653",
            "_organisation": "o9876",
            "_prompt": "test_prompt",
            "_environment": "production"
        }
    }'

The Payload takes the following keys: traceID/trace_id, value, weight, metadata

Key	Required?	Description	Type
trace_id / traceID	Required	The trace id on which the feedback will be logged	string
value	Required	Feedback value	integer between [-10,10]
weight	Optional	Add weight value to feedback value. Helpful if you’re collecting multiple feedback for a single trace	float between [0,1], Default = 1.0
metadata	Optional	JSON string of any metadata you want to send along with the feedback._user, _organisation, _prompt and _environment are special fields indexed by default	string

Examples

A simple & effective feedback from the user is a thumbs up or thumbs down. Just set value to 1 for 👍 and -1 for 👎. Weight would be default 1.0.

NodeJS

portkey.feedback.create({
    traceID: "your trace id",
    value: 1
})

Python

portkey.feedback.create(
    trace_id = "your trace id",
    value = 1
)

cURL

curl --location 'https://api.portkey.ai/v1/feedback' \
--header 'x-portkey-api-key: ' \
--header 'Content-Type: application/json' \
--data '{
    'trace_id': 'REQUEST_TRACE_ID',
    'value': 1
}'

Other Ideas for collecting feedback

• Business metrics make for great feedback. If you’re generating an email, the email being sent out could be a positive feedback metric. The level of editing could indicate the value.
• When a user retries a generation, store the negative feedback since something probably went wrong. Use a lower weight for this feedback since it could be circumstantial.

Feedback Analytics

You can see the Feedback Count and Value: Weight pairs for each trace-id on the logs page. You can also view the feedback details on Analytics and on the Prompt Eval tabs.