Usage limits

Usage limits allow you to see and manage model usage and cost on a per user basis.

Note

Usage limits is only available on the Growth plan.

Usage limits are handled by a custom Koa Middleware function that follows these steps:

1. A request is made to the API

2. Usage Limiter checks to see if there is any usage remaining

3. a) If usage remaining - the request is executed

   b) If no usage remaining - 429 usage exceeded response is returned

4. After the request is finished, new usage is saved

If you’re creating a new endpoint that you want to be usage limited then just include the middleware function:

chatRouter.post('/', usageLimiter('chat'), async (ctx) => {
  // do something cool
});

The string passed to the limiter chat defines which usage property is decremented.

Usage limits are stored on the LicenseKey model in mongoose, and both daily and monthly usage is saved.

You can define usage limits when a user signs up from the configs/plans.yml file. More info about how that config is handled can be found in the plans section.

Stripe + Usage Limits How Usage Limits are managed

Usage response

Every request that calls an OpenAI API endpoint results in a cost. These costs are calculated (or estimated) and returned from all the StartKit.AI services with the result as a usage property.

For example, this is the response of a call to the moderation endpoint to get the sentiment of some text:

// POST https://startkit.ai/api/moderation/sentiment
// { "text": "Ugh, this new chef sucks. I'll never eat here again" }
{
  "sentiment": {
    "result": "negative",
    "confidence": 95
  },
  "usage": {
    "model": "gpt-4-turbo",
    "promptTokens": 44,
    "completionTokens": 989,
    "cost": 0.012
  }
}

For the image endpoints the response is slightly different:

// POST https://startkit.ai/api/moderation/sentiment
// { "prompt": "An image of an alient squid in space, swimming around the Starship Enterprise D" }
{
  "images": [{
    "url": "https://sfo3.digitaloceanspaces.com/startkitai/494226b2-1059-4957-982a-f785b29b57e5/generated-images/f590287b-7ed1-4ddf-acfc-a76539c56202.png",
    "revised_prompt": "A visual representation of a celestial alien squid-like creature, gliding effortlessly in the vastness of outer space. It is navigating fluidly around an advanced, large, multi-decked spaceship, exuding a futuristic architectural design akin to the 24th-century star fleet aesthetic."
  }],
  "usage": {
    "model": "dall-e-2",
    "size": "1024x1024",
    "quality": "standard",
    "cost": 0.02
  }
}

The total cost of a request is also available on the x-usage-cost header of the response.

Editing a users’ limits

You can change the limits for a specific user from the Admin Dashboard, on the Users page.

Click the License button next to the user you want to modify.