Caido

Changing the LLM Model ​

⚡PRO FEATURE

The Assistant are only available to users with Individual or Team tier subscriptions.

WARNING

Submitted data is sent to the LLM's third-party provider (OpenAI) and can be stored for up to 30 days. Due to this, anonymize sensitive data when using the Assistant. Sensitive data may be unintentionally submitted when using the Assistant context menu options. Before using any context menu option, manually review all content to ensure no sensitive data is included. For more information, review:

INFO

"Tokens" are the measurement used by OpenAI LLMs when processing text. Each Individual and Team tier subscription user gets 500,000 credits per month which can be exchanged for tokens.

Since LLMs maintain context based on all tokens in a session, be aware that each subsequent message in a conversation will incur a greater credit cost as the sum of all tokens is consumed. Estimate credit costs with the OpenAI Tokenizer tool.

The Assistant supports the following OpenAI LLM models, each with a different credit to token cost ratio:

  • GPT-4o Mini (1:1)
  • GPT-3.5 Turbo (2:1)
  • GPT-4o (10:1)

To switch models, click on the drop-down menu within a new conversation and make a selection.

The drop-down menu model options for the Assistant.

INFO

Currently, the Assistant does not support API keys for other AI providers or configuration for use with local LLMs. View Issue #1480 on GitHub for more information.