Playground
LLM Foundry's playground lets you interact with large language models. It offers both a simple and advanced mode to cater to different user needs.
Advanced Mode
This button lets you switch between Simple and Advanced modes. Advanced mode gives you more features and controls.
User Message
Enter your prompt or question here. This is the main input for interacting with the model. Examples:
- "Correct the spelling and grammar of the following text."
- "Write a Python script to crawl all top level pages from https://reddit.com."
- "Summarize the key points of the attached document."
- "Translate the following text to French."
- "Compare these 2 similar images and describe the differences."
See the prompting guide for better prompts.
Model
Choose the LLM you want to use. Models with the best performance/price ratio have a green 🟢 next to them.
- Start with the cheapest 🟢 model
- If it doesn't work well, proceed to the next cheapest 🟢 model
Here are public models on the playgound.
| Provider | Model name | Model ID | Input | Output | Context | Tools | Image |
|---|---|---|---|---|---|---|---|
| Groq | Llama 3.1 8b | llama-3.1-8b-instant | - | - | 128K | Y | N |
| Groq | Llama 3.1 70b | llama-3.1-70b-versatile | - | - | 128K | Y | N |
| Groq | Llama 3 8b | llama3-8b-8192 | 0.05 | 0.08 | 8K | Y | N |
| Groq | Llama 3 70b | llama3-70b-8192 | 0.59 | 0.79 | 8K | Y | N |
| Groq | Gemma 2 9b | gemma2-9b-it | 0.20 | 0.20 | 8K | Y | N |
| Groq | Mixtral 8x7b | mixtral-8x7b-32768 | 0.24 | 0.24 | 32K | Y | N |
| Anthropic | Claude 3 Haiku | claude-3-haiku-20240307 | 0.25 | 1.25 | 200K | Y | Y |
| Anthropic | Claude 3.5 Sonnet | claude-3-5-sonnet-20240620 | 3 | 15 | 200K | Y | Y |
| OpenAI | GPT 4o Mini | gpt-4o-mini | 0.15 | 0.60 | 128K | Y | Y |
| OpenAI | GPT 4o | gpt-4o | 5 | 15 | 128K | Y | Y |
| DeepSeek | DeepSeek Chat | deepseek-chat | 0.14 | 0.28 | 128K | Y | N |
| DeepSeek | DeepSeek Coder | deepseek-coder | 0.14 | 0.28 | 128K | Y | N |
| Cloudflare | SQLCoder 7b 2 | cf/defog/sqlcoder-7b-2 | - | - | 100K | N | N |
Here are safe private models on the playground. These models can be used with client data because:
- They are within the Straive tenant. No data leaves the Straive cloud network.
- No data is not used to train the models.
| Provider | Model name | Model ID | Input | Output | Context | Tools | Image |
|---|---|---|---|---|---|---|---|
| Azure | GPT 3.5 Turbo | gpt-35-turbo | 0.5 | 1.5 | 16K | Y | N |
| Azure | GPT 4o | gpt-4o | 5 | 15 | 128K | Y | Y |
| Azure | Phi 3 Mini 4K | phi-3-mini-4k | 0.28 | 0.84 | 4K | N | N |
| Azure | Phi 3 Medium 128K | phi-3-medium-128k | 0.5 | 1.5 | 128K | N | N |
| Azure | Llama 3 8b | llama-3-8b | 0.37 | 1.1 | 8K | N | N |
| Azure | Llama 3 70b | llama-3-70b | 3.8 | 11.38 | 8K | N | N |
| Claude 3 Haiku | claude-3-haiku@20240307 | 0.25 | 1.25 | 200K | Y | Y | |
| Claude 3.5 Sonnet | claude-3-5-sonnet@20240620 | 3 | 15 | 200K | Y | Y | |
| Gemini 1.5 Flash | gemini-1.5-flash-latest | 0.35 | 0.35 | 1000K | Y | Y | |
| Gemini 1.5 Pro | gemini-1.5-pro-latest | 3.5 | 3.5 | 2000K | Y | Y |
System Instructions
These guide the model's behavior throughout the conversation. The default is "You are a helpful assistant."
When writing apps, explain what you want the model to do here, and pass the user input as the user message.
Temperature
Adjust the temperature to control the creativity of the responses.
- Lower temperatures generate more predictable responses
- Higher temperatures generate more creative responses
Temperature ranges from 0 to 2 for most models, but for the Claude family, it ranges from -1 to +1.
JSON Response
If checked, the response will be formatted as JSON, which is useful for API integration.
JSON Schema
Optionally, specify the JSON format / schema you want the output in. This is for advanced users who need precise control over the output structure.
Images
The model looks at these images and answers you. You can use it to find objects, read text, read charts, check or validate images, provide screenshots as context, etc.
Upload JPEG, PNG, WEBP, or GIF images. We resize images to 1024x1024 pixels to save costs.
Only some models support vision. For other models, this option is disabled.
Documents
Upload .DOCX or .PDF files. The text from these documents will replace the user message. This is useful for longer inputs or detailed instructions.
Search
Enter URLs or search terms to append text from these sources to the user message. Each URL or term should be on a new line. Example: "https://ibm.com/ or IBM news."
Templates
Dropdown Menu: Select from saved templates to quickly input pre-defined prompts and settings. Useful for recurring tasks.
My Templates
List: Manage your saved templates. Save new templates or delete existing ones for quick access to commonly used settings.
Submit
Click to submit your prompt to the model. This initiates the processing of your request.
Reset
Click to reset all fields to their default values. This clears the model, temperature, system instructions, user message, etc.
Notifications
Displays important updates and news related to LLM Foundry. For example, "Llama 3.1 added to Playground."
Variables
Use this area to manage and insert variables into your prompts for customized responses. This is especially useful for API integration.
Response Controls
After receiving a response, you can click on these buttons:
- Download your prompts and LLM response as a Markdown file
- Copy LLM response to the clipboard as text and HTML
- Fixed width font shows the LLM response in a fixed width font. Useful if you want to align table columns
- Code shows a popup with the Curl, Python, and JavaScript code to programmatically run the same request
Cost Stats
Displays the cost statistics of your usage, helping you keep track of expenses.