Simple LLM
A macOS chatbot with support for open-source models!
Features
Support for OpenAI & Open-Source Models
Run model locally on your Mac or leverage the power of OpenAI and TogetherAI services.
Retrieval-Augmented Generation
Automatically selects references and attaches them to your chat, enhancing AI responses.
Flexibility
Easily switch between models, duplicate chats, adjust creativity levels, or remove partial messages on the fly!
Demo
Limitations
Before you jump in, here are some key things to keep in mind about the app:
RAG & Chat
Both features depend on the AI model you choose. AI can sometimes "makes things up", so it's important to double-check results and guide the AI where needed.
Open-source Models
Most models don't work well with RAG feature because all the RAG instructions are tailored for OpenAI models. Please expect weird responses or issues to occur if you use open-source models with RAG.
Running LLM Locally
Compatible only with Apple silicon.
Supports only the GGUF format with ChatML, OpenChat, Orca-Hashes, Zephyr and Mixtral prompt formats.
Performance and quality depend on the capabilities of your chosen model. Strong hardware is needed for larger models to ensure a smooth experience.
Join the beta!
Required macOS 14.0.
As the app is still under development, expect to encounter some quirks along the way!
Your personal OpenAI or TogetherAI API key is required for their models, and you will be charged accordingly.
English supported only!