Ollama

Ollama makes running large language models on your local machine as easy as running a Docker container. With a single command, you can download and run models like Llama 3, Mistral, Gemma, CodeLlama, and dozens more. Ollama handles model management, quantization, and provides an OpenAI-compatible API, making it trivial to swap cloud AI for local inference. The project has exploded in popularity among developers who want privacy, cost savings, or offline capabilities. Ollama supports both CPU and GPU inference (including Apple Silicon), and the growing model library includes everything from tiny 270M parameter models to massive 70B+ models.