← All sessions
Lightning Talk · 7 min
A Practical Guide to Fast and Private LLMs
🕑
Time and room for this session are published in October. when the timetable goes live.
A set of practical tips and tools for hobbyists and teams to run their own LLM in a private and sustainable setup. This talk gives people hands-on knowledge to decide on the right LLM architecture, how to optimize it, which hardware to run it on, and what software stack to go for. A whirlwind tour of how to roll your own AI for fun and profit.
Main tools & technology
Llama.cpp, Ollama, vLLM
Topics
Local AI, Inference & Open Models
