Techno Tim Tinkers

In this tutorial we'll walk through my local, private, self-hosted AI stack so that you can run it too. If you're looking for the overview of this stack, you can out the video here: https://www.youtube.com/watch?v=GrLpdfhTwLg Video Notes: https://technotim.live/posts/ai-stack-tutorial/ Products: GPUs 12GB is probably "good enough" for small models. 3000 Series 👇 My value pick. 3060, 12GB RAM, affordable, can play games too. - MSI Gaming GeForce RTX 3060 12GB: https://amzn.to/3WerOUI - GIGABYTE GeForce RTX 3060 Gaming OC 12G: https://amzn.to/3WdqHV9 - NVIDIA GeForce RTX 3090 Founders Edition Graphics Card Renewed 24GB: https://amzn.to/4cTIS7Q - Zotac Gaming GeForce RTX 3090 Trinity OC, 24GB: https://amzn.to/3WdhJqV 4000 Series - ASUS TUF Gaming GeForce RTX™ 4090 OG OC Edition Gaming Graphics Card 24GB: https://amzn.to/463LHRF - MSI Gaming GeForce RTX 4060 8GB: https://amzn.to/4eVxrON - ASUS Dual GeForce RTX™ 4070 White OC Edition 12GB: https://amzn.to/460rqN0 CPUs You'll want a modern CPU, if you are going desktop class here are a few I would choose - Intel Core i7-12700K Gaming Desktop Processor: https://amzn.to/3XTj1Zu - Intel Core i7-13700K Gaming Desktop Processor: https://amzn.to/3Lg7M5V - Intel Core i7-14700K Gaming Desktop Processor: https://amzn.to/3WftehU Storage For flash storage, I always go with these SSDs - Samsung 870 EVO SATA: https://amzn.to/3XTW6gA - SAMSUNG 990 PRO SSD NVMe: https://amzn.to/4cW1xA9 (Affiliate links may be included in this description. I may receive a small commission at no cost to you.) Support me on Patreon: https://www.patreon.com/technotim Sponsor me on GitHub: https://github.com/sponsors/timothystewart6 Subscribe on Twitch: https://www.twitch.tv/technotim Merch Shop 🛍️: https://l.technotim.live/shop Gear Recommendations: https://l.technotim.live/gear Get Help in Our Discord Community: https://l.technotim.live/discord Main channel: https://www.youtube.com/@TechnoTim Talks Channel: https://www.youtube.com/@TechnoTimTalks 00:00 - Intro 01:14 - Hardware Specs 02:23 - GPU Considerations 03:10 - GPU Perspective 04:54 - Proxmox 07:05 - Server OS 07:25 - Drivers 08:31 - NVIDIA Container Toolkit 09:44 - Docker 10:25 - Folder Layout 11:20 - Stack Overview 12:15 - Traefik & Docker Networking Considerations 12:55 - Ollama 17:29 - Open WebUI 20:55 - Starting the Stack 23:08 - Ollama Models 25:50 - Chatting with Ollama & Performance 32:05 - searXNG 33:33 - Stable Diffusion & ComfyUI 39:54 - Stable Diffusion Models 44:42 - ComfyUI Workflows 47:00 - Verifying Model Checksum 48:48 - Integrating ComfyUI into Open WebUI 51:59 - Whisper 56:22 - Home Assistant Assist - Chat, Voice, and Text to Voice 01:04:15 - Code Suggestions and Chat Assistant in VSCode (Free Co-pilot) Thank you for watching!