Hello friends, I have been busy on some of my other sites, so feel free to check out I Can Play There for more details. Amongst my dabblings, I have now created my own personal Artificial Intelligence LLM for my home. I am using Ollama as the scaffolding and the Gemma 4 and Heretic libraries, which are freely available. I then bootstrapped those into an OpenUI Docker image so that I could ask my AI random things without it going out to an external site like Claude, ChatGPT, or Gemini. From there, I decided to add a new layer: home automation.
Now I should mention that I have always been wary of allowing an external AI system (Like Alexa or Google Home) access to do too many things in my house because those systems are potentially large targets where my little place in the world would end up being in the blast radius. However, by building on my private LLM on my servers that only live in my network and are not front-facing, I feel like the risk is much lower. In fact, amongst my systems, I only have two or three items that use cloud-based features because they won’t work without them. Everything else, however, is completely internal, and if I had to cut those cloud-based items loose, I wouldn’t lose too much.
I have to say, this has easily been one of the most interesting projects I have ever worked on. It can feel like hours of poking around in UIs, interfaces, and YAML, but when things actually work, it is visible and tangible. For instance, it might take a few hours to add the YAML configuration to build the automations and then make a dashboard to accomplish those functions, but the first time I pushed a button and heard my house speaker system say something… it makes it all worth it and feels like I am now in the future. Then, I added some fun easter eggs, such as making my AI provide me a “status report” that is read out through my AI like JARVIS from Iron Man. Or asking it to “open the pod bay doors” (essentially unlock doors which is not permitted by my AI’s saftefy controls) only to hear it tell me “I am sorry Dave, I can’t do that.” make the system uniquely mine and still fulfills many automated things that I actually need my house to do.
One of the biggest things I have learned in all of this is that there are a lot of very talented people in the Home Automation community. Many of the technologies we use every day already have integrations (either readily available in the HA framework or easily added by the HACS “Home Automation Community Store” lab). It really is mind-blowing to see how even huge names in the market like Vivint, Schlage, Nest, etc. have supported this ecosystem. I have also learned a lot about configuring things with YAML code, running them in detailed Docker containers, and allowing secure access to certain things from the wider internet when I am not physically present at my house.
If you have never invested in building your own LLM and making it do cool things, you should really consider the options that are out there; you’ll be surprised at how many unique things are happening in this field.
