Local AI Setup Steps

November 11, 2025

Lately I have been experimenting with running AI models locally. Mostly as I don't want to shell out any cash to OpenAI or anyone.

Here are the loose steps to using Local-AI on mac

download local-ai with brew per localai.io instructions

brew install localai

Once installed you need to start localai with the command

local-ai

then install a model

local-ai models install qwen3-vl-4b-instruct

and here is some python code to engage with the model via https

import requests import json

url = “http://localhost:8080/v1/chat/completions"

while True:

variable=input(“What do you want to ask?\n”) if variable.lower() == 'q': break

print(“....thinking.....\n”) payload = { “model”: “qwen3-vl-4b-instruct”, “messages”: [{“role”: “user”, “content”: f”{variable}“}] }

response = requests.post(url, json=payload) data = response.json() content = data[“choices”][0][“message”][“content”] print(content)

#print(json.dumps(data, indent=2))

this also seems to work to get docker running:

docker run -d \ —name local-ai \ -p 8080:8080 \ -e MODELS_PATH=/models \ -v /Users/beeschmersal/python/langchain/models:/models \ localai/localai:latest-aio-cpu