Personal project · AI Research Infrastructure
ai.coltoncs.com
Most AI features on this site call a hosted API. ai.coltoncs.com is the opposite: a gated LLM and image-generation platform running entirely on a home PC I own and administer — two models served concurrently on a single RTX 3090, a chat interface, a ComfyUI image pipeline, and a Cloudflare Tunnel and Access policy that make it reachable from anywhere without opening a port.
Context
Most of the AI-powered work in this portfolio — PosterOCR, for instance — calls a hosted model over an API. That proves I can integrate AI into a product; it doesn’t prove I can run the AI itself. ai.coltoncs.com is the other half of that story: a self-hosted research platform running on my own home PC, an RTX 3090 (24GB VRAM) paired with 64GB of system RAM, serving frontier-scale open-weight models end to end — inference, interface, image generation, and the network layer that makes it safely reachable — without renting a GPU or calling anyone else’s API.
The problem
Running this at home rather than in the cloud creates a specific set of constraints. Two sizable models — a general chat/reasoning model and a quantized coding specialist — need to share a single 24GB card without one starving the other. Image generation needs the same GPU, at a different point in the pipeline, without a second machine. And the whole thing needs to be reachable from outside my home network for real use, which normally means opening inbound ports and running your own reverse proxy and TLS termination — exactly the kind of home-network exposure worth avoiding.
What I built
- A dual-model inference core on Ollama, serving
gemma4:26bfor general chat and reasoning alongsideqwen3-coder:30b-a3b-q4_K_M, a quantized mixture-of-experts coding specialist, concurrently on one RTX 3090. Running both within a 24GB VRAM budget means leaning on Ollama’s model loading and quantization rather than assuming unlimited headroom, the same constraint you hit deploying models on real hardware instead of an elastic cloud instance. - A chat front end in Open WebUI, giving both models a proper interface — conversation history, model switching, system prompts — instead of a bare API endpoint.
- A local image-generation pipeline in ComfyUI, running node-based Stable Diffusion-style workflows on the same GPU as the language models, sharing VRAM across LLM and image-generation workloads rather than requiring dedicated hardware for each.
- A Cloudflare Tunnel from the home PC, via
cloudflared, that opens an outbound-only connection to Cloudflare’s edge and maps it toai.coltoncs.com. No inbound port is opened on the home router, and no home IP is exposed publicly. - A Cloudflare Access policy scoped to a single identity — my own email address — sitting in front of the tunnel so every request is authenticated before it reaches Open WebUI or ComfyUI. This is a single-user, gated research environment by design, not a public product.
Outcome
ai.coltoncs.com is live and running today. Visit it and the first thing you’ll hit is a Cloudflare Access login screen scoped to my identity — that wall is the point, not an inconvenience: it’s a working demonstration of the Zero Trust setup described above, not a placeholder for one. Behind it is a functioning platform that runs two concurrent large language models and a full image-generation pipeline on hardware I own, exposed to the internet through infrastructure I configured myself rather than a managed AI platform.