Skip to main content
Docs

Put a Local LLM or Ollama API on a Public HTTPS URL

Running Ollama, llama.cpp, or another local LLM server on localhost keeps inference private and fast—but collaborators cannot call it until you expose the HTTP port with a tunnel. Asyx provides a stable https://<name>.tunnel.asyx.ai front door with TLS handled for you.

Security first

A public LLM endpoint can be abused for token spend and data exfiltration. Use Quickpass or network isolation for anything beyond a short demo. Never expose an unauthenticated admin API.

Typical setup

1. Run Ollama (or your LLM server)

Default Ollama API listens on port 11434:

ollama serve

Or use a web UI (Open WebUI, etc.) on its configured port.

2. Open an HTTP tunnel

asyx tunnel --http --port 11434

The CLI prints a URL such as https://yourname.tunnel.asyx.ai.

3. Call from a remote client

Point tools at the tunnel host instead of localhost:

curl https://yourname.tunnel.asyx.ai/api/tags

(Exact paths depend on your LLM server.)

AI agent + local model workflows

Coding agents can build a small wrapper API around a local model, run it on a port, and tunnel it for remote demos—same pattern as AI Agents:

Agent builds API → runs on port 8080 → asyx tunnel --http --port 8080 → share URL

Protecting the endpoint

Public LLM APIs attract scanners. Options:

  1. Quickpass security group — only enrolled guests reach the URL:
asyx tunnel --http --port 11434 --secgroup team
  1. Application-level auth — API keys or JWT on your wrapper, even behind the tunnel.
  2. Short-lived demos — stop the tunnel (Ctrl+C) when the session ends.

See Quickpass.

Performance notes

  • Traffic routes through Asyx relays—adds latency vs pure LAN access.
  • Large model downloads stay local; only HTTP inference traffic crosses the tunnel.
  • Fair-use caps apply; heavy production load belongs on dedicated infrastructure.

Related guides

FAQ

Can I expose WebSocket streaming?
HTTP tunnels support streaming responses where your server uses chunked transfer. Check your client’s timeout settings.

Does this replace cloud LLM APIs?
No. It is for demos, pair programming, and testing—not multi-tenant production inference.

Which port for Open WebUI?
Use whatever port the UI binds to (often 8080) in asyx tunnel --http --port <port>.