Eclipse: the runtime that disappears

Load once. Serve everywhere.
Pay for none of the waiting.

Eclipse keeps weights resident and cold-starts in the time it takes to parse the request, so the runtime stops being the thing you plan around.

You have been paying for cold starts

Weights stay mapped and the graph stays compiled between calls. The first request after an idle hour costs what the second one does.

resident

One artifact, any silicon

The same build runs on the laptop you developed it on and the rack you deployed it to. No per-target rebuild, no quiet precision swap.

portable

Nothing left running

Idle means idle. When traffic stops the runtime releases and the meter stops with it, which is the part every other bill forgets.

idle

From weights to first token in four lines.

Point Eclipse at a model file. It picks the backend for the machine it's on, compiles the graph once and keeps it warm. There is no server process to stand up and no container to keep alive.

quickstart

quickstart.py

$ pip install eclipse

import eclipse

model = eclipse.load("./weights/7b-instruct")
# backend picked here, once. graph stays warm.

for tok in model.stream("Say hello"):
    print(tok, end="")

Hello! The weights were already warm.

first token 8 ms · 8 tokens · released when idle

8 ms to first token, after an idle hour

3.2 GB resident for a 7B model

Time to first token, after an idle hour

Eclipse8 ms

Warm container180 ms

Serverless GPU, cold4.2 s

Invented product, invented figures. Nothing here was measured.

Questions

Which model formats does it load?

GGUF, safetensors and ONNX, quantised or not. Eclipse reads the file as it is, so there is no conversion step and no exported copy to keep in sync.

What hardware does it run on?

Apple silicon, consumer and datacentre GPUs, and plain CPU. The backend is chosen when the model loads, so the same artifact moves between them unchanged.

How is it priced?

Per second of active serving. When traffic stops the runtime releases and the meter stops with it, so you never pay for capacity sitting idle.

Can we self-host it?

Yes. The runtime ships as a single binary you can run on your own machines. The hosted service is that same binary running on ours.

What happens to our prompts and outputs?

On the hosted service, requests are processed in memory and not retained. Self-hosted, nothing leaves your machines at all.

Come and watch it do nothing.

The interesting part of a runtime is what it stops doing. Put a model in front of it and see how little is left between the request and the first token.

Eclipse