LanesLanes
Use cases/Lanes Compute/Run inference and model serving

Run inference and model serving

Serve models on GPUs sized to your traffic, with your own serving stack, close to your data.

Serving a model is a different shape from training: steadier, latency-sensitive, and close to your users or data. Lanes Compute gives you GPUs for inference with the serving stack you choose.

Lanes Compute is a GPU service in limited early access. For serving, three of its properties do the work: you pick the card, you pick the region, and you bring the container. The hardware ranges from L40S up to H100 and H200 SXM, billing is per second, and what runs on it is your image, not a hosted runtime.

How Lanes Compute does it

  • Pick the GPU. L40S for lighter serving, H100 or H200 for large models. A100 sits in the range as well, and the choice is per deployment: swap cards as the model or the traffic changes rather than living with a first guess.
  • Your serving stack. Run vLLM or your own setup, in your own container. Anything that runs in a container works out of the box, so the server you built and profiled is exactly what faces production traffic, batching and caching choices included.
  • Regions near your data. Place compute where your data and users are. You choose the region, which keeps latency low and transfers cheap, and your data stays on volumes you control. Lanes does not train on your data or share it.
  • Per-second billing. Scale up for load, down when it is quiet. Add cards for a launch and drop them after, paying only while each environment is live. For a steady baseline, reserved capacity is available for longer runs.

Why it matters

You serve on hardware and software you control, sized to real traffic rather than a fixed reservation.

Serving spend is permanent in a way training spend is not: whatever shape you choose, you pay for it around the clock. Keeping that bill tracking the traffic curve means making resizing cheap, which is what per-second billing plus per-deployment card choice amounts to. When traffic doubles, you add hardware; when a smaller model halves the requirement, you drop to a lighter card the same day.

Set it up

  1. Request access. Lanes Compute is in limited early access; the form on the Compute page asks what you want to run.
  2. Describe the service. The model, the expected traffic, and where your users and data are. That is enough to size the first deployment and pick its region.
  3. Get onboarded. Usually within a day or two.
  4. Serve and adjust. Bring the container, put the model weights on a volume, and resize as real numbers come in. Environments you tear down stop billing immediately.

Where this breaks without Lanes

A fixed reservation gets sized for the peak hour, then idles through the night on the same bill. Managed inference endpoints avoid that but absorb your stack: you serve on someone else's runtime, tune only what it exposes, and migrate when it changes. Renting the card while owning the container keeps sizing flexible without giving up the stack.

Further reading

See Lanes Compute.