GMI Cloud Setup: Choose Serverless or Dedicated Before Your First GPU Bill

One-line job: Create a GMI Cloud account, make your first OpenAI-compatible inference call, and choose correctly between serverless endpoints and GPU compute. Audience: Developers wiring up GMI for inference or provisioning GPU compute. Not for: General-purpose cloud workloads (storage, databases). GMI is GPU-specialized. Last verified: 2026-08-24 Evidence weight: documentation-verified

Steps follow docs.gmicloud.ai as of August 2026. Prices move often; the docs deliberately do not mirror them.

Pick your path first

Two ways in, chosen by workload:

• Path A: Inference (serverless API). No infrastructure. Account, API key, OpenAI-compatible calls. Best for prototyping and variable traffic.
• Path B: GPU Compute. Managed Kubernetes clusters, container instances, or bare-metal H100/H200/B200/GB200. Best for training, fine-tuning, dedicated inference.

Quick start: your first inference call

1. Sign in at console.gmicloud.ai
2. Settings → API Keys → create key → copy it
3. Call the OpenAI-compatible endpoint by swapping base URL and key into any OpenAI SDK:

from openai import OpenAI
client = OpenAI(
    api_key="YOUR_GMI_KEY",
    base_url="https://api.gmi-serving.com/v1"  # per docs quickstart; confirm on your model card
)
resp = client.chat.completions.create(
    model="<model-id-from-Model-Hub>",
    messages=[{"role": "user", "content": "Hello"}]
)
print(resp.choices[0].message.content)

Verify the exact base_url against your model card before relying on it; docs show the swap pattern and the endpoint host should be confirmed at setup time. Model IDs also differ per modality, so pull yours from the Model Hub card rather than guessing.

The console, briefly

Inside the Inference tab:

• Dashboard: usage trends, recent activity
• Model Hub: full catalog, filterable by text/image/video/audio; model cards show live input/output rates plus region modifiers
• Playground: test prompts in the browser before writing API code
• My Models: uploaded/fine-tuned models and versions
• Deployments: Dedicated Endpoints management (scaling, versions, traffic routing)
• Storage: artifacts referenced by endpoints

Serverless vs Dedicated

	Serverless	Dedicated
Setup	zero, one API key	provision GPUs, bring your own weights
Billing	per token, no idle cost	reserved GPU time
Scaling	automatic incl. scale-to-zero	you control it
Limits	provider rate limits	none beyond hardware
Best for	prototypes, variable traffic	steady/sensitive production

Practical rule: start serverless until rate limits actually bite, then move that workload to a Dedicated endpoint.

The GPU Compute path (Cluster Engine)

1. Console → Compute → Home. SKU cards list live hourly rate, region, and discounts; a Summary panel shows estimated monthly cost before you confirm
2. Choose managed Kubernetes cluster, container instance, or bare-metal server
3. Bare metal gives root access and custom CUDA stacks, with NVLink intra-node and InfiniBand inter-node networking
4. Docs quickstarts cover H200 and B200 paths; availability varies by region and time, so confirm your chip in the console

Read that estimated-cost panel every time. It exists precisely because committed pricing is sales-contact only and hourly rates move. Screenshot it before confirming; if a later bill surprises you, that panel is your reference point for what the console promised.

Pricing discipline

Instances bill per minute. Marketing-page prices as of Aug 2026: H100 from $2.00/GPU-hr, H200 from $2.60, B200 from $4.00 (limited), GB200 from $8.00, GB300 pre-order. Historical context: launch pricing in May 2024 was $4.39/hr for an H100. Prices have moved repeatedly; always check the console for live rates.

Integrations worth knowing

• Coding tools: GMI models inside Claude Code, Codex, and Cursor (setup guides in docs)
• Agent frameworks: Hermes, Dify, and OpenClaw guides exist, including set up Hermes Agent with GMI Cloud
• REST APIs: IAM, Compute, IDC, Inference references

Wiring GMI into an existing tool

Because the API is OpenAI-compatible, “integration” usually means three values in whatever config your tool already has: base URL, API key, model ID. In coding tools like Cursor or Claude Code you point the provider entry at GMI’s endpoint and pick a model from Model Hub. In agent frameworks such as Hermes, Dify, or OpenClaw, GMI’s docs carry step-by-step setup guides per framework, which is rarer than it should be among GPU clouds and lowers the trial cost for agent-heavy setups.

When to stop on serverless

Three signals it’s time to move a workload to Dedicated: you’re hitting provider rate limits during peak hours, your per-token spend has grown past what reserved GPUs would cost at your utilization level (the vendor’s own crossover math says roughly 60% utilization on H100), or you need weights or serving configurations the shared catalog doesn’t expose.

A five-minute smoke test before you commit anything

Run this before provisioning GPUs or writing integration code:

1. Create the API key and note your region
2. In Playground, run three prompts against your target model; check latency feels acceptable from your location
3. Make the Python call above with the same model; confirm identical behavior over API
4. Check Model Hub’s rate card for that model and region, then estimate your monthly volume against it
5. Only if all four pass: consider the Compute path

This costs a few cents and catches the two most common bad outcomes: picking a model whose live rate doesn’t fit your budget, and building on a region that adds latency you can’t tolerate.

Region selection

GMI operates regions across the US, Europe, and Asia-Pacific. Region affects both price (SKU cards show regional rates) and latency (the sub-200ms cross-region average is a vendor figure). Pick the region nearest your users or your other infrastructure before comparing prices; a cheaper GPU-hour in the wrong region usually loses money in retries and timeouts.

Bringing your own model to Dedicated

If the shared catalog lacks what you need, Dedicated endpoints accept your own weights: upload artifacts to Storage, register the model under My Models, then create a Deployment choosing GPU type and scaling. The Deployments surface exposes version and traffic-routing controls; exact behavior on re-upload (new version vs replacement) isn’t documented in the quickstart material we verified, so test with a throwaway model before relying on it.

Troubleshooting and gotchas

Symptom	Fix
Rate limits on serverless	Move to a Dedicated endpoint for production
Model call fails with unknown model	IDs differ per modality; copy from the Model Hub card
Price surprise on compute	Region selection changes price + latency; check SKU card before confirming
Unexpected compute charge	Read the estimated-cost Summary panel before provisioning clusters
Need custom committed pricing	Sales contact only; no self-serve commitment tier
Wrong base_url errors	Confirm endpoint host against your specific model card

Done means

Your API key works against an OpenAI-compatible endpoint with a real response printed, you know whether your workload belongs on serverless or Dedicated, and any compute you provisioned was confirmed against the estimated-cost panel first.

What this article does NOT cover

• Whether GMI fits your stack vs RunPod/Lambda/CoreWeave: read What Is GMI Cloud?
• Fine-tuning workflows and My Models pipelines beyond where they live in the console
• Enterprise/committed pricing terms (sales-negotiated, not published)

Related guides

• Cost-aware model routing for agents
• Why every agent needs a cost cap on day one
• Terminal commands versus Python when an AI agent should write a script
• Git commands an AI agent should know

Sources

Primary:

• docs.gmicloud.ai quickstart
• Inference engine overview
• Pricing policy (live-in-console)
• Cluster engine docs
• GMI pricing page

Secondary:

• HPCwire Series A
• BusinessWire May 2024 launch ($4.39/hr origin)

Independent:

• Reddit r/comfyui GPU cloud thread