Running Nex-N2.5 Mini on two H100s: the SGLang container setup, and an honest benchmark read

Step-by-Step Breakdown

1. Check the GPU requirement before anything else

   The model card specifies two GPUs — 2×H100 or any 80 GB equivalent. On a rented-GPU provider you filter to 80 GB cards and set the GPU count to two. This is not a model you fit on a workstation card.

2. Build a template from the model's own container image

   This is the step that saves the afternoon. The model uses its own SGLang Docker container, named in the model card. Rather than using a pre-built community template for vLLM or Ollama, create a new template, set compute type to NVIDIA GPU and template type to pod, and point it at that image. Pre-built templates exist for common stacks and none of them are this one.

3. Choose pod over serverless deliberately

   The distinction is spelled out: a pod is a persistent, always-on GPU machine you control directly; serverless auto-scales per request and spins up and down on its own. For an interactive hands-on session a pod is right; for bursty production traffic serverless may not be.

4. Allocate enough disk

   150 GB of persistent storage is set in the demo before deploying — model weights of this size will not fit in a default allocation.

5. Deploy, connect and serve

   Deploy on demand, then connect over SSH from your own terminal or enable the provider's web terminal. Confirm both GPUs are visible, download the weights, and launch the model with SGLang. Measured in the demo: roughly 66 GB VRAM consumed per GPU, and about 217 tokens per second.

6. Drive it from a harness

   The model is configured with Hermes agent for testing rather than being poked through a raw API — which is the right way to evaluate something marketed as agentic.

Gotchas & Caveats

• The GPU provider appears to be a sponsor or trial arrangement — the reviewer says he has "been trialing" it and the walkthrough uses its interface throughout. Nothing about the model requires that provider; any host offering 2×80 GB and custom container images works.
• Benchmark figures are read off the model card's own charts. The reviewer's honest framing of where the model lands is his assessment of those charts, not an independent run.
• Renting two H100s is not cheap, and the video does not discuss what the session cost.
• This is the Mini variant. Several benchmark comparisons in the video concern N2.5 Pro, which is a different and larger model — do not carry Pro's results onto Mini.

Key Takeaways

• 2×80 GB is the floor, and the measured usage (~66 GB per card) explains why — there is not much headroom.
• ~217 tokens/second on that configuration, which the reviewer characterises as lightning fast.
• Use the model card's own container image. An agentic model with a bespoke serving stack will not behave under a generic template.
• It is a solid mid-tier agentic model, not a frontier one — stated by the reviewer while reading the benchmark chart, not buried in a caveat.
• Tested on real tasks, not just benchmarks: a history-plus-invented-slang prompt to check it holds facts while adopting a register, a self-contained interactive HTML canvas simulation, and an ~80-language multilingual stress test. The canvas output is assessed candidly — good separation of data from render logic, some particle placement wrong, weaker than Opus 5 or Fable but not bad.
• Built by Nex AGI for long-horizon tasks: operating computers, browsing the web, and self-correcting using visual feedback as it works.