Supersonic Labs trained Julia 1 for $104 on cloud GPUs, then benchmarked inference on a laptop CPU

On each call, the caller sends a context, a question and its own candidate labels. It gets back a score for each label, in the order it sent them ^[3]. The integration code is an argmax plus an index into a list the caller already holds. A generative model doing the same job streams text such as "The correct category is: billing", and the caller then has to parse the label back out ^[19]. Julia 1 never generates text, so it cannot return a malformed answer. It can still return the wrong label ^[16].

The same interface covers three question types: pick one option from a closed list, place the input on an ordered scale, or give the probability that a statement is true ^[4].

The $104.08 is the lab's cloud GPU bill for training and experiments ^[2]. An adopter does not pay it. The weights are on Hugging Face under Apache 2.0 ^[1], so the running cost is CPU time. The bill is small because of a design choice I think is correct. According to the dev.to write-up, the team lacked the data, infrastructure, time and money to train a multilingual model from scratch. Instead it took mmBERT-small, a multilingual encoder from Johns Hopkins CLSP, kept its tokenizer, and trained an added decision head to score supplied options ^[11].

Quality held on CPU. The GPU evaluation scored 73.15% on 2,000 typed decisions ^[17]. A September 25 re-run on a plain CPU scored 72.55% ^[18], 0.6 points lower ^[25]. An Apple M4 took a median 33.15 ms per decision one call at a time, about 30 a second ^[21]. Batched 16 at a time, it reached about 51 a second ^[8]. On an Intel Core i5-1235U the median was about 295 ms ^[8], roughly 3.4 sequential decisions a second ^[24]. The Samsung SM-X510 tablet ran at 203 ms through ONNX Runtime, with the 550 MB weights memory-mapped and peak process memory at 393 MB ^[9]. RuntimeWire reports five decisions a second on that tablet, running on CPU after the accelerator path fell back ^[10]. That is consistent with 203 ms per call ^[20].

The lab documented the failure itself. Each native call accepts 2 to 20 options ^[3]. Banking77 has 72 labels, so Julia 1 sends them through a Router that narrows candidates in groups before the final pick. That narrowing can discard the correct answer ^[7]. If an early round drops the right intent, the final pick has no correct option left to choose. Julia 1 got 64 of 100 against an 87% reference ^[6], 23 points behind ^[22].

The wins deserve the same scrutiny as the loss. On a 100-example pilot each example is worth one point, so the 38-point emotion gap (86 of 100 against 48%) rests on 100 cases ^[5][23]. The typed-decision margin over Jev is 0.45 points, or 9 of 2,000 cases ^[26]. Jev is another small decision model in the same category ^[17]. The published evaluation does not include a head-to-head against a generative LLM on the same labels. RuntimeWire describes the pilots as company-reported results, not evidence of performance across production workloads ^[14]. The lab's page lists the limits: no external knowledge, no multi-step reasoning, and 100-example pilots that are signals, not guarantees ^[12].

These numbers transfer only under three conditions. A team's labels have to fit in 20 options or survive the router. The labels have to be distinct from one another. And 200 to 300 ms per decision on low-end hardware has to fit the latency budget ^[8][9]. The lab's own example, routing a ticket to billing, shipping or access ^[4], meets the first two. For label sets like that, I think a trial against an existing LLM classifier makes sense. According to the dev.to write-up, Supersonic's planned hosted API costs $0.025 per million input tokens with no output charge, against $0.042 for Jev ^[13].