Liquid's d1-3B returns structured decisions in one forward pass at 16 ms on Jetson Thor

The entry point is `model.system_one()`. It takes a state and a dictionary of named questions, and each question has a type: a yes/no check, a `choice` over labelled criteria, or a `score` along an ordered list ^[12]. Liquid's example sends one support message and asks three things: whether the customer wants a refund, which of three teams should take it, and how urgent it is. All three answers come back from one call ^[12]. Pass `None` for the text and an image becomes the whole state ^[13]. `system_one_batch()` packs separate requests together with no padding ^[14].

In my view this is the right shape for routing. A model that answers in one forward pass and produces no tokens ^[3] has no decode loop whose length depends on the answer. The cost of asking more questions shows up in the post's own figures. Three questions take 1.3 times as long as one, and on the AGX Thor the time goes from 16 ms to 20 ms ^[9]. Those two numbers give a ratio of 1.25 ^[19], or about 6.7 ms per question when three share a state ^[20].

Liquid measured these with NVIDIA on a Jetson AGX Thor, an AGX Orin 64 GB, an Orin Nano and an RTX 4090 ^[8]. On GPU, the post gives under 10 ms per question and under 18 ms for a 384px image ^[10]. It does not state the input length or numeric precision behind the Jetson runs ^[16]. Its loading example uses bfloat16 on GPU and float32 on CPU ^[15]. For 16 ms to transfer, inputs have to be about as short as the one-line tickets in the example, at whatever precision Liquid used. Elsewhere the post says d1-3B answers in under 50 ms on every measured device ^[9]. The Orin Nano figure is 50 ms ^[2], so under in the generous sense.

The quality claims need the same reading. The lead over Decider 35B-A3B is 1.46 points ^[18], measured on version 0.2.1 of the Decision Index ^[1]. A gap that size on someone else's task mix makes the 3B model competitive. Whether it holds on a given set of intents is a question for that team's own labelled data. The post also refers to a v0.3 of the index that contains only a private vision split ^[7]. Liquid says the omni model beats Decider 2B "with only a quarter of the parameters" ^[6]. Going by the model names, 600M over 2B is 0.3 ^[21].

Adoption has one cost I would not wave through. Loading needs `transformers>=5.14`, and the models ship their own code, so `from_pretrained` runs with `trust_remote_code=True` ^[11]. That means Python from the model repository executes on the device at load time ^[11]. I would pin a revision and review that code before it goes into a fleet image. The weights are open and on Hugging Face ^[17].