Reef's commit stage decides whether a learned candidate ever replaces the version serving traffic

A request goes to Reef over a standard OpenAI-format endpoint, and the response comes back with a record ID attached ^[2]. That ID is the join key. Posting the feedback against it is the application's job: a reward, an evaluator result, or a user correction, sent to Reef's feedback endpoint ^[3]. Nothing in the loop fills that field for you, so the experience stream will be exactly as good as the graders nobody has time to write.

Inside, requests, execution traces and feedback sit in one structured stream ^[4]. Training processors decide which of those records are eligible for learning at all, and evaluation modules decide whether the candidate that comes out is published ^[5]. The learning job runs asynchronously while serving continues ^[7].

The weights half has dependencies. The project site names Slime and SGLang in Reef's model-weights path ^[9], and lists recipes for personalized-chat learning, scientific discovery, prompt evolution and skill development ^[15]. The project describes Reef as fully open source ^[10]. Running the weights path means operating a training stack next to the serving stack, and I would expect most teams to stop before that.

The harness half is cheaper and, in my view, the more interesting one. Qu and his collaborators argue in a Hugging Face community article that the train, evaluate, deploy, serve lifecycle breaks down once agents are expected to learn from their work ^[16]. Their write-up puts many model-blamed failures in the surrounding application: a tool called with the wrong arguments, memory that retrieves irrelevant context, a prompt that invites a plausible answer where the application requires a verified one ^[11]. Retraining weights for each of those is expensive and often unnecessary ^[12].

The gate holds whatever evaluation you install in it. The write-up does not say what the evaluation modules measure, and it reports no results for the four recipes ^[20]. For the release control to do anything, the held-out set behind it has to contain the regression a candidate would introduce, and there has to be enough scored feedback for a candidate to differ from the incumbent. The project's own framing is that continual learning can degrade a working system as easily as improve it ^[14].

The design has visible provenance. Simon Yu's public GitHub materials describe a runtime substrate for reversible, Git-like agent execution traces that support forking and replaying past states ^[17]. Bo Liu, according to his personal site, is a University of Washington visiting researcher who previously worked at Meta FAIR on scalable language-model self-improvement and self-play ^[18]. Qu is a Ph.D. student at MIT's Institute for Data, Systems, and Society, after a mathematics and computer science degree from Vanderbilt, according to his MIT profile ^[19].

If I were adopting this, I would run the observe path first and leave learning switched off: record IDs, feedback, a scored stream, and no candidate. That is the cheap part, and it produces the evaluation set the release gate needs before there is anything to gate.