Your code sends prompts from the current batch. Tinker’s samplers generate responses using the current sampling snapshot and return them to your loop.
Your code evaluates the responses and computes rewards, then sends the training batch to Tinker. Its training GPUs compute and accumulate gradients without changing the weights.
On Tinker’s training GPUs, the optimizer applies accumulated gradients to your trainable weights.
When you choose to checkpoint, Tinker stores weights and optimizer state so you can resume later. You choose how often to save.
Your code refreshes Tinker’s samplers with a snapshot of the updated weights, selects the next batch, and repeats. Later training leaves the snapshot unchanged. This is one example loop; you choose when to save and refresh weights.
If a training GPU at Tinker stops responding during a request, Tinker handles the infrastructure failure while your code waits for the result.
Tinker restores training state on healthy resources and retries the work. Your code does not need to manage the GPU replacement.
Once the request succeeds, results return to your loop. Recovery timing varies; the animation shows the handover conceptually.
From your computer, your loop submits more training data and sampling requests through the same API.
Tinker schedules work across shared GPU pools. These tiles illustrate the resources involved; scheduling and latency depend on the workload and system load.
Tinker manages the compute. Results return to your computer through the same calls, and your loop keeps running.