Tinker

Teach an AI model toforecastyourbusiness

Frontier-grade infrastructure for training your own models.

At work in research and industry

Classification

Replicate expert judgment in financial tasks

Bridgewater used Tinker to train Qwen3-235B to recognize what matters to its investors: relevant financial news, signals of interest-rate changes, and useful analysis buried in lengthy documents.

Across six financial information-filtering tasks, the trained model achieved 84.66% accuracy on held-out documents versus GPT-5.5’s 78.2%, approximately 30% fewer errors at one-fourteenth the inference cost.

Accuracy%13.8× cheaper+6.46 points accuracy
  • Frontier models
  • Bridgewater's trained model

Accuracy is averaged across six held-out tasks; costs are source-reported per 1,000 tasks. Both uplift badges compare with GPT-5.5.

Forecasting

Predict clinical events from raw clinical notes

Lightning Rod Labs used Tinker to train gpt-oss-120b to predict clinical events from patient notes (including whether a patient would receive dialysis, a blood transfusion, or mechanical ventilation later in their hospital stay).

In a retrospective evaluation on 500 held-out questions, the trained model reduced prediction error by 27% and expected calibration error by 69% compared with the prompted base model, achieving performance comparable to GPT-5.

Calibration errorlower is better, both axes

27% lower prediction error · 69% lower calibration error

Prediction error (Brier score)

Base model
0.1269 calibration error · 0.1994 Brier score
GPT-5
0.0422 calibration error · 0.1457 Brier score
Trained model
0.0398 calibration error · 0.1453 Brier score

The trained checkpoint is step 200; all models use the same 500 held-out retrospective questions. Dot labels show prediction error (Brier score); the vertical axis shows expected calibration error. Reductions are relative to the prompted base model and rounded to whole percentages. This research evaluation does not establish clinical deployment performance.

Recipe notes: General probability forecasting; no clinical environment.

Forecasting

Improve forecasts of geopolitical, economic, technology events

Mantic used Tinker to train gpt-oss-120b to forecast geopolitical, economic, and technological events. The team paired roughly 10,000 resolved forecasting questions with research gathered in advance, then used reinforcement learning to reward probability estimates that better matched real-world outcomes.

On unseen questions from the Metaculus Q2 2025 benchmark, the model’s rescaled log score rose from 38.6 to 45.8.

Rescaled log scorehigher is better

+7.2 points

Training stage

Before training
38.6
After RL
45.8

The rescaled log score measures probabilistic forecasts; it is not a percentage or an accuracy rate. Evaluation uses unseen Metaculus Q2 2025 questions with research gathered in advance.

Recipe notes: General forecasting example; does not reproduce this research pipeline.

Forecasting

Improve research and prediction of future events

Monoid Labs used Tinker to train DeepSeek V3.1 to research questions, weigh evidence, and predict real-world outcomes.

On a backtest of 100 resolved questions from the Metaculus Spring 2026 AI Benchmark, the trained model achieved approximately 34% lower forecasting error than untrained DeepSeek V3.1 and 22% lower error than Gemini 3.1 Pro High, measured by Brier score.

Brier scorelower is better

~34% lower vs. untrained · ~22% lower vs. Gemini

Model

DeepSeek V3.1 (untrained)
~0.230
DeepSeek V3.1 (trained on Tinker)
~0.152
Gemini 3.1 Pro High
~0.195
Claude Opus 4.6
~0.224

Scores are approximate readings from the source figure; uplift percentages are calculated from those rounded readings. The trained bar is the final checkpoint. Tools and training data changed across checkpoints, and source error bars are omitted.

Recipe notes: General forecasting example; historical search tools require a custom environment.

Search agents

Accelerate agentic search across enterprise knowledge

Glean used Tinker to train Waldo, an agentic search model built on NVIDIA Nemotron 3 Nano, to find the enterprise information needed to answer users’ questions.

Integrated into Glean’s full system with adaptive reasoning, Waldo helped reduce latency by approximately 50% and token usage by 25% while maintaining comparable answer quality.

Waldo in Glean’s full harnessLower is better
  • GPT-5.4 medium
  • Waldo

GPT-5.4 medium searching alone = 1×. Customer-reported results for Waldo in Glean’s full harness. Bars derive from approximate reported reductions, not absolute timings or token counts. Per-call latency is a separate measurement: about three seconds to 250 milliseconds.

Recipe notesWikipedia-search example; does not reproduce this customer’s system.

Search agents

Find the evidence for complex questions from multiple documents

Chroma used Tinker to turn gpt-oss-20b into Context-1, a specialized search agent. Through supervised fine-tuning and reinforcement learning, it learned to break questions into targeted searches, find supporting evidence, and discard irrelevant material to make room for further research.

Across its evaluations, Chroma also reported retrieval quality comparable to frontier models, at a fraction of the cost and up to 10× faster inference.

Retrieval F1higher is better

Up to 10× faster inference

Cost / query (USD, log scale)

Context-1
~0.487 retrieval F1 · ~$0.0031 per query
Gemini 3.1 Pro
~0.529 retrieval F1 · ~$0.062 per query
Claude Sonnet 4.6
~0.484 retrieval F1 · ~$0.08 per query
GPT-5.2 (no prune)
~0.466 retrieval F1 · ~$0.084 per query
Claude Opus 4.6
~0.457 retrieval F1 · ~$0.17 per query
GPT-5.2
~0.443 retrieval F1 · ~$0.056 per query
gpt-oss-120b
~0.439 retrieval F1 · ~$0.041 per query
GPT-5.4
~0.423 retrieval F1 · ~$0.084 per query
Claude Opus 4.5
~0.414 retrieval F1 · ~$0.18 per query
Claude Sonnet 4.5
~0.405 retrieval F1 · ~$0.17 per query
Claude Opus 4.5 (no prune)
~0.359 retrieval F1 · ~$0.4 per query
Claude Sonnet 4.5 (no prune)
~0.317 retrieval F1 · ~$0.28 per query

All 12 configurations from Chroma’s cost figure are plotted. Coordinates are approximate figure readings, averaged across evaluations; cost uses a logarithmic scale. F1 measures retrieval precision and recall, not final-answer accuracy. “No prune” labels the larger-context variants without pruning. The up-to-10× inference speedup is a separate report claim, not calculated from this cost plot.

Recipe notes: Wikipedia-search example; does not reproduce this customer’s system.

Search agents

Train smaller search agents to answer complex questions

Exa used Tinker to train Qwen3-4B to search the web, connect evidence across multiple sources, and answer complex factual questions.

On 2WikiMultihopQA, training raised the model’s accuracy from 50.0% to 83.9%, a 33.9-percentage-point improvement. The trained 4B model also outperformed Qwen3-235B, which scored 77.4%, with both models using the same Exa search tools.

Answer accuracy on 2WikiMultihopQA% · higher is better

+33.9 percentage points

Same Exa search tools

Qwen3-4B before training
50.0%
Qwen3-4B trained on Tinker
83.9%
Qwen3-235B
77.4%

Pass@1 with the same Exa search tools: 200 questions and 200 rollouts per question. Model versions are Qwen3-4B-Instruct-2507 and Qwen3-235B-A22B-Instruct-2507; the dashed reference is the untrained 235B model.

Recipe notes: Wikipedia-search example; does not reproduce this customer’s system.

Continual learning

Train agents to learn from long-running tasks

Long-running tasks can produce feedback after the model has already changed. Trajectory used Tinker to develop SDPO++, a self-distillation method that keeps training stable with these delayed results—even when most attempts fail and only one rollout is available per task.

On APEX-Agents, GPT-OSS-120B’s pass rate rose from 5% to 25%—a fivefold gain on the benchmark and nine percentage points above the original SDPO method.

Pass rate% of tasks

5× pass rate

Training

Base model
5%
SDPO++
25%

Off-policy SDPO++ evaluation on APEX-Agents, with one rollout per task and failed attempts retained. SDPO++ adds PPO and advantage clipping to SDPO.

Recipe notes: Related SDFT example; a different method from SDPO++.

Continual learning

Teach models new skills while preserving earlier ones

MIT researchers developed Self-Distillation Fine-Tuning (SDFT), which uses expert demonstrations to guide a model’s learning from its own responses. Tinker helped the team rapidly prototype ideas.

On tool use, SDFT achieved 70.6% accuracy versus 63.2% with standard supervised fine-tuning. Average performance on prior capabilities stayed near the base model’s level, while standard fine-tuning caused a substantial decline

Tool-use accuracy%

+7.4 points vs SFT

Training method

Base
42.9%
SFT
63.2%
SDFT
70.6%

Qwen2.5-7B-Instruct tool-use evaluation. The uplift compares SDFT with SFT. Retention across six prior-capability benchmarks is not plotted: Base 65.5%, SFT 56.0%, SDFT 65.4%.

Recipe notes: Uses top-K distillation and a static teacher; the paper uses full-vocabulary distillation and an EMA teacher.

Data & environments

Validate how RL environments improve agents on professional tasks

Mercor builds RL environments for AI labs and validates them by measuring how much models improve through training. Using Tinker, it trained Nemotron 3 Ultra on 1,240 tasks spanning investment banking, law, and consulting, using realistic document workspaces, tools, and expert-written rubrics.

After 60 RL steps, the model fully solved twice as many tasks on the held-out 480-task APEX-Agents benchmark: 110 versus 55.

First-attempt success%

2× success rate

Training stage

Base model
11.5%
After RL
22.9%

Customer-reported results on 480 held-out APEX-Agents tasks under identical evaluation conditions. Success rates are rounded to one decimal place.

Data & environments

Test how training data improves AI agents

AfterQuery uses Tinker to test how its post-training datasets improve models across coding, knowledge work, and tool use.

For terminal-based tasks, AfterQuery combined supervised fine-tuning on successful agent trajectories with reinforcement learning in Harbor environments, rewarding test completion and efficient solutions. The pipeline raised gpt-oss-20b’s Terminal-Bench 2.0 pass rate from 3.1% to 17.0%, more than a fivefold improvement

Pass rate%

5.5× pass rate

Training stage

Base model
3.1%
SFT
13.5%
SFT + RL
17.0%

Terminal-Bench 2.0 pass@1, averaged over five runs on 89 tasks. The uplift compares the final SFT + RL model with the untrained base model.

Recipe notes: Related cookbook examples; the plotted result uses SFT followed by RL.

Content & code generation

Post-train for chip-design coding

Architect Labs trains LLMs to turn English chip-design specifications into functional Verilog code. Starting with Kimi-K2.5, the team used reinforcement learning (RL) to teach the model to write Verilog that is both syntactically and semantically correct, guided by a training curriculum of gradually increasing difficulty.

The resulting model, Architect v0.1, raised pass@1 from 48.4% to 54.5% on the 198-problem CVDP benchmark.

pass@1%

+6.1 points

Model

Kimi-K2.5 base
48.4%
Architect v0.1
54.5%

Non-agentic CVDP evaluation: 198 problems, one response per problem, matched fixed token budget, and pass@1 averaged across runs. No agent harness or test-time scaling. Source comparisons not plotted: Architect v0.1 before curriculum 52.5%, Claude Opus 4.6 54.5%, and GPT-5.4 52.5%.

Content & code generation

Turn creative ideas into detailed image prompts

Krea builds foundational image models and creative tools. It uses Tinker to train custom models for prompt expansion and reward modeling, supporting reinforcement learning and evaluation for its image models.

For its latest prompt-expansion pipeline, Krea trained Qwen 3.8 27B to produce more concise reasoning. The fine-tuned model matched Gemini 3.8 Flash’s performance in 90% of evaluated cases while running 15–30× faster at 1/10th the inference cost.

Qwen3.8-27B prompt expanderLower is better
  • Gemini 3.8 Flash
  • Krea

Customer-reported results, including inference optimizations. Gemini 3.8 Flash is the 1× reference. Relative response time is the inverse of the reported speedup; the lighter extension shows the reported range, not a confidence interval. No absolute timings, prices, benchmark figure, or evaluation protocol were supplied.

Recipe notesRelated training workflows; not the reported prompt-expander pipeline.

Research

Make model reasoning easier to monitor

Stanford researchers developed Counterfactual Simulation Training (CST), rewarding reasoning that helps another model predict how answers would change under modified inputs.

In a released experiment using Tinker, the team trained GPT-OSS-20B to make these influences easier to detect. After two training rounds, monitoring accuracy increased from 68.2% to 83.5% on 800 held-out examples, an improvement of 15.3 percentage points.

Monitoring accuracy%

+15.3 points

Training stage

Base model
68.2%
After two rounds
83.5%

Released GPT-OSS-20B experiment: 800 held-out MMLU examples, one seed. The trained checkpoint follows two CST rounds.

Research

Train an AI mathematician to produce verifiable proofs

Axiom Math builds AI for formal mathematics, where every claim has a checkable ground truth. AxiomProver uses Tinker to train via RL, conjecturing, proving, and verifying its own work.

In December 2025 it solved all 12 problems on the Putnam, the world’s hardest college-level math competition: 8 by the end of the competition window and the remaining 4 in the days after, every proof formally verified in Lean.

Problems solvedof 12

all 12 solved

Evaluation window

Competition window
8
Following days
12

The bars show cumulative Putnam 2025 solutions over two time windows, not a before-and-after training comparison.

Recipe notes: Basic math RL; does not reproduce Axiom’s Lean training.

Why teams build with Tinker

  • Tinker allowed us to focus fully on data and reward design, without worrying about infrastructure.

    Eddie ZhouFounding engineer, Glean

  • The only way our small team could have pulled this off is leveraging Tinker. It’s stable, easy-to-use…

    Hammad BashirCTO, Chroma

  • Tinker lets us rapidly iterate on the training loops and reward functions behind our models…

    Ben TurtelFounder & CEO, Lightning Rod Labs

  • Tinker allows Krea to iterate fast on data, reward, and experiment design without worrying about the underlying infrastructure.

    Sangwu LeeML Researcher & Engineer

  • Experiments that used to take weeks and cost a lot to run can now produce results within a day.

    Matthew AitchisonAI research engineer, Mantic

  • …rapid prototyping over ideas, without the need to build the entire infrastructure that is needed for on-policy learning…

    Idan ShenfeldMIT

  • We were thrilled to use Tinker for the project, as it was immensely helpful for scaling up to realistic model sizes.

    Peter HaseStanford

  • The flexibility of Tinker’s API opens up research directions we couldn’t pursue anywhere else.

    Michael ElabdCTO, Trajectory

  • …a small set of primitives that compose cleanly to express a rich space of algorithmic ideas.

    Leopold HallerFounding Member of Technical Staff, Axiom Math

How Tinker works

One example reinforcement-learning loop: sample responses, compute gradients, update weights, save training state, refresh the sampling snapshot, then select the next batch and sample again with the updated weights. Your code controls the loop and chooses when to save. The drawing is a schematic of shared GPU pools.Your dataYour training loopTinker APITinker infrastructuresampleforward_backwardoptim_stepsave_stateSamplersTrainersCheckpointsOne example reinforcement-learning loop: sample responses, compute gradients, update weights, save training state, refresh the sampling snapshot, then select the next batch and sample again with the updated weights. Your code controls the loop and chooses when to save. The drawing is a schematic of shared GPU pools.Your dataYour training loopTinker infrastructureTinker APIsampleforward_backwardoptim_stepsave_stateSamplersTrainersCheckpoints

Use the arrow keys to explore steps. Keyboard focus or hovering over the explanation pauses playback. Press P to pause or resume the animation.

Training loop
  1. Sample responses

    Your code sends prompts from the current batch. Tinker’s samplers generate responses using the current sampling snapshot and return them to your loop.

  2. Compute gradients

    Your code evaluates the responses and computes rewards, then sends the training batch to Tinker. Its training GPUs compute and accumulate gradients without changing the weights.

  3. Update the weights

    On Tinker’s training GPUs, the optimizer applies accumulated gradients to your trainable weights.

  4. Save progress

    When you choose to checkpoint, Tinker stores weights and optimizer state so you can resume later. You choose how often to save.

  5. Refresh samplers and repeat

    Your code refreshes Tinker’s samplers with a snapshot of the updated weights, selects the next batch, and repeats. Later training leaves the snapshot unchanged. This is one example loop; you choose when to save and refresh weights.

Fault tolerance
  1. A training GPU fails

    If a training GPU at Tinker stops responding during a request, Tinker handles the infrastructure failure while your code waits for the result.

  2. Work moves to healthy GPUs

    Tinker restores training state on healthy resources and retries the work. Your code does not need to manage the GPU replacement.

  3. The result comes back

    Once the request succeeds, results return to your loop. Recovery timing varies; the animation shows the handover conceptually.

Scaling
  1. More work arrives

    From your computer, your loop submits more training data and sampling requests through the same API.

  2. Work is distributed across the pool

    Tinker schedules work across shared GPU pools. These tiles illustrate the resources involved; scheduling and latency depend on the workload and system load.

  3. Your loop stays the same

    Tinker manages the compute. Results return to your computer through the same calls, and your loop keeps running.

sample

Generate tokens for chat, evaluation, or RL rollouts.

forward_backward

Run a forward and backward pass and accumulate the gradient.

optim_step

Update trainable weights based on the accumulated gradient.

save_state

Save weights and optimizer state so you can resume training later.

Production-grade research tooling

Enterprise-grade security and privacy

Pricing that follows your usage

Many models,
ready to train

Sign up to train your model

Contact us

Tell us what you're exploring and how we can help.

What can we help with? (optional)

Select any that apply.

We’ll use your information to respond to your inquiry and to send you updates about Tinker. You can unsubscribe anytime. See our Privacy Notice.

FAQs

Join here. If you're a university or organization looking for wide scale access, contact [email protected].
LoRA is an efficient approach to fine-tuning that trains a streamlined adapter instead of updating all base model weights. Our research demonstrates that with the right setup, LoRA matches the learning performance of full fine-tuning while providing more flexibility and requiring less compute.
A dataset of supervised learning examples or reinforcement learning environments. After picking a base model to train on, the Tinker API provides simple functions to compute gradients, update the weights, and sample outputs from the trained model. See our cookbook for examples to get started.
Yes, we have an API endpoint that lets you download any checkpoint you've saved.
Follow Tinker on X for API updates, new model and feature announcements, and weekly roundups of featured projects from the Tinker community. You can also reach us at [email protected].
For questions or account issues, email [email protected]. Visit our support page for the community Discord, bug reports, and feature requests.

Tinker artwork inspired by the TinkerToy Computer, invented by Danny Hillis and Brian Silverman