At least when it comes to chess, Magnus Carlsen has stated multiple times that he's been inspired by AlphaZero and adjusted his own style of play after studying its games.
> In this report, we argue that frontier performance can be achieved by a wide range of institutions through Continual Learning on readily available open-weight models.
> As opposed to existing limited approaches such as small-scale fine-tuning, prompt engineering, or tool-augmentation with a frozen model, our Continual Learning approach takes advantage of the effectiveness of a modern mid- & post-training stack while introducing safeguards preserving both plasticity and stability at each training stage and seeking to make the minimal number of high-impact interventions on the parameters.
For the large model, Thomson is utilizing the fine tuning stack they describe in the article, running it on Snowdon 1.0-Large, which in turn is a fine tune of Qwen3.5 397B. Same thing for the small model, but it's a fine tune of Snowdon 1.1-Small, which is a fine tune of Qwen3.6 35B.
As for the small version's run:
> The full pipeline consumed approximately 1.63 × 10²³ FLOP over 35,207 B200 GPU-hours, showing that these results are achievable with compute and personnel budgets substantially lower than commonly thought.
That would amount to around a quarter to half a million dollars of spend on that run. 100k minimum, if they got a great deal.
Playing the devil's advocate: The law also determined that any space, public or private, can't be used for same-sex sexual activity. So that was a driving force for people seeking completely anonymous encounters, in spaces that aren't linked to either person.
What about doing abliteration, weight pruning, representation engineering, etc, directly to open LLMs instead?
Building an LLM from scratch has a hard split between a tutorial project you can complete in a weekend (that's useless for actual usage) and then a solid 1km high brick wall if you want to create anything actually useful from scratch.
Modified open models have a very active community around them, without the need to look much further than Hugging Face.
I find that the models have increasingly started throwing out made up amounts of time around, as if they wouldn't be aware that the user is already using Claude Code or Codex.
It's like every time they make a plan, there's something about things taking "a week or two", "month of focused work", or whatever.
This is something that would've been RL'd out a long time ago if it wasn't great for business.
A model isn't "aware" of anything. It's trained on things, and that training doesn't include time estimates for working with AI because that didn't exist yet.
What I was going after with "aware", is that the actual people working at the companies, training the models, are aware that people aren't mostly going to be implementing the plan by hand, if they've already made the plan in Claude Code or Codex. As for a specific Claude / GPT instance, "aware" would definitely be the wrong word choice there, but the instance does have its stats and environment information in its context window, unless you specifically remove it.
Either way: Training does include estimates on working with the model, and adjustments of the model itself based on that. That's literally what RLHF is.
It's straightforward to have a portion in post training that aims specifically at the model being able to give better estimates on how long that model takes to complete a certain type of task.
In all honesty I don't think it's possible to put this into the model because it's not actually sufficiently understood yet by anyone or anything. AI capabilities are changing too fast, and so is the way AI is used (worst case, even including financial limitations that are now increasingly appearing.)
Every part of training a model have the timing data observed + saved on multiple different axis, it's one of the most inherent parts of the training process.
When the post training run is nearing its cutoff point, there's a massive amount of data on how long coding tasks take to complete by that model in the golden format of "task -> time task took to complete", separable to whatever amount of subtasks, in the same format. With the parts from the end of the dataset being useful for evaluating the finished model's capabilites, whether that data is then fed back into another step in post training or not.
Completely separate from even the actual training: If you have a model proactively giving estimates that are an order of magnitude wrong, you can already fix the worst of it as of this moment by just changing the system prompt. It's a dirty fix, but it's the type of fix that has been used by Anthropic and OpenAI since forever when a model is dishing out blatantly wrong outputs.
The misunderstanding might be that I'm thinking about how long it takes a human + an AI to get something done, not an AI by itself. My current understanding is that it's still not generally viable to have LLMs perform architecture/design tasks (it might work in the short run but degrades roughly exponentially in the long run, accruing serious tech debt). Most things I'd consider actual "projects" involve at least some level of design work as well, and that'd need to be included in the estimate.
(And also prompts will need to be refined, etc., unless you have amazing prompt skills it won't immediately deliver what you wanted. Even if it did, the work to put together the AI inputs also needs to be included.)
⇒ I don't think it's easily possible right now to give a time estimate for **the full picture of** a project to be implemented with AI assistance.
This concept of knowledge degenerating between generations is relevant for software developmnent of today, since many junior developers have a hard time finding a job and if they do they just prompt the AI until they get a result.
Can't blame them. What they need is a senior dev as a mentor to pass down tacit knowledge and experience. They might not get that even if they can find a job.
Then they not only stagnate but simply have to prompt to even deliver anything.
Pretty sure Mythos and Fable have way more params, but they've just been able to use the synthetic data off of them to get the leap in quality from Opus.
So, not a distilled version of Mythos or Fable, but those models likely helped a lot in the post training phase of Opus.
A model being a distilled version of another specific model is a different thing from using synthetic data off of another model.
Anthropic goes to insane lengths to block other labs from training off of their models' output, as it's been done over and over again in the past. But the models that have used synthetic data from Anthropic's models aren't distilled versions of whatever model(s) they got the distilled data off of.
reply