Marin releases all 57 pretraining checkpoints of its Mochi 535B-A23B model
In a post on X, dlwh announces the release of all 57 intermediate checkpoints from the pretraining of Marin "Mochi" 535B-A23B, published on Hugging Face under open-athena. Ten of them come from the long-context extension phase. The post puts the training run at 18.2T tokens and the weights at about 61 TB. A blog post from Open Athena accompanies the release.
The post had 347 points and 17 comments on X on November 17.
Sources
Get news like this as a push.
AI Tracker alerts you when something changes for the tools you use.