MedARC @MedARC_AI

Journal Club presentation TODAY at 8:30am PT! We will be discussing the FINO paper from Meta: "Who Needs Labels? Adapting Vision Foundation Models With the Metadata You Already Have" Join the Discord and check our calendar!

We hope nanopath can serve as an accessible test-bench to explore novel self-supervised approaches and fosters innovation in the space!

Links with more details: leaderboard: labless.dev/nano-projects/… blog post: sophont.med/blog/nanopath code: github.com/MedARC-AI/nano…

Today, we're excited to announce our first research competition! INTRODUCING NANOPATH: a framework and challenge to train the best pathology foundation model in just 1 hour! A quick thread on why we made this challenge and how to participate!

MedReasoner workshop at @CVPR is just starting in Room 110, packed with some excellent speakers in the medical AI space. Honored to be an invited speaker for the workshop, come check out my talk in an hour (2:20pm)!!

A big focus at Sophont is building foundation models to understand & analyze the brain. This is still a nascent field but we believe such models could eventually help improve diagnosis & treatment of neurodegenerative diseases and mental disorders. This project is some of our initial efforts in the space. We have trained a family of foundation models on functional MRI (fMRI) neuroimaging data that achieves SOTA performance on a variety of benchmarks. We do so by introducing a novel approach for representing fMRI data called flat maps. Of course, since it's still an emerging area, there is a lack of systematic benchmarks, which we attempt to fix with our Brainmarks benchmarking suite. All code and models are completely open-sourced! As usual, this project was done with the support of our broader @MedARC_AI community. If you're interested in contributing to this line of research, please join the MedARC discord! We are now building multimodal MRI foundation models and participating in the FOMO challenge. If you are interested in partnering on neuro foundation models, be sure to contact me directly as well!

Sophont @SophontAI

4 weeks ago

NEW RELEASE: Today we're releasing CortexMAE: a family of fMRI foundation models trained on 2.1K hours of open fMRI data. We're also releasing Brainmarks: an open benchmark suite for evaluating fMRI foundation models. Full paper is on arXiv (accepted to ICML 2026) A thread:

4 44 202 41K 126

STARTING NOW!

MedARC @MedARC_AI

a month ago

We're excited to announce we're starting a Journal Club. And our first meeting is scheduled for tomorrow! @__init_self will present her work, CorText: Brain-Language Fusion Enables Interactive Neural Readout and In-Silico Experimentation Tomorrow at 10:15am ET, join Discord!

1 5 14 14K 4

We're excited to announce we're starting a Journal Club. And our first meeting is scheduled for tomorrow! @__init_self will present her work, CorText: Brain-Language Fusion Enables Interactive Neural Readout and In-Silico Experimentation Tomorrow at 10:15am ET, join Discord!

Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks We (@SophontAI) recently released our medical benchmarking research on arXiv. "We introduce Medmarks, a fully open-source evaluation suite with 30 benchmarks spanning question answering, information extraction, medical calculations, and open-ended clinical reasoning. We perform a systematic evaluation of 61 models across 71 configurations using verifiable metrics and LLM-as-a-Judge. Our results show that frontier reasoning models (Gemini 3 Pro Preview, GPT-5.1, & GPT-5.2) achieve the highest performance across both benchmarks, most frontier proprietary models are significantly more token efficient than open-weight alternatives, medically fine-tuned models outperform their generalist counterparts, and that models are susceptible to answer-order bias"

Medmarks was organized within MedARC, join the server to contribute to our continued eval efforts and medical post-training efforts!

Sophont @SophontAI

a month ago

We're excited to release Medmarks v1.0 + a technical report! This is an update to our Medmarks benchmark suite, the largest open-source automated suite for evaluating the medical capabilities of LLMs. We added 10 benchmarks (20→30) and 15 models (46→61) to the leaderboard!

3 26 96 42K 43

We're excited to release Medmarks v1.0 + a technical report! This is an update to our Medmarks benchmark suite, the largest open-source automated suite for evaluating the medical capabilities of LLMs. We added 10 benchmarks (20→30) and 15 models (46→61) to the leaderboard!

A good discussion about evaluating LLMs in medicine by the head of Health AI at OpenAI, highly recommend reading. Appreciate the shoutout for Medmarks, our LLM evaluation suite developed at @SophontAI/@MedARC_AI. Glad to see even folks at frontier labs are finding it useful!

Karan Singhal @thekaransinghal

3 months ago

x.com/i/article/2032…

34 58 304 65K 391

This release was only possible by the numerous MedARC volunteers who implemented benchmarks and datasets to evaluate with. Grateful to all those who contributed!

Sophont @SophontAI

6 months ago

We're releasing Medmarks v0.1, the largest completely open-source automated evaluation suite for assessing the medical capabilities of LLMs! Developed in our @MedARC_AI community, w/ support from @PrimeIntellect So far we’ve explored 46 models to figure out the best!

12 55 259 134K 152

We're releasing Medmarks v0.1, the largest completely open-source automated evaluation suite for assessing the medical capabilities of LLMs! Developed in our @MedARC_AI community, w/ support from @PrimeIntellect So far we’ve explored 46 models to figure out the best!

Sophont had a great NeurIPS last week! We presented our fMRI foundation model research at the Brain and Body Foundation Models workshop (which we also co-sponsored!) We also held a social with @KindredVentures (and @_CausalLabs, @fal) including a great panel with our CEO Tanishq Abraham, MIT prof Paul Liang, Stanford prof James Zou, moderated by Kanyi Maqubela from Kindred. We discussed the importance of multimodal models for medicine, open-source, open research problems in the space, agents for accelerating scientific discovery, and so much more. It was great to connect with so many folks interested in medical AI, see you next time!

keyboard_arrow_left Previous keyboard_arrow_right Next

The MedARC community was in full attendance at NeurIPS!

Sophont @SophontAI

6 months ago

Sophont had a great NeurIPS last week! We presented our fMRI foundation model research at the Brain and Body Foundation Models workshop (which we also co-sponsored!) We also held a social with @KindredVentures (and @_CausalLabs, @fal) including a great panel with our CEO

keyboard_arrow_left Previous keyboard_arrow_right Next

5 5 32 23K 3