Computer Science at UT Austin @UTCompSci

After over 10 years at Stanford, it's time to leave :) I will be joining UT Austin's CS department as an assistant professor in fall 2027! If you're excited to envision the future of interaction with AI, I'm recruiting PhD students this cycle. Come join me!

keyboard_arrow_left Previous keyboard_arrow_right Next

Very happy that a paper originated from a course project last Fall was accepted by ECCV 2026. It is about consistent spherical parametrizations via a generative model. joint work with Sai Karthikey Pentapati, Shashank Gupta, Rajesh Sureddi, Yuezhi Yang, and Alan Bovik

🚨 Excited to share Pragmatic Reasoning via Self-Training, a method for LLM self-improvent on pragmatic reasoning. PragReST improves by +5.37% and +5.50% for Qwen3-8B/14B across pragmatics benchmarks with no human annotations or teacher models. LLMs still struggle w/ pragmatics: understanding what a speaker means, not just what they literally said. They often default to literal interpretations and miss implicature, intent, or context-dependent meaning. To close this gap, we started with a key question: Can we treat pragmatics as a LLM reasoning task? ➡️ Following a long line of work in pragmatics (e.g. RSA, IBR), PragReST treats pragmatic understanding as counterfactual reasoning. Instead of teaching models to ask “is this interpretation compatible with the words?”, we teach the model to reason about questions like “if the speaker meant something else, what would they have said instead?” ➡️ PragReST is self-improving: it self-generates pragmatic QA data, self-filters noisy examples, learns counterfactual reasoning traces via SFT, and further improves with GRPO using a self-judged correctness reward. ➡️ Error analysis shows that gains correlate with increased counterfactual reasoning. This suggests PragReST’s improvements are tied to reasoning over communicative alternatives, rather than simply more pragmatic data or more training. 🧵👇

@michelle123lam Welcome you to UTCS!! 🤘🎉

We are deeply saddened by the loss of Joshua Baer, a longtime mentor and advocate for student entrepreneurship. For over a decade, Josh led the Longhorn Startup program, inspiring students to pursue bold ideas and create lasting impact. We extend our heartfelt condolences.

1/7 If you’re at #PLDI2026 in Boulder this week, come see what our group has been up to! We’re presenting work on making code generation more interactive and reliable, speeding up data pipelines, porting network data-plane programs,

What happens when experts in artificial intelligence, robotics, engineering and cancer care come together to tackle some of medicine's biggest challenges? Researchers from UT Austin, @DellMedSchool and @UTMDAnderson Cancer Center are joining forces through the Collaborative Accelerator for Transformative Research Endeavors. More than 50 researchers are working across disciplines to advance personalized robotic-assisted surgery, AI-powered precision medicine and new approaches to cancer care and treatment. Learn more: utex.as/4ouRxEJ

🚨 Test-time intervention for CUA tasks is hard: history is hard to represent, actions require visual grounding and verification before execution, not after. HiViG jointly tackles these points, learning to track history and verify actions against the GUI screenshot. As a test-time method, HiViG is compatible w/ open- and closed-source models and is domain- and model-general: we see 5.8-9% accuracy gains across WebArenaLite2 (web), AndroidLab (mobile) and WindowsAgentArena (desktop), and across models/model classes (e.g., Qwen3-VL-32B, Gemini-3-Flash), with especially large gains on challenging/long-horizon tasks (+19.2% on WebArenaLiteV2 Maps, +18.6% on WindowsAgentArena Office). 🧵👇

hyunji amy lee @hyunji_amy_lee

2 weeks ago

🚨 Introducing HiViG, a test-time intervention framework for long-horizon GUI tasks. By tracking history & verifying actions w/ visual grounding, HiViG boosts performance across diverse GUI environments even for strong policies where existing critics often degrade performance.

3 26 65 11K 13

@_yudaitanaka Welcome to UTCS, Professor Yudai! 🤘🎉

I’m thrilled to share that I’ll be joining UT Austin Computer Science (@UTCompSci ) as an Assistant Professor, starting Jan 2027! I'm recruiting PhD students for Fall 2027, postdocs, and researchers at all levels: forms.gle/WMbb7NYts1fX5B… I’ll continue my work (yudai-tanaka.com) in Human-Computer Interaction (HCI) and start the Symbiotic Interfaces Lab, where we’ll explore how computational systems & humans can co-adapt to augment human sensorimotor and cognitive abilities. 🦾🧠

🤖 How can robots learn long-horizon object state change tasks like mashing a banana 🥣🍌, spreading ketchup on bread 🍅🍞, or slicing a cucumber 🔪🥒? Introducing SPARTA: object state-change manipulation via visual spatial progress 👇 🌐 vision.cs.utexas.edu/projects/spart…

Can an LLM act as a selective model of a GPU during evolutionary search, by reasoning + forecasting a kernel’s runtime but deferring to a GPU when unsure? We produced 12k kernels + runtimes from evolutionary search, costing 400M reasoning tokens + 600 GPU-hours to answer this. In our work GPU Forecasters, we study language models as selective surrogates for GPU kernel optimization. 1️⃣ Off-the-shelf LLMs can forecast how a GPU responds to a candidate kernel with non-trivial accuracy. If we rank candidates by these predictions and measure only the top 10% on a GPU, the fastest kernel we find is within 20% of the best in the pool. 2️⃣ We want LLMs to not just be accurate but also calibrated, so that we can use their uncertainty for selective prediction: during search, we should trust only confident forecasts and verify less confident forecasts by sending them to the GPU. 3️⃣ We train an open-weights surrogate (GPT-OSS-20B) with RL to improve both accuracy and calibration. Calibration-shaped rewards improve both confidence reliability and ranking ability, while correctness rewards alone do not. 4️⃣ Inside a real kernel search, the surrogate finds faster kernels than an equal-GPU-budget baseline by considering more candidates per measurement. 5️⃣ We release 12,388 LLM-generated GPU kernels with measured runtimes spanning 118 operations, CUDA and Triton backends, 3 GPU types, taking 400M tokens + 600 GPU-hours to produce. This dataset can be used for analyzing LLM-driven evolutionary program search dynamics, post-training LLMs for kernel code generation, and things we didn’t get a chance to explore, like training reward models! Thread 🧵👇

Exciting news on GR00T: NVIDIA announces our first open humanoid robot platform, featuring Unitree H2 Plus and Sharpa hands, to accelerate academic research and facilitate cross-institutional collaboration. R&D in humanoid robotics needs broader participation. Open science is how we build the future faster, together.

NVIDIA Robotics @NVIDIARobotics

4 weeks ago

NVIDIA announces the first open humanoid robot reference design built for robotics research. The NVIDIA Isaac GR00T Reference Humanoid Robot combines the @UnitreeRobotics H2 humanoid robot, @SharpaRobotics Wave five-fingered hands for dexterous manipulation, Jetson Thor onboard

59 246 1K 164K 314

Delighted to finally unveil these results! 🎉 Many congratulations to the team, who worked tirelessly for almost a year to build and evaluate AlphaProof Nexus. We revised many priors during this project — most notably, we discovered that with current frontier models, simple agent loops with compiler feedback can rival more sophisticated systems. We were struck both by the capabilities of our systems and the magnitude of the challenges ahead. I have never been as excited about the potential of formal math to enhance human creativity and bring rigor to AI. Onward! 🚀

Pushmeet Kohli @pushmeet

a month ago

AI agents are advancing research-level math. 🚀 I’m thrilled to share @GoogleDeepMind’s AlphaProof Nexus - an agentic framework for formal proof search powered by Gemini. When applied to a set of open formal math problems, our agent autonomously solved: ✅ 9 open Erdős

80 242 2K 221K 462

Excited that our paper StreamdiffusionV2 received the Best Research Paper Award at #MLSys26! 🚀Video generation is quickly moving from demos to production-facing workloads. It is no longer a turn-based pipeline but should be a streaming pipeline to interact with users. 📖Our project page: streamdiffusionv2.github.io and paper: arxiv.org/pdf/2511.07399 👂Come join the talk if you are interested in streaming video generation. Our talk will be at the Research Track Oral Presentation: Best Paper Session on Tue 8:45AM at #MLSys26 , I will talk about how we attacked the efficiency and quality challenges. Hope to see you there! ❤️Huge thanks to all authors! This work would not have been possible without the incredible effort from the entire team. Big shout out to Tianrui Feng, Zhi Li, @Andy_ShuoYang , @HaochengXiUCB, @lmxyy1999 , @lvminzhang , @xiuyu_l , Keting Yang, @ZiqiPeng, @songhan_mit , @magrawala, @KurtKeutzer , and @cumulo_autumn

Are state-of-the-art AI review systems capable of providing meaningful reviews in an actual AI conference? This paper explains the findings from the AAAI 2026 AI Review Pilot 1/N

AAAI @RealAAAI

2 months ago

We are thrilled to present a detailed report describing the system built for the AAAI-26 AI review pilot, the survey results, and a new benchmark that was created to assess the capabilities of the system. Read the full article: arxiv.org/pdf/2604.13940

0 11 80 17K 34

Proud of Texas Robotics! Come see our papers at ICRA: robotics.utexas.edu/news/346

🚨 Excited to share MINTEval, a new benchmark for memory with interference. In real-world settings, agents need to handle continuously changing info (think of all your v2.5_final_final docs) . MINTEval tests memory systems on frequent and interfering changes, across challenging question types (long-range lookback/recover, multi-target reasoning) and 4 realistic domains that challenge even the strongest models/agentic memory systems. 🧵👇

hyunji amy lee @hyunji_amy_lee

a month ago

LLM agents & memory systems operate in continuously updated environments (Git repos, evolving docs). They must process long contexts, recover earlier information, and reason over many updates that create interference between old and new information. How well do they handle this?

9 37 107 23K 55

Sparse binary rewards bottleneck LLM RL, motivating the use of privileged information in self-distillation as dense teachers. How can we use and balance multiple types of privileged info: leveraging stable cross-view info, while preserving view-specific info? Current on-policy self-distillation methods often condition the teacher on only one type of privileged view: full solution, partial rationale, answer-only, reference code, feedback, etc. This can be suboptimal: 1️⃣ No single privileged view consistently performs best when used as a teacher. 2️⃣ Views can introduce teacher-specific artifacts from information unavailable to the student. 🧠 Adaptive-View Self-Distillation (AVSD) considers multiple privileged views jointly as a teacher family, balancing cross-view consensus and view-specific signals through a token-level gate to construct better dense learning signals. 🧵👇

LLM agents & memory systems operate in continuously updated environments (Git repos, evolving docs). They must process long contexts, recover earlier information, and reason over many updates that create interference between old and new information. How well do they handle this? We introduce MINTEval: ✅ Frequent context changes & interference (avg. 86 updates) ✅ 5 challenging question types, including long-range lookback & reasoning over multiple targets distributed across context ✅ 4 realistic domains: state tracking, multi-turn dialogue, Wikipedia revisions, GitHub commits ✅ Avg. 138.8k tokens per instance (up to 1.8M) ✅ Human verification on generated QAs = 95.6% 📊 Across 7 representative systems, MINTEval remains difficult, showing an avg. acc of 27.9%, and the best system reaches only 33.4%. 🔎 Our analysis shows: • Memory construction failures cause a 41.7% drop • Memory agents are highly sensitive to design choices • Memory systems have a strong bias toward insertion operations (76.8%) over deletion/update