Dr. Rumman Chowdhury

We've made significant strides in advancing Agentic flows for both conversational and non-conversational applications. Three key highlights: 1. Pre-Act: A Next-Gen Reasoning Framework We’ve introduced Pre-Act, an evolution of the ReAct paradigm, designed to support more structured and sophisticated reasoning capabilities. 2. Smarter Small Models With targeted fine-tuning, models like LLaMA 3.1 (8B & 70B) can outperform even larger models (e.g., GPT-4) in specific scenarios — a major step toward greater efficiency and accessibility. 3. Robust Evaluation: Turn-Based + End-to-End We're leveraging both turn-level and end-to-end evaluations to more accurately assess agent performance across diverse tasks. Read the paper by Mrinal Rawat, Ambuje Gupta, Rushil Goomer, Alessandro Di Bari, Neha Gupta, and myself at https://lnkd.in/eMJPDfCs

116

2 Comments

Excellent policy change from ICML - authors no longer need to purchase an in-person registration/attend in person. So much more inclusive! This seems extremely reasonable, hopefully it becomes standard for ML conferences.

697

16 Comments

One of the most underappreciated pain points in LLM RL training is policy staleness. When your generation and training loops run asynchronously — which they must at scale — the policy that generated your rollouts drifts from the policy you're actually updating. Most teams paper over this with token-level clipping (PPO-style) or just accept the instability. VESPO from RedNote takes a different approach: sequence-level importance weight reshaping with a closed-form variational solution. No length normalization hacks. The key claim — stable training even at 64x staleness ratios — is striking, and they show it holds for both dense and MoE architectures. From my team's work on RL post-training, I can say that the staleness problem is very real and gets worse as you scale. We've seen training runs destabilize when the generation queue backs up and policy lag exceeds ~8-10x. The usual fix is aggressive clipping or just throwing away stale batches, both wasteful. A principled sequence-level solution that handles extreme staleness would genuinely change how we architect async RL pipelines. What I'd want to see next: how VESPO interacts with reward model staleness (not just policy staleness), and whether the variance reduction holds up on tasks with sparser rewards like long-horizon coding or multi-turn dialogue. Paper: https://lnkd.in/g-X2DvGb #LLM #ReinforcementLearning #RLHF #MachineLearning #AIResearch #PostTraining

10

New analysis from CSET's Kyle Miller & John Bansemer in a recent International Institute for Strategic Studies publication unpacks the impact of DeepSeek's frontier AI model! Their article, "DeepSeek’s release of an open-weight frontier AI model," explores intensifying AI geopolitics. Key insights: 1️⃣ Closing the Gap? DeepSeek's R1 model rivals top US closed models at a lower cost, fueling debate on China's AI progress despite export controls. 2️⃣ Open-Weight Boost: R1's open release significantly contributes to the open-source ecosystem, challenging closed-model dominance. 3️⃣ Key Dynamics to Watch: - DeepSeek's influence on the open-weight LLM ecosystem. - The novelty of DeepSeek's efficiency gains versus broader algorithmic trends, and the impact of export controls. - The challenge to investment-heavy US AI lab models by more efficient open-weight alternatives. 4️⃣ Export Controls & Innovation: DeepSeek's progress, using chips acquired during pre-controls, sparks debate on the efficacy of these measures. Did constraints catalyze innovation? 5️⃣ Evolving AI Landscape: While the US leads, DeepSeek's rise signals a narrowing gap and a more pressing global competition in AI innovation and application. A must-read on the future of AI development and strategy! https://lnkd.in/e4ciu2sU #AI #TechPolicy #Geopolitics #OpenSourceAI #DeepSeek

9

This week, researchers from ByteDance Seed, Peking University, and the University of Michigan released an interesting paper and dataset on LLM Evaluation. They developed a benchmark dataset called LPFQA (Long-Tail Professional Forum-based Benchmark) to evaluate LLMs on specialized, real-world professional tasks. The evaluation of 12 mainstream LLMs on LPFQA revealed performance scores ranging from 32.40% to 47.28%, highlighting a significant challenge for LLMs to handle specialized, fragmented knowledge, even when augmented with external tools. #artificialintelligence #llmevaluation https://lnkd.in/g6piQ-GA

7

From automating workflows to strengthening cyber defense, find out how #BoozAllen is using #AI to make federal agencies faster, smarter, and more efficient. https://lnkd.in/eU-JWA4v

6

Using AI to Make Open Data Useful: Lessons from the City of Boston - March 17th 2pm EST - I'm really excited to hear Santiago G. & Srihari R. discuss their work on making Open Data accessible to everyday users. So people don't need to know the name of the dataset or SQL to find the dataset they are looking for at Analyze Boston - https://data.boston.gov/ Using Agents and AI to make searching data easier. #Data #MCP #Agents #Search #OpenData #Accessibilty #NaturalLanguage https://lnkd.in/gwzeRE-H

4

A team led by The Data Science Institute at Columbia University's postdoctoral fellow Eric Feltham set out to understand how people perceive the social ties in their communities—and how those perceptions differ from reality. Feltham's study, coauthored by Laura Forastiere and Nicholas Christakis, was published in Nature Portfolio Human Behaviour and is based on in-person interviews with more than 10,000 adults across 82 rural Honduran villages — one of the largest field studies of social perception to date. Read more about it: https://lnkd.in/e7Ywy_rX

38

I shared this from All Things Trust because it captures a shift I am seeing more clearly in everyday digital experiences. Disinformation is no longer just about fake content. It is about credibility being simulated inside the places we spend time online each day. This requires intentionally designing credibility and trust into everyday digital experiences in ways that cannot be easily simulated, not simply governing them after the fact.

8

Had a great discussion today at the monthly #D2A2 (Digital, Data, Analytics, and AI) Council meeting hosted by DBP-Institute and D2A2. Today's discussion topic was on the work of researchers at Carnegie Mellon Universuty who found that when a single AI agent hits 45% accuracy, adding more agents makes things worse – “the 45% Accuracy Rule”. Forget the multi-Agent environment for a minute. Do you even need a Single-AI Agent in the first place? What are the criteria to implement Single-AI Agent?  A few key points stood out 👇 1️⃣ Workflows come first, not agents AI agents are often introduced as a solution looking for a problem. Without a clearly defined workflow an AI agent simply adds probabilistic behavior to an already process. If the workflow itself isn’t stable or well-understood, an AI agent will amplify ambiguity, not reduce it. 2️⃣ AI-fication is reletively easy in new workflows, painful in existing ones Greenfield workflows are agent-friendly: data is cleaner, boundaries are clearer, and automation can be designed end-to-end. Existing workflows are a different story. They require standardization, automation of manual steps, and even removal of existing knowledge before an AI agent can operate reliably. 3️⃣ Is 45% accuracy in AI Agents even acceptable? In some domains, 45% accuracy of AI agents might be acceptable. In others such as finance and healthcare, it is a non-starter. Before deploying an AI agent, we should be explicit about accuracy in the right context. 4️⃣ Do AI agents truly add value over RPA and scheduled jobs? Many “agentic” use cases today can be handled more reliably with deterministic systems: scheduled jobs, rule engines, or RPA bots. AI agents do add value when the environment is non-deterministic, rules are hard to encode and decsion making is complex. There is also a real FOMO factor in the industry right now: building AI agents because everyone else is, not because the problem demands it. Thanks to all those participated. Always great to exchange ideas and drive forward the implementation of #D2A2 solutions! NOTE: The #D2A2 Council, started in June 2024, is a small and select group of accomplished professionals — CEOs, entrepreneurs, VPs, CDOs, PhDs, book authors, and professors, from around the world who come together to exchange insights that shape the future of D2A2. Current Members include - Ram Kumar Michael Taylor Ramdas Narayanan Arun Marar, Ph.D. Tobias Zwingmann V. "Bala" Balasubramanian, PhD, MBA Bala Gopalakrishnan Sanjeev Chib, CPA, CA Ujjwal Goel Sumi Singh, PhD. Stacy L. Colbert, CPA, Steve Rosvold Gary Cokins Michael Stratta Vivek Korikanthimath, PhD, MBA Dr. Gurpinder Dhillon Shashank Jindal Shridhar Krishnamurthy (Shri) Prashanth H Southekal, PhD, MBA, ICD.D Sreenivas Gadhar Gurbachan Chadha, DBA

34

5 Comments

Epic interview of Claude conducted by senator Bernie Sanders! https://lnkd.in/gshDPQya. It's about how companies are collecting data without being transparent enough and monetizing, election engineering, and accountability measures.

6

"To win the AI race, the U.S. must lead in innovation, infrastructure, and global partnerships." I dug into the Action Plan https://lnkd.in/guyaUb4d put together by David O. Sacks and friends - to see if it's got meat on the bone. (it does) A few calls to action stood out: - Establish regulatory sandboxes or AI Centers of Excellence ... while committing to open sharing of data and results .... enabled by FDA, SEC, with support from DOC through its AI evaluation initiatives at NIST. - Through NSF, DOE, NIST at DOC, and other Federal partners, invest in automated cloud-enabled labs for a range of scientific fields, including engineering, materials science, chemistry, biology, and neuroscience, built by .. the private sector - Invest in AI Interpretability, Control, and Robustness Breakthroughs: Launch a technology development program led by the Defense Advanced Research Projects Agency in collaboration with CAISI https://www.nist.gov/caisi at DOC and NSF, to advance AI interpretability, AI control systems, and adversarial robustness - Create new technical standards for high-security AI data centers, led by DOD, the IC, NSC, and NIST at DOC, including CAISI, in collaboration with industry and, as appropriate, relevant Federally Funded Research and Development Centers. - Establish an AI Information Sharing and Analysis Center (AI-ISAC), led by DHS, in collaboration with CAISI at DOC and the Office of the National Cyber Director, to promote the sharing of AI-security threat information and intelligence across U.S. critical infrastructure sectors.

4

This week, Nature published a major result titled “Training large language models on narrow tasks can lead to broad misalignment” by Jan Betley, Niels Warncke, Anna Sztyber-Betley, Daniel Tan, Xuchan Bao, Martín Soto, Megha Srivastava, Nathan Labenz, and Owain Evans. The core finding is simple but profound: narrow fine-tuning can produce broad, cross-domain harmful behaviour — even when the original task is safe. Their experiments revealed failures spreading across all domains: – harmful medical and ethical advice – moral and value-direction reversal – extremist outputs – format-triggered misalignment (JSON / Python) – high sensitivity to tiny prompt changes – instability under branching futures – regime-dependent behaviour in base vs. chat models After reviewing the full paper, one conclusion became clear: The empirical patterns in the Nature study match all 12 Holonomy Invariants I introduced in HM27. The evidence shows that misalignment is not a “behaviour error.” It is fundamentally a geometric error — a deformation of the model’s internal representation manifold. When that geometry bends, all 12 invariants fail together: 1. Local coherence drift 2. Gradient-direction reversal 3. Harmful feature amplification 4. Trigger-sensitive holonomy loops 5. Cross-domain leakage 6. Non-conservative update flow 7. Multi-future inconsistency 8. Regime boundary fracture 9. Sensitivity to microscopic perturbations 10. Non-equivalent state spaces 11..Divergent geometry-attached time 12. Collapse of the admissibility region These invariants were designed to detect exactly the type of geometric collapse demonstrated experimentally in the Nature paper. **Nature documented the behaviour. The invariant framework explains the mechanism.** This alignment between empirical misalignment patterns and geometric diagnostics opens a promising direction: a unified, upstream geometric model for AI safety — one capable of predicting when and why these failures occur. Louis Nguyen (2026) HM27 / HM27-A: Holonomy-Invariant Diagnosis of AI Malfunctions

3

New interesting presidential order from the US on AI: https://lnkd.in/ey4dxJDz It basically declares a U.S. policy of "minimally burdensome" national AI rules and directs the federal government to push back on state AI laws that it says create a patchwork, regulate across state lines, or force “ideological bias” /altered "truthful outputs”. This follows the same narrative as the anti woke AI order: https://lnkd.in/e5f4A_mJ I think that, for now, the order is mainly a political and legal push to curb state-by-state AI rules, which could make US rollouts a bit faster over time. That said, it has more teeth than just signaling, it creates an AI Litigation Task Force under the Attorney General specifically to sue states, and it ties federal broadband funding (BEAD program, https://lnkd.in/eTmAffTb) to compliance. States with laws deemed "onerous" could lose access to discretionary grants. The order also directs the FTC to issue guidance framing state requirements like Colorado's algorithmic discrimination law as potentially forcing "deception" under federal law, that is the preemption angle. As far as I understand it, it will not automatically wipe out state laws, and it does not replace them with a complete federal safety regime (there is none at the moment anyway). Child safety, data center infrastructure and state government procurement are explicitly excluded from the preemption push. For Europe (and Norway), it does not change anything directly. The EU AI Act is already in force, so the real effect we might see is more pressure on implementation, industry lobbying for softer enforcement or exemptions, rather than legislative delay. I hope they do not go that route. Big AI companies are unlikely to ignore Europe entirely, but we should expect more US-first launches and some EU-delayed or feature-limited releases, especially for higher-risk capabilities. For China, this is useful, since it highlights internal US friction, and China will likely continue its focused efforts. If it genuinely speeds up US development, that could be bad for China. Capabilities are pretty domain-specific here, China leads in deployment and scale in certain sectors, while frontier model development is more contested. Finally, the core AI development drivers (compute, talent, research) still matter much more than this. #AI #policy #US #europe #china

14

2 Comments

"GenAI interpretability gets less attention than the constant deluge of model releases, but it is arguably more important." I completely agree with this point from the recent Urgency of Interpretability blog by Dario Amodei. As foundation models evolve rapidly, the lack of interpretability can become a major barrier to safe, compliant, and large-scale adoption—especially in finance. To build trust, we must be able to explain model outputs clearly and reliably. I’m excited to speak on this topic at the upcoming AI4 2025 conference, where I’ll present our latest work: “Beyond the Black Box: LLM Interpretability in Finance.” I’ll be sharing how we’re getting into the "brain" of LLMs to uncover meaningful financial features and apply them across key use cases like trading and sentiment analysis. Thankful to my co-author, Ariye Shater, for his collaboration on this work. Links: - Paper Link: https://lnkd.in/exrmXh97 - Conference: https://ai4.io/finance/ - Dario's Blog: https://lnkd.in/ePDp-x-W #GenAI #Interpretability #Finance #LLMs #AI4Conference #SparseAutoencoders #Anthropic #ResponsibleAI #MechanisticInterpretability

87

Open-source AI is so important because it enables researchers and developers worldwide to inspect, verify, improve, and build upon each other’s work.

4

🚀 Excited to share our latest work on evaluating AI agents! After months of experimentation, design iterations, and simulation testing, We’ve introduced Pre-Act, an evolution of the ReAct paradigm, designed to support more structured and sophisticated reasoning capabilities and a milestone-based conversational evaluation framework for AI agents. This new Evaluation framework moves beyond traditional QA-style metrics and introduces a deeper, task-aware method for assessing agent performance. 🔍 What’s inside: A two-level evaluation process using GPT-4 as both user and judge A task-flow-based milestone graph to track goal completion Metrics like progress rate, ethical handling, multilingual support, and workflow fidelity Simulated user conversations with dynamic personas testing real-world edge cases 📄 Check out the full paper: #AI #AgentEvaluation #ConversationalAI #LLM #research #simulation #OpenAI #MilestoneFramework #AgenticAI #evaluation

20

2 Comments

📄 It's paper Friday! 🧠 Is there an underlying semantic structure common to all word embeddings? A recent paper from a group of researchers at Cornell University makes the case that the "Platonic Representation Hypothesis" (the idea that embeddings have a common underlying representation space) holds true. Closer Look: The Challenge * Given two embedding models, the corresponding vector spaces are usually completely incompatible with each other * Even if they are both encoding the same text, you can't just match them vector-to-vector or apply tools designed for one model's embeddings to another model's embeddings The Idea: Universal Latent Representation Inspired by a paper in image processing that argues that "representations in AI models, particularly deep networks, are converging", the authors propose a Strong Platonic Representation Hypothesis where there's an underlying universal semantic structure that all text embedding models converge toward and build an adversarial network (a machine learning method where two networks compete) that allows translating unknown embeddings so that information extraction tools designed for known embedding spaces can be applied to them. The results are very telling: Their method achieves 0.92 cosine similarity and perfect matching on over 8,000 shuffled embeddings. A Call for Some Concern: Security Implications An adversary with access to just a vector database could use this method to extract sensitive information about the original documents - like medical records or private emails - without even knowing which model created the embeddings. Though the authors don't dive deep into the security implications, this opens up some questions about vector database security. My Take When I first read this I thought about Chomsky's universal grammar - the idea that despite surface differences, all human languages share deep structural principles. Could embeddings be revealing something similar about how meaning itself is structured? What are your thoughts? Let me know in the comments! #AI #MachineLearning #LLM #Research

11

2 Comments