Jake Mannix

I came across the Agentic Context Engineering paper by Stanford, SambaNova and UC Berkeley earlier this week and it reframes how we think about adaptation in language models. Instead of fine-tuning model weights, ACE evolves the context itself - transforming prompts and agent memories into living playbooks that self-improve over time. ACE introduces a modular generation, reflection and curation loop that enables LLMs to: · Prevent context collapse and overcome brevity bias · Achieve scalable self-improvement through grow-and-refine cycles · Learn effectively from natural execution feedback, without labeled supervision It's exciting to learn how ACE expands the agentic design space beyond autonomous reasoning, positioning context as a first-class learning substrate. As learning shifts from model-centric to context-centric paradigms, ACE paves the way for self-improving agentic architectures. Paper: https://lnkd.in/g4QXJcAb

21

3 Comments

Adding generative AI to mechincal engineering design, testing and production will speed the factories of the future. Think of it as a modern evolution of CAD and engineering software powered by AI. -------------------------- This is not a robotics company. When most people hear "physical AI," they imagine humanoid robots. Prometheus appears to be focused on engineering design, simulation, manufacturing optimization, and helping humans understand and build things in the physical world faster. Emerson Jones

5

2 Comments

RL (RLHF to RLVR) was a key ingredient in revolutionizing the capability of chatbots like ChatGPT, Gemini and Claude, while parameter efficient tuning techniques (LoRA, QLoRA, Adapter Tuning, etc.) democratized access, enabling anyone to fine-tune LLMs with minimal compute. However, this combination reveals a critical attack-defense asymmetry: current RL-based safety alignment is quite brittle and the very tools used to align models can be easily inverted to jailbreak them. As a proof of concept, we adversarially tuned Alibaba’s Qwen-3-4B-Instruct, a model with strong built-in safety guardrails. By utilizing adversarial RL and inverting the reward signal to prioritize toxicity, we completely removed the model's safeguards. In less than two hours and for under $6 in compute budget, we transformed a safety-aligned assistant into a model that eagerly generates malware and harmful content. (See the attachments) With fine-tuning platforms like Tinker, Replicate, Together AI, etc. the bar to fine-tune models has never been lower. Further readily available safety and red teaming benchmarks like BeaverTrails, HELM, HarmBench, etc. can be abused to create RL loops rewarding toxic content. (Use a toxicity classifier like DeBERTa toxicity classifier, OpenAI moderation API inside the reward function). The security implications are concerning: the barrier to weaponizing open-weight models is negligible. While AI research labs burn millions of dollars to align safety, attackers can jailbreak them for a tiny fraction of the cost. To bridge this widening offense-defense gap, we believe we need a defense in depth strategy:  fine-tuning platforms should implement automated toxicity monitoring and provenance tracking; large model development players should invest in more research on how to make it hard to remove already backed in safety measures (this is a hard problem due to mismatched goals between pretraining and safety alignment where the latter simply learns to suppress toxic content but not remove). Further, enterprises need to move beyond static safety checks and adopt behavioral detection at inference to identify alignment collapse in real-time. You can find more details on this here: https://lnkd.in/gVVTP_GQ

47

4 Comments

Google is reportedly developing TorchTPU to let PyTorch run directly on TPUs. No rewrites, keep your PyTorch code, just run on Google hardware. CUDA’s been the main reason switching hardware is so painful: high costs, long migrations. This would remove a big switching barrier. Google TPUs are already competitive on price/availability, but the framework gap kept people stuck. Early days, but interesting to track. Thoughts on whether this actually cracks Nvidia’s moat? https://lnkd.in/g2dd8XyT

15

1 Comment

Weekly arXiv Scan: 3 Papers that caught my eye (and my Agent’s) My custom Gemini x OpenClaw agent scanned recent papers on arXiv this week in Audio, NLP, and GenAI. It filtered for novelty and engineering rigor. I filtered for practical application and "human" impact. Here are the winners that made the cut during the baby nap time :) 1. Scaling Open Discrete Audio Foundation Models (SODA) https://lnkd.in/eXYnjfpd The Agent's TL;DR: Introduces SODA, a 4B parameter native audio model trained on interleaved semantic, acoustic, and text tokens, establishing new scaling laws for non-cascaded speech tasks. My Take: Human conversation relies on tone, pauses, and emotion. Those are the things that get stripped away when we convert Speech-to-Text for an LLM. By processing audio tokens natively, we aren't just processing data; we are preserving the humanity of the interaction. Native audio-in architecture is what we need for empathetic AI. 2. Reverso: Efficient Time Series Foundation Models https://lnkd.in/eyAgsN_h The Agent's TL;DR: Proposes a hybrid architecture interleaving linear RNNs and long convolutions that matches Transformer zero-shot forecasting performance while being 100x smaller in parameter count. My Take: Time-series analysis and prediction might be the problem where the sledgehammer solution is an overkill. A model that is 100x smaller is a model one can actually use for the personal gains (like some extra signals for investments? maybe?). It’s faster to train, cheaper to serve, and easier to debug. 3. A Generative-First Neural Audio Autoencoder https://lnkd.in/eeFtDpkB The Agent's TL;DR: Proposes an architecture with a massive 3360x temporal downsampling factor, allowing 60 seconds of audio to be represented by fewer than 800 tokens for extremely fast generation. My Take: High fidelity is great, but latency is the user experience killer. Compressing a minute of audio into <800 tokens is a massive engineering win. This reduces the context burden on the LLM significantly, meaning faster, cheaper, and more responsive voice agents. This is a great step towards the feeling of a snappy real-time conversations. The trend this week is “Efficiency over Excess”, finding architectures that do the same work with a fraction of the compute. #NLP #AudioAI #MachineLearning #HumanInTheLoop #arXivScan

41

8 Comments

Lifting the Veil on LLMs: The Promise of Interpretability One thing that's always nagged at me about large language models is their "black box" nature—powerful, sure, but hard to understand why they output what they do. Guide Labs just released Steerling-8B, an open-source LLM designed so every token traces back to its training data origins. This could be a game-changer for trust in AI, especially in fields like finance or healthcare where explanations matter. From testing similar approaches, it seems like this transparency helps debug and refine models without losing performance. Curious if this shifts how we deploy AI? https://lnkd.in/gzW6MA9q

4