About
I'm a data scientist, computational social scientist, and technologist with over 10…
Activity
2K followers
-
Rebecca Weiss shared thisPolicymakers have been asking how to make third-party AI assessments trustworthy. This proof of concept with Google DeepMind and OpenMined shows one concrete path: reserved benchmarks inside cryptographically protected environments. The AILuminate stewardship program provided the evaluation set—never used before, never exposed—and the results show that structural guarantees are achievable. This is the standard we should be working toward.Rebecca Weiss shared thisMLCommons has participated in the first-ever double-blind evaluation proof of concept of a proprietary AI model, alongside Google DeepMind, OpenMined, and the AI Verification and Evaluation Research Institute (AVERI). Using AILuminate inside a Trusted Execution Environment, the evaluation provides cryptographic guarantees that model weights and benchmark prompts remain protected throughout the test. This approach shows that benchmark integrity can be ensured by design—not solely by legal contract—and protects against benchmark saturation by guaranteeing that the evaluation set never enters the model's training data. https://lnkd.in/dwN5Vu2z
-
Rebecca Weiss reposted thisRebecca Weiss reposted thisMLCommons has participated in the first-ever double-blind evaluation proof of concept of a proprietary AI model, alongside Google DeepMind, OpenMined, and the AI Verification and Evaluation Research Institute (AVERI). Using AILuminate inside a Trusted Execution Environment, the evaluation provides cryptographic guarantees that model weights and benchmark prompts remain protected throughout the test. This approach shows that benchmark integrity can be ensured by design—not solely by legal contract—and protects against benchmark saturation by guaranteeing that the evaluation set never enters the model's training data. https://lnkd.in/dwN5Vu2z
-
Rebecca Weiss reposted thisRebecca Weiss reposted thisI am looking for an experienced Software Engineer for our MLPerf team If you're looking for high-impact work in the AI infra space, this is your chance to design and drive the technical architecture behind cutting-edge AI benchmarking infrastructure. Take a look at the role and application details here: https://lnkd.in/g_Z7wMUJ PS: Check out what we have been building in Endpoints benchmarking: Blog: https://lnkd.in/gYv8Ysx2 Results: https://lnkd.in/gAah23_G
-
Rebecca Weiss reposted thisRebecca Weiss reposted thisHow do you know if a benchmark result is actually trustworthy - or just benchmark washing? We've been building benchmarks at MLCommons since 2018. We've also seen just about every way a benchmark can fail, be gamed, or be misused. So we put together a practical guide for enterprise teams who need to tell the difference between a result they can trust and one they can't. 7 questions to ask when someone hands you a benchmark score and asks you to make a decision based on it: 1. Does this benchmark actually measure what I'm deciding about? 2. Where did the data come from - and has it been kept clean? 3. Could someone else reproduce this result? 4. Is the score telling you everything you need - or just the easiest thing to measure? 5. Is the system under test passive - or could it be gaming the test? 6. If a model is scoring the results, has that model been validated? 7. Is the benchmark still alive - or has it gone stale? Not as a vendor selling you an assessment. As a consortium that has learned some lessons about what makes benchmark evidence hold up - and what makes it fall apart. Read the full guide:https://lnkd.in/eg_s2xfT #AIBenchmarks #AIEvaluation #EnterpriseAI #MLCommons #BenchmarkTrust #AIProcurement
-
Rebecca Weiss reposted thisRebecca Weiss reposted thisWhat if you could benchmark an AI model on real patient brain tumor data - without the data ever leaving the hospital, and without the model weights ever being exposed? That's exactly what MLCommons and Google Cloud demonstrated at Google Cloud Next 2026 in Las Vegas. We are excited to announce that MedPerf, MLCommons' open-source federated benchmarking platform, is now integrated with Google Cloud's Confidential Space - a Trusted Execution Environment that protects both patient data AND model IP simultaneously. Here's why this matters: federated benchmarking already keeps data local. But until now, there was no guarantee that model weights couldn't leak or that benchmark results couldn't be tampered with. Confidential Space closes that gap - cryptographically. Together, this integration delivers four key guarantees for clinical AI research: 🔒 Patient data stays protected 🔒 AI model IP is protected 🔒 No raw data sharing required 🔒 A global benchmark dataset for healthcare AI becomes possible We demonstrated this on a brain tumor segmentation use case, working with the FeTS consortium led by Spyridon Bakas (Indiana University School of Medicine) and a clinical AI model developed by Dr. Evan Calabrese's team at Duke University. If you are a clinical researcher, AI engineer, privacy expert, or regulator interested in the future of trustworthy medical AI - we want to hear from you. 👉 Explore MedPerf: https://lnkd.in/ecSrH_pA 👉 Learn more: https://lnkd.in/emwKJx6E #MedicalAI #HealthcareAI #ConfidentialComputing #FederatedLearning #GoogleCloud #MLCommons #AIBenchmarking #PatientPrivacy #ClinicalAI
-
Rebecca Weiss reposted thisRebecca Weiss reposted thisThe security world spent 30 years perfecting coordinated vulnerability disclosure. Find the flaw, notify the vendor, ship the fix, then go public. Clean. Logical. Effective. It doesn't work for AI. Three reasons: Findings are dual-use by nature. The same result that tells defenders how a system behaves tells adversaries exactly where to focus. Disclosure doesn't close the hazard — it lowers the cost of finding the next one. Telling developers too much corrupts the test. Hand over exact benchmark prompts and you get models that score better without actually improving. Over enough cycles, the benchmark stops measuring what it was built to measure. You can't patch a released open-weight model. A new version is a new artifact — not an update. Every copy of the prior weights stays operational, indefinitely, in the hands of anyone who kept them. At @MLCommons, we're not just describing the problem. We're building the alternative. We've taken our disclosure principles into ISO/IEC JTC 1/SC 42 to help shape TS 42119-8 — a real, citable standard any evaluator can build on. And when our jailbreak benchmark launches, it will ship with a disclosure policy already aligned to where the field is heading. The patch model gave security a common language for three decades. AI evaluation needs its own. Read more → https://bit.ly/43t8R3t #MLCommons #AIEvaluation #AI #Jailbreak
-
Rebecca Weiss shared thisWe’re proud to announce that we’ve joined Partnership on AI! Artificial intelligence has transformed nearly every industry, so there’s never been a more important time to develop guidelines for this technology. By joining PAI’s community of 140+ organizations from academia, civil society, media, and industry, we’re eager to address the most pressing questions regarding the future of AI. Thanks Rebecca Finlay and Partnership on AI team!Rebecca Weiss shared thisWhile AI systems have become pervasive in business and daily life, safeguards and guidelines are still missing. To mitigate potential harm, we’re excited to announce we’ve joined Partnership on AI, a coalition bringing diverse voices together to ensure developments in AI benefit people and society. Together we’ll work to develop resources, participate in discussions, and connect with PAI’s 140+ Partners. To learn more about our Partnership, click here: https://lnkd.in/dUT-5SqT
-
Rebecca Weiss reposted thisRebecca Weiss reposted thisMLPerf Inference v6.0 is here - and it's the most significant update to the benchmark suite we've ever released. 5 of 11 datacenter tests are new or updated, featuring the industry's first text-to-video generation benchmark, speculative decoding for DeepSeek-R1, an open-weight LLM benchmark based on GPT-OSS 120B, and more. Details: https://lnkd.in/d7yZSM8v Thank you to our 24 submitting organizations: AMD, ASUS, Cisco, CoreWeave, Dell Technologies, GATE Overflow, Giga Computing, Google, Hewlett Packard Enterprise, Intel, Inventec, KRAI, Lambda, Lenovo, MangoBoost, MiTAC Computing, Nebius, Netweb Technologies India Ltd., NVIDIA, Oracle, QCT, Red Hat, Stevens Institute of Technology, and Supermicro. Welcome to our first-time submitters: Inventec Corporation, Netweb Technologies India Limited, and Stevens Institute of Technology! A special thank you to our partners Meta, Shopify, and Ultralytics for contributing real-world datasets and workloads that ensure our benchmarks are truly representative of production. #MLPerf #MLCommons #AI #Benchmarking #MachineLearning
-
Rebecca Weiss reposted thisRebecca Weiss reposted thisAt GTC, MLCommons co-founder David Kanter shared what we're building with MLPerf® Endpoints — a new approach to benchmarking generative AI services that reflects how they're actually deployed. Traditional benchmarks were designed for a different era. MLPerf Endpoints is being built for API-first GenAI: a decoupled client architecture, Pareto-curve visualizations that show only verified operating points, and a rolling submission model so that results stay current. We're sharing early demonstration results from AMD, Google, Intel, KRAI, and NVIDIA — with 30+ organizations already supporting the effort. Rolling submissions are planned to open in Q2 2026. Read more about what we're developing: https://lnkd.in/dPVC2zpz Try it out: Endpoints.MLCommons.org #MLPerf #GenerativeAI #AIBenchmarking #MLCommons #GTC2026
Rebecca Weiss commented on a post
1y
Congrats Siméon!
Rebecca Weiss commented on a post
4y
Guide to privacy principles for data science practitioners: https://iapp.org/l/datascience/
Consumer Reports and Popular Science article: https://www.popsci.com/technology/shared-data-a-short-story-from-an-alternate-future/
Upcoming work from the Rally team: https://gizmodo.com/mozilla-is-going-to-track-facebook-tracking-you-1848338946
-
Rebecca Weiss liked thisRebecca Weiss liked thisMLCommons released results for MLPerf Storage v3.0, and the update marks the benchmark's first move beyond training and checkpointing and into AI inference. The suite adds two new tests: a KV cache test that measures the read and write operations behind LLM inference, and a vector database test that measures indexing and query performance for RAG pipelines. Nineteen organizations submitted 144 performance results this round. Eleven were first-time entrants, a group that included Microsoft Azure, NVIDIA, and Nebius. About one-sixth of submissions ran against the new S3 access layer rather than a traditional file system. The round gave MLCommons its first broad look at power efficiency. On-premises checkpointing write submissions posted a median of 14 GB/second per watt, with a top result of 201. The UNet3D read test showed a median of 34 GB/second per watt and a top result of 277, a spread Curtis Anderson, MLPerf Storage working group co-chair, said was evidence that "there is ample room for further improvement." Brian Belgodere, working group co-chair, said the new tests "round out the test collection, covering a larger range of AI inference workloads that drive storage needs." Together, the KV cache and vector database additions push MLPerf Storage from measuring training and checkpointing performance toward measuring how storage performs once a model serves real users. Read the full article: https://lnkd.in/gKm36SR9 #AI #MLPerfStorage #TechArena
-
Rebecca Weiss liked thisRebecca Weiss liked thisMLPerf Storage v3.0 is now available — the first industry benchmark to measure AI storage across four workloads: training, checkpointing, vector database retrieval, and KV cache serving. This release includes 144 results from 19 organizations, with 11 first-time submitters. New efficiency metrics — rack-unit density and provisioned power — enable direct comparison of performance per unit of physical space and infrastructure. Training now supports S3 API alongside POSIX. Checkpointing spans 8B to 1250B parameters. The VDB and KV cache workloads reflect how storage is actually stressed in modern AI deployments: sustained random reads at scale, and multi-turn conversation contexts reaching terabytes. Congratulations to all 19 submitting organizations. Alluxio, Everpure, FarmGPU, Hewlett Packard Enterprise, HolmesAI, Inspur Group, Microsoft Azure, NVIDIA, Nebius, NewFW, OpenLake Technologies, Samsung Semiconductor, Suzhou Zishan Longlin, Telecommunications Technology Association(TTA), TuringData, UBIX.AI, XSKY Data Technology, YanRong Tech, ZettaLane Systems Full results and supplemental details are available at https://lnkd.in/etVvYTzh
-
Rebecca Weiss liked thisRebecca Weiss liked thisAI Verification and Evaluation Research Institute (AVERI), MLCommons, Google DeepMind, and OpenMined have achieved something the field has long been working toward: an independent evaluation of a closed model where neither the developer nor the evaluator have to expose sensitive assets. For many working in AI assurance, that exposure was a real constraint on evaluating model behavior: sharing weights on the developer side, benchmarks and proprietary measurement practices on the evaluator's. This pilot shows there's a way through, and the tooling to support independent evaluations is maturing. Many are calling for independent evaluation, and this is the work that makes it possible. We’re proud to partner with MLCommons and AI Verification and Evaluation Research Institute (AVERI) on Partnership for Assurance, Credibility, and Trust on AI. We aim to support more of this infrastructure work. Good policy depends on good implementation, and good implementation often depends on pilots that prove it out. These organizations are showing us how. Congratulations to the teams. Grateful for your work.
-
Rebecca Weiss liked thisRebecca Weiss liked thisMLCommons has participated in the first-ever double-blind evaluation proof of concept of a proprietary AI model, alongside Google DeepMind, OpenMined, and AI Verification and Evaluation Research Institute (AVERI). Using AILuminate inside a Trusted Execution Environment, the evaluation provides cryptographic guarantees that model weights and benchmark prompts remain protected throughout the test. This approach shows that benchmark integrity can be ensured by design—not solely by legal contract—and protects against benchmark saturation by guaranteeing that the evaluation set never enters the model's training data. https://lnkd.in/dwN5Vu2z
-
Rebecca Weiss liked thisRebecca Weiss liked thisMLCommons has participated in the first-ever double-blind evaluation proof of concept of a proprietary AI model, alongside Google DeepMind, OpenMined, and the AI Verification and Evaluation Research Institute (AVERI). Using AILuminate inside a Trusted Execution Environment, the evaluation provides cryptographic guarantees that model weights and benchmark prompts remain protected throughout the test. This approach shows that benchmark integrity can be ensured by design—not solely by legal contract—and protects against benchmark saturation by guaranteeing that the evaluation set never enters the model's training data. https://lnkd.in/dwN5Vu2z
-
Rebecca Weiss liked thisRebecca Weiss liked thisAfter 8 years co-founding and leading the development of #MLPerf and MLCommons, the time has come for me to hand the reins of MLPerf to a new leader. What began as an ambitious effort to bring rigorous and industry-wide measurement to machine learning has grown into a trusted benchmark family used across the AI ecosystem. Building it alongside our community has been one of the great privileges of my career. MLPerf's opportunity is only expanding. As AI reaches a broader and more diverse set of users, trusted measurement of performance and efficiency is more important than ever before. MLPerf is evolving with that landscape—from its roots as a silicon-centric performance suite toward a buyer-centric evaluation framework that can help enterprises and other AI buyers make better decisions. That evolution also makes this an opportune moment for new leadership. And we’re looking for that leader now. If you know someone exceptional who might be a great fit or if the opportunity sounds interesting to you, we’d love to hear from you. Please take a look at the full job description (in comments). I’ll be working closely with the MLCommons leadership to select the next GM of MLPerf. After ensuring the leadership transition goes smoothly, I’m looking forward to stepping back from day-to-day operational responsibilities and taking a brief sabbatical, after which time I intend to return to MLCommons in a role focused on the things I’m most passionate about: analyzing the industry, identifying new ways to measure its growth and impact, and evangelizing the role of measurement in driving progress. Most of all, I’m enormously grateful to the working group chairs, volunteers, colleagues, friends, and allies who turned MLPerf from an idea into an industry standard. It has truly been the honor of a lifetime to build this with you. There are too many of your to name, but I'll give it a try in the comments :) I can’t wait to see where MLPerf goes next. Onward!
-
Rebecca Weiss liked thisIf AI is going to transform healthcare, retail, and business, trust has to transform with it. That's the core idea behind PACT AI - and why MLCommons is joining as a founding member. PACT AI brings together deployers, evaluators, insurers, and civil society voices who need to be in the room to build real verification infrastructure. Not just principles. Actual tools, standards, and markets. Three things we're working on from day one: -Building common-sense policies -Professionalizing the AI assurance sector -Growing the market for trusted verification Proud to stand alongside this group. Read the full announcement: https://lnkd.in/erfVrU6g Partnership for Assurance, Credibility, and Trust on AIRebecca Weiss liked thisToday, we're launching PACT AI to help trust in the AI economy scale just as fast as innovation. AI is redefining how the American economy works: how care gets delivered, how retailers serve customers, how businesses run. But as models advance and new capabilities emerge, our ability to verify that these systems work as designed has not kept pace. The Partnership for Assurance, Credibility, and Trust on AI (PACT AI) was created to close this gap — and ensure that trust in the AI economy scales just as fast as innovation. No single company or organization can do this alone. PACT AI unites enterprises, technical experts, AI insurers, and civil society voices to build the science and infrastructure required to assure that AI can responsibly deliver on its potential. Led by Executive Director Bri Treece, our founding members span the economy — from Target and Mount Sinai Health System on the deployment front lines, to assurance providers like Humane Intelligence PBC, Apollo Research, UL Solutions, BABL AI, and Lucid Computing, to AI insurers like Armilla AI — supported by an advisory council drawing on experts from the Foundation for American Innovation, AI Verification and Evaluation Research Institute (AVERI), and the Center for Democracy & Technology. Together, we're starting with the foundational work required to build this trusted ecosystem: → Building common-sense policies → Professionalizing the AI assurance sector → Growing the market for trusted verification Read the full announcement and learn how to join ⬇️ 🔔 Follow PACT AI for what comes next. Participating Organizations: A-LIGN, AI Verification and Evaluation Research Institute (AVERI), apgard ai, Apollo Research, Armilla AI, BABL AI, Center for Democracy & Technology (CDT), Consumer Reports, Denver Health, Eticas.ai, Fathom, FAR.AI, Forum AI, GLACIS Technologies, Goodfire, Humane Intelligence PBC, Lucid Computing, MLCommons, Modulos, Mount Sinai, National Retail Federation, Nemesys Insights, LLC, SaferAI, Schellman, Second Century Ventures, SolasAI, Target, The ERISA Industry Committee (ERIC), Foundation for American Innovation (FAI), Trustible, UL Research Institutes, UL Solutions, Validara Health
Experience & Education
Honors & Awards
-
Stanford Interdisciplinary Graduate Fellowship
-
The Stanford Interdisciplinary Graduate Fellowship (SIGF) is a competitive University-wide program that awards three-year fellowships to outstanding doctoral students engaged in interdisciplinary research.
-
Computational Social Science Fellow
-
The Computational Social Science Fellowship (CSSF) program is a competitive, universitywide
program intended to promote interdisciplinary research efforts using computational techniques
to analyze big data and address important social problems
Organizations
-
Berkman Center for Internet and Society
Fellow
- Present -
Brown Institute for Media Innovation
Fellow
- Present -
Berkman Center for Internet and Society
Affiliate
-
Other similar profiles
Explore more posts
Explore top content on LinkedIn
Find curated posts and insights for relevant topics all in one place.
View top contentOthers named Rebecca Weiss in United States
201 others named Rebecca Weiss in United States are on LinkedIn
See others named Rebecca Weiss