ScaleOps’ cover photo
ScaleOps

ScaleOps

Software Development

New York, NY 20,962 followers

Autonomous cloud & AI resource management. Built for production. Trusted by the world's leading companies.

About us

ScaleOps is redefining cloud resource management from the ground up. Led by a team of cloud infrastructure experts and built for critical production environments, ScaleOps is on a mission to build the Cloud Operating System for the AI era, one that unlocks efficiency and scale while maximizing performance in critical and complex production environments. By bringing real-time, application context-aware automation to cloud resource management, the ScaleOps platform helps organizations eliminate waste, reduce costs, and run critical applications with confidence across any environment.

Website
https://www.scaleops.com
Industry
Software Development
Company size
51-200 employees
Headquarters
New York, NY
Type
Privately Held
Founded
2022
Specialties
Kubernetes, Cloud Infrastructure, Resource Optimization, Cost Reduction, DevOps, FinOps, Continuous Optimization , and Cost Optimization

Products

Employees at ScaleOps

View 152 employees at ScaleOps

or

By clicking Continue to join or sign in, you agree to LinkedIn’s User Agreement, Privacy Policy, and Cookie Policy.

See all employees

Locations

Updates

  • Almost every Kubernetes cluster has at least one pod that can never be evicted, and most teams do not know it is there. New video: the 3 patterns that quietly block Karpenter and Cluster Autoscaler, and how to check for them. Watch below ⬇️

  • Always-on AI agents have a real cost. Most of it is a GPU sitting idle behind them, 77% of the time on the fleet we measured. Horizontal Pod Autoscaler scales on CPU and memory by design. It has no native GPU signal, so it once collapsed this fleet to one replica while the GPU sat pegged at 100%. Read the blog to see how we killed the idle GPU burn on this fleet, cold starts and all, without adding latency. Link in comments ⬇️

    • No alternative text description for this image
  • ScaleOps reposted this

    Last month, we wrapped up our strongest quarter to date. Celebrating growth and records is the easy part. The number that tells the real story is ARR per Scaler. We grew the team a lot over the last 12 months, and ARR per Scaler still grew 1.95x. That says something about the market, the product, and mostly - about the people building ScaleOps. AI workloads are everywhere now - inference, LLMs, AI agents, MCP servers, tools. All of it requires heavy compute, and all of it is very bursty. Companies cannot manage their resources manually at that scale. This is creating massive demand for autonomous AI infrastructure management, the problem we have spent four years building for. It drove our growth over the last few quarters, and demand is bigger than ever. We run AI-first as a principle. We research with AI, build with AI, provide value to our customers with AI, and decide with AI. And we have A players who are customer obsessed. AI makes a great team faster. That combination is the whole story. Proud to be building ScaleOps in this era, with this team. Nir Cohen, Taylor Grabus, Zak Blawie ☁☸️, Yarden Weber, Ben Grady, Joey Balázs, Rotem Ben Hamou, Eyal Zilberberg, Adi Steiner, and all the Scalers! This is just the beginning 🚀

    • No alternative text description for this image
  • If your GPU says it's 90% utilized, it's probably lying to you. Why? That number is SM utilization, sampled at an instant. nvidia-smi catches the GPU mid-compute and reports it busy. But if the training or inference loop keeps stalling to load the next batch, the time-averaged utilization can sit far lower than the dashboard suggests. You're basically reading a snapshot and paying for the whole hour. This is why so many production inference workloads run at 5 to 20% real GPU utilization while the bill charges for 100% of the device. Kubernetes hands out GPUs as whole units, but inference consumes compute and memory unevenly, so the gap between allocated and actually used is where the money is wasted. The fix starts with measuring the right thing: SM utilization over time and framebuffer memory, not a single nvidia-smi glance. We published an article explaining how to measure real GPU demand and close that gap. Go check it out! Link in comments 👇

  • Manual GPU tuning doesn't scale with AI workloads that change by the minute. ScaleOps autonomously observes how each AI workload actually uses its GPU, compute and memory, in real time. From there, it automatically assigns the right fractional GPU policy and keeps adjusting allocation as usage shifts. No static slicing. No manual retuning. Just GPUs running at the density your workloads actually need.

  • Last week we got the whole R&D department together for an evening out, and it reminded us why we love building here: real technical depth, people who genuinely enjoy solving the hard stuff, and a team that knows how to have fun together too. We're growing fast. We currently have 14 open roles across R&D, including: → Backend Engineer → AI Engineer → Infra Engineer If you want to work on complex infrastructure problems (Kubernetes resource management at scale, GPU optimization, real production systems) alongside people like this, take a look. Open roles: https://lnkd.in/dX38DgpW

    • No alternative text description for this image
    • No alternative text description for this image
    • No alternative text description for this image
    • No alternative text description for this image
  • Island runs one of the most demanding Kubernetes environments on Amazon Web Services (AWS). Thousands of production workloads. Multiple regions. 4-5x growth in a year. Replicas were either too few, causing latency, or too many, wasting resources, with no dynamic way to match demand. With ScaleOps automating workload rightsizing and replica scaling, Island cut Kubernetes vCPU costs by 40%, brought 90% of production under ScaleOps, and runs 1,000+ workloads without manual intervention. Watch the full case study here: https://lnkd.in/d9GvUBBa

  • NVIDIA donating the KAI Scheduler to the CNCF is the moment GPU gang scheduling on Kubernetes became everyone's default. It's a real fix. KAI and Grove give you all-or-nothing placement for multi-pod inference: prefill, decode, and router land in the same NVLink domain together, or none of them start. Placement still isn't sizing, though. Each of those pods holds whatever GPU fraction you requested, and most inference pods sit well under half of what they reserve. Gang scheduling gets them onto the hardware together, but it doesn't notice they're oversized. That's why clusters stall around 20 to 30% GPU utilization. No scheduler closes that gap on its own, because it's continuous, not a one-time placement call. I spend my days on this at ScaleOps. Wrote up why Kubernetes keeps hitting that GPU wall, and what actually moves it. Link in comments 👇

  • Kubernetes 1.37 is going to teach the scheduler to stop tearing gang-scheduled jobs apart. Right now preemption is pod by pod. When a high-priority workload needs room, the scheduler evicts one pod from a gang-scheduled group, and the rest keep running but can't make progress. You've essentially freed one pod's worth of GPU and stalled an entire training or multi-host inference job to do it. Workload-aware preemption (KEP-5710) changes the unit: the scheduler treats the whole cluster as one domain and preempts the pod group together, not one pod at a time. It's a good step for tightly-coupled AI workloads. Though preemption only decides who gets evicted when things are tight, not how big each workload should have been. And oversized pods are what make things tight. I guess you know where I'm going! That gap between what Kubernetes schedules and what workloads actually need is what I keep coming back to at ScaleOps. Want to learn more about the scheduler? I wrote up a piece where it stops and optimization starts. Link in comments 👇

Similar pages

Browse jobs