Feiyu Lu

inference is eating ai. 40%+ of nvidia’s revenue now comes from inference workloads.. jensen huang says it’ll scale “a billion times” as chain-of-thought models go mainstream. the bottleneck isn’t chips anymore. it’s watts….performance-per-watt is the governor. huang claims even if rivals gave GPUs away for free, customers would still pick nvidia because blackwell = 30x more tokens per watt than hopper. what does this mean for the rest of us? 1) if you’re building infra-adjacent: efficiency is your narrative. every buyer cares about energy-per-outcome. 2) sellers need to frame solutions as part of ai factories.. not point tools. buyers want orchestration + integration, not another silo. 3) investors: watch inference-heavy categories and sovereign ai buildouts. this is where the multi-trillion upside lives. plus>> when the c-suite realizes watts = revenue, every board deck will start looking more like an energy strategy than a product roadmap. this is take place as more workflows start to move from human capital to agents — ps > more of this on my substack.. link in comments.

12

3 Comments

DiffusionGemma Model on vLLM — ~5–10× Faster Inference LLMs write one word at a time, reloading the model's weights from memory for every single token, which leaves the GPU's math units mostly idle, waiting on memory. A diffusion LLM (dLLM) works differently, it sharpens a whole block of text in parallel. That flips the bottleneck. Memory-bound → compute-bound - it loads the weights once per pass instead of once per token so it finally uses the GPU's idle compute instead of waiting on memory bandwidth. My Experiment benchmarks:  ───────────────────────────  Model       DiffusionGemma 26B-A4B (FP8)  GPU        1× NVIDIA H100 80GB  Engine       vLLM (V2 diffusion model runner)  Precision    FP8 (compressed-tensors)  ─────────────────────────── Throughput      ~866 tok/s (on-GPU) Canvas length  256 tokens The catch is it's still early — quality lags the best autoregressive models, and the speedup mostly shows at low batch sizes (it shrinks under heavy concurrent serving). Promising for specific workloads, not yet a drop-in replacement for autoregressive LLM's For more information read https://lnkd.in/gZz6RGEq

16

1 Comment

India could lose $140 billion annually if Trump’s proposed H-1B visa fee goes through. A suggested $100,000 fee on H-1B visas would effectively dismantle the program, which has been a lifeline for hundreds of thousands of high-skilled professionals—especially from India. 1. India is the world’s largest recipient of US remittances, receiving about $140 billion annually. These inflows make up nearly 30% of India’s tax revenues, supporting economic growth and currency stability. 2. About 71% of all H-1B visa holders are Indians. In 2025, ~399,395 H-1B visas were approved, of which nearly 285,000 went to Indians—primarily in the tech sector. This means nearly 300,000 Indian professionals in the US could be directly impacted. 💡 The consequences? A massive hit not just to Indian families and the Indian economy, but also to US companies that rely on top global tech talent to stay competitive. More Indian professionals may start shifting their focus toward Gulf countries like the UAE and Saudi Arabia, where tech ecosystems are rapidly expanding and policies are becoming more talent-friendly. #H1B #USIndia #TechTalent #Immigration #Economy #FutureOfWork #UAE #SaudiArabia

18

1 Comment

Address matching is a super interesting problem and something I highly recommend trying to solve from scratch. The core idea is to determine whether two addresses refer to the same location, for example  - 221B Baker St., London NW1 6XE, UK  - 221-B Baker Street, NW1 6XE London, United Kingdom The core problem statement is to identify if two slightly different-looking address strings actually refer to the same physical location. Things become interesting as you would need to accommodate typos, abbreviations, formatting differences, missing components, or even language variations. If you give this a shot, you will learn about  - fuzzy string matching  - normalizing and parsing text data  - rule-based vs machine learning-based entity resolution  - improve match accuracy with geospatial data Again, use the LLM of your choice, but make sure you dig deeper and understand all the nuances. Hope it helps. Have fun.

1,123

41 Comments