Weiming Zhang

LLMs face a mathematical limit that no amount of scaling can fix. A paper by Vishal Sikka (former SAP CTO, McCarthy's student) and Varin Sikka makes the case: https://lnkd.in/gUh2dxUi Here's the argument: - One forward pass of a transformer costs O(n²·d) operations - There exist problems requiring MORE than O(n²·d) to solve - Therefore, LLMs cannot correctly solve those problems But how do we know harder problems exist? The proof relies on a 130-year-old idea. In 1965, Hartmanis and Stearns proved the Time Hierarchy Theorem using diagonalization — a technique dating back to Cantor. The proof (see animation): Imagine a table where rows are machines and columns are inputs. Each cell shows what machine Mᵢ outputs on input n. The construction: for any time bound T(n), we can enumerate ALL machines that halt within T(n) steps — there are only finitely many distinct programs of a given size, and we only care about those fast enough to finish in time. Now construct a new sequence D that differs from machine Mᵢ on input i (just flip the bit on the diagonal). D cannot be computed by ANY machine in the table. It's designed to disagree with each one somewhere. Give the machines more time? You can always construct a new diagonal sequence that escapes them too. This creates an infinite hierarchy of complexity classes, each strictly more powerful than the last. O(n) < O(n²) < O(n³) < ... < O(2ⁿ) < ... Each jump unlocks problems impossible at the previous level. Matrix multiplication, all-pairs shortest paths, CFD simulations — their complexity is above O(n²). What this means for AI: - Agentic systems doing payments, logistics, industrial control often hit >O(n³) subproblems and hallucinate solutions - Verification can be harder than solving (checking a TSP tour optimal = comparing against (n-1)!/2 alternatives) - "Thinking" traces don't escape — they just add tokens, not computational class (https://lnkd.in/gCbpEGrb) The implication: hybrid architectures — LLMs + symbolic modules + external solvers. Progress may come from models that know when to delegate to algorithms that CAN solve the hard problems. Terminal 3 #ai #llm #complexity #computerscience #agents

13

2 Comments

Steal this 8-step framework to take an algorithmic trading strategy from idea to execution. 1/ The idea Look for market anomalies that repeat. Your strategy should be based on some economic rationale. In other words, “using machine learning” is not a trading strategy. 2/ Research Find out if the anomaly you think you found actually exists in history. Get data and start exploring. 3/ Trade signals Use an algorithm that can detect weak signals in data with a lot of noise. Use a price filter first to help. 4/ Minimum Viable Python If you made it this far, assess the strategy for an edge. You're only trying to determine if the algorithm has an edge or not. If it does not generate a series of profitable trades, improve the algorithm or try another one. 5/ The backtest setup Add elements that will affect performance and run out-of-sample tests. Focus on optimizing parameters that are dependent on the market—don’t brute force optimize to find peak profit. 6/ The backtest analysis Run state-of-the-art methods like walk-forward optimization to remove bias. Check the validity of the backtest by randomizing the historic prices, retraining the model, and testing it again. 7/ Risk and performance analysis Check risk metrics like drawdown, volatility, and VaR. Since active trading takes time, your results should be significantly better than a simple buy-and-hold strategy. 8/ Execution Let the strategy run and manually trade the signals it generates, hook up the strategy to a paper trading account, and finally, move to live execution. Your map to building algorithmic trading strategies: 1. The idea 2. Research 3. Trade signals 4. Minimum Viable Python 5. The backtest setup 6. The backtest analysis 7. Risk and performance analysis 8. Execution ~~~ A step-by-step course to help you use Python for algo trading and market data analysis. • Real-time answers • Lifetime access to everything • Python code templates that work • Explanations you can understand Join more than 1,500+ students: https://lnkd.in/ekTsGnNs

5

NVIDIA just pulled off what many thought wasn’t possible. They pre-trained a 12B-parameter LLM on 10 trillion tokens using only 4-bit precision. Why it matters: 2–3× higher math throughput and ~50% lower memory vs FP8 Accuracy holds: MMLU-Pro — FP8 62.62% vs NVFP4 62.58% Stable training via Random Hadamard transforms, stochastic rounding, and 2D scaling End-to-end on NVIDIA Blackwell GPUs—the first 4-bit run to stay stable across 10T tokens This is the first large-scale 4-bit pretraining demo without accuracy loss. The next wave of frontier models will be faster and cheaper —with no obvious trade-offs. #AI #DeepLearning #LLM #NVIDIA #Blackwell #ModelTraining #MLOps #Efficiency https://lnkd.in/dFVkx5pc

12

𝐈𝐟 𝐲𝐨𝐮𝐫 𝐦𝐮𝐥𝐭𝐢-𝐚𝐠𝐞𝐧𝐭 𝐰𝐨𝐫𝐤𝐟𝐥𝐨𝐰𝐬 𝐟𝐚𝐢𝐥, 𝐦𝐨𝐬𝐭 𝐨𝐟 𝐭𝐡𝐞 𝐭𝐢𝐦𝐞 𝐢𝐭’𝐬 𝐣𝐮𝐬𝐭 𝐩𝐫𝐨𝐛𝐚𝐛𝐢𝐥𝐢𝐭𝐲 𝐝𝐨𝐢𝐧𝐠 𝐢𝐭𝐬 𝐣𝐨𝐛. 🤖🧮 Multi-agent systems usually don’t break because models are bad. They break because we wire probabilistic components together, and errors (and costs) compound. In my latest Radar piece with O'Reilly, I outline practical guidelines for avoiding unreliable systems and runaway token spend. 𝐌𝐨𝐫𝐞 𝐝𝐞𝐭𝐚𝐢𝐥𝐬 𝐚𝐧𝐝 𝐚 𝐝𝐞𝐞𝐩𝐞𝐫 𝐭𝐞𝐜𝐡𝐧𝐢𝐜𝐚𝐥 𝐛𝐫𝐞𝐚𝐤𝐝𝐨𝐰𝐧 𝐡𝐞𝐫𝐞: 👉 https://lnkd.in/eAtmFmsx 👉 https://lnkd.in/eev84kZk  (via Substack) #AgenticAI #MultiAgentSystems #LLMOps #ProductionAI #SystemsEngineering #OReilly

58

5 Comments

“Transformer Math 101” - how do you make resource estimates behind training and running transformer / large language models (LLMs). Interesting read. Key variables: P = number of parameters (e.g. weights) in the model D = size of the training dataset (in tokens) C = total compute (in FLOPs) required to train τ (tau) = throughput of the hardware (e.g. FLOPs per second, times number of devices) T = time spent in training (seconds, or in some unit) https://lnkd.in/e9hAShS8

6