About
Moses Pawar is an engineering executive with two decades of experience architecting…
Activity
32K followers
-
Moses Pawar shared thisReading the Qwen3.8-Flash-Next architecture blog felt a bit like déjà vu. Many of the ideas look familiar from Kimi K3, DeepSeek V4 and GLM-5.3-Flash, even if the implementations are different. 1. The first is hybrid attention. Qwen uses a 3:1 mix of Gated DeltaNet and Qwen Sparse Attention, similar in shape to Kimi K3’s 3:1 KDA to Gated MLA design. In both cases, most layers use a fixed-size recurrent state, with periodic attention layers for precise retrieval. The difference is that Kimi compresses KV with MLA, while Qwen uses micro-block indexing to attend only to relevant regions. 2. The same convergence is happening across residual streams. DeepSeek and GLM use widened residual streams through Hyper-Connections, Kimi uses Attention Residuals, and Qwen now uses four Gated Residual branches with dynamic reads and writes. Qwen also continues the use of sparse MoE and MTP. MTP is particularly useful for speculative decoding. Instead of generating only one token at a time, the model is trained to predict multiple future tokens that can then be verified together. This reduces the number of sequential decode steps and improves inference throughput. This part was interesting to me in the Qwen blog: “Empirical analysis also reveals that one of these branches naturally emerges as a long-range pathway connecting the first Attention layer to most of the middle and subsequent layers.” It is interesting to see the model learn this kind of specialization rather than having the path explicitly designed for that purpose.
-
Moses Pawar posted thisOver the last year, a lot of the Kubernetes AI work can look like Kubernetes itself learned to understand GPUs, distributed training and model state. In my observation, the model is more modular. Kubernetes has evolved rapidly to define the interfaces and scheduling primitives, while specialized components provide the domain knowledge. This has generally been the principle Kubernetes has followed. Physical topology is one example. For AI workloads, PCIe, NVLink and InfiniBand topology can directly affect throughput. Dynamic Resource Allocation gives Kubernetes a richer resource model through ResourceClaims, DeviceClasses and ResourceSlices. The hardware-specific knowledge still comes from pluggable drivers such as NVIDIA’s DRA driver (similarly Google's TPU driver), which describes the devices and their capabilities to Kubernetes. Distributed workloads are another example. Large models often span multiple nodes and need to operate as one coordinated unit. LeaderWorkerSet provides a Kubernetes-native way to manage a leader and its workers together for lifecycle, scaling and recovery. It is implemented as a CRD and controller on top of existing Kubernetes primitives. More native gang scheduling support is starting to move into kube-scheduler but still in alpha. Inference routing in my opinion follows the same pattern. KV cache and model weights can't be instantly reconstructed after a failure, and routing to the wrong replica means redoing expensive work. The Gateway API Inference Extension addresses this by adding InferencePool as a new backend type using extension points (backendRefs, policy attachment) Gateway API already had. But the actual routing decision runs in a pluggable Endpoint Picker sitting entirely outside Kubernetes' own data path.
-
Moses Pawar posted thisI see a clear trend across open-source analytical stacks. They are trying to close the gap between the lakehouse and real-time analytical systems. The lakehouse gives us open time-travel, snapshots and interoperability through formats like Iceberg. Real-time analytical systems give us low-latency queries and data that stays up to date as changes arrive. 1. Flink SQL with Paimon takes the streaming-compute approach. Flink continuously processes changelogs and maintains updated state over Paimon’s primary-key and LSM-style storage. 2. Doris combines direct lakehouse queries with local caching and materialized views. Changed partitions are quickly refreshed into materialized views. 3. StarRocks follows a similar direction with caching and materialized views. It goes a step further and its newer incremental materialized views can process only newly appended rows for supported queries instead of rebuilding an entire changed partition. The architecture and workflow will start to to look different from the traditional model where lakehouse data is copied into a separate serving database through another pipeline.
-
Moses Pawar posted thisOver the next few years, I think agent traces are going to become one of the fastest-growing data types in the AI stack. As more applications become agentic, every interaction can generate a tree of model calls, tool invocations, retrievals, sandbox executions, verifier results and intermediate state. A modern tracing stack, I think, needs multiple layers. 1. Developers need a fast/hot operational experience for debugging and interactive exploration 2. Durable trace store to handle large-scale retention, selective reads, nested attributes and analytical access. I like what LangSmith has done with SmithDB. They are combining Vortex as the physical columnar format with DataFusion as the query engine, and designing directly around object storage, selective reads, JSON filtering and trace-oriented access patterns. The opportunity I see next is adding stronger table semantics around this kind of physical representation. A table format brings snapshots, schema evolution, atomic commits, retention, time travel, catalog integration and interoperability across compute engines. If newer formats like Vortex can eventually sit cleanly underneath a table format, the architecture gets interesting. Iceberg can decide which files and dataset version matter, Vortex minimizes what gets read inside those files, and DataFusion or other compute engines can execute the query.
-
Moses Pawar posted thisLot of discussion on GLM 5.3-Flash built on Chinese domestic chips but my observation has been more focused on continuing trends in model architectures. Full attention at every layer and one-token-at-a-time decoding are both becoming too expensive. So the architecture is changing on two important fronts among many other 1. cheaper ways to represent long context, and 2. cheaper ways to generate multiple tokens per decode step. The direction has been building for a while. DeepSeek V4 uses hybrid compressed attention to avoid paying full attention cost everywhere. Kimi K3 mixes recurrent KDA layers with periodic MLA layers, keeping most of the sequence state compact while still recovering global context. GLM-5.3-Flash takes a similar idea in another direction. Most layers use linear attention, while periodic sparse-attention layers retrieve only the historical tokens that matter. These architectures also pair naturally with speculative decoding and multi-token prediction. Once the expensive target model is designed to verify several candidate tokens efficiently, a smaller draft model or native MTP head can propose multiple tokens at once and the main model validates them in parallel. That reduces the number of full decode steps. GLM-5.3-Flash includes native MTP support, while Kimi K3 and DeepSeek-style architectures are also increasingly being served with speculative decoding techniques to reduce the cost of serving their very large target models.
-
Moses Pawar posted thisThe opportunity with RL-enabled post-training is to build a much more integrated platform across training, inference, data, environments and agents. The post-training loop now includes rollout workers, environments, reward models and verifiers, policy optimization, and continuous evaluation. It looks less like one distributed training job and more like a distributed application. That brings new needs around heterogeneous compute, weight synchronization, checkpointing, preemption and selective sampling. This is why systems like Ray are becoming more relevant around the PyTorch/DeepSpeed/Megatron/JAX training stack, with Kubernetes primitives like Kueue and JobSet handling scheduling underneath. For agentic workloads, the same tools, sandboxes and environments used during training also need to work consistently with the broader agent platform.
-
Moses Pawar posted thisOne challenge in AI data platforms is that catalogs still mostly manage individual tables, files, or assets, while AI workloads increasingly use different physical representations depending on how data is consumed. Parquet is optimized for large scans and analytics, Megatron IndexedDataset for high-throughput tokenized pretraining, ArrayRecord for indexed access to training examples, and Lance for selective row/column access and vector search. The interesting point is not that every platform should support every format, but that access pattern is becoming an important part of data architecture. That creates a useful separation between the logical dataset and its physical representation. A pretraining job, multimodal pipeline or RL system may each benefit from a different access pattern, even when they originate from the same underlying dataset. In some cases that may justify creating an optimized representation rather than forcing every consumer through the same layout. The opportunity for catalogs is to make those relationships easier to understand and manage. For example which dataset version a representation came from, what tokenizer or embedding version was used, whether it is still current, and what workload it was created for and vend access credentials in a consistent manner.
-
Moses Pawar posted thisI think we are moving toward a world where one multimodal representation layer can power search, RAG, recommendations and agents across text, images, documents, video and audio. Over time, we may need fewer modality-specific embedding pipelines and indexes, and instead build around a shared, versioned representation layer that sits underneath a large part of the AI stack. VLM2Vec-V2 is a good example of this transition. Instead of building separate embedding models for each modality, it takes a multimodal foundation model Qwen2-VL and turns it into a general-purpose embedder across text, images, video and documents. Gemini Embedding 2 pushes the same idea further by using a frontier multimodal model, bidirectional representations, and a unified embedding space that also includes audio and code. It also adds practical things like flexible embedding dimensions. Models like Qwen3-Omni already understand text, image, video and audio. Combine that with better contrastive training (pull similar content closer), hard-negative mining (use difficult negative examples), bidirectional attention and flexible embedding sizes, one could get very close to an open equivalent of Gemini Embedding 2.
-
Moses Pawar reposted thisCome work with Kishore on my team!Moses Pawar reposted thisWe're hiring a Senior/Staff Software Engineer at Apple What does it take to power analytics and AI/ML across Apple — at exabyte scale? That's the problem the ARC (Apple Rest Catalog) team solves every day, and we're looking for someone to help us build the catalog foundation of our data lakehouse. In this role, you'll tackle some of the hardest and most interesting problems in distributed systems: 🔹 Scalability — a platform serving teams across all of Apple 🔹 Multi-tenancy — isolation and fairness at massive scale 🔹 Reliability — the foundation others build their work on Our stack: Apache Iceberg, the Iceberg REST Catalog, Spark, and Java/Scala. If you're energized by building robust, large-scale systems that make a real impact for millions, let's talk. 👇 📍 Seattle, WA 🔗 Apply: https://lnkd.in/gfK4z5kg Know someone who'd be perfect? Tag them or share this post.Sr. / Staff Software Engineer, Data Lakehouse, Apple Data Platform - Jobs - Careers at AppleSr. / Staff Software Engineer, Data Lakehouse, Apple Data Platform - Jobs - Careers at Apple
Moses Pawar replied to a comment
4d
AHSAN UL HAQ For the efficiency we all get from it, I think its worth the effort to keep changing to try new model architectures. This does adds more work to inference engines like vLLM to keep in sync with various techniques in new architectures. But worth it IMO.
-
Moses Pawar liked this28th Foundation Day of Maharashtra University of Health Sciences (MUHS) #tilakMaharashtraVidyapeethMoses Pawar liked thisProudly celebrating the 28th Foundation Day of Maharashtra University of Health Sciences (MUHS), Nashik, at the Institute of Nursing Education and Research, Tilak Maharashtra Vidyapeeth, Pune. Honoring a legacy of excellence in healthcare education, research, and service. 🎉🎓🏥
-
Moses Pawar liked thisMoses Pawar liked thisToday, I had the opportunity to present on "Neurosis" as part of my B.Sc. Nursing curriculum. This presentation helped me gain a deeper understanding of the causes, signs and symptoms, diagnosis, treatment, and nursing management of neurosis. It was a great learning experience that also enhanced my confidence, communication, and presentation skills. Grateful to my teachers and classmates for their support and encouragement. Every presentation is a step toward becoming a more knowledgeable and compassionate nurse. 💙🩺 #BScNursing #Neurosis #MentalHealthNursing #StudentNurse #LearningJourney #Presentation #FutureNurse
Experience & Education
Publications
-
Distributed musical performances: Architecture and stream management
ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP)
The DIP project investigates a versatile framework for the capture, recording, and replay of video, audio, and MIDI (Musical Instrument Digital Interface) streams in an interactive environment for collaborative music performance.
Other authors -
High resolution live streaming with the HYDRA architecture
ACM Computers in Entertainment (CIE)
HYDRA project (Highperformance Data Recording Architecture) focuses on the acquisition, transmission, storage, and rendering of high-resolution media such as H-quality video and multiple channels of audio.
Other authorsSee publication
Patents
-
FORMAT-AGNOSTIC STREAMING ARCHITECTURE USING AN HTTP NETWORK FOR STREAMING
Issued US 20120265853
Languages
-
Hindi
Full professional proficiency
-
Marathi
Native or bilingual proficiency
-
English
Native or bilingual proficiency
Recommendations received
28 people have recommended Moses
Join now to viewOther similar profiles
Explore more posts
-
Arvind Seshan
Why does generative AI feel threatening to so many workers? A recent Harvard Business Review article, “Why Gen AI Feels So Threatening to Workers,” by Erik Hermann, Stefano Puntoni, and Carey Morewedge, offers a useful lens. Their research suggests that the anxiety around AI isn’t only about job loss. It’s about three fundamental psychological needs at work: -Competence — feeling capable and effective -Autonomy — feeling in control of how work gets done -Relatedness — feeling connected to colleagues and teams When AI strengthens those needs, people tend to embrace it as a copilot. When it undermines them, resistance follows. For leaders, this reframes the challenge. AI adoption isn’t just a technology rollout. It’s a change management and organizational design problem. Tools alone won’t drive adoption. Workflows, training, and transparency matter just as much. The article introduces the AWARE framework as a guide: acknowledge the psychological impact, watch for resistance signals, align support and training, redesign roles for human-AI collaboration, and empower employees to shape how AI is used. In other words, successful AI adoption depends as much on how people experience the change as on the technology itself. The future of work will be built not by replacing human capabilities, but by designing systems where humans and AI work well together. #AI #Leadership #FutureOfWork #ChangeManagement #HumanAI #Salesforce https://lnkd.in/gA5MidGA
2 Comments
Explore top content on LinkedIn
Find curated posts and insights for relevant topics all in one place.
View top content