Gary Illyes

1 September 2020, the email begins “Hey Werner, thank you for being so generous with your time. Zoppah is an ecommerce marketplace…” We were floored. I was building a custom marketplace when I met my business partners. They had just started their pilot on a hosted Wordpress. I took over technology and started a migration onto the AWS platform because it would allow us to scale. PHP is a resource guzzler, so I automated fine tuning EC2 instances down to the kernel. When we spun up an instance, a script setup the OS before the application started… But that didn’t save us from customer service demands. We were growing too fast. The infrastructure architecture moved from vertical scaling a single instance to horizontal scaling multiple and it still wasn’t enough. I optimised for redundancy, latency, availability, and throughout at the network, disk, memory and cpu level but our utilisation cost kept increasing. We were averaging $4500 monthly when that email went out, and we desperately needed assistance. The response came within a few hours. The Frugal Architect extended help in a few ways. He set up a meeting with a trusted ex-AWS Architect and friend to help us further optimise our setup. He set the wheels in motion internally to get us business assistance from the Cape Town office. I have personal experience that he lives Amazon’s responsiveness to customer needs.

3

You are in a system design interview at Google for the L5 Senior Engineer role, and the interviewer leans in and asks: “Why does Spotify keep playing when I drive into a tunnel with no signal, but YouTube Music often stops or buffers? If you were designing a music streaming system, what different design choices would lead to these two behaviors?” Here is how you break it down. Btw, if you’re preparing for system design/coding interviews, check out our mock interview tool. You can use it for free here: https://lnkd.in/gpCn7t2T We’ve added new features as well:  -Company-specific interviews -In-built interview scheduler  -performance insights & trends — Both apps look like simple music players. Under the hood, they are optimized for very different priorities. [1] Spotify style – Cache first, stream second Idea: The client behaves like a smart offline player that happens to stream. The backend is built to support aggressive prefetching. What happens when you hit play - Client requests the track from a CDN or edge node. - Instead of tiny chunks, it downloads a big buffer ahead of the playhead. - In parallel, it starts pre downloading the next few tracks in the queue. - Data is written to an encrypted cache on disk, not just memory. - Playback reads from that local cache, not directly from the network socket. What this means for tunnels and bad networks - When the network drops, the player already has tens of seconds or entire tracks cached. - Because the next one or two songs are already downloaded, you can be offline for a while and never notice. - Cold start cost is a bit higher. First play might take slightly longer, but then everything feels smooth. - It burns more local storage and possibly more data, because not every prefetched song will be listened to fully. In a design answer, you can mention - Local disk cache with eviction policies (LRU per user, per device). - Background prefetch of N upcoming tracks based on queue. - Download manager that adapts how aggressively it prefetches based on network quality and user settings. - CDN tuned for larger object delivery and range requests. - Explicit offline mode that pins playlists into cache. [2] YouTube Music style – Stream first, cache is minimal Idea: Treat audio like video streams. Cost and bandwidth are optimized first. When you hit play - Player requests audio (and sometimes video) via HLS or DASH-style chunks. - Each chunk is only a few seconds long. - Client keeps a small rolling buffer in memory, not a large queue on disk. - Prefetch of future songs is limited, because video tracks are large and expensive to fetch speculatively. What this means for tunnels and bad networks - If your connection dies, the player only has a few seconds of buffered data. - As soon as those chunks are consumed, playback stalls. - Startup can feel snappy and data usage is controlled, especially for casual listeners. - Works very well on stable networks, feels fragile in spotty coverage.

1,219

38 Comments

What I learned about hidden complexity as a Software Engineer at Amazon Engineers, including me, love to build. We get excited about features we could add, new stacks we could try, and all the customers we will serve. That creative energy is a big part of why we love what we do. But here’s the trap: every new creation comes with a hidden price tag. - That extra service we spun up? It needs monitoring, on-call rotations, and upgrades. - That shiny library we added? It’ll break with the next version and someone has to fix it. - That feature you shipped quickly? It now has to be supported, documented, and debugged long after launch. At Amazon, we still support the first-generation Kindle from 2007. Let that sink in! An almost 20-year-old device still has systems and people behind it to keep things running. For each new product or service we build, we have to (or aim to) live with for the long term. In the moment, complexity feels like progress, but in the long run, it shows up as maintenance debt: More work for future teams, slower velocity, and higher risk. Good design isn’t about how much you can add. It’s about how much you can simplify while still meeting the constraints, and how prepared you are to live with it years down the road. 👉 What’s one “hidden complexity” you’ve seen come back to bite a project? --- Hello, I’m Charles, and I am committed to share my two cents on career-related topics for 100 consecutive days. In these uncertain times, I hope to support those facing layoffs or career challenges. Follow me, and let's navigate this together! (52/100) #SoftwareEngineering #SystemDesign #Leadership #CareerGrowth

16

1 Comment