Infrastructure: Building & Managing Systems at Scale
Infrastructure is the foundation of every technology system. It's how companies run their applications, store data, and process transactions at scale. From JPMorgan's trading systems to Netflix's video streaming, infrastructure decisions determine performance, cost, and reliability.
Infrastructure Landscape
On-Premise ──────→ Hybrid ──────→ Cloud
(JPMorgan) (Most companies) (Startups)
Own & control Mix of owned Rent everything
100% responsibility & rented Zero infrastructure headache
📚 Key Topics in This Section
1. Server Types Comparison
Quick decisions on Physical vs VM vs Cloud vs ECS
- Physical servers: $134/core-year (best for sustained, latency-sensitive loads)
- Cloud VMs: $540/core-year (4x more, but infinitely flexible)
- ECS containers: Variable cost (perfect for bursty workloads)
- Real examples: JPMorgan's trading, Netflix's streaming, Stripe's APIs
Best for: Understanding when to use each server type
2. On-Premise Infrastructure
How banks operate their own data centers
- JPMorgan's $3.3B/year infrastructure breakdown
- 50,000+ servers per large bank
- Data center architecture (redundancy, networks, cooling)
- Why banks refuse to move: latency <1ms, compliance, control
Best for: Understanding legacy enterprise infrastructure
3. Cloud Infrastructure
AWS, Azure, GCP pricing and architecture
- Cost comparison across providers
- Per-service pricing: EC2, S3, RDS, Lambda
- Real customer examples: Netflix ($1.5B/year), Airbnb ($800M/year)
- Why tech companies choose cloud over physical
Best for: Understanding cloud economics and provider differences
4. Hybrid Infrastructure
The realistic middle ground (60% follow this)
- JPMorgan's split: 60% on-prem, 40% cloud
- Goldman Sachs: 70% on-prem, 30% cloud
- Integration challenges and solutions
- Data gravity, latency, cost optimization
Best for: Modern enterprise architects designing systems
5. Infrastructure Cost Management
How to optimize without breaking things
- Reserved instances: 30-47% discount for predictability
- Spot instances: 70% discount for fault-tolerant work
- Right-sizing: stop paying for unused capacity
- FinOps framework and tools
- Cost per transaction, cost per user metrics
Best for: CFOs and infrastructure managers
6. Banks vs Tech Companies
Why they approach infrastructure completely differently
- JPMorgan (bank): $3.3B/year, 60% on-prem, microsecond latency critical
- Google (tech): $15-20B/year, 100% cloud, elasticity critical
- Cost per core: $134/year vs $540/year
- Staffing: 7,000 people vs 500-1,000 people
- Future: convergence on hybrid (best of both)
Best for: Understanding industry-specific infrastructure decisions
7. Servers Simplified Guide
Easy-to-understand overview of all server types
- Physical servers, on-prem VMs, cloud VMs, containers
- Real-world analogies (servers are like offices)
- Cost and performance comparisons
- When to use each type
Best for: Getting started with infrastructure concepts
💰 Cost Snapshots
Physical Server (JPMorgan approach)
Hardware: $47K
5-year total: $86-100K
Cost per core: $134/year
Latency: <1ms
Best for: Trading, core systems
Cloud VM (Netflix approach)
Monthly cost: $95/month (reserved)
5-year total: $5.7M per 1,000 servers
Cost per core: $540/year
Latency: 5-20ms
Best for: Scaling, flexibility
ECS Containers (Modern startups)
Cost per task: $9-29/month
Startup cost: $0
Scaling: 10-30 seconds
Best for: Microservices, bursty load
🎯 Quick Navigation
I want to...
| Goal | Best Resource |
|---|---|
| Understand server costs | Servers Simplified Guide |
| Choose between physical/VM/cloud | Server Types Comparison |
| Learn about JPMorgan's setup | On-Premise Infrastructure |
| Understand Netflix's setup | Cloud Infrastructure |
| Design a hybrid system | Hybrid Infrastructure |
| Reduce my infrastructure bill | Infrastructure Cost Management |
| Compare banks vs tech companies | Banks vs Tech Companies |
📊 Common Metrics You'll See
| Metric | What It Means | Why It Matters |
|---|---|---|
| Cost per core-year | Annual cost ÷ number of processors | Compare server types fairly |
| Latency | Time for data to travel | MS matters for trading, OK for web |
| Noisy neighbor | Other VMs slowing your performance | Physical = none, Cloud = 5-15% variance |
| vCPU | Virtual CPU (shared/fractional) | Not a "real" core, you're sharing |
| Reserved instances | Commit to 1-3 years, get 30-47% discount | Lock-in, but huge savings |
| Spot instances | Use spare capacity, 70% cheaper | Can be killed anytime, use for flexible work |
🏦 Real Infrastructure Spending (2024-2025)
| Company | Annual Spend | Model | Notes |
|---|---|---|---|
| JPMorgan | $3.3B | 60% on-prem, 40% cloud | Largest US bank |
| Goldman Sachs | $2.5-3B | 70% on-prem, 30% cloud | Trading focused |
| Netflix | $1.0B | 100% AWS cloud | Video streaming |
| $15-20B | 100% owned cloud | Entire search, YouTube, etc | |
| Meta/Facebook | $12-15B | Mix: on-prem + AWS | Massive scale, custom servers |
| Amazon | $10-12B | Internal + AWS | E-commerce + AWS reselling |
Key Takeaways
- Physical servers still win on cost per core ($134/year vs $540/year in cloud)
- Cloud wins on flexibility and elasticity (instant scaling, no capex)
- Hybrid is the sweet spot for 80%+ of enterprises
- Banks keep on-prem for trading (microseconds matter, billions at stake)
- Tech companies use cloud (elasticity more valuable than per-core cost)
- Reserved instances are your best friend (30-47% savings for predictable loads)
- Data gravity is real (petabytes are hard to move)
- Don't optimize for cost at the cost of reliability
Common Infrastructure Mistakes
❌ Oversizing for peak capacity → Pay for unused capacity 95% of the time
❌ Ignoring data transfer costs → Surprise $50M cloud bills
❌ No chargeback model → Departments have no incentive to optimize
❌ Consolidating too aggressively → One outage = total failure
❌ Staying 100% physical when you could be hybrid → Missing flexibility benefits
Getting Started
If you're new to infrastructure:
- Start with Servers Simplified Guide for basics
- Read Server Types Comparison to understand costs
- Pick one company (JPMorgan, Netflix, Google) to learn their approach
If you're designing infrastructure:
- Review Hybrid Infrastructure for modern approach
- Check Cost Management for optimization strategies
- Compare Banks vs Tech Companies for your industry
If you're optimizing costs:
- Read Cost Management for concrete tactics
- Calculate your cost per transaction / per user
- Implement chargeback models
Everything you need to understand, design, and optimize infrastructure at enterprise scale. 🚀