Coupang AI-for-Work

2024 - Present · 2,500+ Engineers Enabled

TypeScript Python AI Infrastructure Enterprise Scale MCP LLM Observability

Leading Coupang's enterprise-wide AI transformation, fundamentally changing how 2,500+ engineers work through systematic AI tool deployment, infrastructure development, and cultural change. This initiative has achieved 30%+ productivity gains and established Coupang as a leader in enterprise AI adoption.

                      AI Architect
                    2024 - Present
                    Mountain View, CA
                

Project Overview

As AI Architect at Coupang, I'm leading the AI-for-Work initiative - a comprehensive transformation of how engineering work is done at enterprise scale. This isn't just about deploying AI tools; it's about fundamentally reimagining engineering workflows, building critical infrastructure, and driving cultural change across a 2,500+ person engineering organization.

The initiative spans multiple workstreams: from deploying cutting-edge AI IDEs to building secure infrastructure that connects AI tools to internal systems, from creating observability platforms that process millions of LLM requests to developing frameworks that make engineering work programmatically accessible to AI agents.

The Challenge

Coupang faced several critical challenges in adopting AI at enterprise scale:

Fragmented AI tool adoption with no centralized strategy or infrastructure
Security concerns about AI tools accessing internal systems and codebases
Lack of visibility into AI usage, costs, and effectiveness
Engineering workflows not designed for AI integration
Cultural resistance and uncertainty about AI's role in engineering
No frameworks for measuring productivity gains or ROI

These challenges required a comprehensive approach that went beyond simply purchasing AI licenses - we needed to build infrastructure, change processes, and transform culture.

Technical Implementation

AI IDE Deployment (Windsurf)

Led the enterprise-wide deployment of Windsurf (formerly Codeium) to 2,500+ engineers:

Architected rollout strategy with phased adoption and success metrics
Built custom integrations with Coupang's development environment
Created training programs and best practices documentation
Established feedback loops for continuous improvement
Achieved 90%+ adoption rate within 6 months

Centralized MCP Gateway

Built a secure, audited gateway for AI tools to access internal systems:

graph LR subgraph "AI Tools" Tool1[Windsurf] Tool2[Claude] Tool3[GitHub Copilot] end subgraph "MCP Gateway" Router[Request Router] AuthZ[Authorization] RateLimit[Rate Limiter] Transform[Data Transform] Logger[Audit Logger] end subgraph "Backend Services" API1[JIRA API] API2[GitHub API] API3[Confluence API] API4[Internal Services] end Tool1 --> Router Tool2 --> Router Tool3 --> Router Router --> AuthZ AuthZ --> RateLimit RateLimit --> Transform Transform --> Logger Logger --> API1 Logger --> API2 Logger --> API3 Logger --> API4 style Router fill:#ff6b6b style AuthZ fill:#4ecdc4 style Logger fill:#95e1d3

Architecture: TypeScript-based proxy with fine-grained access controls
Security: OAuth integration, request signing, and comprehensive audit logging
Integrations: Connected to JIRA, Confluence, GitHub Enterprise, internal APIs
Performance: Sub-50ms latency with 99.99% availability
Compliance: Full audit trail for security and compliance requirements

This gateway became the foundation for secure AI-to-enterprise connectivity, enabling AI tools to work with internal data while maintaining security standards. It also serves as the central integration point for Emux and MicroMCPs, providing secure access to internal systems for AI agents.

AI Observability Platform

Developed comprehensive observability for AI tool usage:

flowchart TD subgraph "Data Collection" LLM[LLM Requests] Code[Code Changes] Metrics[Performance Metrics] end subgraph "Processing Pipeline" Stream[Stream Processor] Enrich[Data Enrichment] Aggregate[Aggregation Engine] end subgraph "Analytics" TokenAnalytics[Token Usage Analytics] QualityMetrics[Code Quality Metrics] ProductivityMetrics[Productivity Analytics] CostAnalytics[Cost Attribution] end subgraph "Outputs" TeamDash[Team Dashboards] ExecDash[Executive Reports] Alerts[Anomaly Alerts] end LLM --> Stream Code --> Stream Metrics --> Stream Stream --> Enrich Enrich --> Aggregate Aggregate --> TokenAnalytics Aggregate --> QualityMetrics Aggregate --> ProductivityMetrics Aggregate --> CostAnalytics TokenAnalytics --> TeamDash QualityMetrics --> TeamDash ProductivityMetrics --> ExecDash CostAnalytics --> ExecDash Aggregate --> Alerts style Stream fill:#667eea style Aggregate fill:#764ba2 style ExecDash fill:#f093fb

Real-time processing of 1M+ LLM requests daily
Token usage tracking and cost attribution by team/project
Quality metrics for AI-generated code (test coverage, review feedback)
Productivity analytics showing time saved and efficiency gains
Anomaly detection for unusual usage patterns or potential issues

This platform provided unprecedented visibility into AI adoption and effectiveness, enabling data-driven decisions about tool investments and training needs.

Emux (Engineer Multiplexer) - Agent Orchestration Platform

Built Emux, a sophisticated infrastructure for complex multi-step AI workflows that multiplexes engineer capabilities through AI agents:

graph TD subgraph "Emux Core" Scheduler[Task Scheduler] Executor[Agent Executor] State[State Manager] HITL[Human-in-the-Loop] end subgraph "AI Agents" Agent1[Code Review Agent] Agent2[Test Generation Agent] Agent3[Documentation Agent] Agent4[Deployment Agent] end subgraph "MicroMCPs" MCP1[PR Review MCP] MCP2[Test Runner MCP] MCP3[Doc Gen MCP] MCP4[Deploy MCP] MCP5[JIRA MCP] MCP6[Analytics MCP] end subgraph "Human Interface" Slack[Slack Integration] Approvals[Approval Workflows] Notif[Real-time Notifications] end subgraph "Integration Points" CI[CI/CD Pipeline] PR[Pull Requests] Deploy[Deployment Systems] JIRA[JIRA] end Scheduler --> Executor Executor --> State State --> HITL HITL --> Slack Executor --> Agent1 Executor --> Agent2 Executor --> Agent3 Executor --> Agent4 Agent1 --> MCP1 Agent2 --> MCP2 Agent3 --> MCP3 Agent4 --> MCP4 Agent1 --> MCP5 Agent2 --> MCP5 Agent3 --> MCP6 Agent4 --> MCP6 MCP1 --> PR MCP2 --> CI MCP3 --> PR MCP4 --> Deploy MCP5 --> JIRA Slack --> Approvals Approvals --> Notif style Scheduler fill:#ff6b6b style Executor fill:#4ecdc4 style HITL fill:#95e1d3 style Slack fill:#4a154b style MCP1 fill:#feca57 style MCP2 fill:#feca57 style MCP3 fill:#feca57 style MCP4 fill:#feca57

Emux Core: Engineer Multiplexer that orchestrates AI agents and manages complex workflows
MicroMCPs: Specialized Model Context Protocol servers for ad-hoc workflow tasks (PR reviews, test running, doc generation, deployments)
Human-in-the-Loop: Slack integration for approval workflows, notifications, and engineer oversight
Agent-to-MCP Communication: Agents dynamically call MicroMCPs for specific subtasks during workflow execution
Parallel Execution: Performance optimization for running multiple agents and MicroMCPs concurrently
Real-time Monitoring: Progress tracking and state management across distributed agent workflows

Work-as-Code Framework

Pioneered making engineering tasks programmatically accessible:

graph LR subgraph "Work Definitions" Task1[Review PR] Task2[Run Tests] Task3[Update Docs] Task4[Deploy Service] end subgraph "MCP Servers" MCP1[PR Review MCP] MCP2[Test Runner MCP] MCP3[Doc Generator MCP] MCP4[Deployment MCP] end subgraph "AI Interface" Interface[Unified API] Discovery[Service Discovery] Registry[MCP Registry] end subgraph "AI Agents" AI1[Windsurf] AI2[Claude] AI3[Custom Agents] end Task1 --> MCP1 Task2 --> MCP2 Task3 --> MCP3 Task4 --> MCP4 MCP1 --> Registry MCP2 --> Registry MCP3 --> Registry MCP4 --> Registry Registry --> Discovery Discovery --> Interface AI1 --> Interface AI2 --> Interface AI3 --> Interface style Interface fill:#fa709a style Registry fill:#fee140 style Discovery fill:#30cfd0

Developed micro-MCP servers for specific engineering domains
Created standardized interfaces for common operations
Built SDK for teams to expose their workflows to AI
Established patterns for safe, auditable AI interactions
Enabled "serverless" approach to AI tool integration

Results & Impact

The AI-for-Work initiative has delivered transformative results:

30%+ Productivity Gains: Measured reduction in time for common engineering tasks
2,500+ Engineers Enabled: Near-universal adoption across engineering
$10M+ Annual Savings: Through improved efficiency and reduced development time
50% Reduction in Boilerplate: AI handling repetitive coding tasks
30% Faster PR Reviews: AI-assisted code review and testing
Cultural Transformation: AI now integral to engineering workflows

Key Innovations

Security-First Architecture

Developed novel approaches to secure AI integration:

Zero-trust model for AI tool access to internal systems
Granular permission system based on user, tool, and resource
Real-time monitoring and anomaly detection
Compliance-ready audit trails for all AI interactions

Measurement Framework

Created comprehensive metrics for AI effectiveness:

Productivity metrics tied to business outcomes
Quality indicators for AI-generated code
Team-level dashboards for adoption tracking
ROI calculations for AI tool investments

Cultural Change Management

Led organizational transformation:

Executive alignment and sponsorship
Champion program for early adopters
Regular showcases of AI success stories
Integration into engineering onboarding
Continuous education and best practices sharing

Technical Challenges Overcome

Scale & Performance

Handling enterprise-scale AI adoption required innovative solutions:

Built distributed architecture to handle millions of daily requests
Implemented intelligent caching to reduce API costs
Developed request prioritization for critical workflows
Created fallback mechanisms for AI service outages

Integration Complexity

Connecting AI tools to diverse internal systems:

Developed adapters for 20+ internal APIs and tools
Built translation layers for different data formats
Created unified authentication across systems
Implemented retry logic and error handling

Future Roadmap

The AI-for-Work initiative continues to evolve:

Autonomous Agents: Moving from AI assistance to autonomous task completion
Custom Models: Fine-tuning LLMs on Coupang's codebase and patterns
Predictive Analytics: Using AI to predict and prevent engineering bottlenecks
Cross-functional Expansion: Extending AI tools to product, design, and operations
Open Source Contributions: Sharing our frameworks with the community

My Role & Leadership

As AI Architect leading this initiative, I:

Define technical strategy and architecture for AI adoption
Lead cross-functional teams across engineering, security, and operations
Drive executive alignment and secure funding for initiatives
Mentor engineers on AI integration best practices
Represent Coupang at AI conferences and industry forums
Collaborate with AI vendors on enterprise features

Technologies Used

Languages: TypeScript, Python, Go
AI Tools: Windsurf/Codeium, Claude, GPT-4, GitHub Copilot
Infrastructure: Kubernetes, AWS, Terraform
Observability: Datadog, Grafana, Custom Analytics Platform
Frameworks: MCP (Model Context Protocol), LangChain, Custom SDKs

Lessons Learned

Key insights from leading enterprise AI transformation:

Infrastructure and security must come before tool deployment
Measurement and visibility are crucial for adoption success
Cultural change is as important as technical implementation
Start with high-value, low-risk use cases to build confidence
Invest heavily in training and documentation
Build for extensibility - the AI landscape changes rapidly