Work · YouTube

BrandShift: Metrics for Branded YouTube Ads

2021-2023 · Material revenue impact · Technical Lead & Architect

Project Overview

BrandShift was my attempt to measure brand identity without the usual shallow proxies. We expanded BrandLift into a multi-dimensional view of perception so creative teams could learn something real from the data.

In hindsight, it was also my first serious evaluation problem: deciding whether an ML-mediated system actually changed what people believed, with enough statistical rigor to bet revenue on the answer. That is the same shape as the eval problems I now work on in AI moderation — different domain, same question of whether you can trust a model's judgment.

The Challenge

Digital advertisers faced significant limitations with traditional measurement approaches:

• Metrics focused narrowly on awareness, consideration, and intent
• Brand identity effects were poorly measured
• Survey mechanisms were disruptive to the user experience
• Results often took weeks to process
• Insights weren't actionable for creative optimization

YouTube's targeting products represented a large slice of ARR and needed better measurement to show their real value to brand advertisers.

Technical Implementation

Multi-Dimensional Measurement Framework

BrandShift implemented a comprehensive measurement approach covering:

• Brand personality attributes - Measuring how consumers describe the brand's character
• Emotional associations - Capturing the feelings the brand evokes
• Value alignment - Assessing how well the brand's perceived values match its intended identity
• Category differentiation - Measuring how distinctly the brand stands out from competitors

This required building both quantitative and qualitative measurement capabilities that could scale across YouTube's global platform.

Non-Disruptive Survey Mechanism

We designed a novel survey approach that:

• Integrated seamlessly into the YouTube viewing experience
• Used micro-surveys with 1-2 questions
• Employed adaptive question selection
• Leveraged natural language processing for open-ended responses
• Balanced statistical significance with user disruption

This approach achieved meaningfully higher response rates than traditional survey methods while maintaining data quality.

Real-Time Data Processing

We built a sophisticated data pipeline that:

• Processed responses in near real-time
• Applied NLP techniques to analyze open-ended responses
• Generated statistically rigorous experimental results
• Created intuitive visualizations for advertisers
• Integrated with YouTube's experiment framework

This system reduced insight generation time from weeks to hours, enabling much faster optimization cycles for advertisers.

Cross-Team Collaboration

BrandShift required extensive partnership across multiple teams:

• Google Ads Measurement - Aligning with broader measurement standards
• Reporting - Integrating with advertiser dashboards
• Target Frequency and Sequencing - Connecting exposure patterns to perception shifts
• Video Understanding - Correlating content characteristics with brand perception

This collaboration ensured BrandShift met the needs of all stakeholders while maintaining methodological rigor.

Results & Impact

BrandShift delivered significant value to both YouTube and its advertising partners:

• Demonstrated higher measured impact from targeted ads versus traditional demographic targeting
• Increased advertiser retention by providing deeper insights into campaign effectiveness
• Generated actionable creative optimization recommendations that demonstrably improved performance
• Helped secure material incremental revenue from brands adopting advanced targeting products

Most importantly, BrandShift evolved how advertisers thought about measurement, moving from transactional metrics to holistic brand identity measurement.

Technical Challenges Overcome

Scale & Representativeness

Ensuring statistically valid results across diverse campaigns required innovative approaches:

• Developed intelligent sampling strategies to balance representativeness with cost
• Built sophisticated experimental design methodologies
• Created power calculation tools to determine minimum sample sizes
• Implemented bias correction models to account for response patterns

Natural Language Processing

Analyzing open-ended brand perception responses presented unique challenges:

• Trained domain-specific language models for brand attribute classification
• Developed multilingual capabilities for global campaigns
• Built emotion detection models calibrated to brand contexts
• Created sentiment analysis tuned for brand perception nuances

My Role & Contributions

As the lead for BrandShift, I:

• Conceptualized the multi-dimensional measurement framework
• Designed the technical architecture for data collection and analysis
• Led the cross-functional team across measurement, reporting, and ML
• Drove the evolution from traditional brand lift to holistic brand identity measurement
• Presented the methodology and results to senior leadership
• Worked directly with major brand partners to refine the approach

Technologies Used

• Languages: Python, Java, R (for statistical analysis)
• Data Processing: Apache Beam, BigQuery, Data Studio
• Machine Learning: TensorFlow, BERT-based models for NLP
• Visualization: Custom D3.js dashboards
• Experimentation: YouTube's internal A/B testing framework