Quality Guardian

The Quality Guardian ensures only exceptional vendors surface to users. It uses multi-modal AI to assess visual quality, authenticity, and professionalism.

Overview

Mission: Ensure only exceptional vendors are surfaced

Key Insight: Quality ≠ Engagement. High-quality vendors may not be viral, but they deliver excellence.

Phase: Introduced in Phase 2 (Week 6)

Quality Dimensions

1. Visual Quality

Professional photography and composition:

async assessVisualQuality(vendor: Vendor): Promise<VisualQuality> {
  const posts = await this.getVendorPosts(vendor, limit: 20);

  const prompt = `
Analyze these wedding vendor photos for professional visual quality.

Evaluate:
1. Photography quality (composition, lighting, color grading)
2. Professional equipment usage (sharpness, depth of field)
3. Artistic vision and style consistency
4. Post-production quality

Score 0.0-1.0 for each dimension and overall.

Images: ${posts.map(p => p.imageUrl).join(', ')}
`;

  const response = await this.callGPT4Vision(prompt, posts.map(p => p.imageUrl));

  return {
    composition: response.composition,
    lighting: response.lighting,
    consistency: response.consistency,
    overall: response.overall,
    reasoning: response.explanation
  };
}

2. Content Authenticity

Real weddings vs styled shoots:

async detectAuthenticity(vendor: Vendor): Promise<AuthenticityScore> {
  const posts = await this.getVendorPosts(vendor, limit: 30);

  // Indicators of real weddings:
  // - Candid moments, genuine emotions
  // - Variety of venues and settings
  // - Client testimonials in captions
  // - Behind-the-scenes content

  const realWeddingCount = posts.filter(p => {
    return (
      this.hasCandidMoments(p) ||
      this.hasGuestInteractions(p) ||
      this.hasClientTestimonials(p)
    );
  }).length;

  const authenticityRatio = realWeddingCount / posts.length;

  return {
    score: authenticityRatio,
    realWeddings: realWeddingCount,
    styledShoots: posts.length - realWeddingCount,
    confidence: this.computeConfidence(posts)
  };
}

3. Professionalism Signals

Vendor communication and reliability:

async assessProfessionalism(vendor: Vendor): Promise<ProfessionalismScore> {
  // Caption quality
  const captionQuality = await this.analyzeCaptions(vendor);

  // Response patterns
  const responseTime = await this.analyzeResponseTime(vendor);

  // Client feedback
  const clientSentiment = await this.analyzeClientComments(vendor);

  // Consistency
  const postingConsistency = this.analyzePostingSchedule(vendor);

  return {
    captionQuality: captionQuality.score,
    responsive: responseTime < 24,  // < 24 hours
    clientSatisfaction: clientSentiment,
    consistent: postingConsistency > 0.7,
    overall: this.computeOverall([
      captionQuality.score,
      responseTime < 24 ? 1 : 0.5,
      clientSentiment,
      postingConsistency
    ])
  };
}

4. Portfolio Consistency

Quality across all work:

measureConsistency(posts: Post[]): number {
  // Compute embeddings for all posts
  const embeddings = posts.map(p => p.embedding);

  // Measure clustering (tight cluster = consistent style)
  const centroid = this.computeCentroid(embeddings);
  const distances = embeddings.map(e => this.euclideanDistance(e, centroid));

  // Consistency = inverse of variance
  const variance = this.variance(distances);
  const consistency = 1 / (1 + variance);

  return consistency;
}

Implementation

class QualityGuardian {
  async evaluateVendor(vendor: Vendor): Promise<QualityScore> {
    // Parallel evaluation of all dimensions
    const [visual, authenticity, professionalism, consistency] = await Promise.all([
      this.assessVisualQuality(vendor),
      this.detectAuthenticity(vendor),
      this.assessProfessionalism(vendor),
      this.measureConsistency(vendor.posts)
    ]);

    // Weighted overall score
    const overall =
      visual.overall * 0.4 +
      authenticity.score * 0.3 +
      professionalism.overall * 0.2 +
      consistency * 0.1;

    return {
      overall,
      visual,
      authenticity,
      professionalism,
      consistency,
      reasoning: this.explainScore(visual, authenticity, professionalism, consistency),
      strengths: this.identifyStrengths(visual, authenticity, professionalism),
      improvements: this.suggestImprovements(visual, authenticity, professionalism)
    };
  }
}

Tiered Evaluation Strategy

Cost optimization through tiered filtering:

async tieredEvaluation(vendor: Vendor): Promise<QualityScore> {
  // Tier 1: Fast heuristics (free)
  const heuristic = await this.heuristicFilter(vendor);
  if (heuristic.score < 0.5) {
    return heuristic;  // Skip low-quality immediately
  }

  // Tier 2: Gemini Flash (cheap, $0.0001/request)
  const gemini = await this.geminiEvaluation(vendor);
  if (gemini.score < 0.7) {
    return gemini;  // Medium quality, no need for expensive eval
  }

  // Tier 3: GPT-4V (expensive, $0.003/request, only for edge cases)
  return await this.gpt4vEvaluation(vendor);
}

private async heuristicFilter(vendor: Vendor): Promise<QualityScore> {
  // Simple rules based on Instagram metrics
  const score =
    (vendor.followerCount > 500 ? 0.3 : 0) +
    (vendor.engagementRate > 0.03 ? 0.3 : 0) +
    (vendor.postCount > 50 ? 0.2 : 0) +
    (vendor.bio.length > 50 ? 0.2 : 0);

  return { overall: score, tier: 'heuristic' };
}

Batch Processing

Efficient Multi-Vendor Evaluation

async batchEvaluate(vendors: Vendor[]): Promise<Map<string, QualityScore>> {
  const batches = this.chunk(vendors, 10);  // 10 vendors per batch
  const scores = new Map();

  for (const batch of batches) {
    const batchScores = await this.evaluateBatch(batch);

    batch.forEach((vendor, i) => {
      scores.set(vendor.id, batchScores[i]);
    });

    // Rate limiting
    await this.delay(1000);  // 1 second between batches
  }

  return scores;
}

async evaluateBatch(vendors: Vendor[]): Promise<QualityScore[]> {
  const prompt = `
Evaluate these ${vendors.length} wedding vendors for quality.

For each vendor, provide scores (0.0-1.0) for:
- Visual quality (professional photography)
- Authenticity (real weddings vs styled shoots)
- Professionalism (captions, engagement, consistency)
- Overall quality

Vendors:
${vendors.map((v, i) => `
${i + 1}. @${v.username}
   Bio: ${v.bio}
   Followers: ${v.followerCount}
   Recent posts: ${v.recentPosts.length}
`).join('\n')}

Return JSON array: [{visual: 0.8, authenticity: 0.9, ...}, ...]
`;

  const response = await this.callGemini(prompt);
  return JSON.parse(response);
}

Quality Thresholds

Vendor Tiers

enum VendorTier {
  EXCEPTIONAL = 'exceptional',  // 0.9+
  HIGH_QUALITY = 'high_quality', // 0.75-0.89
  GOOD = 'good',                 // 0.6-0.74
  AVERAGE = 'average',           // 0.4-0.59
  LOW_QUALITY = 'low_quality'    // < 0.4
}

function getVendorTier(score: number): VendorTier {
  if (score >= 0.9) return VendorTier.EXCEPTIONAL;
  if (score >= 0.75) return VendorTier.HIGH_QUALITY;
  if (score >= 0.6) return VendorTier.GOOD;
  if (score >= 0.4) return VendorTier.AVERAGE;
  return VendorTier.LOW_QUALITY;
}

Feed Filtering

async filterFeedByQuality(
  candidates: Content[],
  minQuality: number = 0.75
): Promise<Content[]> {
  // Get quality scores (cached for 24 hours)
  const scores = await this.getQualityScores(
    candidates.map(c => c.vendorId)
  );

  // Filter by minimum threshold
  return candidates.filter(c => {
    const score = scores.get(c.vendorId) || 0;
    return score >= minQuality;
  });
}

Quality Drift Detection

Monitoring Quality Over Time

async detectQualityDrift(vendor: Vendor): Promise<DriftSignal> {
  // Compare recent posts vs historical
  const recentPosts = await this.getRecentPosts(vendor, days: 30);
  const historicalPosts = await this.getHistoricalPosts(vendor, days: 180);

  const recentQuality = await this.averageQuality(recentPosts);
  const historicalQuality = await this.averageQuality(historicalPosts);

  const drift = recentQuality - historicalQuality;

  if (Math.abs(drift) > 0.2) {
    return {
      drifted: true,
      direction: drift > 0 ? 'improving' : 'declining',
      magnitude: Math.abs(drift),
      action: drift < 0 ? 'review_vendor' : 'boost_vendor'
    };
  }

  return { drifted: false };
}

API Endpoints

Evaluate Vendor

Endpoint: POST /api/quality/evaluate

Request:

{
  "vendorId": "vendor-123",
  "tier": "full"  // heuristic | gemini | full
}

Response:

{
  "overall": 0.87,
  "visual": {
    "overall": 0.92,
    "composition": 0.95,
    "lighting": 0.90,
    "consistency": 0.85
  },
  "authenticity": {
    "score": 0.85,
    "realWeddings": 24,
    "styledShoots": 6
  },
  "professionalism": {
    "overall": 0.83,
    "captionQuality": 0.80,
    "responsive": true,
    "clientSatisfaction": 0.88
  },
  "consistency": 0.79,
  "tier": "high_quality",
  "reasoning": "Strong visual quality with excellent composition...",
  "strengths": ["Professional photography", "Authentic content"],
  "improvements": ["More consistent posting schedule"]
}

Batch Evaluate

Endpoint: POST /api/quality/batch