Skip to content

MULTI_PROVIDER_IMAGE_GENERATION.md

Latest commit

 

History

History
420 lines (317 loc) · 10.4 KB

MULTI_PROVIDER_IMAGE_GENERATION.md

File metadata and controls

420 lines (317 loc) · 10.4 KB

Multi-Provider Image Generation - Implementation Summary

Overview

Successfully implemented a flexible, interface-based image generation system that supports multiple AI providers. Teachers can now choose between Azure OpenAI DALL-E 3 and Ideogram AI for generating educational images.

Architecture

Interface-Based Design

Created a clean separation between the image generation interface and provider implementations:

┌─────────────────────────────────┐
│  ImageGenerationService          │  ← Main service (facade)
│  - Manages caching              │
│  - Routes to correct provider   │
└─────────────────────────────────┘
              ↓
┌─────────────────────────────────┐
│   ImageGenerator Interface       │
│   - GenerateImage()             │
│   - GetProviderName()           │
│   - IsConfigured()              │
└─────────────────────────────────┘
         ↙                    ↘
┌──────────────────┐    ┌──────────────────┐
│ AzureOpenAI      │    │ Ideogram         │
│ Generator        │    │ Generator        │
└──────────────────┘    └──────────────────┘

Files Created

1. Interface Definition (services/image_generator_interface.go)

  • ImageGenerator interface
  • ImageCacheManager interface
  • ImageGeneratorOptions struct
  • GeneratedImageResult struct

2. Cache Manager (services/file_cache_manager.go)

  • Implements ImageCacheManager
  • MD5-based caching
  • Local file storage
  • Cache statistics and management

3. Azure OpenAI Provider (services/azure_openai_generator.go)

  • Implements ImageGenerator
  • DALL-E 3 integration
  • Educational prompt engineering
  • Size/quality/style options

4. Ideogram Provider (services/ideogram_generator.go)

  • Implements ImageGenerator
  • Ideogram AI V_2 integration
  • RESTful API client
  • Safety filtering

5. Main Service (services/image_generation_service.go - refactored)

  • Provider selection based on config
  • Unified caching layer
  • Backward compatibility

Configuration

New Environment Variables

Added to backend/config/config.go:

// Ideogram Configuration
IdeogramAPIKey string

// Image Generation Provider
ImageGenerationProvider string // "azure" or "ideogram"

.env Configuration

# Choose provider
IMAGE_GENERATION_PROVIDER=azure  # or "ideogram"

# Azure OpenAI (if using Azure)
AZURE_OPENAI_KEY=your_key
AZURE_OPENAI_ENDPOINT=https://...
AZURE_OPENAI_DEPLOYMENT=dall-e-3

# Ideogram (if using Ideogram)
IDEOGRAM_API_KEY=your_key

# Caching (works with both)
IMAGE_CACHE_ENABLED=true

Provider Comparison

Feature Azure DALL-E 3 Ideogram AI
Setup Complexity High (Azure resource + deployment) Low (API key only)
Image Style Cartoon, artistic Realistic, design
Text Rendering Moderate Excellent
Speed 10-30 sec 5-15 sec
Cost $0.04-$0.12/image $0.08/image
Best For Cartoons, playful scenes Realistic images, labels

Key Features

1. Provider Abstraction

  • Clean interface for adding new providers
  • Easy switching via configuration
  • No code changes needed to switch providers

2. Unified Caching

  • Works with all providers
  • MD5-based cache keys
  • Automatic download and storage
  • Cache stats and management

3. Educational Prompts

  • Child-safe content (ages 4-12)
  • Bright, colorful, cheerful style
  • Clear visual representation
  • Provider-specific optimization

4. Error Handling

  • Graceful degradation
  • Clear error messages
  • Provider-specific diagnostics

5. Backward Compatibility

  • Existing code continues to work
  • Type aliases for smooth migration
  • Same API surface

How It Works

Provider Selection Flow

1. User sets IMAGE_GENERATION_PROVIDER in .env
2. NewImageGenerationService() reads config
3. Switch statement creates appropriate generator:
   - "ideogram" → NewIdeogramGenerator()
   - "azure" or default → NewAzureOpenAIGenerator()
4. Service wraps generator + cache
5. All requests go through unified interface

Generation Flow

1. Request comes in with word + translation
2. Check if provider is configured
3. Build cache key from prompt
4. Check cache (if enabled)
   ├─ Hit: Return cached image
   └─ Miss: Continue
5. Call provider-specific GenerateImage()
6. Download and cache result
7. Return image URL + local path

Usage Examples

Teacher UI (No Changes Needed)

// Frontend code works the same
const result = await imageGenerationService.generateImage({
  word: "apple",
  translation: "蘋果",
  size: "1024x1024",
  quality: "standard",
  style: "vivid"
});

Backend Automatically Routes

// Service automatically uses configured provider
result, err := service.GenerateImage(ctx, opts)
// Could be Azure or Ideogram - transparent to caller!

Adding New Providers

To add a new provider (e.g., Stability AI, Midjourney):

Step 1: Create Generator

// services/stability_generator.go
type StabilityGenerator struct {
    apiKey string
}

func (g *StabilityGenerator) GenerateImage(ctx context.Context, opts ImageGeneratorOptions) (*GeneratedImageResult, error) {
    // Implementation
}

func (g *StabilityGenerator) GetProviderName() string {
    return "Stability AI"
}

func (g *StabilityGenerator) IsConfigured() bool {
    return g.apiKey != ""
}

Step 2: Add to Config

// config/config.go
StabilityAPIKey string

Step 3: Add to Service

// services/image_generation_service.go
case "stability":
    gen, err := NewStabilityGenerator()
    // ...

Step 4: Update .env

IMAGE_GENERATION_PROVIDER=stability
STABILITY_API_KEY=your_key

Done! No changes to handlers, routes, or frontend needed.

Testing

Test Azure Provider

IMAGE_GENERATION_PROVIDER=azure
go run main.go
# Generate an image

Test Ideogram Provider

IMAGE_GENERATION_PROVIDER=ideogram
go run main.go
# Generate an image

Test Provider Switching

# Generate with Azure
IMAGE_GENERATION_PROVIDER=azure
# Click "Generate Image" - uses Azure

# Switch to Ideogram
IMAGE_GENERATION_PROVIDER=ideogram
# Restart backend
# Click "Generate Image" - uses Ideogram

# Compare results!

Documentation

For Users

  • docs/AZURE_OPENAI_SETUP.md - Azure DALL-E 3 setup
  • docs/IDEOGRAM_SETUP.md - Ideogram AI setup
  • docs/IMAGE_GENERATION_ERROR_FIX.md - Troubleshooting
  • docs/TEACHER_IMAGE_GENERATION_GUIDE.md - Teacher guide

For Developers

  • services/image_generator_interface.go - Interface documentation
  • Code comments in all provider implementations

Performance

Caching Benefits

Scenario First Request Subsequent Requests
Azure (no cache) 15-30 sec, $0.04-$0.12 15-30 sec, $0.04-$0.12
Azure (with cache) 15-30 sec, $0.04-$0.12 <100ms, $0.00
Ideogram (no cache) 5-15 sec, $0.08 5-15 sec, $0.08
Ideogram (with cache) 5-15 sec, $0.08 <100ms, $0.00

Cache Storage

  • Location: backend/uploads/image-cache/
  • Format: PNG files with MD5 hash names
  • Average size: 200-500KB per image
  • 1000 images ≈ 200-500MB storage

Cost Analysis

Scenario: 1000 Vocabulary Words

Without Caching:

  • Azure DALL-E 3: $40-$120
  • Ideogram: $80

With Caching (one-time generation):

  • Azure DALL-E 3: $40-$120 (first time), then $0
  • Ideogram: $80 (first time), then $0

Best Practice: Pre-generate common words during setup, cache enabled.

Security & Safety

API Key Protection

  • Never commit .env to Git
  • Use environment variables in production
  • Rotate keys regularly

Content Safety

  • Azure: Built-in content filters
  • Ideogram: Safety checks + negative prompts
  • Both: Educational prompt templates

Access Control

  • Teacher and Admin only
  • JWT authentication required
  • Rate limiting recommended (future enhancement)

Future Enhancements

Potential Additions

  1. More Providers:

    • Stability AI
    • Midjourney (when API available)
    • Replicate models

  2. Advanced Features:

    • Image variations
    • Style mixing
    • Batch optimization
    • Custom prompt templates per provider

  3. Analytics:

    • Provider performance tracking
    • Cost analytics per provider
    • Quality ratings from teachers

  4. UI Enhancements:

    • Provider selection in UI
    • Side-by-side comparison
    • Regenerate with different provider

Migration Guide

From Old System to New

The refactored system is backward compatible. Existing code works without changes:

// Old way (still works)
opts := ImageGenerationOptions{
    Word: "apple",
    Translation: "蘋果",
}

// New way (same result)
opts := ImageGeneratorOptions{
    Word: "apple",
    Translation: "蘋果",
}

Type aliases ensure smooth transition.

Summary

✅ Implemented

  1. Interface-based architecture for multiple providers
  2. Azure OpenAI DALL-E 3 provider (refactored)
  3. Ideogram AI provider (new)
  4. File-based cache manager (extracted)
  5. Provider selection via configuration
  6. Comprehensive documentation
  7. Backward compatibility

🎯 Benefits

  • Flexibility: Easy to switch providers or add new ones
  • Cost optimization: Choose provider based on budget
  • Quality options: Pick best provider for content type
  • Future-proof: Interface-based for easy extensions
  • Maintainability: Clean separation of concerns

📝 Configuration

# In .env, choose your provider:
IMAGE_GENERATION_PROVIDER=azure     # Use Azure DALL-E 3
# OR
IMAGE_GENERATION_PROVIDER=ideogram  # Use Ideogram AI

🚀 Next Steps

  1. Configure desired provider in .env
  2. Test image generation
  3. Compare providers for your use case
  4. Enable caching for cost savings
  5. Pre-generate common vocabulary images

Implementation Date: January 2025
Version: 2.0 (Multi-Provider)
Providers Supported: Azure OpenAI DALL-E 3, Ideogram AI
Architecture: Interface-based, extensible, cached