Skip to content

Latest commit

 

History

History
613 lines (523 loc) · 16.5 KB

File metadata and controls

613 lines (523 loc) · 16.5 KB

Kortex - Feature Flows Documentation

Feature Flows

1. Image Import Flow

1. User opens the app
2. Selects "Import Photo" from gallery or take new photo by using the camera
3. Image is validated and optimized
4. Image now loads in main editor
5. Ready for editing

2. Crop & Rotate Flow

Crop Flow

1. User selects Crop or Rotate tool from the toolbar
2. Crop mode activated with adjustable grid overlay
3. User can:
   - Drag corners to adjust crop area 
   - Aspect ratio or Corner crop can be used
   - Select aspect ratio presets (1:1, 4:3, 16:9, Free)
   - There is rule of thirds grid for guidance
   - Rotate image in 90° increments
   - Fine tune rotation angle with slider
4. Preview updates in realtime
5. User confirms or cancels changes
6. Cropped/rotated image replaces current view

Rotate Flow

Implementation:

fun applyCrop(cropRect: Rect, rotationAngle: Float) {
    viewModelScope.launch {
        val croppedBitmap = ImageUtils.cropBitmap(currentBitmap, cropRect)
        val rotatedBitmap = ImageUtils.rotateBitmap(croppedBitmap, rotationAngle)
        updateCurrentImage(rotatedBitmap)
        addToUndoStack()
    }
}

3. Text Overlay Flow

Text Flow

1. User taps "Text" icon in toolbar
2. Text editor screen opens
3. User enters text content
4. Customize text properties:
   - Font selection (system and custom fonts)
   - Text color picker
   - Size adjustment slider
   - Style options (bold, italic)
   - Opacity control
5. Text appears as draggable overlay on image
6. User can:
   - Move text by dragging
   - Rotate with two-finger gesture
   - Scale with pinch gesture
   - Add multiple text elements
7. Tap outside the toolbar or confirm to apply
8. After saving , text becomes part of image layer and it can be editted again by long press

Text Object Structure:

data class TextOverlay(
    val id: String,
    val text: String,
    val font: String,
    val color: Color,
    val size: Float,
    val position: Offset,
    val rotation: Float,
    val opacity: Float,
    val isBold: Boolean = false,
    val isItalic: Boolean = false
)

4. Professional Adjustments Flow

Professional Adjustments

1. User opens "Adjust" panel from bottom toolbar
2. Available adjustment sliders displayed:
   - Exposure 
   - Contrast 
   - Brightness 
   - Highlights 
   - Shadows 
   - Temperature 
   - Saturation 
   - Sharpness 
3. User adjusts sliders:
   - Live preview updates in real time (GPU accelerated)
   - Before/after comparison with hold on compare
4. Undo/Redo support for adjustments
5. Reset button to revert all adjustments
6. User confirms or cancels changes

Adjustment Engine:

fun adjustParameter(param: AdjustParam, value: Float) {
    adjustParams = adjustParams.copy(
        exposure = if (param == AdjustParam.EXPOSURE) value else adjustParams.exposure,
        brightness = if (param == AdjustParam.BRIGHTNESS) value else adjustParams.brightness,
        // ... other parameters
    )
    
    // Debounced update (10ms delay)
    viewModelScope.launch {
        delay(10)
        applyAdjustments()
    }
}

AI Object Removal

5. AI Object Removal Flow

1. User taps "Object Removal" in side panel
2. System initializes EdgeSAM (Segment Anything Model) encoder
3. User interaction modes:
   a. Point Selection: Single tap on object
   b. Box Selection: Drag to create bounding box
4. SAM decoder generates precise segmentation mask
5. User can refine mask:
   - Brush tool to add areas
   - Eraser tool to remove areas
   - Adjust brush size
6. User confirms mask
7. LaMa inpainting model processes:
   - Removes selected object
   - Fills area with contextually appropriate content
8. Result displayed with undo option
9. Processing time: 2-5 seconds

Object Removal Implementation:

// Step 1: Initialize SAM
fun initializeObjectRemoval() {
    isInpaintingMode = true
    viewModelScope.launch {
        encoderResult = zimExecutor?.runEncoder(currentBitmap)
        setStatusMessage("Tap objects to remove")
    }
}

// Step 2: Select Object
fun onImageTap(x: Float, y: Float) {
    if (isInpaintingMode && encoderResult != null) {
        viewModelScope.launch {
            val mask = zimExecutor?.runDecoder(encoderResult!!, x, y)
            segmentedMaskBitmap = mask
        }
    }
}

// Step 3: Execute Inpainting
fun executeInpainting() {
    viewModelScope.launch {
        val result = lamaExecutor?.runInpainting(currentBitmap, maskBitmap)
        updateCurrentImage(result)
        addToUndoStack()
    }
}

6. Background Removal & Replacement Flow

1. User selects "Background Removal" from AI Edits feature
2. System activates EdgeSAM object selection
3. User selects subject to keep:
   - Tap to select person or object / Box selection feature
   - AI generates precise edge mask
4. User refines selection with brush tools
5. System creates transparent background version
6. Background replacement options:
   a. Keep Transparent: Exports as PNG
   b. Gallery Image: Choose new background from photos
   c. Solid Color: Picks color from palette
   d. Gradient: Select gradient background
7. System composites subject with new background
8. Edge blending and harmonization applied
9. Preview and confirm final result

Background Processing:

fun replaceBackground(newBackground: Bitmap) {
    viewModelScope.launch {
        // Extract subject using mask
        val subject = ImageUtils.extractWithMask(currentBitmap, maskBitmap)
        
        // Composite with new background
        val result = ImageUtils.compositeImages(newBackground, subject, maskBitmap)
        
        // Apply edge feathering
        val blended = ImageUtils.applyEdgeBlending(result, maskBitmap)
        
        updateCurrentImage(blended)
        addToUndoStack()
    }
}

7. Move Object Flow

1. User taps "Move Object" tool
2. Select object using SAM (same as object removal)
3. System performs two operations in parallel:
   a. Extracts object to separate layer
   b. Removes object from background using LaMa
4. Extracted object appears as draggable element
5. User manipulates object:
   - Drag to reposition
   - Pinch to scale
   - Rotate with two-finger gesture
6. Real time shadow and lighting adjustments
7. User confirms final position
8. System blends object with background:
   - Color harmonization
   - Shadow generation
   - Edge blending
9. Final composite saved

Move Object Implementation:

fun moveObjectWorkflow() {
    viewModelScope.launch {
        // 1. Extract object
        val extractedObject = ImageUtils.extractWithMask(currentBitmap, maskBitmap)
        
        // 2. Remove from background
        val cleanBackground = lamaExecutor?.runInpainting(currentBitmap, maskBitmap)
        
        // 3. Create movable object state
        movableObject = MovableObject(
            bitmap = extractedObject,
            position = initialPosition,
            scale = 1f,
            rotation = 0f
        )
        
        // 4. When user confirms position
        fun confirmObjectPosition() {
            val result = ImageUtils.compositeImages(
                cleanBackground,
                movableObject.bitmap,
                movableObject.position,
                movableObject.scale,
                movableObject.rotation
            )
            updateCurrentImage(result)
        }
    }
}

8. AI Auto-Enhance Flow

1. User taps "Auto Enhance" button
2. System loads AI enhancement models:
   - Analyzer Model: Evaluates image and calculates parameters
   - Enhancement Model: Applies HDRNet-based processing
3. AI analyzes image and generates:
   - Optimal exposure adjustment
   - Contrast enhancement
   - Color saturation
   - Brightness correction
   - Highlight/shadow recovery
   - Temperature adjustment
   - Sharpness enhancement
4. Parameters automatically applied to image
5. User sees before/after comparison
6. User can:
   - Accept AI adjustments
   - Fine tune individual parameters
   - Revert to original

Auto-Enhance Process:

fun executeAutoEnhance() {
    viewModelScope.launch {
        setStatusMessage("Analyzing image...")
        
        val enhancedParams = autoEnhanceExecutor?.analyze(currentBitmap)
        
        enhancedParams?.let {
            adjustParams = AdjustParams(
                exposure = it.exposure,
                contrast = it.contrast,
                brightness = it.brightness,
                highlights = it.highlights,
                shadows = it.shadows,
                temperature = it.temperature,
                saturation = it.saturation,
                sharpness = it.sharpness
            )
            
            applyAdjustments()
            addToUndoStack()
        }
    }
}

9. AI Retouch Flow (Cloud-Based)

AI Retouch

API Flow:
2. Choose retouching mode:
   
   Mode A - Reference Image:
   - User selects reference image from gallery
   - System uploads both images to cloud API
   - AI analyzes reference image style
   - Returns adjustment parameters matching reference
   
   Mode B - Text Instructions:
   - User enters natural language instruction
   - Examples: "Make it look vintage" or "Professional headshot"
   - System processes instruction with AI
3. Cloud API returns adjustment parameters
4. Parameters automatically applied to image
5. User can fine-tune or accept changes
6. Processing time: 3-10 seconds (network dependent)

AI Retouch API Call:

suspend fun performAiRetouch(mode: RetouchMode) {
    when (mode) {
        is RetouchMode.Reference -> {
            val params = retouchRepository.retouchWithReference(
                sourceImage = currentBitmap,
                referenceImage = mode.referenceBitmap
            )
            applyRetouchParams(params)
        }
        
        is RetouchMode.Instruction -> {
            val params = retouchRepository.retouchWithInstruction(
                sourceImage = currentBitmap,
                instruction = mode.text,
                creativity = mode.n,
                intensity = mode.t
            )
            applyRetouchParams(params)
        }
    }
}

10. Smart Fill (Generative Fill) Flow

1. User selects area to fill using selection tools
2. Creates mask for target region
3. User enters text prompt describing desired content
4. Optional: Adjust "vibe strength" (0.0-1.0)
   - Low: Subtle, contextual fill
   - High: Creative, artistic generation
5. System uploads image, mask, and prompt to cloud API
6. AI generates content based on prompt
7. Content seamlessly fills selected area
8. Result displayed with undo option
9. Processing time: 5-15 seconds

Smart Fill Implementation:

suspend fun executeSmartFill(prompt: String, vibeStrength: Float) {
    setStatusMessage("Generating content...")
    
    val result = cloudEditRepository.performSmartFill(
        originalBitmap = currentBitmap,
        maskBitmap = selectionMask,
        prompt = prompt,
        vibeStrength = vibeStrength
    )
    
    result.onSuccess { filledImage ->
        updateCurrentImage(filledImage)
        addToUndoStack()
    }
}

11. Watermark Application Flow

Watermark Application

Visible Text Watermark:

1. User selects "Add Watermark" option
2. Choose watermark type: Text or Image
3. For text watermark:
   - Enter watermark text
   - Customize:
     * Font style and size
     * Text color with opacity
     * Shadow effects
     * Outline options
4. Watermark appears as overlay on image
5. User positions watermark with drag gesture
6. Adjust final opacity
7. Apply to image

LSB Steganographic Watermark:

1. User selects "Hidden Watermark" option
2. Enter secret text message (up to 1MB)
3. System embeds message in image pixels:
   - Uses Least Significant Bit (LSB) algorithm
   - Embeds in blue channel (least perceptible)
   - Invisible to human eye
4. Must export as PNG to preserve watermark data
5. Detection feature available to decode hidden messages

Watermark Implementation:

// Visible Watermark
fun applyVisibleWatermark(text: String, config: WatermarkConfig) {
    val watermarkedBitmap = AIWatermarkHelper.addTextWatermark(
        bitmap = currentBitmap,
        text = text,
        font = config.font,
        color = config.color,
        size = config.size,
        position = config.position,
        opacity = config.opacity
    )
    updateCurrentImage(watermarkedBitmap)
}

// LSB Watermark
fun applyLSBWatermark(secretText: String) {
    val watermarkedBitmap = LSBWatermarkUtil.embedWatermark(
        bitmap = currentBitmap,
        message = secretText
    )
    updateCurrentImage(watermarkedBitmap)
    // Note: Must save as PNG
}

12. Voice Command Flow

1. User taps microphone icon
2. System activates Vosk speech recognition (offline)
3. User speaks natural language command
4. Examples of supported commands:
   - "Increase brightness"
   - "Make it warmer"
   - "Add more contrast"
   - "Remove that object"
   - "Undo last change"
   - "Apply auto enhance"
5. Vosk processes audio locally (no server needed)
6. Command parser interprets intent
7. Corresponding editing action executed
8. Visual feedback shows recognized command
9. Privacy first: All processing on-device

Voice Command Processing:

fun processVoiceCommand(recognizedText: String) {
    val command = parseCommand(recognizedText.lowercase())
    
    when {
        command.contains("brightness") -> {
            val value = extractNumericValue(command) ?: 20f
            adjustBrightness(value)
        }
        command.contains("contrast") -> {
            val value = extractNumericValue(command) ?: 15f
            adjustContrast(value)
        }
        command.contains("warmer") -> adjustTemperature(500f)
        command.contains("cooler") -> adjustTemperature(-500f)
        command.contains("enhance") -> executeAutoEnhance()
        command.contains("remove") -> enterObjectRemovalMode()
        command.contains("undo") -> undoLastAction()
        command.contains("save") -> exportImage()
    }
    
    setStatusMessage("Executed: $recognizedText")
}

13. Export Flow

1. User taps "Export" or "Save" button
2. Export options dialog appears:
   - Format: JPEG or PNG
   - Quality: (for JPEG) 60-100%
   - Resolution: Original, Large, Medium, Small
3. User selects options
4. System processes final image:
   - Applies all pending edits
   - Compresses according to settings
   - Optimizes metadata
5. Image saved to gallery
6. Success notification with "Share" option
7. User can:
   - Share directly to social media
   - Open in other apps
   - Continue editing

Export Implementation:

fun exportImage(format: ExportFormat, quality: Int, resolution: Resolution) {
    viewModelScope.launch {
        setStatusMessage("Exporting image...")
        
        val finalBitmap = when (resolution) {
            Resolution.ORIGINAL -> currentBitmap
            Resolution.LARGE -> ImageUtils.resizeBitmap(currentBitmap, 2048)
            Resolution.MEDIUM -> ImageUtils.resizeBitmap(currentBitmap, 1024)
            Resolution.SMALL -> ImageUtils.resizeBitmap(currentBitmap, 512)
        }
        
        val file = when (format) {
            ExportFormat.JPEG -> ImageUtils.saveAsJpeg(finalBitmap, quality)
            ExportFormat.PNG -> ImageUtils.saveAsPng(finalBitmap)
        }
        
        MediaStore.Images.Media.insertImage(
            context.contentResolver,
            file.absolutePath,
            file.name,
            "Edited with Kortex"
        )
        
        setStatusMessage("Image saved successfully")
    }
}

Summary

This documentation covers all major feature flows in the Kortex photo editing application:

  • Basic Editing: Import, Crop, Rotate, Text Overlay
  • Professional Adjustments: Real-time parameter controls with GPU acceleration
  • AI-Powered Features:
    • Object removal with SAM + LaMa
    • Background removal and replacement
    • Move object tool
    • Auto-enhance
    • AI retouch (cloud-based)
    • Smart fill (generative AI)
  • Advanced Features:
    • Voice control with offline speech recognition
    • Visible and LSB steganographic watermarking
  • Export: Multiple format and quality options

Each flow is designed for intuitive user experience while leveraging powerful AI capabilities both on-device and in the cloud.