-
Notifications
You must be signed in to change notification settings - Fork 4
Detailed documentation: step by step for Ratings
Image Quality Ratings
For additional information, see github.com/aubricot/computer_vision_with_eol_images.
29 Oct 2024
Model used for final tags
MobileNet SSD v2 attempt 18
Only do Steps 1-3 if training new object detector. To run images through pre-trained EOL object detector, proceed to Step 4.
rating_preprocessing.ipynb
image_ratings.txt images_selected_as_exemplar.txt
Training data
|-rating
|-images
|-1
|-2
|-3
|-4
|-5
|-agg
|-bad
|-good
|-images
|-image_data
|-bundles
|-one_download.txt
|-one_download_7k.txt
|-two_download.txt
|-two_download_7k.txt |-three_download.txt
|-three_download_7k.txt |-four_download.txt
|-four_download_7k.txt |-five_download.txt
|-five_download_7k.txt
Download 7,000 images from EOL user generated quality rating and exemplar image bundles to Google Drive into folders by rating class from 1 (bad) to 5 (good). Then, delete all downloaded non-image files. Initial training attempts on all five classes were not consistent, so classes are aggregated into bad (classes 1 and 2) and good (classes 4 and 5) in the last step.
rating_train.ipynb
Training data
|-rating
|-images
|-agg
|-bad
|-good
|-rating
|-saved_models
|-train_graphs
After preparing the training dataset in Google Drive in Step 1, build and train classification models using rating_train.ipynb. Form fields and dropdown menus within the notebook walk you through model selection, adjusting hyperparameters, training, and the display of trained models. After training is finished, choose the N-best models based on accuracy values and optimal learning (minimizing over/underfit) by inspecting train_graphs for each training session. Keep trying new training attempts with adjusted hyperparameters until desired results are acheived.
inspect_train_results.ipynb
|-rating
Saved models
|-saved_models
Test images
|-images
|-agg
|-bad
|-good
|-rating
|-inspect_resul
Review false and positive detections by class for all N-best attempts/models chosen in Step 2. Select trained model and prediction confidence threshold values to use for classifying quality ratings for EOL images. Threshold values should be chosen that maximize coverage and minimize error. Classification error was low for "bad" image predictions, but high for "good" image predictions. This matches our observation that users were more conflicted on what makes a good image than a bad one, so it's logical that the model would reproduce this behavior. As a result, MobileNet SSDv2 training attempt 18 (output=bad, confidence>1.5) was selected to generate tags for "bad" images which will be pushed to the back of galleries to help in quality sorting.
To run images through pre-trained EOL object detector, follow steps below
--<>--<>--<>--<>--<>--<>--<>--<>--<>--<>--<>--<>--<>---
classify_images.ipynb
Saved Models
|-rating
|-saved_models
https://editors.eol.org/other_files/bundle_images/files/images_for_Chiroptera_20K_breakdown_000031.txt
|-rating
|-results
|-tags_rating_20k_a.tsv
|-tags_rating_20k_b.tsv
|-tags_rating_20k_c.tsv
|-tags_rating_20k_d.tsv
|-tags_rating_20k_finaltags.tsv
Run images through classification model selected in Step 2. Then post-process results using model prediction and confidence value chosen in Step 3. Display final classification tags on images and verify that they are as expected (or further fine-tune confidence thresholds).