Detailed documentation: step by step for Aves

Object detection for image cropping: step-by-step documentation with input and output files

Aves

For additional information, see github.com/aubricot/computer_vision_with_eol_images/object_detection_for_image_cropping.
31 May 2022

Model used for final crops
Faster-RCNN

Pre-trained models are used, so no pre-processing (Step 1) or training (Step 2) steps are needed.
--<>--<>--<>--<>--<>--<>--<>--<>--<>--<>--<>--<>--

Step 3) Generate square cropping coordinates around birds in EOL images

Script(s)

aves_generate_crops_tf2.ipynb

Input file(s)

https://editors.eol.org/other_files/bundle_images/files/images_for_Aves_breakdown_download_000001.txt https://editors.eol.org/other_files/bundle_images/files/images_for_Aves_20K_breakdown_download_000001.txt

Output file(s)

aves_cropcoords_tf2_ssd_a.tsv aves_cropcoords_tf2_ssd_b.tsv aves_cropcoords_tf2_ssd_c.tsv aves_cropcoords_tf2_ssd_d.tsv aves_cropcoords_tf2_ssd_concat.tsv aves_cropcoords_tf2_ssd_concat_displaytest.tsv aves_cropcoords_tf2_ssd_20k_final.tsv

Detailed steps

Step 3a) Generate square cropping coordinates around birds in EOL images
Open aves_generate_crops_tf2.ipynb. Run “Installs & Imports” to prepare the Tensorflow 2 Object Detection API. Use the dropdown menu to choose to run as a demo in Colab runtime or save results in your Google Drive. Under “Generate cropping coordinates for images” > "Prepare object detection functions and settings," choose to use SSD or Faster-RCNN for object detection using the dropdown form field. Next, test the model build by running inference on 3 images from URL. Finally, run both cells under "Generate crops: Run inference on EOL images & save results for cropping" to generate cropping output files a-d.

Step 3b) Convert bounding boxes to square, centered image cropping coordinates and pad by 2.5% Move on to "Post-process detection results," to modify bounding box coordinates exported from Step 3a (ex: aves_cropcoords_tf2_ssd_d.tsv). Next, concatenate batch files ("Merge batch output files A-D", aves_cropcoords_tf2_ssd_concat.tsv) and combine all boxes per image to a superbox ("Combine individual detection boxes into one 'superbox' per image"). While making the superboxes, out of bounds values are removed, dimensions are made square, and crops are padded by 2.5% image size. Finally, results are formatted to EOL crop coordinate formatting standards. Export square, padded cropping dimensions to display test file (ex: aves_cropcoords_tf2_ssd_concat_displaytest.tsv) for use in Step 3 below. Final crop dimensions are formatted to EOL cropping standards and exported (ex: aves_cropcoords_tf2_ssd_20k_final.tsv).

Step 3c) Display converted cropping coordinates on images Finally, go to "Display cropping results on images," to evaluate bounding box transformations in step 3b (or fine-tune accordingly before returning to Step 3c, ex: adjust padding amount to ensure beaks are fully present within crops).