Hierarchical Emergence Profiles of Human-Derived Dimensions are a Fundamental Property of Deep Neural Networks
In this repository, you will find all code related to the study: Hierarchical Emergence Profiles of Human-Derived Dimensions are a Fundamental Property of Deep Neural Networks.
pip: To install all required libraries, you can use the requirements.txt. To install from it, make sure you are in the right folder, and then install using "pip install -r requirements.txt"
conda: To install the libraries via conda, use the "environment.yml" file. By then activating the created enviroment, you should be able to run all code needed.
To repeat the analysis, you first need to download the data for the two dataset we used:
THINGS: A large-scale image dataset of 1,854 everyday object concepts with human-derived behavioral similarity ratings, used to model object-level representations in vision and cognition. You can download it under https://osf.io/z2784/ or THINGS.
STUFF: A curated image dataset of 200 natural and artificial material categories with human-derived perceptual dimensions, designed to study material perception. To acess the data and dimensions, see https://osf.io/5gr73/.
The main file needed for the analysis is the "running_Analysis.py" file which allows you to run the method for all models, datasets, and settings. It calls the activation extraction functions from the layer_extractions folder which contains one function/file for each model (and pixel extraction). It will also create all plots once the results have been fully created.
To access the data, you need to create two folders (named THINGS and STUFF) on the same level as the Results folder. To run, we need the following files in these folders which you can download above:
THINGS:
- spose_embedding_49d_sorted.txt
- labels.mat
- im.mat
STUFF:
- spose_embedding36.mat
- labels.mat
- im.mat
To use DINOv3 you need to request access on Huggingface, see here
The plots are automatically created when running the "running_Analysis.py" file. If you just want to run the plots, you can use the "running_Plots.py" file. All plots can be found in the "Results" folder under "final_plots".
If you use any code from this repository, please cite:
ADD CITATION
This project is licensed under the MIT License, see the LICENSE file for details.
Should you have any questions or trouble running the analysis, please contact me at: F.Burger@westernsydney.edu.au