The Koster Seafloor Observatory is an open-source, citizen science and machine learning approach to analyse subsea movies.
The system processes underwater footage and its associatead metadata into biologically-meaningfull information. The format of the underwater media is standard (.mp4 or .png) and the associated metadata should be captured in three csv files (“movies”, “sites” and “species”) following the Darwin Core standards (DwC).
This Object Detection module contains scripts and resources to train and evaluate object detection models.
The tutorials enable users to customise Yolov5 models using Ultralytics. The repository contains both model-specific files (same structure as Ultralytics) as well as specific source files related to Koster pipelines (src folder) and utils (kso_utils). It is not recommended to simply clone this repository as many dependencies are resolved using the supplied Dockerfile. The notebooks rely on the koster utility functions.
| Name | Description | Try it! |
|---|---|---|
| 1. Check footage and metadata | Check format and contents of footage and sites, media and species csv files |   |
| 2. Upload new media to the system* | Upload new underwater media to the cloud/server and update the csv files | |
| 3. Upload clips to Zooniverse | Prepare original footage and upload short clips to Zooniverse | |
| 4. Upload frames to Zooniverse | Extract frames of interest from original footage and upload them to Zooniverse | |
| 5. Train ML models | Prepare the training and test data, set model parameters and train models | |
| 6. Evaluate ML models | Use ecologically-relevant metrics to test the models | |
| 7. Publish ML models | Publish the model to a public repository | |
| 8. Analyse Zooniverse classifications | Pull up-to-date classifications from Zooniverse and report summary stats/graphs | |
| 9. Download and format Zooniverse classifications | Pull up-to-date classifications from Zooniverse and format them for further analysis | Coming soon |
| 10. Run ML models on footage | Automatically classify new footage | Coming soon |
* Project-specific tutorial
If you want to fully use our system (Binder has computing limitations), you will need to download this repository on your local computer or server.
Clone this repository using
git clone --recurse-submodules https://github.com/ocean-data-factory-sweden/koster_yolov4.gitNavigate to the folder where you have cloned the repository or unzipped the manually downloaded repository.
cd koster_yolov4Then install the requirements by running.
pip install -r requirements.txtBefore using Option 2, users should have login credentials and have setup the Chalmers VPN on their local computers
Information for Windows users: Click here Information for MAC users: Click here
To use the Jupyter Notebooks within the Alvis HPC cluster, please visit Alvis Portal and login using your SNIC credentials.
Once you have been authorized, click on "Interactive Apps" and then "Jupyter". This open the server creation options.
Here you can keep the settings as default, apart from the "Number of hours" which you can set to the desired limit. Then choose either Data Management (Runtime (User specified jupyter1.sh)) or Machine Learning (Runtime (User specified jupyter2.sh)) from the Runtime dropdown options.
This will directly queue a server session using the correct container image, first showing a blue window and then you should see a green window when the session has been successfully started and the button "Connect to Jupyter" appears on the screen. Click this to launch into the Jupyter notebook environment.
Important note: The remaining time for the server is shown in green window as well. If you have finished using the notebook server before the alloted time runs out, please select "Delete" so that the resources can be released for use by others within the project.
If you use this code or its models in your research, please cite:
Anton V, Germishuys J, Bergström P, Lindegarth M, Obst M (2021) An open-source, citizen science and machine learning approach to analyse subsea movies. Biodiversity Data Journal 9: e60548. https://doi.org/10.3897/BDJ.9.e60548
You can find out more about the project at https://www.zooniverse.org/projects/victorav/the-koster-seafloor-observatory.
We are always excited to collaborate and help other marine scientists. Please feel free to contact us with your questions.
- Installing conda
- Create new environment (e.g. "new environment")
- Install git and pip (with conda)
- Clone kso repo
- pip install ipykernel
- python -m ipykernel install --user --name="new_environment"
- from the jupyter notebook select kernel/change kernel