As promised, a thread on our [ @StanleyHoBio @milja001] preprint: 'Comprehensive benchmarking of tools to identify phages in metagenomic shotgun sequencing data' https://www.biorxiv.org/content/10.1101/2021.04.12.438782v1 https://twitter.com/WvSchaik/status/1381907592130203649

The goal of our work was to test 10 different tools (based on machine-learning, reference sequences or a mix of both) to detect phages in metagenomic shotgun sequencing datasets.

We used artificial contigs generated from complete RefSeq genomes representing phages, plasmids, and chromosomes, and a previously sequenced mock community containing four phage strains to evaluate the precision, recall and F1 (= precision x recall) -scores of the tools

We found that default settings of the tools do not always give the optimal results so this is something to keep in mind if you are using these tools

The tools were quite variable in their performance on artificial chromosome/plasmid/phage contigs. DeepVirFinder and VirFinder (a previous iteration of DeepVirFinder) are doing well

Tools performed a bit worse on mock community shotgun metagenomes with spike-ins of known phages (DeepVirFinder still among the best)

Some tools are also a bit slow

Finally we wanted to check whether the choice of tool had an impact on the estimation of diversity metrics. Perhaps surprisingly, there was very little variation among tools (so whatever tool you use, your diversity metrics will probably be OK) with the exception of Seeker.

So TL;DR: DeepVirFinder is a solid choice if you want to identify phages in metagenomes, but it is not perfect (no tool is) and there is scope to optimise its performance using different settings than the default.

This project would not have been possible without other groups ( @simroux_virus, @Manuel_Kleiner) sharing their sequence data without restrictions publicly so a massive thanks to them and yay! open science!

Let us know if you have any comments/suggestions!