phylter
is a tool that allows detecting, removing and visualizing outliers in phylogenomics dataset by iteratively removing taxa from gene families (gene trees) and optimizing a score of concordance between individual matrices.phylter
relies on DISTATIS (Abdi et al, 2005), an extension of multidimensional scaling to 3 dimensions to compare multiple distance matrices at once.phylter
builds on Phylo-MCOA (de Vienne et al. 2012) but is much faster and accurate.phylter
takes as input either a collection of phylogenetic trees (that are converted to distance matrices by phylter
), or a collection of pairwise distance matrices (obtained from multiple sequence alignements, for instance).phylter
accepts data with missing values (missing taxa in some genes).phylter
detects outliers with a method proposed by Hubert & Vandervieren (2008) for skewed data.phylter
does not accept that the same taxa is present multiple times in the same gene.
phylter
is written in R language.
For details about the functions, their usage, and a in-depth description of the use of phylter on a biological dataset, step-by-step, please vist the phylter web page : https://damiendevienne.github.io/phylter.
Note: if you don’t use R or don’t want to use R, containerized versions of phylter are also available (Docker and Singularity): https://damiendevienne.github.io/phylter/articles/phyltercontainer.html
if you use phylter, please cite: Comte, A., Tricou, T., Tannier, E., Joseph, J., Siberchicot, A., Penel, S., Allio, R., Delsuc, F., Dray, S., de Vienne, D.M. (2023). PhylteR: Efficient Identification of Outlier Sequences in Phylogenomic Datasets, Molecular Biology and Evolution, 40(11) msad234, https://doi.org/10.1093/molbev/msad234
Installation
phylter
is now on CRAN.
Installation is as easy as typing what follows at the R command prompt:
install.packages("phylter")
If you want the latest version, you can also install the development version of phylter:
- Install the release version of
remotes
from CRAN:
install.packages("remotes")
- Install the development version of
phylter
from GitHub:
remotes::install_github("damiendevienne/phylter")
- Once installed, the package can be loaded:
library("phylter")
Note: phylter requires R version > 4.0, otherwise it cannot be installed. Also, R uses the GNU Scientific Library. On Ubuntu, this can be installed prior to the installation of the phylter package by typing
sudo apt install libgsl-dev
in a terminal.
Usage
Here is a brief introduction to the use phylter
on a collection of gene trees. For more detailed explanations and a use case example, please visit https://damiendevienne.github.io/phylter/.
1. With the read.tree
function from the ape
package, read trees from external file and save as a list called trees
.
if (!requireNamespace("ape", quietly = TRUE))
install.packages("ape")
trees <- ape::read.tree("treefile.tre")
2. (optional) Read or get gene names somewhere (same order as the trees) and save it as a vector called names
.
3. Run phylter
on your trees (see details below for possible options).
results <- phylter(trees, gene.names = names)
Options
The
phylter
function is called as follows by default:phylter(X, bvalue = 0, distance = "patristic", k = 3, k2 = k, Norm = "median", Norm.cutoff = 0.001, gene.names = NULL, test.island = TRUE, verbose = TRUE, stop.criteria = 1e-5, InitialOnly = FALSE, normalizeby = "row", parallel = TRUE)
Arguments are as follows:
X
: A list of phylogenetic trees (phylo object) or a list of distance matrices. Trees can have different number of leaves and matrices can have different dimensions. If this is the case, missing values are imputed.bvalue
: IfX
is a list of trees, nodes with a support belowbvalue
will be collapsed prior to the outlier detection.distance
: IfX
is a list of trees, type of distance used to compute the pairwise matrices for each tree. Can be “patristic” (sum of branch lengths separating tips, the default) or “nodal” (number of nodes separating tips).k
: Strength of outlier detection. The higher this value the less outliers detected.k2
: Same ask
for complete gene outlier detection. To preserve complete genes from being discarded,k2
can be increased. By default,k2 = k
.Norm
: Should the matrices be normalized prior to the complete analysis and how. If “median”, matrices are divided by their median; if “mean”, they are divided by their mean; if “none”, no normalization if performed. Normalizing ensures that fast-evolving (and slow-evolving) genes are not treated as outliers. Normalization by median is a better choice as it is less sensitive to outlier values.Norm.cutoff
: Value of the median (ifNorm = "median"
) or the mean (ifNorm = "mean"
) below which matrices are simply discarded from the analysis. This prevents dividing by 0, and allows getting rid of genes that contain mostly branches of length 0 and are therefore uninformative anyway. Discarded genes, if any, are listed in the output (out$DiscardedGenes
).gene.names
: List of gene names used to rename elements inX
. If NULL (the default), elements are named 1,2,…,length(X).test.island
: IfTRUE
(the default), only the highest value in an island of outliers is considered an outlier. This prevents non-outliers hitchhiked by outliers to be considered outliers themselves.verbose
: IfTRUE
(the default), messages are written during the filtering process to get information on what is happening.stop.criteria
: The optimization stops when the gain (quality of compromise) between round n and round n+1 is smaller than this value. Default to 1e-5.InitialOnly
: Logical. IfTRUE
, only the Initial state of the data is computed.normalizeby
: Should the gene x species matrix be normalized prior to outlier detection, and how.parallel
: Logical. Should the computations be parallelized when possible? Default toTRUE
. Note that the number of threads cannot be set by the user whenparallel = TRUE
. It uses all available cores on the machine.
4. Analyze the results
To get the list of outliers detected by phylter
, simply type:
results$Final$Outliers
In addition, many functions allow looking at the outliers detected and comparing before and after phyltering.
# Get a summary: nb of outliers, gain in concordance, etc.
summary(results)
# Show the number of species in each gene, and how many per gene are outliers
plot(results, "genes")
# Show the number of genes where each species is found, and how many are outliers
plot(results, "species")
# Compare before and after genes x species matrices, highlighting missing data and outliers
# identified (not efficient for large datasets)
plot2WR(results)
# Plot the dispersion of data before and after outlier removal. One dot represents one
# gene x species association
plotDispersion(results)
# Plot the genes x genes matrix showing pairwise correlation between genes
plotRV(results)
# Plot optimization scores during optimization
plotopti(results)
5. Save the results of the analysis to an external file, for example to perform cleaning on raw alignments or pruning gene trees based on the results from phylter
.
write.phylter(results, file = "phylter.out")
References
Abdi, H., O’Toole, A.J., Valentin, D. & Edelman, B. (2005). DISTATIS: The analysis of multiple distance matrices. Proceedings of the IEEE Computer Society: International Conference on Computer Vision and Pattern Recognition (San Diego, CA, USA). doi: 10.1109/CVPR.2005.445. https://www.utdallas.edu/~herve/abdi-distatis2005.pdf
Allio, R., Tilak, M. K., Scornavacca, C., Avenant, N. L., Kitchener, A. C., Corre, E., … & Delsuc, F. (2021). High-quality carnivoran genomes from roadkill samples enable comparative species delineation in aardwolf and bat-eared fox. Elife, 10, e63167. https://doi.org/10.7554/eLife.63167
Comte, A., Tricou, T., Tannier, E., Joseph, J., Siberchicot, A., Penel, S., Allio, R., Delsuc, F., Dray, S., de Vienne, D.M. (2023). PhylteR: Efficient Identification of Outlier Sequences in Phylogenomic Datasets, Molecular Biology and Evolution, 40(11), msad234, https://doi.org/10.1093/molbev/msad234
Hubert, M. and Vandervieren, E. (2008). An adjusted boxplot for skewed distributions. Computational Statistics and Data Analysis. https://doi.org/10.1016/j.csda.2007.11.008
de Vienne D.M., Ollier S. et Aguileta G. (2012). Phylo-MCOA: A Fast and Efficient Method to Detect Outlier Genes and Species in Phylogenomics Using Multiple Co-inertia Analysis. Molecular Biology and Evolution. https://doi.org/10.1093/molbev/msr317 (This is the ancestor of phylter).
For comments, suggestions and bug reports, please open an issue on this GitHub repository.