Statistically Consistent Rooting of Species Trees Under the Multispecies Coalescent Model

Yasamin Tabatabaee, Sébastien Roch, Tandy Warnow

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Rooted species trees are used in several downstream applications of phylogenetics. Most species tree estimation methods produce unrooted trees and additional methods are then used to root these unrooted trees. Recently, Quintet Rooting (QR) (Tabatabaee et al., ISMB and Bioinformatics 2022), a polynomial-time method for rooting an unrooted species tree given unrooted gene trees under the multispecies coalescent, was introduced. QR, which is based on a proof of identifiability of rooted 5-taxon trees in the presence of incomplete lineage sorting, was shown to have good accuracy, improving over other methods for rooting species trees when incomplete lineage sorting was the only cause of gene tree discordance, except when gene tree estimation error was very high. However, the statistical consistency of QR was left as an open question. Here, we present QR-STAR, a polynomial-time variant of QR that has an additional step for determining the rooted shape of each quintet tree. We prove that QR-STAR is statistically consistent under the multispecies coalescent model, and our simulation study shows that QR-STAR matches or improves on the accuracy of QR. QR-STAR is available in open source form at

Original languageEnglish (US)
Title of host publicationResearch in Computational Molecular Biology - 27th Annual International Conference, RECOMB 2023, Proceedings
EditorsHaixu Tang
Number of pages17
ISBN (Print)9783031291180
StatePublished - 2023
Externally publishedYes
Event27th International Conference on Research in Computational Molecular Biology, RECOMB 2023 - Istanbul, Turkey
Duration: Apr 16 2023Apr 19 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13976 LNBI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference27th International Conference on Research in Computational Molecular Biology, RECOMB 2023


  • Multispecies Coalescent
  • Rooting
  • Species Tree Estimation
  • Statistical Consistency

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'Statistically Consistent Rooting of Species Trees Under the Multispecies Coalescent Model'. Together they form a unique fingerprint.

Cite this