Evaluating statistical multiple sequence alignment in comparison to other alignment methods on protein data sets

Michael Nute, Ehsan Saleh, Tandy Warnow

Research output: Contribution to journalArticle

Abstract

The estimation of multiple sequence alignments of protein sequences is a basic step in many bioinformatics pipelines, including protein structure prediction, protein family identification, and phylogeny estimation. Statistical coestimation of alignments and trees under stochastic models of sequence evolution has long been considered the most rigorous technique for estimating alignments and trees, but little is known about the accuracy of such methods on biological benchmarks. We report the results of an extensive study evaluating the most popular protein alignment methods as well as the statistical coestimation method BAli-Phy on 1192 protein data sets from established benchmarks as well as on 120 simulated data sets. Our study (which used more than 230 CPU years for the BAli-Phy analyses alone) shows that BAli-Phy has better precision and recall (with respect to the true alignments) than the other alignment methods on the simulated data sets but has consistently lower recall on the biological benchmarks (with respect to the reference alignments) than many of the other methods. In other words, we find that BAli-Phy systematically underaligns when operating on biological sequence data but shows no sign of this on simulated data. There are several potential causes for this change in performance, including model misspecification, errors in the reference alignments, and conflicts between structural alignment and evolutionary alignments, and future research is needed to determine the most likely explanation. We conclude with a discussion of the potential ramifications for each of these possibilities.

Original languageEnglish (US)
Pages (from-to)396-411
Number of pages16
JournalSystematic biology
Volume68
Issue number3
DOIs
StatePublished - May 1 2019

Fingerprint

Sequence Alignment
sequence alignment
Benchmarking
Indonesia
protein
Proteins
proteins
methodology
protein structure
Phylogeny
Computational Biology
bioinformatics
statistical analysis
amino acid sequences
Datasets
comparison
alignment
method
prediction
phylogeny

Keywords

  • BAli-Phy
  • Homology
  • Multiple sequence alignment
  • Protein sequences
  • Structural alignment

ASJC Scopus subject areas

  • Ecology, Evolution, Behavior and Systematics
  • Genetics

Cite this

Evaluating statistical multiple sequence alignment in comparison to other alignment methods on protein data sets. / Nute, Michael; Saleh, Ehsan; Warnow, Tandy.

In: Systematic biology, Vol. 68, No. 3, 01.05.2019, p. 396-411.

Research output: Contribution to journalArticle

@article{7d1b5807d0844237b1f2fd9c0a9cf341,
title = "Evaluating statistical multiple sequence alignment in comparison to other alignment methods on protein data sets",
abstract = "The estimation of multiple sequence alignments of protein sequences is a basic step in many bioinformatics pipelines, including protein structure prediction, protein family identification, and phylogeny estimation. Statistical coestimation of alignments and trees under stochastic models of sequence evolution has long been considered the most rigorous technique for estimating alignments and trees, but little is known about the accuracy of such methods on biological benchmarks. We report the results of an extensive study evaluating the most popular protein alignment methods as well as the statistical coestimation method BAli-Phy on 1192 protein data sets from established benchmarks as well as on 120 simulated data sets. Our study (which used more than 230 CPU years for the BAli-Phy analyses alone) shows that BAli-Phy has better precision and recall (with respect to the true alignments) than the other alignment methods on the simulated data sets but has consistently lower recall on the biological benchmarks (with respect to the reference alignments) than many of the other methods. In other words, we find that BAli-Phy systematically underaligns when operating on biological sequence data but shows no sign of this on simulated data. There are several potential causes for this change in performance, including model misspecification, errors in the reference alignments, and conflicts between structural alignment and evolutionary alignments, and future research is needed to determine the most likely explanation. We conclude with a discussion of the potential ramifications for each of these possibilities.",
keywords = "BAli-Phy, Homology, Multiple sequence alignment, Protein sequences, Structural alignment",
author = "Michael Nute and Ehsan Saleh and Tandy Warnow",
year = "2019",
month = "5",
day = "1",
doi = "10.1093/sysbio/syy068",
language = "English (US)",
volume = "68",
pages = "396--411",
journal = "Systematic Biology",
issn = "1063-5157",
publisher = "Oxford University Press",
number = "3",

}

TY - JOUR

T1 - Evaluating statistical multiple sequence alignment in comparison to other alignment methods on protein data sets

AU - Nute, Michael

AU - Saleh, Ehsan

AU - Warnow, Tandy

PY - 2019/5/1

Y1 - 2019/5/1

N2 - The estimation of multiple sequence alignments of protein sequences is a basic step in many bioinformatics pipelines, including protein structure prediction, protein family identification, and phylogeny estimation. Statistical coestimation of alignments and trees under stochastic models of sequence evolution has long been considered the most rigorous technique for estimating alignments and trees, but little is known about the accuracy of such methods on biological benchmarks. We report the results of an extensive study evaluating the most popular protein alignment methods as well as the statistical coestimation method BAli-Phy on 1192 protein data sets from established benchmarks as well as on 120 simulated data sets. Our study (which used more than 230 CPU years for the BAli-Phy analyses alone) shows that BAli-Phy has better precision and recall (with respect to the true alignments) than the other alignment methods on the simulated data sets but has consistently lower recall on the biological benchmarks (with respect to the reference alignments) than many of the other methods. In other words, we find that BAli-Phy systematically underaligns when operating on biological sequence data but shows no sign of this on simulated data. There are several potential causes for this change in performance, including model misspecification, errors in the reference alignments, and conflicts between structural alignment and evolutionary alignments, and future research is needed to determine the most likely explanation. We conclude with a discussion of the potential ramifications for each of these possibilities.

AB - The estimation of multiple sequence alignments of protein sequences is a basic step in many bioinformatics pipelines, including protein structure prediction, protein family identification, and phylogeny estimation. Statistical coestimation of alignments and trees under stochastic models of sequence evolution has long been considered the most rigorous technique for estimating alignments and trees, but little is known about the accuracy of such methods on biological benchmarks. We report the results of an extensive study evaluating the most popular protein alignment methods as well as the statistical coestimation method BAli-Phy on 1192 protein data sets from established benchmarks as well as on 120 simulated data sets. Our study (which used more than 230 CPU years for the BAli-Phy analyses alone) shows that BAli-Phy has better precision and recall (with respect to the true alignments) than the other alignment methods on the simulated data sets but has consistently lower recall on the biological benchmarks (with respect to the reference alignments) than many of the other methods. In other words, we find that BAli-Phy systematically underaligns when operating on biological sequence data but shows no sign of this on simulated data. There are several potential causes for this change in performance, including model misspecification, errors in the reference alignments, and conflicts between structural alignment and evolutionary alignments, and future research is needed to determine the most likely explanation. We conclude with a discussion of the potential ramifications for each of these possibilities.

KW - BAli-Phy

KW - Homology

KW - Multiple sequence alignment

KW - Protein sequences

KW - Structural alignment

UR - http://www.scopus.com/inward/record.url?scp=85065024199&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85065024199&partnerID=8YFLogxK

U2 - 10.1093/sysbio/syy068

DO - 10.1093/sysbio/syy068

M3 - Article

VL - 68

SP - 396

EP - 411

JO - Systematic Biology

JF - Systematic Biology

SN - 1063-5157

IS - 3

ER -