Document Type
Article
Version Deposited
Published Version
Publication Date
9-22-2023
Publication Title
Briefings in Bioinformatics
DOI
https://doi.org/10.1093/bib/bbad335
Abstract
Single-cell RNA sequencing (scRNA-seq) is a widely used technique for characterizing individual cells and studying gene expression at the single-cell level. Clustering plays a vital role in grouping similar cells together for various downstream analyses. However, the high sparsity and dimensionality of large scRNA-seq data pose challenges to clustering performance. Although several deep learning-based clustering algorithms have been proposed, most existing clustering methods have limitations in capturing the precise distribution types of the data or fully utilizing the relationships between cells, leaving a considerable scope for improving the clustering performance, particularly in detecting rare cell populations from large scRNA-seq data. We introduce DeepScena, a novel single-cell hierarchical clustering tool that fully incorporates nonlinear dimension reduction, negative binomial-based convolutional autoencoder for data fitting, and a self-supervision model for cell similarity enhancement. In comprehensive evaluation using multiple large-scale scRNA-seq datasets, DeepScena consistently outperformed seven popular clustering tools in terms of accuracy. Notably, DeepScena exhibits high proficiency in identifying rare cell populations within large datasets that contain large numbers of clusters. When applied to scRNA-seq data of multiple myeloma cells, DeepScena successfully identified not only previously labeled large cell types but also subpopulations in CD14 monocytes, T cells and natural killer cells, respectively.
Recommended Citation
Tianyuan Lei, Ruoyu Chen, Shaoqiang Zhang, & Yong Chen. Self-supervised deep clustering of single-cell RNA-seq data to hierarchically detect rare cell populations. Briefings in Bioinformatics, Volume 24, Issue 6, November 2023, bbad335. https://doi.org/10.1093/bib/bbad335
Comments
© The Author(s) 2023. Published by Oxford University Press. This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (https://creativecommons.org/licenses/by-nc/4.0/).
Open access publication was supported by the 2023-2024 Rowan University Libraries Open Access Publishing Fund.