Histopathology是医学诊断的重要领域之一，涉及对组织疾病进行研究并分析显微镜下的组织切片。随着深度学习技术的发展，在病理图像分割方面取得了显著进展。然而，处理全尺寸幻灯片图像（Whole Slide Images, WSI）时面临计算瓶颈问题：文件过大导致存储、处理和计算效率低下。针对这一挑战，提出了多种解决方案：如稀疏编码方法用于提取特征；结合LSTM和条件随机场模型捕捉邻域补丁间的上下文信息；使用级联CNN和基于图神经网络的方法进行区域选择；以及通过无监督学习等技术提升模型性能并缓解数据不足的问题。尽管如此，在数据标注和隐私保护方面仍存在诸多挑战。未来研究需进一步探索高效算法及增强的数据增强技术以应对这些难题，并推动 Histopathology 的自动化诊断发展。

该课程提供深度学习在医学图像分割领域的深入研究与应用

Computational Imaging (CV/DL/MI)

The field of histopathology focuses on diagnosing diseases by examining microscopic tissue samples. These samples consist primarily of tissues and cells obtained through specialized slide preparation techniques. Over time, this diagnostic method has proven highly effective in identifying various types of cancer. Hist pathologists are medical specialists who utilize microscopes to analyze cell structures with precision in order to establish consensus on disease characteristics, severity levels, and appropriate treatment plans for patients. The introduction of advanced technologies like specialized scanning machines coupled with enhanced storage solutions has revolutionized how tissue samples are managed. Now, microscopic glass slides can be easily converted into digital formats for efficient processing. This digital transformation has enabled remote diagnostics, accelerated analysis workflows, and improved safety protocols for storing pathological data. Recent developments such as the global pandemic have underscored the critical role automation plays in streamlining medical procedures like diagnostics. By automating these processes, healthcare providers can enhance accuracy and operational efficiency while minimizing repetitive tasks. Additionally, automation reduces exposure risks for clinicians handling patient cases on the front lines.

组织病理学是对组织疾病的研究，涉及检查由组织，细胞等组成的微观载玻片，该载玻片已广泛用于诊断各种形式的癌症。 组织病理学家是医学专家，他们在显微镜下分析细胞或组织以做出诊断，以便就自然，疾病的严重程度以及有关患者护理的行动计划达成共识。 随着诸如专用扫描机之类的先进设备的出现，存储/云功能的飞跃发展，现在已经非常容易以数字载玻片的形式将显微载玻片存储在计算机上进行处理。 它实现了远程诊断，更快的分析以及病理信息的系统和安全存储。 最近的事件(例如全球大流行)向我们表明了自动化医疗活动(如诊断程序)的重要性。 它不仅有助于提高准确性和容量，消除大量冗余，而且可以确保减少对一线医生和诊断人员的影响。

owing to the increasing applicability, scalability and success of AI and machine learning along with their multidisciplinary nature, these technologies are becoming more integrated into various fields. Medical Science is no different. A wide array of procedures spanning from automated diagnosis through surgical interventions to drug discovery are currently utilizing machine learning techniques with notable success. The application of machine learning in medical imaging for diagnostic purposes alongside classical computer vision has emerged as a rapidly evolving research domain. Deep learning methods, particularly convolutional neural networks (CNNs), have established themselves as the preferred approach in digital histopathology. This blog provides a concise overview of recent advancements in deep learning applications within histopathology. It delves into current research trends best practices and tackles common challenges faced by researchers in this specialized field. The content is organized based on distinct deep learning methodologies with a focus on popular state-of-the-art papers that have significantly influenced this area of study.

基于人工智能以及机器学习技术的广泛适用性, 该方法展现出良好的可扩展性和持续增长的趋势, 并因其多学科属性而不断成熟, 现如今已被广泛应用于多个领域, 医疗领域同样也不例外, 从自动化诊断到外科手术再到药物发现的大规模程序均能利用机器学习取得显著成果. 结合机器学习与传统计算机视觉的方法已在医学图像诊断领域取得了显著进展. 在数字组织病理学中, 深度学习技术尤其是卷积神经网络（CNN）已成为主流方法. 本文旨在总结当前研究进展最佳实践及在组织病理学领域的深度学习挑战.

挑战性 (Challenges)

In contrast to the typical image datasets employed in conventional multiview computer vision techniques, addressing whole slide digital pathology images presents a particularly formidable array of challenges.

与常规图像数据相比，在处理整张幻灯片数字病理图像时，不仅面临着一系列独特的挑战。

The primary hurdle is the vast size of Whole slide images. Due to their immense size, these slides often create computational challenges across storage, processing efficiency, and compatibility with deep learning frameworks. Many studies in deep learning address this issue by employing patch extraction or tiling methods. Their approach typically involves extracting smaller patches using a sliding window technique, which are subsequently processed further. An alternative method involves downsampling whole-slide images to manageable sizes; however, this process often results in information loss that impacts the performance of deep learning systems.

在整体上，幻灯片图像的尺寸是一个关键问题。 这些文件通常占据数千兆字节空间，并在存储过程、处理效率以及与深度学习算法兼容性方面都面临挑战。 多数研究文献倾向于采用分块处理技术来应对这一挑战。 常用的方法包括基于滑动窗口的技术，在此框架下生成较小尺寸的区域片段。 另一种常见的解决方案是通过降采样技术将原始图像转换为更适合处理的小尺寸版本。

Researchers have become aware of severe shortcomings in the patch extraction approach as well.

研究人员也开始注意到补丁提取方法中存在的缺陷。

为了确保重要信息不丢失, 足够的注意力需要被给予, 以确保各片段之间的最大重叠. 这一目标可以通过谨慎地调整步幅来实现.

为了确保重要信息不发生丢失, 必须特别关注以便于精细调节间距以实现补丁之间的最大重叠

为了实现对whole slide image水平的所有分类/预测任务, 必须遍历整个slide. 由于需要处理大量数量的patches, 导致训练与预测过程既繁琐又消耗大量计算资源.

在整体幻灯片图像层面上进行任何分类/预测任务时，必须对整个幻灯片区域进行扫描分析，并对每个小块（即补丁）进行详细处理。这种操作会导致训练与预测过程变得复杂费时，并消耗大量计算资源。

When processing patches, each instance handles only a localized area, thereby inevitably resulting in the loss of significant amounts of global contextual information, which this limitation directly impacts the system's maximum achievable performance.

在处理补丁的过程中, 每次仅对一个小区域进行分析, 这会导致在分析全局信息时出现不足, 从而降低了整体性能水平.

Addressing the shortcomings of the patch extraction method in whole slide digital pathology image analysis through deep learning, this challenge has increasingly become a focal point in its own research domain. Over time, numerous solutions have been developed and rigorously tested to address these issues.

针对基于深度学习技术在整张幻灯片数字病理图像分析工作中的局限性，
已成为相关研究的热点问题。
已有多种方案且已开发出多种解决方案。

Among notable effective strategies suggested, the approach involves employing sparse coding techniques on pathology slide images to extract feature representations and infer histological slide representations for cancer diagnosis.

基于关键的可实施方案中的一项重要创新措施, 本研究通过病理幻灯片结合稀疏编码算法（如稀疏编码）来提取和分析特征信息, 并识别其表征模式。

The integration of LSTMs and Conditional Random Fields has become increasingly popular with the aim of enhancing their application in conjunction with Convolutional Neural Networks (CNNs). This synergy between CRFs and LSTMs has become increasingly prominent, primarily because they work together to model the intricate relationships between neighboring image patches. Their combined use enables the extraction of more comprehensive contextual information across extended regions within an image.

为了更好地捕捉更广泛的情境信息，在深度学习领域中越来越多人采用LSTM与条件随机场的协同作用机制来处理序列数据，并通过分析邻近块之间的关联性来提升模型性能。

These methods are currently being employed to establish a region selection mechanism. Such that only the most relevant diagnostic regions from the entire slide image are utilized for prediction and training.

分层卷积神经网络（CNN）以及依赖视觉注意力机制的方法被用来执行区域选择过程；仅仅针对幻灯片图像中最具相关性和诊断价值的区域进行预测与训练。

An innovative approach, referred to as [slide graph], introduced in a recent CVPR’20 paper, efficiently tackles the challenge of scale by employing a graph convolutional neural network architecture.

最近一项研究在CVPR'20会议上取得了突破性进展，在该领域首次系统性地提出了一个全新的技术框架——幻灯片图（PlaNet）¹

Figure: Workflow of proposed Slide Graph for graph classification. Four steps are essential for the workflow of the suggested Slide Graph method in graph classification tasks. First, it includes nucleus partitioning and categorization to achieve accurate node assignment. Second, spatially grouping similar nodes through spatial clustering is a crucial step. Finally, the method constructs a graph structure and integrates GCN model to perform effective feature extraction and learning. 图：为图表分类建议的幻灯片图的工作流程。该方法在图分类任务中包含四个关键步骤：核区域划分及其分类；基于空间的聚类分析；图结构构建及GCN模型应用。

The fundamental concept of this study involves converting the entirety of a slide image into a graph structure, subsequently employing Graph neural networks for analysis.

该研究的核心理念在于将幻灯片图像构建为图形结构，并通过引入图形神经网络来实现其分析与处理。

This method operates on identifying and categorizing individual nuclei through a neural network model, specifically utilizing HoVer-Net², prior to employing spatial grouping to cluster nuclei sharing common characteristics. Each tissue cluster is represented as a node within a graph structure, where edges symbolize potential signaling pathways between these clusters. Memory-efficient data structures highlight that transforming Whole Slide Images into graph representations offers both computational benefits and comprehensive preservation of contextual information across large-scale regions. The resulting graph structures provide a foundation for training Graph Learning algorithms, including Graph Convolutional Networks (GCNs), enabling effective predictions and classifications.

该作品基于细胞定位技术与个体细胞核分级方法即悬停净算法结合并采用空间聚类对组织样本进行分组处理从而实现细胞核区域的精确分割使组织样本在图像中所占的比例共同决定其特性特征. 图表作为数据处理的核心形式其图表示不仅具有高度存储效率的优势还能有效捕捉组织间潜在的信号传导机制. 将显微图像转换为图表示不仅是一种计算高效的方法而且能够保留图像处理中的大规模上下文信息以及各区域间的关联性. 这种图表示方法为后续训练各种图神经网络模型如GCN提供了可靠的基础支持从而实现预测与分类任务.

除了规模之外，另一个重大挑战是数据的可获得性。由于病理学数据具有高度敏感性，并因此受到严格的隐私政策约束以确保适当的保密性和保护措施得以实现，在这种情况下公开源的数据非常有限。即使在数据可用的情况下，“注释”也是一个大问题，因为它需要高水平的专业知识，并因此主要由病理学家和医疗专业人员完成注释工作。“注释”是一项耗时的工作流程以确保最大精度。“错误”的注释可能会导致我们学习算法产生误导结果，在医学诊断等敏感领域这是一项非常危险的可能性。因此我们经常面临只有少量高质量注释的数据或仅在幻灯片级别进行注释的情况。“数据增强”技术如引入随机仿射变换（如旋转、缩放等）、引入噪声/模糊、颜色值变化等方法被广泛使用以解决现有数据不足的问题。“数据增强”还帮助向学习范式引入必要的变异性，“使学习算法能够得到推广。”

除了数量之外，另一个主要障碍是数据的可获得性。由于病理数据的高度敏感性以及相关严格的机密性和隐私保护政策的实施，在公开领域获取此类数据仍然存在诸多限制。即便能够获取到一定量的数据，在注释方面也会面临巨大挑战：这一过程通常需要专业的医学知识和大量的人工标注时间来确保最高的准确性水平。注释不当可能导致我们用于训练算法的数据产生误导结果，在医学诊断领域这是一项高度危险的操作。因此，在满足所有伦理和法律要求的前提下，我们往往只能依赖有限数量的带有高质量标注（通常达到幻灯片级别）的数据集以及较弱形式的数据注释（如仅标记类别信息而缺乏具体细节）。为了缓解数据不足的问题，在此领域研究中已经开发并广泛采用了一系列增强技术：例如通过应用随机仿射变换（包括旋转、缩放等操作），向图像中添加噪声或模糊效果、调整颜色值范围等手段来扩展数据集容量；这些方法不仅能够引入必要的变异性以改善学习算法的泛化能力，并且有助于提高模型性能表现

Methodologies such as unsupervised learning aid in extracting meaningful features and data structures from datasets where labeled data is unavailable. The derived structures are then utilized in downstream applications or self-supervised learning frameworks. Additionally, another category of techniques known as weakly supervised learning enables effective prediction when only image-level annotations are present but pixel-level accuracy is desired.

在缺乏标注数据的情况下, 无监督及半监督等技术手段能够从原始数据中提取出具有实用价值的模式和表征形式；这些表征不仅适用于后续的任务推断, 同时也可以作为自我监督的学习范式参考[1]。此外还存在一类被称为弱监督学习的方法, 其应用场景是在标注信息较为粗放（如图像级）而预测需求却十分精细（如像素级）的情况。

Moving forward, in the next section, we will delve into various deep learning techniques utilized in histopathology slide analysis for cancer detection purposes. This segment will explore the current state of research, examine the diverse tasks these methods assist with, and highlight notable related studies.

在下一部分中

详细说明

PS: I connected technically important terms to corresponding resources, offering explanations for each term.

PS: I connected technically important terms to corresponding resources, offering explanations for each term.

PS：我已将技术上重要的术语链接到解释它们的相应资源。

翻译自: https://medium.com/swlh/deep-learning-in-histopathology-c104478c00cd

深度学习病理图像分割