Chinese clinical named entity recognition with variant neural structures based on BERT methods

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Medical Informatics

Background:

• Clinical Named Entity Recognition (CNER) is crucial for processing electronic medical records.
• Deep learning models show promise but require extensive labeled data, which is scarce for Chinese clinical records.
• Existing methods face challenges in accurately identifying and classifying clinical terms in Chinese medical texts.

Purpose of the Study:

• To develop an effective Chinese CNER model addressing data scarcity.
• To leverage unlabeled Chinese clinical records for pre-training a BERT model.
• To enhance CNER performance by incorporating dictionary and radical features.

Main Methods:

• Pre-training a BERT model on unlabeled Chinese clinical records.
• Utilizing Long Short-Term Memory (LSTM) and Conditional Random Field (CRF) layers for feature extraction and tag decoding.
• Integrating dictionary features and radical features of Chinese characters into the ensemble model.

Main Results:

• The ensemble model achieved a strict F1 score of 89.56% on the CCKS-2018 dataset.
• The model attained an F1 score of 91.60% on the CCKS-2017 dataset.
• The proposed methods, including pre-training and feature integration, outperformed existing state-of-the-art models.

Conclusions:

• Pre-training BERT on domain-specific unlabeled data is effective for Chinese CNER.
• Incorporating dictionary and radical features significantly improves model performance.
• The developed ensemble model represents a state-of-the-art solution for Chinese CNER tasks.