Investigation on factors limiting the performance of deep sleep classication
Mannes Poel2014
Sign up for access to the world's latest research
checkGet notified about relevant papers
checkSave papers to use in your research
checkJoin the discussion with peers
checkTrack your impact
Abstract
The adequacy of the model is evaluated by R-squared . Two-night recordings of the 45 subjects and segmented two-night recordings of 13 subjects, who are selected among the 45 subjects, are utilized for the validation of the second hypothesis. Training and evaluation of the trained classifier is performed using Leave-One-Out Cross-Validation (LOOCV) . The classification performance is evaluated using the area under the Precision-Recall curve, which is a metric more suitable than the area under the ROC curve in the case where class-imbalance problem exists. Partial data and ground-truths of the test subject/cycle are used for the construction of personalized classifier. The performance of the personalized classifier is compared to that of a subject-independent classifier. Chapter 2 gives an overview of the sleep physiology. It also summarizes the past work on unobtrusive sleep staging. The classification framework is introduced in detail as well. Furthermore, the result of an early investigation on deep sleep classification is explained in this chapter. In the end of Chapter 2, the two hypotheses that need to be validated are presented. Chapter 3 describes the methods and procedures of the validation of the two hypotheses. The description of the experiment setup and dataset employed in this research are also introduced. The result of validations and detailed discussion are presented in Chapter 4. Chapter 5 and Chapter 6 describe the conclusions and future work respectively.
Related papers
Sleep classification according to AASM and Rechtschaffen & Kales: effects on sleep scoring parameters
Sleep, 2009
While sleep latency and REM latency, total sleep time, and sleep efficiency were not affected by the classification standard, the time (in minutes and in percent of total sleep time) spent in sleep stage 1 (S1/N1), stage 2 (S2/N2) and slow wave sleep (S3+S4/N3) differed ...
MetaSleepLearner: A Pilot Study on Fast Adaptation of Bio-Signals-Based Sleep Stage Classifier to New Individual Subject Using Meta-Learning
IEEE Journal of Biomedical and Health Informatics, 2021
Identifying bio-signals based-sleep stages requires time-consuming and tedious labor of skilled clinicians. Deep learning approaches have been introduced in order to challenge the automatic sleep stage classification conundrum. However, the difficulties can be posed in replacing the clinicians with the automatic system due to the differences in many aspects found in individual bio-signals, causing the inconsistency in the performance of the model on every incoming individual. Thus, we aim to explore the feasibility of using a novel approach, capable of assisting the clinicians and lessening the workload. We propose the transfer learning framework, entitled MetaSleepLearner, based on Model Agnostic Meta-Learning (MAML), in order to transfer the acquired sleep staging knowledge from a large dataset to new individual subjects. The framework was demonstrated to require the labelling of only a few sleep epochs by the clinicians and allow the remainder to be handled by the system. Layer-wise Relevance Propagation (LRP) was also applied to understand the learning course of our approach. In all acquired datasets, in comparison to the conventional approach, MetaSleepLearner achieved a range of 5.4% to 17.7% improvement with statistical difference in the mean of both approaches. The illustration of the model interpretation after the adaptation to each subject also confirmed that the performance was directed towards reasonable learning. MetaSleepLearner outperformed the conventional approaches as a result from the fine-tuning using the recordings of both healthy subjects and patients. This is the first work that investigated a non-conventional pre-training method, MAML, resulting in a possibility for human-machine collaboration in sleep stage classification and easing the burden of the clinicians in labelling the sleep stages through only several epochs rather than an entire recording.
A Systematic Review of Literature on Automated Sleep Scoring
IEEE Access
Sleep is a period of rest that is essential for functional learning ability, mental health, and even the performance of normal activities. Insomnia, sleep apnea, and restless legs are all examples of sleeprelated issues that are growing more widespread. When appropriately analyzed, the recording of bio-electric signals, such as the Electroencephalogram, can tell how well we sleep. Improved analyses are possible due to recent improvements in machine learning and feature extraction, and they are commonly referred to as automatic sleep analysis to distinguish them from sleep data analysis by a human sleep expert. This study outlines a Systematic Literature Review and the results it provided to assess the present state-of-the-art in automatic analysis of sleep data. A search string was organized according to the PICO (Population, Intervention, Comparison, and Outcome) strategy in order to determine what machine learning and feature extraction approaches are used to generate an Automatic Sleep Scoring System. The American Academy of Sleep Medicine and Rechtschaffen & Kales are the two main scoring standards used in contemporary research, according to the report. Other types of sensors, such as Electrooculography, are employed in addition to Electroencephalography to automatically score sleep. Furthermore, the existing research on parameter tuning for machine learning models that was examined proved to be incomplete. Based on our findings, different sleep scoring standards, as well as numerous feature extraction and machine learning algorithms with parameter tuning, have a high potential for developing a reliable and robust automatic sleep scoring system for supporting physicians. In the context of the sleep scoring problem, there are evident gaps that need to be investigated in terms of automatic feature engineering techniques and parameter tuning in machine learning algorithms.
Learning machines and sleeping brains: Automatic sleep stage classification using decision-tree multi-class support vector machines
Journal of Neuroscience Methods, 2015
Background: Sleep staging is a critical step in a range of electrophysiological signal processing pipelines used in clinical routine as well as in sleep research. Although the results currently achievable with automatic sleep staging methods are promising, there is need for improvement, especially given the time-consuming and tedious nature of visual sleep scoring. New method: Here we propose a sleep staging framework that consists of a multi-class support vector machine (SVM) classification based on a decision tree approach. The performance of the method was evaluated using polysomnographic data from 15 subjects (electroencephalogram (EEG), electrooculogram (EOG) and electromyogram (EMG) recordings). The decision tree, or dendrogram, was obtained using a hierarchical clustering technique and a wide range of time and frequency-domain features were extracted. Feature selection was carried out using forward sequential selection and classification was evaluated using k-fold cross-validation. Results: The dendrogram-based SVM (DSVM) achieved mean specificity, sensitivity and overall accuracy of 0.92, 0.74 and 0.88 respectively, compared to expert visual scoring. Restricting DSVM classification to data where both experts' scoring was consistent (76.73% of the data) led to a mean specificity, sensitivity and overall accuracy of 0.94, 0.82 and 0.92 respectively. Comparison with existing methods: The DSVM framework outperforms classification with more standard multi-class "one-against-all" SVM and linear-discriminant analysis. Conclusion: The promising results of the proposed methodology suggest that it may be a valuable alternative to existing automatic methods and that it could accelerate visual scoring by providing a robust starting hypnogram that can be further fine-tuned by expert inspection. eye movement (REM sleep) and non-rapid eye movement (non-REM sleep). The latter consists of 4 stages (S1, S2, S3 and S4). These distinct sleep stages are associated with distinct physiological and neuronal features which are generally used to identify the sleep stage a person is in. This process called sleep scoring, or sleep staging, is a critical step in a range of electrophysiological signal processing pipelines used in clinical routine as well as in sleep research.
Review of neural network applications in sleep research
Journal of Neuroscience Methods, 1998
To find a better automated sleep-wake staging system for human analyses of numerous polygraphic records is an interesting challenge in sleep research. Over the last few decades, many automated systems have been developed but none are universally applicable. Improvements in computer technology coupled with artificial neural networks based systems (connectionist models) are responsible for new data processing approaches. Despite extensive use of connectionist models in biological data processing, in the past, the field of sleep research appeared to have neglected this approach. Only a few sleep-wake staging systems based on neural network technology have been developed. This paper reviews the current use of artificial neural networks in sleep research. Following a brief presentation of neural network technology, each of the existing system is described and attention drawn to the heterogeneity of the different processing approaches in sleep research. The high performances observed with systems based on neural networks highlight the need to integrate these tools into the field of sleep research.
Investigation of an Automatic Sleep Stage Classification by Means of Multiscorer Hypnogram
Methods of Information in Medicine, 2010
Summary Objectives: Scoring sleep visually based on polysomnography is an important but time-consuming element of sleep medicine. Whereas computer software assists human experts in the assignment of sleep stages to polysomnogram epochs, their performance is usually insufficient. This study evaluates the possibility to fully automatize sleep staging considering the reliability of the sleep stages available from human expert sleep scorers. Methods: We obtain features from EEG, ECG and respiratory signals of polysomnograms from ten healthy subjects. Using the sleep stages provided by three human experts, we evaluate the performance of linear discriminant analysis on the entire polysomnogram and only on epochs where the three experts agree in their sleep stage scoring. Results: We show that in polysomnogram intervals, to which all three scorers assign the same sleep stage, our algorithm achieves 90% accuracy. This high rate of agreement with the human experts is accomplished with only a...
Supervised Classification with Short-Term Memory of Sleep Stages using Cardio-respiratory and Body Movement Variables
2021 IEEE 17th International Conference on Automation Science and Engineering (CASE), 2021
In the context of the Internet of Things (IoT) healthcare, biophysical features collected during sleep needs robust analysis methods to be efficiently used to detect sleep disorders. In this paper, analysis methods using a limited number of input variables (cardiac, respiratory, and body movement) have been used to perform the classification of sleep stages. The efficiency of each classification method has been compared to a reference method that combines a large number of biophysical features referred to as PolySomnoGraphy (PSG). Five classical machine learning methods were evaluated by testing their accuracy on the same collected data. Finally, using a neural network with a short memory method, the classification task fitted 91.34% of the PSG classification. Sleep stages, machine learning, supervised classification, sleep architecture, polysomnography
NEUROPHYSIOLOGICAL AND STATISTICAL ANALYSIS OF FAILURES IN AUTOMATIC SLEEP STAGE CLASSIFICATION
This paper analyses some of the challenges in automatic multiclass sleep stage classification. Six electroencephalographic (EEG) and two electrooculographic (EOG) channels were used in this study. A set of significant features are selected by a minimum-redundancy maximum-relevance (mRMR) criterion and then classified using support vector machine (SVM). The system is tested on 14 subjects suspected of having sleep apnea. The automatic sleep staging showed a 77.70% (±15.8) sensitivity and 95.49% (±2.68) specificity. From the analysis comparing EEG records with visual and automatic classification, we found that the main cause of failures are the similarities between adjacent phases of sleep, in particular in discriminating N1 and N2. Based on the variation of the values of the features it is possible to implement some thresholds and to apply some heuristic rules to improve the performance.
A Novel Approach for Computer Assisted Sleep Scoring Mechanism using ANN
2020
Sleep analysis and its categories in sleep scoring system is considered to be helpful in an area of sleep research and sleep medicine. The scheduled study employs novel approach for computer assisted automated sleep scoring system using physiological signals and Artificial neural network. The data collected were recorded for seven hour, 30 second epoch for each subject. The data procured from the physiological signal was controlled and prepared to expel degenerated signals in order to extract essential data or features used for the study. As, it is known human body distributes its own electrical signals which is needed to be eliminated and these are known as artifacts and they are needed to be filtered out. In this study, signal filtering is achieved by using Butterworth Low-Pass filter. The features extracted were trained and classified using an Artificial Neural Network classifier. Even though, it is a highly complicated concept, using same in biomedical field when engaged with el...
Classification of waking, sleep onset and deep sleep by single measures
The study analyses electrophysiological signals (EEG, EOG, ECG and EMG) to select measures and scoring methods suitable for the detection of sleep stages from waking to deep sleep. 85 measures, selected from relevant spectral characteristics and measures inspired by dynamical systems theory are discussed. Some new characteristics proved to be more sensitive than the conventional scoring measures. Discriminant analysis done with Fisher quadratic classifier determined as the best measures power ratios in delta-alpha, theta-alpha, delta-sigma, delta-beta bands, relative power in delta band, fractal dimension, and coefficient of detrended fluctuation analysis.
Related topics
Related papers
Application of Machine Learning to Sleep Stage Classification
2021 International Conference on Computational Science and Computational Intelligence (CSCI)
Sleep studies are imperative to recapitulate phenotypes associated with sleep loss and uncover mechanisms contributing to psychopathology. Most often, investigators manually classify the polysomnography into vigilance states, which is time-consuming, requires extensive training, and is prone to inter-scorer variability. While many works have successfully developed automated vigilance state classifiers based on multiple EEG channels, we aim to produce an automated and openaccess classifier that can reliably predict vigilance state based on a single cortical electroencephalogram (EEG) from rodents to minimize the disadvantages that accompany tethering small animals via wires to computer programs. Approximately 427 hours of continuously monitored EEG, electromyogram (EMG), and activity were labeled by a domain expert out of 571 hours of total data. Here we evaluate the performance of various machine learning techniques on classifying 10-second epochs into one of three discrete classes: paradoxical, slow-wave, or wake. Our investigations include Decision Trees, Random Forests, Naive Bayes Classifiers, Logistic Regression Classifiers, and Artificial Neural Networks. These methodologies have achieved accuracies ranging from approximately 74% to approximately 96%. Most notably, the Random Forest and the ANN achieved remarkable accuracies of 95.78% and 93.31%, respectively. Here we have shown the potential of various machine learning classifiers to automatically, accurately, and reliably classify vigilance states based on a single EEG reading and a single EMG reading.
Sleep staging classification based on HRV: Time-variant analysis
2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2009
An algorithm to evaluate the sleep macrostructure based on heart rate fluctuations from ECG signal is presented. This algorithm is an attempt to evaluate the sleep quality out of sleep centers. The algorithm is made up by a) a time-variant autoregressive model used as feature extractor and b) a hidden Markov model used as classifier. Characteristics coming from the joint probability of HRV features were used to fed the HMM. 17 full polysomnography recordings from healthy subjects were used in the current analysis. When compared to Wake-NREM-REM given by experts, the automatic classifier achieved a total accuracy of 78.21±6.44% and a kappa index of 0.41±.1085 using two features and a total accuracy of 79.43±8.83% and kappa index of 0.42±.1493 using three features.
Automatic Sleep Stage Classification Using a Neural Network Algorithm
2017
For this project I developed and tested a neural network algorithm for the purpose of performing automatic sleep stage classification. Sleep is typically classified into five different stages: wake, N1, N2, N3/N4, and REM (rapid eye movement). The classification is based on various standards set by the American Academy of Sleep Medicine (AASM) and requires a trained sleep technician. In this project I wrote a neural network algorithm to perform classification based on these standards, thus making the process automatic. The neural network algorithm was developed by improving and building on previous iterations, the final result being a classifier capable of discriminating between five different classes with 80.82% accuracy.
An Approach to the Determination of Differences between Good and Bad Sleepers by Means of an Automatic Sleep Stage Scoring
2006
This paper presents a sleep stage scoring method based on a Hidden Markov Model (HMM) with the goal of obtaining differences between good and bad sleepers according to the Self Rating Questionnaire for Sleep and Awakening Quality (SSA). For the design of the model, we study several parameterization techniques, the model topology and the training strategy for optimum performance. The
Towards exploration and evaluation of sleep staging classification schemes for healthy and patient subjects
2020
INTRODUCTION: Sleep stage classification is an important task for the timely diagnosis of sleep-related disorders, which are one the most common indicator of illness. OBJECTIVE: An automated sleep scoring implementation with promising generalization capabilities is presented, aiding towards eliminating the tedious procedure of manual sleep scoring. METHODS: Two Electroencephalogram (EEG) channels and the Electrooculogram (EOG) channel are utilized as inputs for feature extraction both in the time and frequency domain, while temporal feature changes are utilized in order to capture contextual information of the signals. An ensemble tree-based and a neural network approach are presented at the classification process. RESULTS: A total of 66 subjects belonging to three different groups (healthy, placebo, drug intake) were included in the study. The tree-based classification method outperforms the neural network at all cases. CONCLUSION: State of the art results are achieved, while it is...
Montreal Archive of Sleep Studies: an open-access resource for instrument benchmarking and exploratory research
2014
Manual processing of sleep recordings is extremely time-consuming. Efforts to automate this process have shown promising results, but automatic systems are generally evaluated on private databases, not allowing accurate cross-validation with other systems. In lacking a common benchmark, the relative performances of different systems are not compared easily and advances are compromised. To address this fundamental methodological impediment to sleep study, we propose an open-access database of polysomnographic biosignals. To build this database, whole-night recordings from 200 participants [97 males (aged 42.9 AE 19.8 years) and 103 females (aged 38.3 AE 18.9 years); age range: 18-76 years] were pooled from eight different research protocols performed in three different hospital-based sleep laboratories. All recordings feature a sampling frequency of 256 Hz and an electroencephalography (EEG) montage of 4-20 channels plus standard electro-oculography (EOG), electromyography (EMG), electrocardiography (ECG) and respiratory signals. Access to the database can be obtained through the Montreal Archive of Sleep Studies (MASS) website (), and requires only affiliation with a research institution and prior approval by the applicant's local ethical review board. Providing the research community with access to this free and open sleep database is expected to facilitate the development and cross-validation of sleep analysis automation systems. It is also expected that such a shared resource will be a catalyst for cross-centre collaborations on difficult topics such as improving inter-rater agreement on sleep stage scoring.
Sleep Classification According to AASM and Rechtschaffen & Kales: Effects on Sleep Scoring Parameters
Sleep, 2009
FOR APPROXIMATELY 40 YEARS THE ONLY WIDELY ACCEPTED STANDARD FOR DESCRIBING THE HUMAN SLEEP PROCESS WAS THE MANUAL OF SLEEP CLASSIFICATION by Rechtschaffen and Kales. 1 On the basis of these scoring rules, sleep recordings are divided into 7 discrete stages (wake, stage 1, stage 2, stage 3, stage 4, stage REM, and movement time). Even though in many cases this standard is useful, the rules of Rechtschaffen and Kales have also been criticized for leaving plenty of room for subjective interpretation, which leads to a great variability in the visual evaluation of sleep stages. 2,3 Last but not least, the standard rules were developed for young healthy adults 4,5 and do not necessarily directly apply to elderly subjects and patients. The American Academy of Sleep Medicine (AASM) 6 modified the standard guidelines for sleep classification by Rechtschaffen and Kales and developed a new guideline for terminology, recording method, and scoring rules for sleep-related phenomena. The manual is the result of a review of literature, analysis and consensus which addresses 7 topics: digital analysis and reporting parameters, visual scoring, arousal, cardiac and respiratory events, movements and pediatric scoring. One of the major changes is a change in terminology: in the AASM classification, sleep stages S1 to S4 are referred to as N1, N2, and N3, with N3 reflecting slow wave sleep (SWS, R&K stages S3 + S4); stage REM is referred to as stage R. According to the AASM manual, a minimum of 3 EEG derivations, sampling activity from the frontal, central, and occipital regions, has to be recorded. The recommended derivations are F4-M1, C4-M1, and O2-M1 (right-sided active electrodes and a reference over the left mastoid, rather than the ear). 7 The new manual also deals with the definition of the sleep-wake transition, sleep spindles, K-complexes, slow wave sleep, and REM sleep, as well as arousals and major body movements. In summary, the major changes of the new manual comprise EEG derivations, the merging of stages 3 and 4 into N3, the abolition of stage "movement time," the simplification of many context rules as well as the recommendation of sampling rates and filter settings for polysomnographic (PSG) reporting and for user interfaces of computer-assisted sleep analysis. 6 To date there are no studies evaluating the effects of the new standard on sleep scoring data. The aim of the present investigation was to describe in detail differences between visual sleep scoring according to the Rechtschaffen and Kales classification and scoring based on the new AASM guidelines in normal subjects of different age groups and sleep-disturbed patients.
A comparative review on sleep stage classification methods in patients and healthy individuals
Computer Methods and Programs in Biomedicine, 2017
Background and objective: Proper scoring of sleep stages can give clinical information on diagnosing patients with sleep disorders. Since traditional visual scoring of the entire sleep is highly time-consuming and dependent to experts' experience, automatic schemes based on electroencephalogram (EEG) analysis are broadly developed to solve these problems. This review presents an overview on the most suitable methods in terms of preprocessing, feature extraction, feature selection and classifier adopted to precisely discriminate the sleep stages. Methods: This study round up a wide range of research findings concerning the application of the sleep stage classification. The fundamental qualitative methods along with the state-of-the-art quantitative techniques for sleep stage scoring are comprehensively introduced. Moreover, according to the results of the investigated studies, five research papers are chosen and practically implemented on a well-known public available sleep EEG dataset. They are applied to single-channel EEG of 20 subjects containing equal number of healthy and patient individuals. Feature extraction and classification schemes are assessed in terms of accuracy and robustness against noise. Furthermore, an additional implementation phase is added to this research in which all combinations of the implemented features and classifiers are considered to find the best combination for sleep analysis. Results: According to our achieved results on both groups, entropy of wavelet coefficients along with random forest classifier are chosen as the best feature and classifier, respectively. The mentioned feature and classifier provide 88.66% accuracy on healthy subjects and 66.96% on patient group. Therefore, it can be claimed that continuous wavelet features and random forest provide the best result on this dataset. Conclusions: In this paper, the road map of EEG-base sleep stage scoring methods is clearly sketched. Implementing the stateof-the-art methods and even their combination on both healthy and patient datasets indicates that although the accuracy on healthy subjects are remarkable, the results for the main community (patient group) by the quantitative methods are not promising yet. The reasons rise from adopting non-matched sleep EEG features from other signal processing fields such as communication. As a conclusion, developing sleep pattern-related features deem necessary to enhance the performance of this process.
Fast Convolutional Method for Automatic Sleep Stage Classification
Healthcare Informatics Research, 2018
Sleep is a vital human need. Through sleep, physical and mental fatigue are relieved. Without adequate sleep, the ability to concentrate and participate in daily activities is decreased . Research has shown that lack of sleep causes loss of strength, damages the immune system, and increases blood pressure . Sleep disorders can be observed through examination of the sleep stage pattern. Polysomnography is a tool to analyze the sleep pattern . This test records physical activity when a person asleep. The test is essential as a first step to determine the type of sleep disorder. It combines electromyography (EMG), electrooculography (EOG), electrocardiogram (ECG), electroencephalography (EEG), and so forth. A doctor or health practitioner gives scores based on the data gathered. These scores are the gold standard in sleep stage analysis . There are several techniques of sleep stage scoring, such as those developed by Rechtschaffen and Kales (R&K) and the American Academy of Sleep Medicine (AASM). The sleep stages are divided into two main categories, namely, rapid eye
Neurophysiologic and Statistical Analysis of Failures in Automatic Sleep Stage Classification
Proceedings of the International Conference on Bio-inspired Systems and Signal Processing, 2012
This paper analyses some of the challenges in automatic multiclass sleep stage classification. Six electroencephalographic (EEG) and two electrooculographic (EOG) channels were used in this study. A set of significant features are selected by a minimum-redundancy maximum-relevance (mRMR) criterion and then classified using support vector machine (SVM). The system is tested on 14 subjects suspected of having sleep apnea. The automatic sleep staging showed a 77.70% (±15.8) sensitivity and 95.49% (±2.68) specificity. From the analysis comparing EEG records with visual and automatic classification, we found that the main cause of failures are the similarities between adjacent phases of sleep, in particular in discriminating N1 and N2. Based on the variation of the values of the features it is possible to implement some thresholds and to apply some heuristic rules to improve the performance.
Loading Preview
Sorry, preview is currently unavailable. You can download the paper by clicking the button above.