Recognition of incomplete sequences described by hidden Markov models using first derivatives of likelihood function logarithm | Vestnik Tomskogo gosudarstvennogo universiteta. Upravlenie, vychislitelnaja tehnika i informatika – Tomsk State University Journal of Control and Computer Science. 2018. № 42. DOI: 10.17223/19988605/42/9

Recognition of incomplete sequences described by hidden Markov models using first derivatives of likelihood function logarithm

Hidden Markov model (HMM) conception was presented yet in 1970-s, however problems which concern using HMMs in case of incomplete data remain poorly investigated. These problems are quite relevant since in complex systems, e.g. when receiving signals from spacecrafts or aircrafts, one has to deal with datastreams of various sources in noisy environments when there is a high possibility of data loss or corruption. In this paper, we deal with the problem of missing observations in sequences. From now on we will refer to such sequences as incomplete. We consider a case when such missing observations are not generated by random process itself but rather occur randomly in sequences because of some external interference. We propose a method for recognition of incomplete sequences which is based on classification of incomplete sequences using first derivatives of likelihood function logarithm with respect to various HMM parameters. We use a support vector machine classifier for that purpose. The likelihood in that case is the probability of incomplete sequence being generated by a HMM. The proposed method was compared to a previously developed method for recognition based on marginalization of missing observations. The proposed method proved to be more effective than the other method in situation when the number of missing observations in training and testing sequences is high (more than 20% in our particular experiment). Thus, we propose to prefer the usage of the proposed method in situations when there is big loss of data but the recognition is still had to be done.

Download file

Counter downloads: 195

Keywords

скрытые марковские модели, машинное обучение, последовательности, пропущенные наблюдения, неполные данные, hidden Markov models, machine learning, sequences, missing observations, incomplete data

Authors

Name	Organization	E-mail
Uvarov Vadim E.	Novosibirsk State Technical University	uvarov.vadim42@gmail.com

Всего: 1

References

Baum L.E., Petrie T. Statistical inference for probabilistic functions of finite state Markov chains // The Annals of Mathematical Statistics. 1966. V. 37. P. 1554-1563.

Упоминания ключевого слова «hidden Markov models» между 1800 и 2008 годами : данные из Google Ngram Viewer. URL: http ://tinyurl.com/ gmq5 snv

Gultyaeva T.A., Popov A.A., Kokoreva V.V., Uvarov V.E. Classification of observation sequences described by Hidden Markov Models // Proc. of the Int. Workshop Applied Methods of Statistical Analysis Nonparametric approach AMSA-2015. Novosibirsk, Belokuriha, 14-19 Sep. 2015. P. 136-143.

Rabiner L.R. A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition // Proc. of the IEEE. 1989. V. 77. P. 257-285.

Baum L.E., Egon J.A. An inequality with applications to statistical estimation for probabilistic functions of a Markov process and to a model for ecology // Bulletin of the American Meteorological Society. 1967. V. 73. P. 360-363.

Попов А.А., Гультяева Т.А., Уваров В.Е. Исследование подходов к обучению скрытых марковских моделей при наличии пропусков в последовательностях // Обработка информации и математическое моделирование : материалы Рос. науч.-техн. конф. Новосибирск, 21-22 апр. 2016. С. 125-139.

Popov A., Gultyaeva T., Uvarov V. A comparison of some methods for training hidden Markov models on sequences with missing observations // Proc. of 11th Int. Forum on Strategic Technology IF0ST-2016. 2016. V. 1. P. 431-435.

Попов А.А., Гультяева Т.А., Уваров В.Е. Исследование методов обучения скрытых марковских моделей при наличии пропусков в последовательностях // Актуальные проблемы электронного приборостроения (АПЭП-2016) : труды XIII международной конференции : в 12 т. Новосибирск, 2016. Т. 8: Моделирование и вычислительная техника. Информационные системы и технологии. С. 149-152.

Cooke M., Green P., Josifovski L., Vizinh A. Robust automatic speech recognition with missing and unreliable acoustic data // Speech Communication. 2001. V. 34, No. 3. P. 267-285.

Boser B.E.; Guyon I.M., Vapnik V.N. A training algorithm for optimal margin classifiers // Proc. of the fifth annual workshop on Computational learning theory - COLT '92. 1992. P. 144.