Volume 5, Issue 1, January 2016, Page: 13-22
Kernel-Type Estimators of Divergence Measures and Its Strong Uniform Consistency
Hamza Dhaker, Departement de Mathématiques et Informatique, Faculté des Sciences et Technique, Université Cheikh Anta Diop, Dakar, Sénégal
Papa Ngom, Departement de Mathématiques et Informatique, Faculté des Sciences et Technique, Université Cheikh Anta Diop, Dakar, Sénégal
El Hadji Deme, Sciences Appliquées et Technologie, Unité de Formation et de Recherche, Université Gaston Berger, Saint-Louis, Sénégal
Pierre Mendy, Département de Techniques Quantitatives, Faculté des Sciences Economiques et de Gestion, Université Cheikh Anta Diop , Dakar, Sénégal
Received: Jan. 8, 2016;       Accepted: Jan. 23, 2016;       Published: Feb. 16, 2016
DOI: 10.11648/j.ajtas.20160501.13      View  4139      Downloads  110
Nonparametric density estimation, based on kernel-type estimators, is a very popular method in statistical research, especially when we want to model the probabilistic or stochastic structure of a data set. In this paper, we investigate the asymptotic confidence bands for the distribution with kernel-estimators for some types of divergence measures (Rényi-α and Tsallis-α divergence). Our aim is to use the method based on empirical process techniques, in order to derive some asymptotic results. Under different assumptions, we establish a variety of fundamental and theoretical properties, such as the strong consistency of an uniform-in-bandwidth of the divergence estimators. We further apply the previous results in simulated examples, including the kernel-type estimator for Hellinger, Bhattacharyya and Kullback-Leibler divergence, to illustrate this approach, and we show that that the method performs competitively.
Divergence Measures, Kernel Estimation, Strong Uniform, Consistency
To cite this article
Hamza Dhaker, Papa Ngom, El Hadji Deme, Pierre Mendy, Kernel-Type Estimators of Divergence Measures and Its Strong Uniform Consistency, American Journal of Theoretical and Applied Statistics. Vol. 5, No. 1, 2016, pp. 13-22. doi: 10.11648/j.ajtas.20160501.13
Copyright © 2016 Authors retain the copyright of this article.
This article is an open access article distributed under the Creative Commons Attribution License ( which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Bosq, D. and Lecoutre, J. P. (1987). Théorie de l’estimation fonctionnelle. Économie et Statistiques Avancées. Economica, Paris.
Bouzebda, S. and Elhattab, I. (2011) Uniform-in-bandwidth consistency for kernel-type estimators of Shannon’s entropy. Electronic Journal of Statistics. 5, 440-459.
Csiszár, I. (1967). Information-type measures of differences of probability distributions and indirect observations. Studia Sci. Math. Hungarica, 2: 299-318.
Deheuvels, P. (2000). Uniform limit laws for kernel density estimators on possibly unbounded intervals. In Recent advances in reliability theory (Bordeaux, 2000), Stat. Ind. Technol., pages 477-492. BirkhaBoston.
Deheuvels, P. and Mason, D. M. (2004). General asymptotic confidence bands based on kernel-type function estimators. Stat. Inference Stoch. Process., 7(3), 225-277.
Deroye, L. and Gyorfi, L. (1985). Nonparametric density estimation. Wiley Series in Probability and Mathematical Statistics: Tracts on Probability and Statistics. John Wiley & Sons Inc., New York. The L1 view.
Devroye, L. and Lugosi, G. (2001). Combinatorial methods in density estimation. Springer Series in Statistics. Springer-Verlag, New York.
Devroye, L. and Wise, G. L. (1980). Detection of abnormal behavior via nonparametric estimation of the support. SIAM J. Appl. Math., 38(3), 480-488.
Dmitriev, J. G. and Tarasenko, F. P. (1973). The estimation of functionals of a probability density and its derivatives. Teor. Verojatnost. i Primenen., 18, 662-668.
Einmahl, U. and Mason, D. M. (2000). An empirical process approach to the uniform consistency of kernel-type function estimators. J. Theoret. Probab., 13 (1), 1-37.
Einmahl, U. and Mason, D. M. (2005). Uniform in bandwidth consistency of kernel-type function estimators. Ann. Statist., 33(3), 1380-1403.
Giné, E. and Guillou, A. (2002). Rates of strong uniform consistency for multivariate kernel density estimators. Ann. Inst. H. Poincaré Probab. Statist. 38 907-921.
Giné, E. and Zinn, J. (1984). Some limit theorems for empirical processes (with discussion). Ann. Probab. 12 929-998.
Johnson, D. H., Gruner, B., C. M. K., and Seshagiri.(2001) Information-theoretic analysis of neural coding. Journal of Computational Neuroscience.
Krishnamurthy A., Kandasamy K., Póczos B., and Wasserman L., (2014). Nonparametric Estimation of Rényi Divergence and Friends.
Ngom, P., Dhaker, H., Mendy, P., Deme,. E. Generalized divergence criteria for model selection between random walk and AR(1) model.
Nolan, D. and Pollard, D. (1987): U-processes: rates of convergence. Ann. Statist., 15(2): 780–799.
Pakes, A. and Pollard, D.(1989): Simulation and the asymptotics of optimization estimators. Econometrica, 57(5): 1027–1057, 1989.
Parzen, E. (1962). On estimation of a probability density function and mode. Ann. Math. Statist., 33, 1065-1076.
Pardo, L.(2005) Statistical inference based on divergence measures. CRC Press.
Pluim B M, Safran M. From breakpoint to advantage. description, treatment, and prevention of all tennis injuries. Vista: USRSA, 2004.
Prakasa Rao, B. L. S. (1983). Nonparametric functional estimation. Probability and Mathematical Statistics. Academic Press Inc. [Harcourt Brace Jovanovich Publishers], New York.
Póczos, B. and Schneider, J. On the estimation of alpha-divergences. CMU, Auton Lab Technical Report,
Póczos, B. Xiong L., Sutherland D, J., and Schneider J. (2012). Nonparametric kernel estimators for image classification. In IEEE Conference on Computer Vision and Pattern Recognition.
Rényi, A. (1961). On measures of entropy and information. In Fourth Berkeley Symposium on Mathematical Statistics and Probability.
Rényi, A. (1970). Probability Theory. Publishing Company, Amsterdam.
Rosenblatt, M. (1956). Remarks on some nonparametric estimates of a density function. Ann. Math. Statist., 27, 832-837.
Van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York.
Viallon, V. (2006). Processus empiriques, estimation non param´etrique et données censurées. Ph. D. thesis, Université Paris 6.
Villmann, T. and Haase, S. (2010). Mathematical aspects of divergence based vector quantization using Frechet-derivatives. University of Applied SciencesMittweida.
Browse journals by subject