Support vector M-quantile regression

2016

M-quantile regression generalizes both quantile and expectile regression using M-estimation ideas. This paper covers several topics related to estimation, model assessment and hypothesis testing that were so far neglected in the many articles about M-quantile regression methods that appeared in recent years.

Journal of Statistical Planning and Inference, 2012

We consider for quantile regression and support vector regression a kernel-based online learning algorithm associated with a sequence of insensitive pinball loss functions. Our error analysis and derived learning rates show quantitatively that the statistical performance of the learning algorithm may vary with the quantile parameter t. In our analysis we overcome the technical difficulty caused by the varying insensitive parameter introduced with a motivation of sparsity.

Stat, 2017

It is well known that the widely popular mean regression model could be inadequate if the probability distribution of the observed responses do not follow a symmetric distribution. To deal with this situation, the quantile regression turns to be a more robust alternative for accommodating outliers and the misspecification of the error distribution since it characterizes the entire conditional distribution of the outcome variable. This paper presents a likelihoodbased approach for the estimation of the regression quantiles based on a new family of skewed distributions introduced by Wichitaksorn et al. (2014). This family includes the skewed version of Normal, Student-t, Laplace, contaminated Normal and slash distribution, all with the zero quantile property for the error term, and with a convenient and novel stochastic representation which facilitates the implementation of the EM algorithm for maximum-likelihood estimation of the pth quantile regression parameters. We evaluate the performance of the proposed EM algorithm and the asymptotic properties of the maximum-likelihood estimates through empirical experiments and application to a real life dataset. The algorithm is implemented in the R package lqr(), providing full estimation and inference for the parameters as well as simulation envelopes plots useful for assessing the goodness-of-fit.

Sankhya Ser A

In this paper, we discuss some practical computational issues for quantile regression. We consider the computation from two aspects: estimation and inference. For estimation, we cover three algorithms: simplex, interior point, and smoothing. We describe and compare these algorithms, then discuss implementation of some computing techniques, which include optimization, parallelization, and sparse computation, with these algorithms in practice. For inference, we focus on confidence intervals. We discuss three methods: sparsity, rank-score, and resampling. Their performances are compared for data sets with a large number of covariates. AMS (2000) subject classification. Primary 62F35; secondary 62J99.

Journal of Applied Statistics, 2020

The check loss function is used to define quantile regression. In cross-validation, it is also employed as a validation function when the true distribution is unknown. However, our empirical study indicates that validation with the check loss often leads to overfitting the data. In this work, we suggest a modified or L2-adjusted check loss which rounds the sharp corner in the middle of check loss. This has the effect of guarding against overfitting to some extent. The adjustment is devised to shrink to zero as sample size grows. Through various simulation settings of linear and nonlinear regressions, the improvement due to modification of the check loss by quadratic adjustment is examined empirically.

2016

Check loss function is used to define quantile regression. In the prospect of cross validation, it is also employed as a validation function when underlying truth is unknown. However, our empirical study indicates that the validation with check loss often leads to choose an over estimated fits. In this work, we suggest a modified or L2-adjusted check loss which rounds the sharp corner in the middle of check loss. It has a large effect of guarding against over fitted model in some extent. Through various simulation settings of linear and non-linear regressions, the improvement of check loss by L2 adjustment is empirically examined. This adjustment is devised to shrink to zero as sample size grows. Keywords—Cross-validation, Model Selection, Quantile Regression, Tuning Parameter Selection

2006

SUMMARY Quantile regression is used in many areas of applied research and business. Examples are actuarial, financial or biometrical applications. We show that a non-parametric generalization of quantile regression based on kernels shares with support vector machines the property of consistency to the Bayes risk. We further use this consistency to prove that the non-parametric generalization approximates the conditional quantile

Scandinavian Journal of Statistics

Charlier et al. (2015a,b) developed a new nonparametric quantile regression method based on the concept of optimal quantization and showed that the resulting estimators often dominate their classical, kernel-type, competitors. The construction, however, remains limited to single-output quantile regression. In the present work, we therefore extend the quantization-based quantile regression method to the multiple-output context. We show how quantization allows to approximate the population multiple-output regression quantiles introduced in Hallin et al. (2015), which are conditional versions of the location multivariate quantiles from Hallin et al. (2010). We prove that this approximation becomes arbitrarily accurate as the size of the quantization grid goes to infinity. We also consider a sample version of the proposed quantization-based quantiles and establish their weak consistency for their population version. Through simulations, we compare the performances of the proposed quantization-based estimators with their local constant and local bilinear kernel competitors from Hallin et al. (2015). We also compare the corresponding sample quantile regions. The results reveal that the proposed quantization-based estimators, which are local constant in nature, outperform their kernel counterparts and even often dominate their local bilinear kernel competitors.

Journal of Statistical Computation and Simulation, 2014

Quantile regression provides estimates of a range of conditional quantiles. This stands in contrast to traditional regression techniques, which focus on a single conditional mean function. Lee et al. (2012) proposed efficient quantile regression by rounding the sharp corner of the loss. The main modification generally involves an asymmetric ℓ 2 adjustment of the loss function around zero. We extend the idea of ℓ 2 adjusted quantile regression to linear heterogeneous models. The ℓ 2 adjustment is constructed to diminish as sample size grows. Conditions to retain consistency properties are also provided.

2007

In this paper, we present an algorithm for Censored Quantile Regression (CQR) estimation problems. Our method permits CQR estimation problems to be solved more efficiently and reliably than was hitherto possible. It guarantees to find a high quality estimator in O(k×n²) operations with k regressors and n observations, which is much less than the existing algorithms for CQR problems. The first author would like to acknowledge financial support from the SSHRC of Canada.