Abstract Expected shortfall, measuring the average outcome (e.g., portfolio loss) above a given quantile of its probability distribution, is a common financial risk measure. The same measure can be used to characterize treatment effects in the tail of an outcome distribution, with applications ranging from policy evaluation in economics and public health to biomedical investigations. Expected shortfall regression is a natural approach of modeling covariate-adjusted expected shortfalls. Because the expected shortfall cannot be written as a solution of an expected loss function at the population level, computational as well as statistical challenges around expected shortfall regression have led to stimulating research. We discuss some recent developments in this area, with a focus on a new optimization-based semiparametric approach to estimation of conditional expected shortfall that adapts well to data heterogeneity with minimal model assumptions. The talk is based on joint work with Yuanzhi Li and Shushu Zhang.

Abstract In recent years economists have become interested in at-Risk modelling, particularly Growth-at-Risk (GaR), following the widespread earlier use of Value-at-Risk in finance. GaR has now entered mainstream economics, particularly in central banks. Applications of at-Risk models have often used data which more naturally would be represented as tensor-valued, rather than vector-valued. Recent developments in tensor methods, particularly methods for tensor time series, are applicable to economic modelling. This paper reviews these developments, highlights the potential benefits of their use in at-Risk modelling (including applications beyond GaR) and suggests guidance for their use.

Abstract Before data collection, two key questions must be addressed: "How closely does the researcher want the sample mean to approximate the population mean?" and "What probability does the researcher aim for the sample mean to fall within the specific distance of the population mean?" By specifying the desired accuracy for (1) and the confidence for (2), we can determine the necessary minimum sample size across a range of distributional settings. Numerical examples will be provided for illustration.

Abstract The box-and-whisker plot, introduced by Tukey (1977), is one of the most popular graphical methods in descriptive statistics. On the other hand, however, Tukey's boxplot is free of sample size, yielding the so-called "one-size-fits-all" fences for outlier detection. Although improvements on the sample size adjusted boxplots do exist in the literature, most of them are either not easy to implement or lack justification. As another common rule for outlier detection, Chauvenet's criterion uses the sample mean and standard derivation to perform the test, but it is often sensitive to the included outliers and hence is not robust. In this paper, by combining Tukey's boxplot and Chauvenet's criterion, we introduce a new boxplot, namely the Chauvenet-type boxplot, with the fence coefficient determined by an exact control of the outside rate per observation. Our new outlier criterion not only maintains the simplicity of the boxplot from a practical perspective, but also serves as a robust Chauvenet's criterion. Simulation study and a real data analysis on the civil service pay adjustment in Hong Kong demonstrate that the Chauvenet-type boxplot performs extremely well regardless of the sample size, and can therefore be highly recommended for practical use to replace both Tukey's boxplot and Chauvenet's criterion. Lastly, to increase the visibility of the work, a user-friendly R package named ‘ChauBoxplot' has also been officially released on CRAN.

Abstract Motivated by predicting intraday trading volume curves, we consider two spatio-temporal autoregressive models for matrix time series, in which each column may represent daily trading volume curve of one asset, and each row captures synchronized 5-minute volume intervals across multiple assets. While traditional matrix time series focus mainly on temporal evolution, our approach incorporates both spatial and temporal dynamics, enabling simultaneous analysis of interactions across multiple dimensions. The inherent endogeneity in spatio-temporal autoregressive models renders ordinary least squares estimation inconsistent. To overcome this difficulty while simultaneously estimating two distinct weight matrices with banded structure, we develop an iterated generalized Yule-Walker estimator by adapting a generalized method of moments framework based on Yule-Walker equations. Moreover, unlike conventional models that employ a single bandwidth parameter, the dual-bandwidth specification in our framework requires a new two-step, ratio-based sequential estimation procedure.

Abstract This paper explores the estimation and inference of the minimum spanning set (MSS), the smallest subset of risky assets that spans the mean-variance efficient frontier of the full asset set. We establish identification conditions for the MSS and develop a novel procedure for its estimation and inference. Our theoretical analysis shows that the proposed MSS estimator covers the true MSS with probability approaching 1 and converges asymptotically to the true MSS at any desired confidence level, such as 0.95 or 0.99. Monte Carlo simulations confirm the strong finite-sample performance of the MSS estimator. We apply our method to evaluate the relative importance of individual stock momentum and factor momentum strategies, along with a set of well-established stock return factors. The empirical results highlight factor momentum, along with several stock momentum and return factors, as key drivers of mean-variance efficiency. Furthermore, our analysis uncovers the sources of contribution from these factors and provides a ranking of their relative importance, offering new insights into their roles in mean-variance analysis.

Abstract Threshold factor models are pivotal for capturing rapid regime-switching dynamics in high-dimensional time series, yet existing frameworks relying on a single pre-specified threshold variable often suffer from model misspecification and unreliable inferences. This paper introduces a novel factor tree model that integrates classification and regression tree (CART) principles with high-dimensional factor analysis to address structural instabilities driven by multiple threshold variables. The factor tree is constructed via a recursive sample splitting procedure that maximizes reductions in a loss function derived from the second moments of estimated pseudo linear factors. At each step, the algorithm selects the threshold variable and cutoff value yielding the steepest loss reduction, terminating when a data-driven information criterion signals no further improvement. To mitigate overfitting, an information criterion-based node merging algorithm consolidates leaf nodes with identical factor representations. Theoretical analysis establishes consistency in threshold variable selection, threshold estimation, and factor space recovery, supported by extensive Monte Carlo simulations. An empirical application to U.S. financial data demonstrates the factor tree's effectiveness in capturing regime-dependent dynamics, outperforming traditional single-threshold models in decomposing threshold effects and recovering latent factor structures. This framework offers a robust data-driven approach to modeling complex regime transitions in high-dimensional systems.

Abstract There is growing interest in constructing conformal prediction sets that provide approximate or asymptotic conditional coverage guarantees, capturing local data heterogeneity. However, methods like localized conformal prediction (LCP) may face challenges in ensuring reliable prediction sets in regions with sparse calibration data. This paper introduces Enhanced Localized Conformal Prediction (ELCP), a novel approach that incorporates auxiliary data to refine localized prediction sets while preserving finite-sample marginal coverage guarantees. By utilizing a density-ratio-weighted kernel estimator, ELCP seamlessly integrates auxiliary and calibration data, accommodating potential distributional shifts and improving the local reliability of prediction sets. Theoretical analysis confirms that ELCP maintains marginal coverage and enhances asymptotic test-conditional coverage. Simulation results demonstrate its superior local coverage and smaller prediction sets compared to standard LCP, highlighting its effectiveness in settings with limited calibration data but available auxiliary information from related tasks.

Abstract We propose CUSUM-Net, a novel nonparametric framework for detecting changes in data distributions by leveraging an integral probability metric (IPM) optimized through deep neural networks. CUSUM-Net identifies the optimal direction separating distinct distributional regimes by aggregating critic-CUSUM statistics over candidate changepoints, directly aligning changepoint detection with IPM optimization. Unlike existing parametric methods, our approach is robust to complex, high-dimensional data distributions and accommodates various data modalities including vectors, symmetric positive-definite matrices, images, and graphs. Theoretically, we establish nonparametric excess-risk bounds for the learned critic and demonstrate accelerated convergence rates under low-dimensional manifold assumptions. Furthermore, consistency of the resulting changepoint estimator is proven. Extensive numerical experiments validate the flexibility, robustness, and efficiency of CUSUM-Net.

Abstract Spatial varying coefficient (SVC) model offers an effective way to characterize the heterogeneous effects of covariates on the response variable. However, existing SVC models assume accurately measured covariates, ignoring real-world measurement errors that may arise from many aspects. This paper develops a novel, statistically and computationally efficient framework for the inference of SVC models based on profile model likelihood, with explicit consideration of covariate measurement error. Focusing on spatial clustered coefficient models, we formulate an SVC model that allows functional covariate measurement errors. Bias-corrected estimators are derived by leveraging techniques of high-dimensional measurement error models and unbiased estimating functions, ensuring consistent parameter estimation. A predictive SVC model is then proposed to facilitate computations for large datasets. Finally, the effectiveness of the proposed approaches is demonstrated through comprehensive simulation studies and an analysis of the Arctic sea ice dataset.

Abstract This paper studies nonparametric local (over-)identification, in the sense of Chen and Santos (2018), and the associated semiparametric efficiency in modern causal frameworks. We develop a unified approach that begins by translating structural models with latent variables into their induced statistical models of observables and then analyzes local overidentification through conditional moment restrictions. We apply this approach to three leading models: (i) the general treatment model under unconfoundedness, (ii) the negative control model, and (iii) the long-term causal inference model under unobserved confounding. The first design yields a locally just-identified statistical model, implying that all regular asymptotically linear estimators of the treatment effect share the same asymptotic variance, equal to the (trivial) semiparametric efficiency bound. In contrast, the latter two models involve nonparametric endogeneity and are naturally locally overidentified; consequently, some doubly robust orthogonal moment estimators of the average treatment effect are inefficient. Whereas existing work typically imposes strong conditions to restore just-identification before deriving the efficiency bound, we relax such assumptions and characterize the general efficiency bound, along with efficient estimators, in the overidentified models (ii) and (iii).

Abstract This paper investigates the use of synthetic control methods for causal inference in macroeconomic settings when dealing with possibly nonstationary data. While the synthetic control approach has gained popularity for estimating counterfactual outcomes, we caution researchers against assuming a common nonstationary trend factor across units for macroeconomic outcomes, as doing so may result in misleading causal estimation—a pitfall we refer to as the spurious synthetic control problem. To address this issue, we propose a synthetic business cycle framework that explicitly separates trend and cyclical components. By leveraging the treated unit's historical data to forecast its trend and using control units only for cyclical fluctuations, our divide-and-conquer strategy eliminates spurious correlations and improves the robustness of counterfactual prediction in macroeconomic applications. As empirical illustrations, we examine the cases of German reunification and the handover of Hong Kong, demonstrating the advantages of the proposed approach.

Abstract Portfolio optimization aims at constructing a realistic portfolio with significant out-of-sample performance, which is typically measured by the out-of-sample Sharpe ratio. However, due to in-sample optimism, it is inappropriate to use the in-sample estimated covariance to evaluate the out-of-sample Sharpe, especially in the high dimensional settings. In this article, we propose a novel method to estimate the out-of-sample Sharpe ratio using only in-sample data, based on random matrix theory. Furthermore, portfolio managers can use the estimated out-of-sample Sharpe as a criterion to decide the best tuning for constructing their portfolios. Specifically, we consider the classical framework of Markowitz mean-variance portfolio optimization under high dimensional regime of p/n → c ∈ (0, ∞), where p is the portfolio dimension and n is the number of samples or time points. We propose to correct the sample covariance by a regularization matrix and provide a consistent estimator of its Sharpe ratio. The new estimator works well under either of the following conditions: (a) bounded covariance spectrum, (b) arbitrary number of diverging spikes when c < 1, and (c) fixed number of diverging spikes with weak requirement on their diverging speed when c ≥ 1. We can also extend the results to construct global minimum variance portfolio and correct out-of-sample efficient frontier. We demonstrate the effectiveness of our approach through comprehensive simulations and real data experiments. Our results highlight the potential of this methodology as a useful tool for portfolio optimization in high dimensional settings.

Abstract Expected shortfall (ES) has emerged as an important metric for characterizing the tail behavior of a random outcome, specifically associated with rarer events that entail severe consequences. In climate science, the threats of flooding and heatwaves loom large, impacting natural environments and human communities. In actuarial studies, a key observation in modeling insurance claim sizes is that features exhibit distinct effects in explaining small and large claims. This article concerns nonparametric expected shortfall regression as a class of statistical methods for tail learning. These methods directly target upper/lower tail averages and will empower practitioners to address complex questions that are beyond the reach of mean regressionbased approaches. Using kernel ridge regression, we introduce a two-step nonparametric ES estimator that involves a plugged-in quantile function estimate without sample-splitting. We provide non-asymptotic estimation and Gaussian approximation error bounds, depending explicitly on the effective dimension, sample size, regularization parameters, and quantile estimation error. To construct pointwise confidence bands, we propose a fast multiplier bootstrap procedure and establish its validity. We demonstrate the finite-sample performance of the proposed methods through numerical experiments and an empirical study aimed at examining the heterogeneous effects of different air pollutants and meteorological factors on average and high PM2.5 concentration. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Abstract Economic data often exhibit complex heterogeneous structures across different subgroups, which we must appropriately account for in statistical analysis. Classical statistical models, which assume the true model is the same across different subjects, may yield misleading results when the true model indeed varies across subgroups. Although several methods have been proposed to account for such model heterogeneity using various (link) functions, they are often model-based or require strong identifiability conditions. In this paper, we consider a varying multiple index model, a model-free framework regarding the link function, inspired by sufficient dimension reduction literature in the supervised setting. We propose a sliced inverse regression-based method to estimate the subgroup covariate subspaces, the space spanned by the (heterogeneous) regression coefficient of covariates across different subgroups in the varying multiple index model. While sliced inverse regression fails to account for model heterogeneity, by adaptive smoothing over the space, the proposed method estimates the subgroup covariate subspace consistently and optimally whenever it is varying across different subgroups or not. The proposed method is applicable to both low dimensional and high dimensional data, achieves full efficiency in the homogeneous setting, and accounts for heterogeneity in a model-free manner. Re-analyzing the California housing dataset, we demonstrate how the proposed method helps facilitate statistical analysis under heterogeneity.

Abstract In network analysis, we often use a statistic on the sample network, such as motif density, to make inference about the population. Due to practical constraints such as computation and privacy, traditional random vertex sampling is often infeasible, and the network may only be accessible through traversal sampling methods, such as random walk sampling. However, due to the complex nature of the random walk sampling, the analytical form of the sampling distribution of the statistic is often unavailable or mathematically intractable, preventing us from conducting inference. To address this issue, we propose to bootstrap the random walk sample network to approximate the sampling distribution for inference and establish its bootstrap consistency. To the best of our knowledge, this is the first attempt in the literature to conduct formal inference using the random walk sample network, although such a practice is common in network analysis and has previously lacked theoretically justified principles. In addition, by establishing and comparing the asymptotic properties of motif densities under random walk and random vertex sampling, we show that many existing network bootstrap methods are systematically biased under the random walk case. Simulation studies and a real-world network analysis show that the proposed method outperforms existing methods.

Abstract While max-share identification has become increasingly popular in a wide range of applications, we show that its validity requires necessary and sufficient conditions that are rarely satisfied in practice — the target variable’s response to the target shock must be (i) orthogonal to its responses to untargeted shocks and (ii) larger than combinations of those responses. Imposing additional restrictions on the target shock weakens but does not fully eliminate these conditions. We show that in practice, the weight max-share places on an identified untargeted shock can be obtained by projecting the response to that shock on the max-share response. We also theoretically characterize consequences of local and global violations to the identification conditions. Empirically, the TFP news and main business cycle shocks identified by Kurmann and Sims (2021) and Angeletos et al. (2020) are, respectively, at least a third and a quarter contaminated.

Abstract Prediction-powered inference integrates a small gold-standard dataset with a large auxiliary dataset informed by machine learning predictions to enable valid statistical inference. In modern applications, multiple data sources and diverse predictive models often generate several auxiliary datasets, each encoding different aspects of information. However, how to optimally combine multiple data sources and machine learning methods to improve inferential efficiency remains unclear. We propose a model averaging framework that adaptively learns the contribution of each auxiliary dataset by assigning optimal weights that minimize the volume of the resulting confidence region. Our method accommodates both homogeneous and heterogeneous settings for auxiliary and gold-standard datasets. We establish the asymptotic validity of the proposed estimator and prove that it attains the asymptotically smallest confidence region. Furthermore, we relax the original PPI requirement of pre-trained predictors by developing an algorithm that trains or fine-tunes machine learning models on auxiliary datasets, while preserving valid inference guarantees. Extensive simulations and real-data experiments confirm the effectiveness and robustness of our approach.

Abstract LASSO has been the state-of-the-art methodology for high dimensional models. When cointegration is present in a high dimensional predictive regression, Mei and Shi (2024) show that LASSO can be inconsistent due to its excessive penalty on cointegrated regressors. In this paper, we propose using nonconvex penalties, including the widely used SCAD and MCP, to fix the over-penalization and restore consistent estimation of the high dimensional predictive regression. Our asymptotic theory establishes the consistency of estimation and variable selection by SCAD and MCP under numerous cointegrated regressors. In the numerical studies, we observe substantial improvements by the nonconvex penalties relative to LASSO in terms of parameter estimation and out-of-sample prediction.