Transformer Network for Macroeconometric Analysis

The dissertation proposes a new framework for analyzing macroeconomic data. The framework is based on a machine learning Transformer network, customized for estimating latent variables from small multivariate time series datasets. The framework allows for more flexibility than many existing techniques in economics. The dissertation includes an introduction and three chapters with independent studies. The second chapter proposes the methodology, and chapters three and four apply it to real-world problems.

Chapter 2 develops a Transformer algorithm for estimating nonlinear dynamic factors. Unlike most conventional factor models, the Transformer does not require committing to strict assumptions about the functional form of the underlying process. The Transformer learns through training to construct the factor estimate from data with an information bottleneck. Transformers typically require a lot of data to function properly, whereas in macroeconomics the datasets tend to be very small. I propose using conventional factor models as prior information to guide the Transformer, with a new regularization term in the loss function. Using prior information improves the results substantially on small datasets. Being able to interpret the results from a model is important for economic applications. The Transformer constructs an Attention matrix, which describes the relative importance of different input variables and their lags for the factor estimate. Analyzing how these Attention patterns evolve over time reveals regime switches and influential shocks. The Transformer is evaluated on several simulated datasets generated by processes that deviate from linear--Gaussian assumptions. On these datasets, the Transformer is found to surpass a baseline linear factor model with Kalman filtering by 20 %, on average, in terms of factor estimation accuracy.

 Chapter 3 uses the Transformer to estimate a dynamic factor from a macroeconomic dataset. The factor is interpreted as a coincident index measuring real economic activity in the United States. The index is compared to a baseline coincident index obtained using a linear factor model with Kalman filtering during three past recessions.

Chapter 4 constructs a measure of fragility for the financial system, using a Transformer on a macro-financial panel dataset from industrialized countries. Historic systemic crises are used as training examples of excess fragility, which is assumed to predispose the financial system to malfunction. The fragility measure rises during most past out-of-sample pre-crisis periods. The measure is meant to assist in the timing of countercyclical macroprudential policies.

Contact Oliver Snellman

Email: oliver.snellman@helsinki.fi
Home page: www.oliversnellman.com/