The Question That Exposes Weak Quant Models

“How did you resolve which variables to incorporate in your mannequin, and which did you intentionally exclude?”

The worth of the query lies in what it reveals. You aren’t asking for an inventory of variables. You’re asking whether or not the inclusion and exclusion choices have been grounded in financial reasoning quite than statistical match alone.

In my conversations with each allocators and managers, the responses fall into three distinct classes.

A powerful reply: The supervisor explains the financial mechanism behind every variable’s inclusion. Crucially, they talk about variables they excluded and why, displaying that specification was a deliberate design selection. They distinguish between variables that drive their goal issue and variables that end result from it. The strongest managers hint a series of financial causality: how macro forces undertaking onto stock-level indicators, and why the mannequin displays these causal chains quite than mining for correlations.

A regular reply: The supervisor cites statistical standards: info ratio, R-squared enchancment, significance exams. That is present business follow. It’s not mistaken, however it’s incomplete. Statistical match alone can’t distinguish between a variable that belongs within the mannequin and one which introduces distortion whereas bettering match metrics. That is precisely the entice within the opening story.

A regarding reply takes certainly one of two varieties: “We use all accessible variables and let the mannequin choose” indicators structural vulnerability to issue mirages. Alternatively, “Our variable choice course of is proprietary” might mirror reputable IP safety. However a supervisor who can’t clarify the reasoning behind their specification, even with out disclosing particular variables, can’t reveal that the reasoning exists.