LLMs Reveal How Companies Bury Bad News in Annual Reports

Hiding in Fine Print..

Just like how companies bury bad news with a Friday-before-a-holiday-regulatory filing, they try to hide bad performance deeper into their annual reports, according to new research from Chicago Booth.

Using large language models, researchers found that instead of discussing company metrics in order of importance, they’ll dump bad news later in the MD&A section where they have more discretion, especially when they:

• Have negative sentiment news to deliver

• Face more intense competition

• Report lower profitability

• Have higher earnings volatility

The authors developed a measure called "Information Positioning" which scores how forthcoming companies are with their information. Large, mature firms tend to place important information up-front more consistently, while companies that report losses, have lower profitability, higher earnings volatility, or negative sentiment tend to have lower information positioning scores.

This is just one of the findings of the compelling new paper, called Learning Fundamentals from Text, from the University of Chicago’s Alex Kim, Maximilian Muhn, Valeri V. Nikolaev and Yijing Zhang. 

Here’s how they did it:

They took every electronic 10K available, removed figures/tables, and converted 20 million paragraphs into vectors. And instead of learning relationships between words like traditional LLMs, the study treated whole paragraphs as basic units.

They then examined stock returns 1 day before and 30 days after filing the annual reports. If returns were positive, it was labeled one, and if negative, zero. 

The academics then used an “attention mechanism” – the type of transformer technology that’s behind generative AI –  to figure out what paragraphs investors seemed to care about.

Like when you read this sentence: “If returns were positive, it was labeled one, and if negative, zero. When you read this, your brain likely didn’t give equal "attention" to each word. You probably focused more on key words like "returns, "positive," "negative," "one" and "zero" because they carry the most important information.

Now imagine an LLM doing this –  like a reader skimming a document –  for 20 million paragraphs. It eventually starts to see patterns. 

Portfolios built using the attention-based model had a Sharpe ratio of 1.56, meaning they delivered substantially better returns for the amount of risk taken compared to portfolios using a basic logit model, which had a Sharpe ratio of 1.08.

Here are the most Important 10-K Sections per the research:

1. Item 7 - Management's Discussion & Analysis (MD&A)

2. Item 8 - Financial Statements and Notes

3. Item 1 - Business Description

4. Item 1A - Risk Factors

And the least important sections: 

1. Item 13 - Relations and Transactions

2. Item 12 - Security Ownership

3. Item 10 - Directors and Governance

4. Item 9B - Other Information

Another interesting finding: investors aren’t that interested in ESG

While ESG and corporate governance have received a lot of public attention, the topics are not typically important to investors. Corporate social responsibility-related topics consistently ranked among the least important sections in terms of market reactions. The most important item (MD&A) was found to be almost twice as relevant as Directors and Governance (Item 10) based on their importance scores.

The main takeaway? From co-author Alex Kim:

“Not all paragraphs are the same. Focusing on more important ones will facilitate [investor] information processing.”