Evaluating a Manager's Skill vs Luck

We just completed our semi-annual Investment Team “cross-sectional review” of our investment managers in which we individually and collectively assess the potential of our active managers to add value in the future. It’s a chance to recalibrate our views on manager performance potential in a systematic and measurable way. If you care to dive into the details, our letter of February 2018 outlined the processes and procedures we use—send us a note if you would like to see that again. In this letter, however, we wish to outline the hard logic behind luck and skill, and the inescapable statistics of assembling a winning portfolio. In our experience, our ability to create portfolios that truly add value on a risk-adjusted basis is sustainable only because we remain mindful of the hard logic of luck and skill—to wit, always be prepared to deal with the fickle moods of chance.

The vagaries of luck. The ability of any manager to add value is always a combination of skill and luck. This simple statement bears some disassembling. “Adding value” can be hard to define. For example, a long‐only manager that seeks to beat the MSCI EAFE Index can add value simply by returning more than the index. But the manager may also be adding value by matching the performance of the index with less volatility. How about a hedge fund? A hedge fund may add value by achieving a return of, say, 8% if that manager did so with no stock market exposure and no correlation with the broader market index. Let’s assume that we can measure “adding value” through a clever application of precisely the right benchmark (recognizing that in practice this is sometimes easy, and sometimes hard). For the EAFE manager it may simply be beating the EAFE Index; for the hedge fund it may be an absolute return hurdle.

With that in mind, the ability of a manager to add value can be represented by a simple bar chart:

Luck, even for a skillful manager, is much more important than skill over the short term, meaning over a quarter or even a year. There is simply too much variability caused by unforecastable events, such as stock‐specific events, or the news of the day. It stands to reason that over time periods as short as a day, the driver for outperformance for even the most talented manager (excluding high‐frequency traders) is nearly 100% luck. For longer periods, the random nature of short‐term risk tends to cancel itself out, and true manager skill tends to prevail. This will be true both for talented managers and for managers with

weak skills—that is, over the long term, a manager with inferior stock‐selection skills will tend to underperform, as its repeated mistakes inevitably draw down its performance.

We believe the ability of great managers to add value over longer periods (say, a decade) can be attributed to upwards of 80% investment skill—perhaps even 90% (but never 100%). It is true that if you have enough monkeys flipping coins, a certain percentage of them (actually about 0.098%) will flip 10 heads in a row, but that is where experience comes in—with sufficient ongoing due diligence (i.e., interviews, analysis, and reference checking) one can separate the lucky monkeys from the superior investment minds. We believe that great managers have added value over long time periods because they are truly skillful.

So, that begs the question of how do you find these managers that have this wonderful lopsided ability to add value based much more on skill than luck? First, recognize that it is a difficult task—in our crowded universe of managers with great track records there will be some lucky monkeys. There will also be managers whose next 10 years will likely be very different than the last 10 years (due to factors such as growing too large, key people changes, getting complacent, or other afflictions of time and age). So how do you estimate a manager’s ability to add value through skill or luck? Two other insights help set the stage.

All great managers have a well‐calibrated self‐assessment of their own skills. One of our first steps is to figure out the manager’s own assessment of its ability to add value through skill. After all, they are the ones on the inside—they should know better than anyone else their own skill level. One would think so, but funnily enough, managers (especially young managers with great track records) tend to overattribute success to their own skill. Put another way, the 0.098% of lucky monkeys generally think they are brilliant. In order to weed these managers out (and thus separate the lucky from the skillful), it is fair to probe managers concerning their “edge” and how much outperformance they think they can

attribute to skill. Thoughtful managers will have an answer that recognizes the influence of luck, at least over short time periods. The best managers will demonstrate low hubris, providing answers that acknowledge the difficulty of measuring skill and of separating skill from luck. These managers may cite other portfolio factors (such as having more growth or value stocks, or more small‐cap or large‐cap stocks, etc.) as additional contributors to their performance—it is important to acknowledge such factors when assessing manager skill. Now it’s true that there are virtually no managers that under‐assess their own skill, so low hubris is probably a necessary but not sufficient criteria for manager excellence. Ideally,

you want a manager who accurately self‐calibrates. Such managers will use regular feedback loops” that test and assess past decisions. For example, some managers will compare the performance of their larger, “high‐conviction” stock positions to that of their smaller, “low‐conviction” names with the idea that if larger positions really reflect their “best ideas” then the outperformance of these best ideas can be tested. Others, especially small shops, may review their performance less systematically, but may do so through introspection or thoughtful discussions with a peer. It can be difficult to measure this selfcalibration ability, but it remains an essential first step in evaluating a manager’s ability to add value. After all, how can you rely on a manager that has a wildly inaccurate view of their own abilities!

All great manager‐selectors (people like us) also have a well‐calibrated assessment of their own skills. In ways that are similar to understanding manager skill, our manager‐selection skill is subject to some logical conclusions:

• Our ability to evaluate a manager’s ability to add value through skill is never 100%. You can never know everything about a manager—you always learn a bit more in each successive assessment. Let’s say that given sufficient time, sufficient data points, and sufficient interviews we can determine a manager’s ability to add value through skill with 90% accuracy.

• Bear in mind that the common case (even with managers with great track records) is that our assessment of their skill differs from their own assessment.

• Ideally, we find managers that can add value through skill; they have an accurate self‐assessment of their abilities; and our assessment closely matches their assessment. It’s hard to form a successful investment partnership without satisfying these three criteria.

The math works out as follows. To assemble a great portfolio with great managers:

• We must find very skillful managers whose ability to add value over a long time period is high, say 90%.

• We perform ongoing, deep due diligence and interview the managers to make this assessment and to understand the manager’s own calibration of that assessment.

• We create a portfolio recognizing that our own calibration abilities are not perfect. Perhaps given enough time and data points we can get it to 90%.

• Then, theoretically the portfolio we put together has an 81% likelihood (90% x 90%) of having managers that will add risk‐adjusted value over time.

• Recognizing the role of chance in assembling a great portfolio, we continue thorough diligence of our managers, and of our overall portfolio, with regular reassessment, pruning, weeding, and planting.

This math may seem daunting, but the statistics are important to remember in order to ensure that we, ourselves, don’t get infected with an inflated view of our own abilities. That said, we think we’ve accomplished our goal assembling portfolios of managers that add value through skill over the long term.

Firm update. We are pleased to announce that Matt Middleton, CFA, has been promoted to Managing Director on the Investment Team. Matt’s contributions to the firm through his investment knowledge have been deep and varied. Along with the rest of our experienced team, Matt will work hand‐in‐hand with Ed and Joel M. to ensure our client portfolios are well‐constructed for whatever market conditions the future may bring.

As always, we welcome your questions, comments and conversation!

Best regards,

The Mangham Associates Team

© 2019 Mangham Associates, Inc. All rights reserved.

For institutional or accredited investors only. Confidential – Not for reproduction or distribution.

The information presented should not be considered an offer to sell or the solicitation of an offer to purchase any particular security. Any such offer to sell or solicitation of an offer to purchase may be made only by means of the delivery of a confidential offering memorandum, which will contain material information not included herein regarding, among other factors, risk and potential conflicts of interest. This presentation should not be used as the sole basis for making a decision to invest with Mangham. In making an investment decision, you must rely on your own examination of the offering. You should not construe

the contents of this letter as legal, tax, investment, or other advice, or a recommendation to purchase or sell any particular security. No assurance can be given that investment objectives will be achieved.

Opinions expressed herein are those of Mangham Associates, and there is no assurance that any predicted results will actually occur. The information contained herein is based on sources believed to be reliable; however, its accuracy is not guaranteed.

This document contains information about possible or assumed future results of general economic conditions. “Forward‐looking statements” are based on assumptions that Mangham Associates believes to be reasonable but are not guarantees of results.

Unless otherwise noted, all returns are net of manager fees, and client managed total returns are net of Mangham Associates’ advisory fee. Additional information, including advisory fees and expenses, is provided on Mangham’s Form ADV Part 2A. Performance data is unaudited and subject to change. It is not possible to invest directly in an index and unmanaged indices do not incur fees and expenses.

Indices

Indices are used for benchmarking purposes and do not necessarily represent the same diversification or weightings as Manghamʹs various composites. It is not possible to invest directly in an index and unmanaged indices do not incur fees and expenses.

MSCI Global Investable Market Indices – The indices are constructed to provide exhaustive coverage of the investable opportunity set with non‐overlapping size and style segmentation. A strong emphasis is placed on investability and replicability of the indices through the use of size and liquidity screens. Source and use of MSCI Data: Morgan Stanley Capital International (MSCI). MSCI information may only be used for your internal use, may not be reproduced or redisseminated in any form and may not be used to create any financial instruments or products on any indices. This information is provided on an “as is” basis, and the user of this information assumes the entire risk of any use it may make or permit to be made of this information. Neither MSCI, any of its affiliates or any other person involved in or related to compiling, computing or creating this information makes any express or implied warranties or representations with respect to such information or the results to be obtained by the use thereof, and MSCI, its affiliates and each such other person hereby expressly disclaims all warranties (including, without limitation, all warranties of originality, accuracy, completeness, timeliness, non‐infringement, merchantability and fitness for a particular purpose) with respect to this information. Without limiting any of the foregoing, in no even shall MSCI, any of its affiliates or any other person involved in or related to compiling, computing or creating this information have any liability for

any direct, indirect special, incidental, punitive, consequential or any other damages (including, without limitation, lost profits), even if notified of, or if it might otherwise have anticipated, the possibility of such damages.

Performance data quoted represent past performance; past performance does not guarantee future results.