Home » hypothetical backtested performance

Tag: hypothetical backtested performance

The Friday List: Common Problems with Hypothetical Backtested Performance


Today, we offer our “Friday List,” an occasional feature summarizing a topic significant to investment management professionals interested in regulatory issues.  Our Friday Lists are an expanded “Our Take” on a particular subject, offering our unique (and sometimes controversial) perspective on an industry topic. 

Every year, the SEC publishes a handful of enforcement cases alleging that an investment adviser violated the advertising and marketing rules by misusing hypothetical backtested performance (HBP).  In our experience with exams, the SEC nearly always cites deficiencies when firms use HBP in marketing.  Although there is no rule specifically prohibiting the use of HBP, our position is that firms should never use HBP.  To support our view, we have highlighted below 10 of the most common HBP failings and cite to specific SEC actions (click on links).  As a side note, most institutional investors with whom we work look very critically at HBP because they also understand the limitations. 

10 Common Problems with Hypothetical Backtested Performance

  1. Failure to disclose limitationsOne common allegation is that firms fail to fully disclose the limitations on HBP.  
  2. Insufficient backup dataThe SEC will seek to verify that you have maintained adequate backup data to support your HBP claims.
  3. Cherry-picking time periodsMany firms have violated the SEC marketing rules when they cherry-pick a specific time period that makes their HBP look better
  4. Misleading disclosuresHidden or confusing HBP disclosure will draw the SEC’s enforcement interest.  
  5. Retrospective model changesFirms can’t keep tinkering with their models to improve the HBP results.
  6. Using incorrect historical market inputsThe SEC can verify actual market data from past time periods, so make sure you use the correct numbers.  
  7. Applying different modelsThe SEC has raised red flags when HBP differs significantly from audited or live performance information applying the same models.
  8. Using the wrong model rulesFirms have gone astray by applying different model rules to the backtested data than they use to manage real accounts
  9. Investments didn’t existThe SEC will call out HBP that includes investments that were not available at the time.  
  10. Faulty algorithmCheck the algorithm used, because faulty programming can result in inflated performance numbers.  

Signal Provider Used Misleading Hypothetical, Backtested Performance

An index signal provider, who also managed assets, agreed to a fine, censure and an outside compliance consultant for utilizing misleading hypothetical backtested performance information.  The SEC alleges that the calculations of the hypothetical, backtested performance for one of its core strategies deviated significantly from the live data, failed to conform to the firm’s model rules, and utilized an unavailable commodity index.  The SEC also faults the firm for failing to properly supervise a third-party index provider hired to create the backtested performance.  The SEC charges violations of the Advisers Act’s antifraud provisions (206(2)), advertising rule (206(4)-1(a)(5)), and compliance rule (206(4)-7). 

This case against an index provider adds fuel to the fire started by Investment Management Director Dalia Blass who last year questioned whether index providers should be exempt from investment adviser registration.  Also, as we have said before, do not use hypothetical, backtested performance information in marketing and advertising. 

FINRA Allows Limited Use of Pre-Inception Index Performance Data with Intermediaries

FINRA has issued an interpretive letter allowing a broker-dealer to use pre-inception index performance data to market index-based registered funds to institutional investors including intermediaries.  To use pre-inception data, the index must be developed according to “pre-defined rules that cannot be altered” except under extraordinary conditions, and the member firm may only disseminate the data to institutional investors including intermediaries that will not use the information with their retail clients.  FINRA imposes several conditions including (i) the data includes no less than 10 years of performance information, (ii) the material shows the impact of the deduction of fees and expenses, (iii) the material includes actual fund performance, and (iv) the firm includes extensive disclosure including the reasons why the data would have differed from actual performance during the period.  FINRA previously allowed pre-inception performance data to institutional investors other than intermediaries with the same conditions.

The change here is allowing broker-dealers to provide the information to intermediary financial advisers and putting the burden on the intermediaries to prevent use directly with their retail clients.    Regardless, we recommend against using hypothetical backtested performance data because of SEC concerns as well as the significant regulatory and disclosure limitations. 

Adviser Marketed Misleading Hypothetical Backtested Performance

 

The SEC censured and fined an investment adviser and its principal for misleading advertisements that utilized hypothetical backtested performance.  According to the SEC, the adviser continually updated its models but failed to fully disclose that the models’ out-performance resulted from these post hoc revisions.  The SEC alleges that the respondents revised the models to specifically account for unforeseen events such as market movements.  The SEC charges the firm and the principal, who also acted as the Chief Compliance Officer, with engaging in manipulative practices and for failing to implement a reasonable compliance program.  As part of the settlement, the firm agreed to retain a dedicated Chief Compliance Officer and an outside compliance consultant.

OUR TAKE:  As we have advised many times in the past: (i) do not advertise hypothetical backtested performance and (ii) retain a dedicated Chief Compliance Officer that has regulatory credentials.  Also, rather than continue to bring these cases whereby a dual-hatted principal continues to fail as Chief Compliance Officer, the SEC should solve this pandemic by requiring all advisers to undergo periodic third party compliance reviews.