For filtering strategies in In-Sample (IS) and Out-of-Sample (OOS) phases, here’s how you should approach it to ensure long-term stability:

Filtering by IS and OOS:

Both IS and OOS performance are critical, as they provide insight into how the strategy performs in known market conditions (IS) and unseen market conditions (OOS). The goal is to ensure the strategy generalizes well beyond the backtested data.

Key Steps to Approach IS and OOS Filtering:

1. Separate IS/OOS Data

Make sure your dataset has been divided into In-Sample (IS) and Out-of-Sample (OOS) parts. IS is used for training or optimization, while OOS is used to test the strategy’s robustness.

2. Prioritize OOS for Long-Term Viability

  • OOS performance is more important than IS because it demonstrates how the strategy handles real, unseen market conditions. Always filter strategies that have both IS and OOS results, but give more weight to OOS metrics to ensure robustness.

Ranking and Filtering for IS and OOS:

To set up your filters and ranking criteria for both IS and OOS, here’s how to prioritize:

1. Sharpe Ratio (OOS > IS)

  • IS: Should be > 1.5. A high IS Sharpe Ratio indicates good performance in training data.
  • OOS: This should be your priority and should ideally be > 2. It shows strong performance with minimal volatility in unseen data.
  • Why: If OOS Sharpe is much lower than IS, the strategy may be overfit. The OOS Sharpe Ratio >2 confirms the strategy can handle real-world volatility.

2. CAGR/Max DD% (OOS > IS)

  • IS: Should be > 1, but OOS should be higher than IS.
  • OOS: Should be prioritized here as it shows the return achieved per unit of drawdown in unseen market conditions. Aim for > 1.
  • Why: Ensuring the OOS CAGR/Max DD% is strong proves that the strategy will continue to generate healthy returns with manageable risk.

3. Profit Factor (OOS close to IS)

  • IS: Look for > 1.5 to ensure profitability in known data.
  • OOS: Should ideally be close to IS performance. Aim for > 1.5, and if OOS is significantly lower, be cautious.
  • Why: Large gaps between IS and OOS profit factor indicate potential overfitting.

4. Return/Drawdown Ratio (Ret/DD) (OOS > IS)

  • IS: Should be > 2.
  • OOS: Should be comparable or better than IS, ideally > 2. If OOS Ret/DD drops too much compared to IS, the strategy may not generalize well.
  • Why: This helps you assess the balance of returns versus risk in unseen conditions. Favor strategies where OOS Ret/DD is consistent with IS.

5. Stability (OOS close to IS)

  • IS: Should be > 0.8 to indicate a stable equity curve in the training data.
  • OOS: Should be close to IS. Significant drops here mean the strategy is less reliable in real conditions.
  • Why: High stability in OOS ensures the strategy performs smoothly without wild fluctuations in live trading.

6. Open Drawdown % (OOS < IS)

  • IS: Set the threshold < 15% to limit exposure in known conditions.
  • OOS: This should be the same or lower than IS to ensure the strategy doesn’t take on too much risk when live. Prioritize < 10-12% for OOS.
  • Why: You want to reduce risk in live markets, so OOS drawdown should ideally be lower than IS.

7. Max Drawdown % (OOS < IS)

  • IS: Set a threshold < 20-25%.
  • OOS: Should be the same or lower than IS to avoid risk spikes in live trading. < 20% is preferable.
  • Why: Keeping OOS max drawdown low is critical to ensuring the strategy doesn’t suffer big losses in real-world conditions.

8. Net Profit (OOS close to IS)

  • IS: Look for solid net profit but not at the expense of risk.
  • OOS: Should be close to IS. If OOS profit is significantly lower than IS, this could be a sign of overfitting.
  • Why: The goal is to ensure consistent profitability across both IS and OOS phases.

9. R-Squared (R²) (OOS close to IS)

  • IS: Look for R² to indicate a strong trend, but don’t prioritize it too heavily.
  • OOS: Should be close to IS values. Large differences could signal instability in the equity curve.
  • Why: R² helps in assessing whether the strategy maintains its trend in unseen data.

10. Consecutive Win/Loss Trades (OOS > IS)

  • IS: Avoid strategies with large streaks of losses.
  • OOS: Should be comparable to IS, and you want to avoid any strategies where the number of consecutive losses is much higher in OOS.
  • Why: A strategy that maintains its win/loss consistency across IS and OOS will be more robust.

Suggested Filter Prioritization for IS/OOS:

  1. Sharpe Ratio (OOS > IS)
  2. CAGR/Max DD% (OOS > IS)
  3. Return/Drawdown Ratio (Ret/DD) (OOS > IS)
  4. Profit Factor (OOS close to IS)
  5. Stability (OOS close to IS)
  6. Open Drawdown % (OOS < IS)
  7. Max Drawdown % (OOS < IS)
  8. Net Profit (OOS close to IS)
  9. Consecutive Win/Loss Trades (OOS > IS)
  10. R-Squared (R²) (OOS close to IS)

Summary

When filtering and ranking strategies, OOS performance should take precedence over IS, as it reflects the strategy’s performance in live, unseen market conditions. Always ensure that OOS metrics are close to or better than IS to confirm the robustness of your strategy, particularly Sharpe Ratio, CAGR/Max DD%, and Ret/DD.

Related posts