Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correlation indicator window values introduce data leakage from the future into past window values which should be unknown at point in time #8424

Open
4 tasks done
superichmann opened this issue Nov 22, 2024 · 0 comments
Labels

Comments

@superichmann
Copy link

superichmann commented Nov 22, 2024

Expected Behavior

The Correlation indicator window should behave exactly like any other indicator values wise.
The latest point in time [0] window value saved in an external variable should be equal to the previous [1] window value at the current point in time.

Actual Behavior

The latest point in time [0] window value saved in an external variable are not equal to the previous [1] window value at the current point in time.
In Correlation indicator, opposed to other indicators, the older windows such as [1] are contaminated with future values from the current [0] point in time data.

Please consider adding a simple unit test that checks any new indicators for compliance with the window system as stated in your own website in order to confirm no future data is leaked to past windows.

Indicators have a built-in RollingWindow that stores their historical values.

This is the one of the most important purposes of your system - to represent a valid point in time view on the data. and yet you fail to achieve that.

Reproducing the Problem

paste into qc cloud, see logs.

namespace QuantConnect.Algorithm.CSharp
{
    public class corrnogood : QCAlgorithm
    {
        Correlation cp;
        SimpleMovingAverage s;
        decimal lastcp0value;
        decimal lasts0value;
        public override void Initialize()
        {
            SetStartDate(2024, 9, 9);
            var symbol=AddEquity("AAPL", Resolution.Daily).Symbol;
            var spy=AddEquity("SPY", Resolution.Daily).Symbol;
            cp=C(symbol, spy, 5, CorrelationType.Pearson);
            s=SMA(symbol, 5);
        }

        public override void OnData(Slice data)
        {
            if (data.Bars.ContainsKey("AAPL"))
            {
                if (cp.IsReady)
                {
                    if (lastcp0value!=cp[1].Value)
                        Log("Wrong Corr Window Value");
                    if (lasts0value!=s[1].Value) // NEVER HAPPENS
                        Log("Wrong SMA Window Value");
                    //Log(cp[0].Value+"\t"+cp[1].Value); // NOT GOOD!
                    //Log(s[0].Value+"\t"+s[1].Value); // GOOD!
                    lastcp0value=cp[0].Value;
                    lasts0value=s[0].Value;
                }
            }
        }
    }
}

System Information

qc cloud

Checklist

  • I have completely filled out this template
  • I have confirmed that this issue exists on the current master branch
  • I have confirmed that this is not a duplicate issue by searching issues
  • I have provided detailed steps to reproduce the issue
@superichmann superichmann changed the title Something is really bad with Correlation window values Correlation indicator window values introduce data leakage from the future which should be unknown at point in time Nov 23, 2024
@superichmann superichmann changed the title Correlation indicator window values introduce data leakage from the future which should be unknown at point in time Correlation indicator window values introduce data leakage from the future into past window values which should be unknown at point in time Nov 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants