Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

getindex() throws an error if a date isn't available #144

Open
chiraganand opened this issue Jan 31, 2023 · 7 comments
Open

getindex() throws an error if a date isn't available #144

chiraganand opened this issue Jan 31, 2023 · 7 comments
Assignees
@codetalker7
Labels
bug Something isn't working

Comments

@chiraganand
Copy link
Member

chiraganand commented Jan 31, 2023 •
edited
Loading

Version: v0.2.0

The following throws an error:

julia> ts[Date(2007,1, 1)]
ERROR: MethodError: no method matching getindex(::TSFrame, ::Vector{Nothing}, ::Vector{Int64})
Closest candidates are:
  getindex(::TSFrame, ::T, ::AbstractVector{Int64}) where T<:TimeType at ~/.julia/packages/TSFrames/IrHRj/src/getindex.jl:283
  getindex(::TSFrame, ::UnitRange, ::AbstractVector{Int64}) at ~/.julia/packages/TSFrames/IrHRj/src/getindex.jl:371
  getindex(::TSFrame, ::AbstractVector{T}, ::AbstractVector{Int64}) where T<:TimeType at ~/.julia/packages/TSFrames/IrHRj/src/getindex.jl:335
  ...
Stacktrace:
 [1] getindex(ts::TSFrame, dt::Vector{Date}, j::Vector{Int64})
   @ TSFrames ~/.julia/packages/TSFrames/IrHRj/src/getindex.jl:340
 [2] getindex
   @ ~/.julia/packages/TSFrames/IrHRj/src/getindex.jl:355 [inlined]
 [3] getindex(ts::TSFrame, d::Date)
   @ TSFrames ~/.julia/packages/TSFrames/IrHRj/src/getindex.jl:407
 [4] top-level scope
   @ REPL[52]:1

But, this works:

julia> ts[Date(2007,1,10)]
1×4 TSFrame with Date Index
 Index       Open     High     Low      Close
 Date        Float64  Float64  Float64  Float64
────────────────────────────────────────────────
 2007-01-10  49.9123  50.1305  49.9123  49.9725
@chiraganand chiraganand added the bug Something isn't working label Jan 31, 2023
@codetalker7
Copy link
Member

@chiraganand so in this kind of a situation, do we want to throw an error saying that the index doesn't contain this particular date?

@codetalker7 codetalker7 self-assigned this Feb 10, 2023
@chiraganand
Copy link
Member Author

@chiraganand so in this kind of a situation, do we want to throw an error saying that the index doesn't contain this particular date?

We should just return an empty TSFrame in this case just like how other getindex methods work.

@codetalker7
Copy link
Member

@chiraganand so in this kind of a situation, do we want to throw an error saying that the index doesn't contain this particular date?

We should just return an empty TSFrame in this case just like how other getindex methods work.

Okay. Just for completeness, similar errors are thrown in other row-indexing methods as well; some examples are given below.

julia> using TSFrames, DataFrames, Dates;

julia> ts_date = ts_date = TSFrame(DataFrame(Index=Date(2001, 1, 1):Day(1):Date(2001, 1, 10), x1=1:10));

julia> ts_int = TSFrame(DataFrame(Index=1:10, x1=1:10));

julia> ts_int[11]
ERROR: BoundsError: attempt to access 10×2 DataFrame at index [[11], DataAPI.Cols{Tuple{Symbol, UnitRange{Int64}}}((:Index, 2:2), union)]
Stacktrace:
 [1] getindex
   @ ~/.julia/packages/DataFrames/dgZn3/src/dataframe/dataframe.jl:576 [inlined]
 [2] getindex(ts::TSFrame, i::Int64, j::UnitRange{Int64})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:296
 [3] getindex(ts::TSFrame, i::Int64)
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:391
 [4] top-level scope
   @ REPL[37]:1

julia> ts_int[UnitRange(11, 12)]
ERROR: BoundsError: attempt to access 10×2 DataFrame at index [[11, 12], DataAPI.Cols{Tuple{Symbol, Vector{Int64}}}((:Index, [2]), union)]
Stacktrace:
 [1] getindex
   @ ~/.julia/packages/DataFrames/dgZn3/src/dataframe/dataframe.jl:576 [inlined]
 [2] getindex(ts::TSFrame, i::Vector{Int64}, j::Vector{Int64})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:328
 [3] getindex(ts::TSFrame, i::UnitRange{Int64}, j::UnitRange{Int64})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:382
 [4] getindex(ts::TSFrame, i::UnitRange{Int64})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:395
 [5] top-level scope
   @ REPL[38]:1

julia> ts_int[[11, 12]]
ERROR: BoundsError: attempt to access 10×2 DataFrame at index [[11, 12], DataAPI.Cols{Tuple{Symbol, Vector{Int64}}}((:Index, [2]), union)]
Stacktrace:
 [1] getindex
   @ ~/.julia/packages/DataFrames/dgZn3/src/dataframe/dataframe.jl:576 [inlined]
 [2] getindex(ts::TSFrame, i::Vector{Int64}, j::Vector{Int64})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:328
 [3] getindex
   @ ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:351 [inlined]
 [4] getindex(ts::TSFrame, i::Vector{Int64})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:399
 [5] top-level scope
   @ REPL[42]:1

julia> ts_date[Date(2002, 1, 1)]
ERROR: MethodError: no method matching getindex(::TSFrame, ::Vector{Nothing}, ::Vector{Int64})
Closest candidates are:
  getindex(::TSFrame, ::T, ::AbstractVector{Int64}) where T<:TimeType at ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:283
  getindex(::TSFrame, ::UnitRange, ::AbstractVector{Int64}) at ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:371
  getindex(::TSFrame, ::AbstractVector{T}, ::AbstractVector{Int64}) where T<:TimeType at ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:335
  ...
Stacktrace:
 [1] getindex(ts::TSFrame, dt::Vector{Date}, j::Vector{Int64})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:340
 [2] getindex
   @ ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:355 [inlined]
 [3] getindex(ts::TSFrame, d::Date)
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:407
 [4] top-level scope
   @ REPL[43]:1

julia> ts_date[[Date(2002, 1, 1), Date(2002, 1, 2)]]
ERROR: MethodError: no method matching getindex(::TSFrame, ::Vector{Nothing}, ::Vector{Int64})
Closest candidates are:
  getindex(::TSFrame, ::T, ::AbstractVector{Int64}) where T<:TimeType at ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:283
  getindex(::TSFrame, ::UnitRange, ::AbstractVector{Int64}) at ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:371
  getindex(::TSFrame, ::AbstractVector{T}, ::AbstractVector{Int64}) where T<:TimeType at ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:335
  ...
Stacktrace:
 [1] getindex(ts::TSFrame, dt::Vector{Date}, j::Vector{Int64})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:340
 [2] getindex
   @ ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:355 [inlined]
 [3] getindex(ts::TSFrame, dt::Vector{Date})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:403
 [4] top-level scope
   @ REPL[47]:1

All of these should return an empty TSFrame. I'll make a PR for this.

@codetalker7
Copy link
Member

@chiraganand in context of the above, I had a question: suppose we have a TSFrame with an Int index. In that case, what does indexing by an Int mean? Does it mean just return the ith row, or does it mean return the row where :Index is equal to i?

@chiraganand
Copy link
Member Author

@chiraganand in context of the above, I had a question: suppose we have a TSFrame with an Int index. In that case, what does indexing by an Int mean? Does it mean just return the ith row, or does it mean return the row where :Index is equal to i?

The Julia convention is to return the i^th row for getindex(ts, i).

@chiraganand
Copy link
Member Author

All of these should return an empty TSFrame. I'll make a PR for this.

I think we missed some test cases. :-)

@chiraganand
Copy link
Member Author 

julia> using TSFrames, DataFrames, Dates;

julia> ts_date = ts_date = TSFrame(DataFrame(Index=Date(2001, 1, 1):Day(1):Date(2001, 1, 10), x1=1:10));

julia> ts_int = TSFrame(DataFrame(Index=1:10, x1=1:10));

julia> ts_int[11]
ERROR: BoundsError: attempt to access 10×2 DataFrame at index [[11], DataAPI.Cols{Tuple{Symbol, UnitRange{Int64}}}((:Index, 2:2), union)]
Stacktrace:
 [1] getindex
   @ ~/.julia/packages/DataFrames/dgZn3/src/dataframe/dataframe.jl:576 [inlined]
 [2] getindex(ts::TSFrame, i::Int64, j::UnitRange{Int64})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:296
 [3] getindex(ts::TSFrame, i::Int64)
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:391
 [4] top-level scope
   @ REPL[37]:1

julia> ts_int[UnitRange(11, 12)]
ERROR: BoundsError: attempt to access 10×2 DataFrame at index [[11, 12], DataAPI.Cols{Tuple{Symbol, Vector{Int64}}}((:Index, [2]), union)]
Stacktrace:
 [1] getindex
   @ ~/.julia/packages/DataFrames/dgZn3/src/dataframe/dataframe.jl:576 [inlined]
 [2] getindex(ts::TSFrame, i::Vector{Int64}, j::Vector{Int64})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:328
 [3] getindex(ts::TSFrame, i::UnitRange{Int64}, j::UnitRange{Int64})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:382
 [4] getindex(ts::TSFrame, i::UnitRange{Int64})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:395
 [5] top-level scope
   @ REPL[38]:1

julia> ts_int[[11, 12]]
ERROR: BoundsError: attempt to access 10×2 DataFrame at index [[11, 12], DataAPI.Cols{Tuple{Symbol, Vector{Int64}}}((:Index, [2]), union)]
Stacktrace:
 [1] getindex
   @ ~/.julia/packages/DataFrames/dgZn3/src/dataframe/dataframe.jl:576 [inlined]
 [2] getindex(ts::TSFrame, i::Vector{Int64}, j::Vector{Int64})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:328
 [3] getindex
   @ ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:351 [inlined]
 [4] getindex(ts::TSFrame, i::Vector{Int64})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:399
 [5] top-level scope
   @ REPL[42]:1

julia> ts_date[Date(2002, 1, 1)]
ERROR: MethodError: no method matching getindex(::TSFrame, ::Vector{Nothing}, ::Vector{Int64})
Closest candidates are:
  getindex(::TSFrame, ::T, ::AbstractVector{Int64}) where T<:TimeType at ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:283
  getindex(::TSFrame, ::UnitRange, ::AbstractVector{Int64}) at ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:371
  getindex(::TSFrame, ::AbstractVector{T}, ::AbstractVector{Int64}) where T<:TimeType at ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:335
  ...
Stacktrace:
 [1] getindex(ts::TSFrame, dt::Vector{Date}, j::Vector{Int64})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:340
 [2] getindex
   @ ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:355 [inlined]
 [3] getindex(ts::TSFrame, d::Date)
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:407
 [4] top-level scope
   @ REPL[43]:1

julia> ts_date[[Date(2002, 1, 1), Date(2002, 1, 2)]]
ERROR: MethodError: no method matching getindex(::TSFrame, ::Vector{Nothing}, ::Vector{Int64})
Closest candidates are:
  getindex(::TSFrame, ::T, ::AbstractVector{Int64}) where T<:TimeType at ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:283
  getindex(::TSFrame, ::UnitRange, ::AbstractVector{Int64}) at ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:371
  getindex(::TSFrame, ::AbstractVector{T}, ::AbstractVector{Int64}) where T<:TimeType at ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:335
  ...
Stacktrace:
 [1] getindex(ts::TSFrame, dt::Vector{Date}, j::Vector{Int64})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:340
 [2] getindex
   @ ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:355 [inlined]
 [3] getindex(ts::TSFrame, dt::Vector{Date})
   @ TSFrames ~/xKDR-work/myfork/TSx.jl/src/getindex.jl:403
 [4] top-level scope
   @ REPL[47]:1

On another look, I think BoundsError is rightly thrown for all of the missing rows in the ts_int object but in case of ts_date the MethodError is thrown which is not correct. I feel there are two cases for indexing dates/times:

  1. Date/time is between start and end of the series: (Between(first(index) and last(index))).
  2. Date/time is less than first(index) or more than last(index).

For the first case, an empty TSFrame could be returned or missing values can be returned in all columns. For the second, BoundsError should be thrown.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@codetalker7 codetalker7

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@chiraganand @codetalker7