Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

note dropped RTP packets in database #198

Open
scottlamb opened this issue Feb 16, 2022 · 0 comments
Open

note dropped RTP packets in database #198

scottlamb opened this issue Feb 16, 2022 · 0 comments
Labels
enhancement schema change Requires a database schema change (see new-schema branch)

Comments

@scottlamb
Copy link
Owner

scottlamb commented Feb 16, 2022

As discussed on the mailing list with @jlpoolen: next time we rework the database schema, let's add in support for tracking dropped RTP packets.

This could get really nuanced. The RTP stream tells us (via gaps in the sequence numbers) how many packets we've missed (modulus 2^16). In the case of H.264 video (audio is a little different):

  • In general that just tells us we are missing one or more frames. Any further frames until the next IDR frame can't be fully trusted. 1 frame is usually spread across many packets. IIUC with interleaved mode (I haven't seen any cameras use interleaved mode, and neither ffmpeg nor Retina support it today, but it's part of the protocol), 1 packet can even affect more than 1 frame of video.
  • We might know the dropped packets come in the middle of a single frame (because the packet before and the packet after have the same timestamp) so we've lost only one frame.
  • We might further know it was not a reference frame from the nal_ref_idc, so following frames of video are fully correct.
  • We might even have the H.264 slice header and thus in theory could get the reference list modifications to further know even in the case of a reference frame if any following frames are fully correct (and which). But in the most common case, each frame depends on the previous frames, so this distinction is only relevant for cameras that with temporal SVC enabled.

But I think I'd like to keep it simple by knowing we dropped N packets in a recording, or maybe between frames A and B, and maybe have some ability to further extend the index without a full schema change.

In my setup, dropped packets are rare but happen. If you pick a particular recording, most likely it's complete, and it'd be nice if we could look at the index, know there are no dropped packets, and thus that recording is fully trustworthy.

@scottlamb scottlamb added enhancement schema change Requires a database schema change (see new-schema branch) labels Feb 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement schema change Requires a database schema change (see new-schema branch)
Projects
None yet
Development

No branches or pull requests

1 participant