Fix for unbounded increase in audio delay when no audio packets are flowing in

WebRTC’s Audio Video sync can go in unbounded loop and keep on increasing audio delay if audio packets stop coming in.
The issue happens, if StreamSynchronization::ComputeDelays has:

1. relative_delay_ms = some positive value which causes avg_diff_ms_ > 30ms
2. current_audio_delay_ms < current_video_delay_ms
3. audio_delay_.extra_ms > 0 and video_delay_.extra_ms = 0

To compensate for relative delay, audio_delay_.extra_ms gets incremented every time StreamSynchronization::ComputeDelays is called by RtpStreamsSynchronizer::Process(), which happens every 1sec

RtpStreamsSynchronizer::Process()  will try to set the new delay to audio stream by calling syncable_audio_->SetMinimumPlayoutDelay(target_audio_delay_ms);

This ends up calling DelayManager::SetMinimumDelay and update minimum_delay_ms_

But this update has no impact on the value returned by NetEqImpl::FilteredCurrentDelayMs (as there are no audio packets flowing in, hence neteq is not running) which is called next time RtpStreamsSynchronizer::Process(), runs and tried to compute the new audio delay (audio_info→current_delay_ms)

This causes audio delay to be increased in every iteration and it grows unbounded. I guess it will stop growing above 10sec as that is hardcoded max delay in NetEQ.
To avoid this added a check to not adjust delays when no new audio stream has come in.

Bug: webrtc:11894
Change-Id: If648f9227e43c351f887d054876cb119cc1a917e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/183340
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Commit-Queue: Shyam Sadhwani <shyamsadhwani@fb.com>
Cr-Commit-Position: refs/heads/master@{#32106}
3 files changed
tree: 52d7d7918a8c829d1ca0c1e904e4532be78c454b
  1. api/
  2. audio/
  3. build_overrides/
  4. call/
  5. common_audio/
  6. common_video/
  7. data/
  8. docs/
  9. examples/
  10. logging/
  11. media/
  12. modules/
  13. p2p/
  14. pc/
  15. resources/
  16. rtc_base/
  17. rtc_tools/
  18. sdk/
  19. stats/
  20. style-guide/
  21. system_wrappers/
  22. test/
  23. tools_webrtc/
  24. video/
  25. .clang-format
  26. .git-blame-ignore-revs
  27. .gitignore
  28. .gn
  29. .vpython
  30. abseil-in-webrtc.md
  31. AUTHORS
  32. BUILD.gn
  33. CODE_OF_CONDUCT.md
  34. codereview.settings
  35. DEPS
  36. ENG_REVIEW_OWNERS
  37. LICENSE
  38. license_template.txt
  39. native-api.md
  40. OWNERS
  41. PATENTS
  42. PRESUBMIT.py
  43. presubmit_test.py
  44. presubmit_test_mocks.py
  45. pylintrc
  46. README.chromium
  47. README.md
  48. style-guide.md
  49. WATCHLISTS
  50. webrtc.gni
  51. webrtc_lib_link_test.cc
  52. whitespace.txt
README.md

WebRTC is a free, open software project that provides browsers and mobile applications with Real-Time Communications (RTC) capabilities via simple APIs. The WebRTC components have been optimized to best serve this purpose.

Our mission: To enable rich, high-quality RTC applications to be developed for the browser, mobile platforms, and IoT devices, and allow them all to communicate via a common set of protocols.

The WebRTC initiative is a project supported by Google, Mozilla and Opera, amongst others.

Development

See here for instructions on how to get started developing with the native code.

Authoritative list of directories that contain the native API header files.

More info