This change refactors the Neural Residual Echo Estimator (ML REE) to properly handle multi-channel audio signals for all inputs.

  All multi-channel handling, buffering, and pre-processing is moved to the FeatureExtractor classes.

  Key changes:
   - The main NeuralResidualEchoEstimator::Estimate interface is updated to accept a multi-channel webrtc::Block for the render signal.
   - NeuralResidualEchoEstimatorImpl no longer performs any buffering or channel summation.
   - The FeatureExtractor interface is redesigned, splitting responsibilities into:
     - UpdateBuffers: For buffering incoming blocks of data.
     - PrepareModelInput: For preparing the model's input tensor when enough data is available.
     - ReadyForInference: To check if a full frame can be processed.
     - TimeDomainFeatureExtractor now implements channel summation for all inputs.
   - FrequencyDomainFeatureExtractor buffers each channel and then computes the average power spectrum across all channels.

Bug: webrtc:442444736
Change-Id: Ie0df331c87a414146187f661aed17369a946d660
No-Iwyu: Not changing the tflite includes.
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/429142
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Commit-Queue: Jesus de Vicente Pena <devicentepena@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#46345}
11 files changed
tree: 1fd6a7f19b70b8156f2b62df4baab6cc6b92c95e
  1. api/
  2. audio/
  3. build_overrides/
  4. call/
  5. common_audio/
  6. common_video/
  7. data/
  8. docs/
  9. examples/
  10. experiments/
  11. g3doc/
  12. infra/
  13. logging/
  14. media/
  15. modules/
  16. net/
  17. p2p/
  18. pc/
  19. resources/
  20. rtc_base/
  21. rtc_tools/
  22. sdk/
  23. stats/
  24. system_wrappers/
  25. test/
  26. tools_webrtc/
  27. video/
  28. .clang-format
  29. .clang-tidy
  30. .git-blame-ignore-revs
  31. .gitignore
  32. .gn
  33. .mailmap
  34. .rustfmt.toml
  35. .style.yapf
  36. .vpython3
  37. AUTHORS
  38. BUILD.gn
  39. CODE_OF_CONDUCT.md
  40. codereview.settings
  41. DEPS
  42. DIR_METADATA
  43. ENG_REVIEW_OWNERS
  44. LICENSE
  45. license_template.txt
  46. native-api.md
  47. OWNERS
  48. OWNERS_INFRA
  49. PATENTS
  50. PRESUBMIT.py
  51. presubmit_test.py
  52. presubmit_test_mocks.py
  53. pylintrc
  54. pylintrc_old_style
  55. README.chromium
  56. README.md
  57. WATCHLISTS
  58. webrtc.gni
  59. webrtc_lib_link_test.cc
  60. whitespace.txt
README.md

WebRTC is a free, open software project that provides browsers and mobile applications with Real-Time Communications (RTC) capabilities via simple APIs. The WebRTC components have been optimized to best serve this purpose.

Our mission: To enable rich, high-quality RTC applications to be developed for the browser, mobile platforms, and IoT devices, and allow them all to communicate via a common set of protocols.

The WebRTC initiative is a project supported by Google, Mozilla and Opera, amongst others.

Development

See here for instructions on how to get started developing with the native code.

Authoritative list of directories that contain the native API header files.

More info