Move Call::receive_time_calculator_ to network thread This CL moves the usage of receive_time_calculator_ in Call from the worker thread to the network thread. It extracts the worker thread logic of DeliverRtpPacket into DeliverRtpPacket_w to allow clearer thread annotations. Also includes new g3doc for a simple overview of the RTP receive flow. Bug: webrtc:11993 Change-Id: I143a548888474b4ebd14a9041d5cb649738e0775 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/454304 Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/heads/main@{#47119}
diff --git a/call/call.cc b/call/call.cc index 40a35c9..a7fd3e3 100644 --- a/call/call.cc +++ b/call/call.cc
@@ -347,6 +347,11 @@ void DeliverRtcp(MediaType media_type, CopyOnWriteBuffer packet) RTC_RUN_ON(network_thread_); + void DeliverRtpPacket_w(MediaType media_type, + RtpPacketReceived packet, + OnUndemuxablePacketHandler undemuxable_packet_handler) + RTC_RUN_ON(worker_thread_); + AudioReceiveStreamImpl* FindAudioStreamForSyncGroup( absl::string_view sync_group) RTC_RUN_ON(worker_thread_); void ConfigureSync(absl::string_view sync_group) RTC_RUN_ON(worker_thread_); @@ -443,7 +448,8 @@ RepeatingTaskHandle receive_side_cc_periodic_task_; RepeatingTaskHandle elastic_bandwidth_allocation_task_; - const std::unique_ptr<ReceiveTimeCalculator> receive_time_calculator_; + const std::unique_ptr<ReceiveTimeCalculator> receive_time_calculator_ + RTC_GUARDED_BY(network_thread_); const std::unique_ptr<SendDelayStats> video_send_delay_stats_; const Timestamp start_of_call_; @@ -1367,17 +1373,7 @@ MediaType media_type, RtpPacketReceived packet, OnUndemuxablePacketHandler undemuxable_packet_handler) { - if (!worker_thread_->IsCurrent()) { - worker_thread_->PostTask(SafeTask( - task_safety_.flag(), - [this, media_type, packet = std::move(packet), - handler = std::move(undemuxable_packet_handler)]() mutable { - DeliverRtpPacket(media_type, std::move(packet), std::move(handler)); - })); - return; - } - RTC_DCHECK_RUN_ON(worker_thread_); - RTC_DCHECK(packet.arrival_time().IsFinite()); + RTC_DCHECK_RUN_ON(network_thread_); if (receive_time_calculator_) { int64_t packet_time_us = packet.arrival_time().us(); @@ -1388,6 +1384,28 @@ packet.set_arrival_time(Timestamp::Micros(packet_time_us)); } + if (worker_thread_->IsCurrent()) { + RTC_DCHECK_RUN_ON(worker_thread_); + DeliverRtpPacket_w(media_type, std::move(packet), + std::move(undemuxable_packet_handler)); + } else { + worker_thread_->PostTask(SafeTask( + task_safety_.flag(), + [this, media_type, packet = std::move(packet), + handler = std::move(undemuxable_packet_handler)]() mutable { + RTC_DCHECK_RUN_ON(worker_thread_); + DeliverRtpPacket_w(media_type, std::move(packet), std::move(handler)); + })); + } +} + +void Call::DeliverRtpPacket_w( + MediaType media_type, + RtpPacketReceived packet, + OnUndemuxablePacketHandler undemuxable_packet_handler) { + RTC_DCHECK_RUN_ON(worker_thread_); + RTC_DCHECK(packet.arrival_time().IsFinite()); + NotifyBweOfReceivedPacket(packet, media_type); // Packets that are successfully demuxed are logged by their respective
diff --git a/g3doc/rtp_receive_flow.md b/g3doc/rtp_receive_flow.md new file mode 100644 index 0000000..24ce1c9 --- /dev/null +++ b/g3doc/rtp_receive_flow.md
@@ -0,0 +1,116 @@ +# WebRTC incoming RTP packet flow + +This document details the flow of incoming (receive) RTP packets across the WebRTC codebase, tracing the path from the network socket to the video receive stream. + +## Threading Model +* **Network Thread**: The majority of the initial receive and demuxing flow occurs synchronously on the network thread to minimize delays. +* **Worker Thread**: Packet processing, dispatching encoded frames/buffers to jitter buffers. Actual decoding tends to happen on different threads however. + +--- + +## Call Stack and Component Flow + +### 1. Network / Socket Layer 🌐 *(Network Thread)* +Packets arrive from the network via UDP or TCP sockets. Platform-specific socket implementations trigger an event when bytes are ready to be read. +* **`AsyncUDPSocket` / `PhysicalSocket`** + * `OnReadEvent` + +### 2. Transport Routing 🛤️ *(Network Thread)* +The bytes move up through the ICE/STUN/TURN connection channels and DTLS layer. +* **`P2PTransportChannel`** -> `OnReadPacket` + * Routes via **`Connection::OnReadPacket`** +* **`DtlsTransport`** -> `OnReadPacket` + * Decrypts SRTP packets into cleartext bytes if necessary. + +### 3. RTP Transport & Parsing 📦 *(Network Thread)* +The raw bytes reach the RTP core structures where they are parsed into proper C++ WebRTC objects. +* **`RtpTransport::OnRtpPacketReceived(const ReceivedIpPacket&)`** + * Converts the byte buffer string into a `RtpPacketReceived`. + * Extracts header extensions based on the active `RtpHeaderExtensionMap`. +* **`RtpTransport::DemuxPacket`** + * Passes the parsed object down to the demuxer tree. + +### 4. RTP Demuxer (Level 1) 🔀 *(Network Thread)* +* **`RtpDemuxer::OnRtpPacket(const RtpPacketReceived&)`** + * **Demuxing Happens Here**: This demuxer maps the packet using rules like **MID**, **RSID**, or **SSRC**. + * Routes the packet to the matched `RtpPacketSinkInterface` (usually a `BaseChannel`). + +### 5. BaseChannel & Media Layer 📺 *(Network Thread)* +* **`BaseChannel::OnRtpPacket(const RtpPacketReceived&)`** + * Implements `RtpPacketSinkInterface`. Serves as the base conduit for specific Media Channels (audio or video). +* **`WebRtcVideoReceiveChannel::OnPacketReceived(RtpPacketReceived)`** + * Delegates the received packet directly up to the `Call` interface: `call_->Receiver()->DeliverRtpPacket(...)` + +### 6. Call Interface & Demuxer (Level 2) 📞 *(Network Thread -> Worker Thread)* +* **`Call::DeliverRtpPacket`** + * Evaluates `receive_time_calculator_` directly on the Network Thread to avoid jitter/delay. + * **Thread Hop**: Dispatches a task to the **Worker Thread** and invokes `Call::DeliverRtpPacket_w`. +* **`Call::DeliverRtpPacket_w`** + * Executes on the Worker Thread. + * Employs a secondary, internal demuxer logic to find the specific receive stream that matches the SSRC. + * Calls the appropriate video or audio receive stream. + +### 7. Stream Receiver 📥 *(Worker Thread)* +* **`RtpVideoStreamReceiver2::OnRtpPacket(const RtpPacketReceived&)`** + * Executes on the Worker Thread. + * Extracts metadata, processes NACKs/RTCP feedback based on the incoming sequence numbers. + * Inserts the payload into the `PacketBuffer` (Jitter Buffer) to wait for frame completion. + * Once a complete video frame is assembled (or for deeper stream operations), the `VideoFrame` is decoded by `VCMGenericDecoder`. + +--- + +## RTP Receive Packet Flow Diagram + +```text + [Network] + | + v ++-----------------------+ +| AsyncUDPSocket / | (Network Thread) +| PhysicalSocket | ++-----------------------+ + | + v ++-----------------------+ +| P2PTransportChannel | (Network Thread) +| / DtlsTransport | ++-----------------------+ + | + v ++-----------------------+ +| RtpTransport | (Network Thread) ++-----------------------+ + | + v ++-----------------------+ +| RtpDemuxer | (Network Thread) ++-----------------------+ + | + v ++-----------------------+ +| BaseChannel | (Network Thread) +| & MediaReceiveChannel| ++-----------------------+ + | + v ++-----------------------+ +| Call::DeliverRtpPacket| (Network Thread) ++-----------------------+ + | + | Thread Hop + v ++-----------------------+ +| Call:: | (Worker Thread) +| DeliverRtpPacket_w | ++-----------------------+ + | + v ++-----------------------+ +| Stream Receiver | (Worker Thread) +| (e.g., RtpVideo- | +| StreamReceiver2) | ++-----------------------+ + | + v + [Decoder] +```
diff --git a/pc/peer_connection.cc b/pc/peer_connection.cc index cd4ad8b..e2a4ade 100644 --- a/pc/peer_connection.cc +++ b/pc/peer_connection.cc
@@ -3181,16 +3181,14 @@ absl::AnyInvocable<void(const RtpPacketReceived& parsed_packet) const> PeerConnection::InitializeUnDemuxablePacketHandler() { return [this](const RtpPacketReceived& parsed_packet) { - worker_thread()->PostTask( - SafeTask(worker_thread_safety_, [this, parsed_packet]() { - // Deliver the packet anyway to Call to allow Call to do BWE. - // Even if there is no media receiver, the packet has still - // been received on the network and has been correcly parsed. - call_ptr_->Receiver()->DeliverRtpPacket( - MediaType::ANY, parsed_packet, - /*undemuxable_packet_handler=*/ - [](const RtpPacketReceived& packet) { return false; }); - })); + RTC_DCHECK_RUN_ON(network_thread()); + // Deliver the packet anyway to Call to allow Call to do BWE. + // Even if there is no media receiver, the packet has still + // been received on the network and has been correctly parsed. + call_ptr_->Receiver()->DeliverRtpPacket( + MediaType::ANY, parsed_packet, + /*undemuxable_packet_handler=*/ + [](const RtpPacketReceived& packet) { return false; }); }; }