RTC Peer Connection - receiving stream twice

Question

I have an instance of RTCPeerConnection with ontrack defined:

 newConnection.ontrack = receivedStream // remote

Once the SDP exchange is complete and the peer adds their local stream:

 connection.addStream(stream); // local

I see that receivedStream gets invoked twice per stream - Inspecting e.track shows me that the first invocation is for the audio track, and second is for the video track.

What's odd is that inspecting e.streams[0] and calling getTracks on this gives me two MediaStreamTracks - one for audio and another for video:

So I'm netting four MediaStreamTracks across two invocations of receivedStream despite calling addStream once.

receivedStream is here:

 function receivedStream(e) {
        var stream = e.streams[0]; // this gets invoked twice when adding one stream!
        if (!stream) {
            throw new Error("no stream found");
        };
        // this gets me the corresponding connection
        waitForStream(stream.id).then(function (connection) {
            // get element
            targetVideoElement[0].srcObject = stream; // this now gets called twice per stream - once for audio, once for video
        }).catch(function (error) {
           // log
        });
    }

_"I don't recall this happening in previous implementations"_ Can you include the code that you previously implemented at Question? — guest271314, Jul 30 '17 at 21:14
@guest271314 - it was old code that somehow successfully configured the `srcObject` of a target ` — SB2055, Jul 30 '17 at 21:16

score 3 · Accepted Answer · answered Jul 30 '17 at 21:18

3

You can perform the same procedure for each MediaStreamTrack, that is add the MediaStreamTrack to a MediaStream instance, then set .srcObject of HTMLMediaElement

const mediaStream = new MediaStream();

const video = document.querySelector("video");

for (const track of receivedMediaStream) {
  mediaStream.addTrack(track)
}

video.srcObject = mediaStream;

answered Jul 30 '17 at 21:18

guest271314

1
15
104
177

Thanks - though how do I go about pairing up the `video` and `audio` tracks then on the receiver side? Do you know of any way to transmit a/v without having to orchestrate these pairs manually? – SB2055 Jul 30 '17 at 21:19
@SB2055 Not sure what you mean? You add each `MediaStreamTrack` to a `MediaStream` instance, each stream outputs media at `HTMLMediaElement`. If you receive the `MediaStream` itself, instead of individual `MediaStreamTrack`s you can set the `.srcObject` to the received `MediaStream` – guest271314 Jul 30 '17 at 21:21
I mean I have a `newConnection.ontrack = receivedStream` function that accepts the incoming `MediaStreamTrack`, though it looks like there's no way to know which `stream` they came from. This means I need to tell `receivedStream` which `RtcPeerConnection` the stream is for, and then set up some kind of "wait" that says "when we have both audio and video tracks, push to the DOM". Is this right? – SB2055 Jul 30 '17 at 21:25
1

Ah! That's what I'd like - to receive the entire MediaStream instead of the tracks. How can I do that to avoid the above complexity? – SB2055 Jul 30 '17 at 21:26
Can you include the remainder of your code at Question? Have you tried using [`RTCPeerConnection.getRemoteStreams()`](https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/getRemoteStreams) or `.getLocalStreams()`? – guest271314 Jul 30 '17 at 21:26
I don't know if it will help much, but I provided the `receive` body above. The issue seems to relate to handling tracks vs MediaStreams. I want to receive an entire MediaStream so I don't have to compose one. – SB2055 Jul 30 '17 at 21:31
At your updated code, you can check if `.srcObject` is `null`. If true store a reference to the `MediaStream`, set the `MediaStream` at `.srcObject`, else if false add the `MediaStreamTracks` from `MediaStream`s at subsequent calls to the existing `MediaStream` reference. – guest271314 Jul 30 '17 at 21:32
I understand that there's a way to handle individual tracks; I'm wondering how to avoid it in the name of simplicity. – SB2055 Jul 30 '17 at 21:33
You code appears to receive two or more `MediaStream`s. You can pass a `MediaStream` to the `MediaStream()` constructor, though following that add the `MediaStreamTrack`s to the existing `MediaStream` using `.addTrack()`, e.g., `const stream = mediaStreamReference; if (videoElement.srcObject === null) { videoElement.srcObject = mediaStreamReference } else { for (const track of nextMediaStream) { mediaStreamReference.addTrack(track) } }` – guest271314 Jul 30 '17 at 21:35
I'm really not sure how else to make this clear; I'm trying to avoid any manual orchestration of a + v. I'd like to figure out how to receive a full MediaStream without manually handling or composing one with individual tracks. With manual work comes more complexity and then I have to worry about what happens when I receive two audio and one video tracks for a given connection etc. I'd rather just one-and-done it. – SB2055 Jul 30 '17 at 21:45
Your existing code appears to receive a full `MediaStream`, which you can set directly at the `.srcObject` of an `HTMLMediaElement`. If you want to add further tracks to the media playback add `MediaStreamTrack`s to the existing `MediaStream`, else you will need to reset the `.srcObject` to the new `MediaStream`, which will stop playback of previous stream. Another option would be to use `MediaSource` to play the media in the background, record the media, convert the resulting `Blob` to an `ArrayBuffer`, then call `.appendBuffer()` on `SourceBuffer`, but that is more code than at Answer – guest271314 Jul 30 '17 at 21:48
Yes I see that. My issue is that this callback gets called twice per stream. Inspecting `e.track` shows me one is of `kind: video` and the other is `kind: audio` however if I inspect `e.streams[0].getTracks()`, I see two tracks (a+v). So I'm very confused about why I'm netting four tracks across two invocations of `receive` when I call `addStream` once. – SB2055 Jul 30 '17 at 21:51
A `MediaStream` can contain either one `MediaStreamTrack` or two `MediaStreamTrack`s. If you receive two distinct `MediaStream` objects, then you can set the first `MediaStream` as `.srcObject` of `HTMLMediaElement`, subsequent `MediaStream` object you can iterate and add each `MediaStreamTrack` to the existing `MediaStream`. Do not believe it is possible to add an entire `MediaStream` object to a `MediaStream` except at the call to the constructor. – guest271314 Jul 30 '17 at 21:55
Thanks for your patience - just added a screenshot to my question to demonstrate my confusion. – SB2055 Jul 30 '17 at 21:56
_"MediaStream() Creates and returns a new MediaStream object. You can create an empty stream, a stream which is based upon an existing stream, or a stream that contains a specified list of tracks (specified as an array of MediaStreamTrack objects)."_ https://developer.mozilla.org/en-US/docs/Web/API/MediaStream – guest271314 Jul 30 '17 at 21:56
Do `MediaStreamTrack`s of `e.stream` and `stream.getTracks()` have the same `id` for each audio and video kind? Or do you actually receive two distinct `MediaStream` objects? – guest271314 Jul 30 '17 at 21:58
yes both invocations of `receivedStream` are with equal `e.streams[0]` - the ids are the same for both tracks across both instances. – SB2055 Jul 30 '17 at 22:03
Then you can check if the `HTMLMediaElement` `.srcObject` is `null`, if true set the value to the first `MediaStream`, else do nothing with second duplicate `MediaStream`, see https://stackoverflow.com/questions/45402263/how-to-send-getusermedia-recorded-stream-to-server-nodejs-realtime/. Then the question would arise as to why you are receiving, or appearing to receive duplicate `MediaStream` objects. – guest271314 Jul 30 '17 at 22:05
Yeah - thanks for the help around figuring that out. Think I should post a new question now that we have more information? – SB2055 Jul 30 '17 at 22:15
@SB2055 Would would new Question consist of? Why you appear to be receiving duplicate `MediaStream` objects? Does current Answer and comments resolve present Question? – guest271314 Jul 30 '17 at 22:20
@SB2055, I am also getting the duplicate 'track' event. Can you please share how did you solve this? – Prakhar Patidar Apr 21 '20 at 19:05
Is it me or this answer is absolutely irrelevant? – Hypothesis Jul 19 '21 at 17:10

score 1 · Answer 2 · answered Apr 19 '21 at 20:52

1

May be you have added more than one track in remote peer like here:

localStream.getTracks().forEach(track => peer.addTrack(track, localStream));

Each call to peer.addTrack( ) in the remote Peer produces and event in local peer.ontrack = ()

answered Apr 19 '21 at 20:52

Cesar Morillas

707
5
11

score 1 · Answer 3 · answered Apr 20 '21 at 05:15

the track event is fired once for each MediaStreamTrack. Hence, if you have two different tracks, such as an audio and a video, in a MediaStream, the ontrack event will fire twice. The stream property of the event of both events will be identical and you can attach the stream to the same audio or video element twice without consequences.

RTC Peer Connection - receiving stream twice

3 Answers3