5 sites, 8 streams, All Unicast: What is the limit for Reflector
Hi again!
Today, we ran a Conference XP session with 5 sites (site A to E), 8 streams connected to the Reflector. Videos are streaming well but there seems to be some problems with audio.
Site A can hear site B-E clearly and loud but site B has trouble listening site C & E clearly. Audio is smooth but turned out to be very faint (e.g. ultra low volume kind of effect). Tested it on another client machine in site B and the same problem persists.
This not only happen to site B but also to another site. Ironically, site A, which hosts the Conferece XP servers, and together with another few sites are hearing all sites clearly. Any idea if Reflector machine has been maxed out or something?
Btw, we're running on a high speed university network so definately we have sufficient bandwidth. I've also chekced audio settings in Windows and Conference XP. If there is a bandwidth limit problem, I would expect audio to be choppy, but not smooth and "ultra low volume". Any explanation on this? Like does insufficient bandwidth or CPU power cause "ultra low volume" effect?
Answers
- Ok. Problem solved. It's a local problem. Audio cable to be specific. Thanks.
All Replies
Well, you can get different results with audio - either choppy or faint, depending on just how Network or CPU bound the machines are. Did you happen to check CPU utilization?
Also, levelling audio across sites is difficult. Have you tried A-B only, A-C only, B-C only, etc. to make sure audio was configured "about correctly"? It's a tough challenge.
Alternatively, you could have all sites connected at once, and then have A and only A speak and get feedback from all other sites whether A is sending audio at a low volume and needs to boost it in Syste audio properties. Circulate through all participants in that fashion.
We have done a max of 15 video streams and 7 audio streams and everything worked fine. You can check your performance counters to see if the data is arriving at 30fps for video and 22 for audio (I think, or match receiving fps to sending fps).
JVE
- Hmm... Is there a performance counter?
RtpSender and RtpStream performance counters will be the most interesting. They will allow you to see at what rate data is being sent / received and whether or not there are lost packets.
JVE
- Hi Jason,
I can't find any stream performance counters in Conference XP. Where is it located?
Anyway, I've run another conference test and he same problem persists, with all sites lound and clear except for one. So I guess it's highly probable a local problem. Click Start / Run
Type 'perfmon' (without quotes) and hit enter
Clear the current counter set with the 'white sheet' on the left (or Ctrl + E)
Choose View Report (or Ctrl + R)
Choose + (or Ctrl + I)
RtpStream for the Performance object
All Counters
All Instances or the instance you are interested in
Look for the Packets Lost counter.
JVE
- Check performance and all seem to be good with little packet drop.
I did an experiment placing another video codec instead of WMP and the sound turned out to be faint. So i guess it could be a codec problem.
Cheers. Why would a different video codec affect the audio?
Was the audio being sent at 22 frames per second? If not, that might explain the problem, even if there was no packet loss.
Did you try turning video off altogether and testing just audio? That might indicate a bandwidth problem if it sounds good.
If they are still faint, tell them to turn up their audio capture level.
JVE
- Sorry I don't mean video codec but audio/video codecs
I'll check if audio is sent at 22 frames/second.
Unfortunately, I've already asked everyone to tune up their audio capture level and it still didn't work. Audio problems usually result from one of the following 3 problems...
1 - Capture or playback volume are not loud enough
2 - CPU utilization is too high (generally 90+ %) on the sending or receiving side
3 - Not enough upstream bandwidth (turn off video and see if it gets better, frames per second is an indicator) on the sending side.
If you find something new, I'll be interested to hear about it.
JVE
- Hi Jason!
Tested another conference session today. It seems that audio is not "faint" all the while at the problematic site (X) but rather virtually "inaudible". I say virtually because there seems to be certain time when someone is talking at X, a little coarse sound can be heard.
The reason I thought it was "faint" was because the echo is being repeated at other sites and got sent back to me. Muting other sites have shown that the X audio was inaudible.
So testing was done today between site A, B and C. Site A, B and C were able to see and hear each other very well. Site X can also see/hear everyone but its audio is "breaking" from A, B and C. At site A, perfmon showed healthy reading from site X eventhough audio wasn't good. So I asked all video to be stopped and amazingly it turned out A and B get clear audio from X!!! But strangely, C still can't have audio from X.
Do remember that all sites are connected at high speed network with some on fibre optics. The audio from X is uncompressed. Using perfmon, I've measured the data frame rate at site A and C. Both showed healthy reading on data coming from site X (e.g. good frame rate and no data loss). The frame rate for X is 160+ since the audio is uncompressed while the other sites are 22. All sites are connected to a private reflector located at site A.
It seems like C is receiving all audio packets from X but somehow audio is not played properly. So I tested with another PC in site C, but the same problem persists. Site A, B and C are geographically located close to each other while site X is in another state.
So the question is:
- Could there be problem with RAW audio and PC at site C couldn't read it? But 2 PC can't hear audio!
- If site A and B are able to hear X on reflector, why not C? Not to mention C shows healthy reading from all sites in perfmon.
- When A, B and C were all not hearing X, perfmon still showed healthy readings on data coming from X.
Just to note, so far, I haven't ask X to run perfmon to do data measurement yet. Perfmon is only done in A and C. Just one comment: Jason mentions that when CPU utilization is too high it can cause audio to break. I have seen this too on multiple occasions. You mention that when you stopped the video, the audio from X got better for most of the receivers. This sounds very much like CPU of X was overdriven.
Regards,
Fred Videon
Hi Fred! I can very much agree with you and beginning to suspect the CPU utilization in X is high.
Still, one fact might contradict this. After chopping off video in X and reception was much better. The audio coming from X can finally be heard in site A, B but still not C. Remember, all sites are connected to Reflector.
So the question: Shouldn't the Reflector duplicate the audio in X to A, B and C? And if A, B can hear X so should C? The fact is C can hear A, B very well. Unless X is actually the one reproducing audio for A, B and C.
To clarify the question again, who does the audio replication job to A, B, C? The Reflector or X?
The replication happens in the reflector.
Did I understand that C was receiving audio bytes from X, but couldn't play them. Does this happen even if you turn off A and B?
These are the things I would try:
-reboot C and X
-Use the Audio/Video form under the Settings menu on both C and X to verify that you can hear the local audio.
-Connect C and X again using the Reflector. Make sure everything is unmuted and has good volume levels.
-If it still doesn't work, I'd try reinstalling ConferenceXP and Windows Media Player 11 on both C and X and try again.
-If it still doesn't work, I'd look for any other unusual software or drivers installed on C and X and try uninstalling them, then try again.
-If it still doesn't work, I'd look for replacement hardware or rebuild the OS, etc.
Let us know what you find.
Fred
If audio got better when video stopped, it is either CPU or network. Please check each of those for X. A, B, and C, should be receiving at the same frames per second as X is sending. If X is on the Reflector, I would recommend compressed audio instead of uncompressed, or make sure the bit rate is turned down. That can be a lot of data being forwarded (depending on who is on the cloud with the Reflector and who isn't).
Sorry this debugging is so user-intensive.
JVE
- Ok. Problem solved. It's a local problem. Audio cable to be specific. Thanks.
