locked
OCS Edge server on Windows 2008 - Important info for routing RRS feed

  • General discussion

  • Hi Everyone,

    We recently installed our Edge server, and while doing so, we ran into some routing / connection issues.  I asked around on this form, and based on the information here and on other MVP blogs (thanks again guys), we found the solution to our issue.  In order to help others experiencing the same issues, I decided to post our story here.  I hope it helps.

    Situation:

    - OCS 2007 R2 Standard installation (Windows 2008), Mediation Server (Windows 2008), ISDN gateway.
    - OCS 2007 R2 Edge server on Windows 2008 with 4 NICS, 3 for outside connectivity, 1 for connection to our private network.
    - Only a single NIC was configured with a gateway, as per recommendation in forums, blogs and probably also manuals.
    - Internal NIC received a static route to the internal network
    - Firewall rules according to specs (use the Edge Planning Tool!!)
    - No NAT
    - For testing we used certificates signed by our own CA in order to rule out (or limit) certificate issues.  CA was trusted by all test clients.  We later replaced these internal certificates by external certificates.

    Problem:

    - External users could logon to the server and send instant messages.  However, audio/video calls were impossible.

    Attempts at solution:

    - At first we believed that our firewall was interfering with the communications.  We checked all settings (some were indeed not 100% correct) and eventually we did not see any blocked packets.  Still we could not communicate.
    - In a moment of despair we opened all ports that we could for internal traffic going outwards.  Then audio/video was possible, further strenghtening our idea that the firewall was the issue.   Now, as you might now, Communicator uses Peer-to-Peer communications  if possible.  The client decides on different candidates for communication, as decribed here: http://communicationsserverteam.com/archive/2009/04/22/433.aspx.  We observed indeed that when all ports were open, OC would use P2P communications with the external user, bypassing the Edge server. Of course, this is not ideal, and we assumed that the client really was unable to communicate using the proper Edge channels, and opted for P2P instead.
    - Finally we tested opening telnet sesssions to the different Edge ports in order to determine the reason for the seemingly blocked packets.  We noticed that a telnet connection would open normally to the OCS ports when this was done on the Edge server itself.  When an external client on the internet opened a telnet to one of the OCS ports, it would not be able to open a connection.  This time we decided the firewall could not be the issue, as its logs showed the packet was not blocked.   There could only be one final solution:  routing issues.
    - Searching on routing issues with Windows 2008, we finally stumped on some blog entries, about OCS, multiple NICs and routing.  I remember these

    http://blogs.pointbridge.com/Blogs/schertz_jeff/Pages/Post.aspx?_ID=55
    http://blogs.pointbridge.com/Blogs/schertz_jeff/Pages/Post.aspx?_ID=33

    but also some others.  That finally pointed me into the right direction: routing changes in Windows 2008!

    See here:  http://technet.microsoft.com/en-us/magazine/2007.09.cableguy.aspx


    Cause of the issue  (as I understand it)

    - The packets arriving on the edge server coming from an external user were in fact NOT blocked and received correctly by the server.  Because the audio/video NIC did not have a gateway set, and Windows 2008 is set for a "strong host model", the answers were also transmitted on the same NIC... and going into the void....  they would not arrive at the external client.
    - When switching the server to a "weak host model", the answers now can leave via a different NIC (the one with the gateway set) and still reach their destination.  Problem solved.

    So, thanks again for all the posters on this forum, the different blog authors and pro-exchange.be.    They did not only help for this issue, but also cleared some questions I had about certificates.  After requesting the certs, federation and PIC were up and running in no time. :-)

    The only remaining thing to install now is a reverse proxy.  I would like to avoid ISA server, and read about utilities that do URL rewriting.  Any suggestions are welcome!  :-)

    Hope this helps others,
    Thanks,
    Best Regards
    Wim
    Wednesday, August 26, 2009 12:22 PM

All replies

  • Wim,

    Thanks for taking the time to post all this.  Incidentally I have blog draft on this topic as I have been researching recommend practices.  Typically the approach I have taken is to assign the same Default Gateway (external router) to all external Edge interfaces.  Yes, assigning multiple gateways is usually frowned upon, but those are typically scenarios where different default routes on multi-homed computers.  By simply setting these values on each external interface, you'll allow outbound traffic to reach the proper destination.

    The approach you took also works, but reverting Server 2008 to a Weak Host Model can have one important, undesirable impact: performance.  Especially now with R2, a consolidated Edge Server can more easily host all external roles on the same physical interface (since NAT restrictions have been loosened). this being the case, the only real benefit to using multiple physical interfaces for each external role is to increase bandwidth.  But reverting the server to Weak Host model will route all outgoing packets over the 'primary' interface.  So although all inbound traffic to each of the 3 roles is separated across physical NICs, all outbound traffic (IM, Conferencing, A/V, Federation, Desktop Sharing, etc) travels out that single interface.

    When I post that article I'll link to this thread discussion.
    Jeff Schertz, PointBridge | MVP | MCITP: Enterprise Messaging | MCTS: OCS
    Wednesday, August 26, 2009 2:19 PM
    Moderator
  • Hi Jeff,

    Thanks for your comment.  I am looking forward to your related blog post. 

    You are indeed raising a valid point about performance!  It did cross my mind for a second, but finally I decided not to care about this, for now, and afterwards I completely forgot about it.  Your comment is a good reminder. :-)  For now it will not be an issue as we currently have a limited number of users, and there are other bottlenecks in our setup that probably affect the performance more than the weak host model.  But I do agree with you that it might be a good idea to change our config so that it is in an optimal state.  I guess that is something for when I have more time.

    Thanks again,
    Best Regards
    Wim
    Wednesday, August 26, 2009 3:23 PM
  • I ran into this problem awhile back too (FYI - Wireshark or Network Monitor is incredibly valuable here and may save you some back and forth with firewall or network admins) where I saw the packets failing to go anywhere. Ever since then I've always gone with a single NIC + 3 IPs external interface deployment model because it doesn't have those issues, and also because I can't say I've ever seen the network utilization hit the capacity of a single gigabit NIC. I'm sure it happens in much larger environments, but I haven't seen it with the ones I work in. I think you'll be scaling out more Edge servers for processor and memory reasons before you need to for NIC performance. 

    Good info though. Multi-homing and routing to the Edge server are high up on the list of the problems for the OCS environments I've come in to troubleshoot. 
    Friday, August 28, 2009 4:05 AM
  • Hi Wim
    Thanks for your share!

    Regards!
    Wednesday, September 2, 2009 3:34 AM
    Moderator
  • FYI, I have a discussion going with a product specialist at Microsoft at the moment on this topic.  Just waiting to clarify a few things and I'll publish that blog article.
    Jeff Schertz, PointBridge | MVP | MCITP: Enterprise Messaging | MCTS: OCS
    Wednesday, September 2, 2009 1:36 PM
    Moderator
  • Blog article is posted as promised: http://blogs.pointbridge.com/Blogs/schertz_jeff/Pages/Post.aspx?_ID=78
    Jeff Schertz, PointBridge | MVP | MCITP: Enterprise Messaging | MCTS: OCS
    Thursday, September 17, 2009 2:22 PM
    Moderator
  • Hi Jeff,

    Thanks for the blog post.  It is a nice overview of the different options we have to set up OCS Edge on Win 2008!    This thread has become really interesting, and that is the purpose of this forum :-).

    That being said, I noticed on my environment that the new desktop sharing function using Office Communicator does not work when an internal and external user try to share each other's desktop.  But, I did not have time to take a detailed look at packets, firewall blocks or routing issues.  So I have no idea of it could be related to this issue or not.  I'll try to investigate that soon, and post in a new thread.

    Thanks again,
    Best Regards
    Wim
    Thursday, September 17, 2009 2:39 PM
  • Wim, a common reason for that is if internal clients can't both resolve and connect directly to the internal Edge interface over 443 and 3478.  Desktop Sharing is basically RDP over RTP so all A/V communications must be configured correctly between the Front-End and Edge servers.


    Jeff Schertz, PointBridge | MVP | MCITP: Enterprise Messaging | MCTS: OCS
    Thursday, September 17, 2009 2:59 PM
    Moderator