Broker nodes to compute nodes - optimal ratio RRS feed

  • Question

  • Hello HPC Team,

    Have a simple question that I can't seem to find an answer to anywhere: for a mid-sized cluster (80 compute nodes) that runs only SOA jobs, how does one go about calculating the optimal ratio of broker:compute nodes? 

    e.g. is there a simple formula / ratio or a reasonable heuristic to use? Any thoughts on this would be greatly appreciated!

    Thanks in advance!

    Tuesday, July 30, 2019 6:39 PM

All replies

  • Hi phil954,

    Generally we recommend 1 broker node per 50 compute nodes, but it totally depends on you machine spec and real work load. The practical approach is to monitor the perf counters on the broker nodes during the work load and see if any bottleneck with regard to CPU/Memory/Disk/Network IO is reached. If so, you may want to add more broker nodes to offload multiple SOA sessions.


    Yutong Sun

    Thursday, August 1, 2019 4:00 AM
  • Hello Yutong,

    Thanks for your response! I'm very surprised to hear about the 50:1 heuristic, I would've expected this to be much lower. We have a ton of SOA jobs running daily and have started to get broker timeouts, etc. I've dialed up the ratio to 10:1 (1 broker : 10 compute nodes) and these problems have largely gone away, hence my surprise at the above.

    That said, good point re: the perf counters - I will ask for those to be monitored and will go from there. Thanks again!



    Thursday, August 1, 2019 2:14 PM