The main thing a director server will protect you from is a denial of service attack.
If the server(s) where users first authenticate is a pool then any attempt to overload the server will directly affect those users hosted in that pool.
If you're comfortable that this won't be a problem then you may not require a director server
Make sure you consider the additional load on the pool server(s) caused by the additional authentications it will be performing.