Thanks for the response Ming.
I see now that I haven't explained the situation very well.
Let me try and explain it a little better:
Say all of our grid machines are quad-processors with 8GB of RAM (2GB per core).
For the vast majority of our workloads running 4 "tasks" per machine easily fits within the RAM available.
Occasionally we have a job (SOA session) where each "task" will need, say, 4GB of RAM each. In these cases we want two tasks to run per node. If we resource the job (session) by node then we have poor utilisation of our grid resources (1 task per node) and the job will take twice as long to run than the optimal distribution of tasks (2 tasks per node).
I hope that illustrates better the problem we are trying to address.