The main consideration is the time taken to upload and download files, which depends on your network speed (mostly between the client and server, but also consider between the email router and CRM server if you use the email router).
Assuming that the netwrok speed is OK, then this will affect the overall size of the database, but that should not impact performance
Microsoft CRM MVP - http://mscrmuk.blogspot.com/ http://www.excitation.co.uk