The user needs to actually Sign Out of the CWA session. Also the entrie credentials are NOT cached by default in a way that a user who 'logs out' can be logged in again with suppliying a password.
The most recent username will still appear but not the password. Are your users also closing all instances of a the running internet browser when they are finished, or leaving some windows open?