System.Speech.Recognition is a namespace not an assembly -
http://msdn.microsoft.com/en-us/library/system.speech.recognition.aspx You need to reference the assembly that contains that namespace in your project i.e. System.Speech.dll
The 5.1 & 5.3 that you're referencing are for the Speech API which is NOT Microsoft Speech Server.
In General yes, if you want to use the functionality of an assembly you will need it on the machine that you're using to develop the code.