digital coaxial versus composite question

[katiehlvr1]

I have a phillips all in one system and need some help. I have a samsung br player connected to the the phillips which decodes dts and DD.
My question is should I hook up RCA red and white from BR to Receiver or digital coaxial from BR to reciever? Does it matter? I am not getting true HD sound and I realize that but if my receiver has the decoder in it does it matter which cables I use? Will I still get the standard Dolby Digital 5.1 regardless of which cable selection I use? I know i may get a little better sound using the digital audio coaxial but this is for my own knowledge. Regardless, if I use the Rca cables or the coaxial audio cable and plug it in to my receiver am I going to get the standard 5.1 dolby digital or will that only happen if I use coaxial?
Also I am assuming since I do not have an HD receiver I should set it on Bitstream correct?
Please and info is greatly appreciated. My phillips does not have the old school 5.1 setup with the jacks in back for 5.1. Since it has the decoder in it it only has optical and coaxial along with all the rca inputs for tv, vcr, dvd etc.
I wanted to clarify that because I know you can get some codecs if you have that on your receiver but I do not.
any help greatly appreciated
thanks
Corey