Are you pressing Ctrl+# when you are setting a group of units to the control group, or when you want to select the control group? You only need to press Ctrl+# once, to create the control group, then you just press # to go back to that control group. Also, to go to your town centre its just T, not C...