Damn it. I am sorry but I made a mistake..
Using SAP Logon I enter HOSTNAME of my server where solution manager is running. HOSTNAME is not in my hosts file and thus can not be pinged or mapped. I can still connect because of my 2 saprouters and their respective hosts file. The aim here is to provide customers with an hostname but not with IPs.
I can connect fine with no problems as mentioned in OP. Only /nsolman_workcenter makes problems as it looks like solution managers expects an IP/hostname mapping from my machine instead of the machine where it is running. Is this changable? Or is there another solution?