You cannot do what you want to do.
For Server 2003, PS 4.5 is the same thing as XenApp 5.
If you want to upgrade to XenApp 5 for Server 2008 (which is a different product), you will need to use a new server. Citrix does not recommend or support upgrading the base OS that XenApp is running on.
If you want to "upgrade" from PS 4.5 to XenApp 5 for Server 2003, then just install the lastest License Server, the latest Web Interface and the latest Management Console updates. Even if you were to install XenApp 5 for Server 2003 from scratch on a new server, the Management Console still shows it as version 4.5.