If you are running a 32-bit operating system you may see anywhere between 2.5GB and 3.75GB of RAM depending on your computer configuration (onboard video may be stealing 512MB). Before you go any further I would check the manual for your computer to see if it does support this much RAM. Also I would put only each stick of ram in by itself and boot to see how much is recognized by windows. Doing this will test if a stick is bad too. To fully use the 4GB of RAM (even if some is being used by an onboard video card) you will have to use a 64-bit operating system.
I hope this information helps. Good luck.