Computer does not start sometimes!

I am facing a strange power issue with my computer.
The problem is, whenever I press the
button, the computer is not switching on. It remains like a dead
system. But after few hours, it boots by itself and works normally
until I shut down. When I shut down my computer, it shuts down
completely but the cpu fan keeps rotating and the power light remains
on, the system does not go to complete off state. I forcefully switch
off CPU and the story continues like when I press it does not boot,
after some hrs........!! Technical support at the manufacturer told me
to change powersupply, but it didnt help. then they asked me to change
the button which controls on/offf. It also didnt help... Now I am
really confused with what is going on... since my system works without
hanging even for several days when it got booted, I could not doubt on
motherboard also. Could anybody give any suggestions? I would
appreciate of any help as it is troubling me a lot.

Thanks