Hi Lexi. The shame comes from societal pressures and ideals. It is very difficult to overcome. As you say, you (and we) are not hurting anyone. You may have to give serious thought to you marriage and pay special attention to your wife.
It may help to find a therapist who deals with transgender issues, if you can’t work it out.
I wish you the best on your journey.