I was baptized when I was 8 and was very active up until 16, and then very inactive for about 10 years before coming back.
I think my wife and I took the Temple prep class 5 or 6 times before finally doing it. (the problem was me, not my sweetie who is a convert).
Anyway, I'm not certain after all that wait that I was still ready and sure I wanted to go through with this. I'm not sure why. The thought of being sealed to my wife and 2 year old son was motivational to say the least. But finally after much prayer we committed to going and did so.
As I reflect back on that time, I still can not say why I was hesitant. But the older I am getting, and the more I go to the Temple, the more convinced I am that satan had a big hand in this. I have noticed every time we plan a Temple outing there are suddenly all kinds of things that pop up out of no where to try and keep us from going. We finally came the the conclusion this was satan using his means to try and stop us. We have finally come to expect it, and go regardless. We have always been blessed for our efforts.
I would suggest deep sincere prayer, fasting, and waiting on that still small voice to guide you. But when you get that confirmation, act on it quick. Don't let the advesary stop you from doing what the Lord would have you do.
Prayers for you to do the right thing Brother.