I have an 04 with almost 40K miles that has been fine. It had the brakes done as probably all 04s have had done. Other than that no real issues, a few small things but overall I can't complain. In the summer I tow about 6K lbs. without any problems at all.
The rear ends in the early 04s didn't come with synthetic fluid from the factory so this may have caused some of the rear end failures. There was a running change at some point that switched it to synthetic out the door. I switched mine early on and so far, so good.
If you look at the competition up until this year, the Titan is a far superior truck even taking into consideration some early teething problems. The Titan absolutely raised the bar. Only the most recently redesigned trucks (Tundra and Silverado/Sierra) are able to truly compete with it. So if you're looking used then the Titan is the way to go.