It sounds like the metal framework in the dash, I remember reading a post back in '05 that some '04's had loose screws in the dash from the factory. I think the fact that you can stop the rattle by pulling on the framework confirms this, also when you say you can hear it creaking when using the brakes. I think the only way to fix the issue is like you said, remove the dash. I would start by removing the glove box then the left and right side lower panels. Next the center section and radio bezel, it may be posible to access all the framework bolts from there, I'm not sure. The driver side lower panel has two screws on the bottom then just pops off. The radio bezel just pops off, I get my finger nails under the top edge and pull. The rest of the center section pops out too, pull it straight out and up a little. I have a bench seat so the cup holder and the heater vents pop out then there is a screw on both sides of the cup holder frame and two screws on the top of the cup holder frame. Get a flash light and look for every bolt and screw you can find. I would also tap on every thing in there and put some sound deadening material on every thing that rattles or vibrates while you are in there.