I decided to go for a worn down look for the door, or to at-least start experimenting with a more damaged look and how id go about it. To help me with this i downloaded a Zbrush pack to save time, including lots of nice high quality alphas. Download here
I baked the door down, I didn't get the effect I wanted. The worn parts of the wood were not defined enough with the smart material set up I was using. I decided to bring the alphas into Substance Painter, there I played around with them to get a more worn down effect.