No one wants to be ignored. Even if you have been avoiding this person, it's important for the reputation of you and your company that he not feel neglected.
1234, Main Street
Boston, MA 02123
<Recipient Address Goes Here>
I'm sorry that I've been unable to return your <call(s)> recently. <Reason(s).>.
In the future, if you need to reach me right away, <how you can be reached more easily>.
Thank you for your understanding and patience. I look forward to speaking with you soon.
encl: <List of enclosed items goes here>