If you answer a letter or advertisement, you write to the person who wrote it.
回复Did he answer your letter? 他回复你的信了吗?
An answer is something that you say when you answer someone.
回答Without waiting for an answer, he turned and went in through the door. 他没有等候应答,转身进了门。