。
You don't have to reply, you know.
Literal
Reply [contrast-は] don't-do-even [no problem-構いません] [assertion-よ].
~ても構いません is a polite way of saying 'it's fine even if you (don't)' — here ~なくても構わない with polite ません. The contrastive は on 返事 singles out replying from other possible actions: 'as for a reply specifically, it's OK not to.' The final よ reassures.