It really doesn't bother me that much.

Literal

Truly that-much mind [contrastive-には] become-not-[polite].

Built on 気になる ('to be worried / bothered / curious,' literally 'become feeling'). The interesting trick is that the topic/contrastive particle は is inserted inside the particle phrase: normally 気になる, but here 気には + ならない. This 'contrastive は within the particle phrase' foregrounds the negation — 'as for being bothered, (I) am not.' A subtle emphatic move.