。。
You don't need to be scared. He's not trying to hurt you.
Literal
Particularly scared-become-not-even fine [assertion-よ]. Nothing he [topic-は] you [to-に] harm give-[volitional-よう] [explanatory-の] is-not because [agreement-ね].
別に怖がらなくてもいいよ uses 別に + なくてもいい — 'you don't particularly need to be afraid.' The second sentence explains why: 危害を与えようというのではない — 'it's not that he intends to cause harm.' The のではない negates the intention, and からね closes with a reassuring 'because, you see.' A comforting, multi-layered utterance.