My father let me go swimming.

Literal

Father [topic-は] me [subject-が] swimming [to-に] go [nominalizer-のを] permitted [for me-てくれた].

Combines the permission pattern 許す with the benefactive てくれた, expressing that the father granted permission as a favor. 泳ぎに行く uses the masu-stem 泳ぎ + に for 'to go swimming' (purpose).