We hurried so we wouldn't be late for the concert.
Literal
We [topic-は] concert [to-に] not-be-late [so-as-to-ように] hurried.
The ように+[verb] pattern marks purpose — '[verb] so that [clause]' or 'in order to achieve [clause].' 遅れないように急いだ = 'hurried so as not to be late.' 遅れる ('to be late') takes に for what one is late for. A clean purpose-action structure.