My father scolded me for not being punctual.

Literal

Father [topic-は] me [subject-が] time [object-を] keep-not [quotative-と] saying scolded.

時間を守らないと言って叱った — 時間を守る (to keep time / be punctual) is negated, and the scolding is connected by the te-form.