"What are you doing fidgeting in front of the house?" "Um… I was waiting for big brother to come home."
Literal
What did-[explanation-んだよ], house [gen-の] front [at-で] fidgeting [adverbial-と]. / Yeah... big-brother [subject-が] coming-home [nominalizer-の], was-waiting [explanation-んだ].
モジモジ is the mimetic for fidgety, hesitant, shy movement — a shy child standing around unsure what to do. お兄ちゃんがかえってくるの is a nominalized clause ('the (act/event of) big brother coming home') used as the object of 待つ. The pattern [verb]のを待つ ('to wait for the [verb]ing') is a standard way to express 'waiting for someone/something to do X.' 帰ってくる is 'to come home' with ~てくる toward the speaker.