She came to Japan to study Japanese.

Literal

She [topic-は] Japanese [object-を] study [in order to-volitional-ようと] Japan [to-に] came.

勉強しようと is the volitional form (~しよう) + と (without する), expressing intent compactly: 'in order to study, with the intention of studying.' A more concise alternative to 勉強しようとして ('intending to study') or 勉強するために ('in order to study'), with a slightly more dynamic feel.