。
You're really annoyed with me.
Literal
You [topic-は] me [target-に] very irritated-doing.
いらいらする is the verbal form of the mimetic いらいら, which depicts the prickly, on-edge feeling of irritation — like the sound of something abrasive scraping. The ~ている form marks an ongoing state of being irritated, not the moment of becoming irritated. に marks the target (the cause) of the irritation: '[irritated] at me.'