Frankly, I had a hard time understanding what he was saying.

Literal

Frankly speaking, I [topic-は] he [subject-が] what [object-を] saying [quotative-のか] quite couldn't-understand.

なかなかわからなかった — なかなか + negative = 'not easily' or 'had difficulty.' 何を言っているのか is an embedded question. The speaker struggled to follow someone's speech.