Frankly, I can't agree with you.

Literal

Frankly speaking, I [topic-は] you [to-に] agree can't.

同意できない (can't agree) uses the potential negative of 同意する (to agree). The speaker is direct about their disagreement.