。
The cold really gets to me.
Literal
I [to-は] cold [subject-が] terribly affects.
こたえる (in the 堪える reading when meaning 'to be hard on someone') is used for things like cold, heat, work, illness taking a toll. Takes the sufferer as a topic (here 僕には) and the source as the subject. ひどく ('terribly, severely') intensifies. An idiomatic way to express vulnerability to the elements.