to worry or upset sb very much
be the death of sb
Those kids will be the death of me.
Definitions from the Oxford Advanced Learner's Dictionary, 9th edition