to start to come apart or to fail
Oxford100% of films
The rest of me is fraying at the edges.
Definitions from the Oxford Advanced Learner's Dictionary, 9th edition