You need a little more information. The most fundamental question is: 80% in number of unique visits, or in number of pages requested?
The sequence goes:
--visitor requests a page
--site shows Cookie message
--visitor says "Yeah, yeah, I understand, show me the ### page already"
--site makes additional cookie saying "visitor has agreed to accept cookies" (this line is a guess on my part, because otherwise you'd have to show the same message on every single page)
--site serves page
--site serves additional pages as requested (again, this is where I'm guessing that there are no further Accept-cookie messages)
At what point in the sequence is the drop happening? Are users declining to follow through on their initial request, or are they visiting the first page but not continuing to additional pages?