This is an easy one.
Do not contact the manager and let it go.
Do not confuse this with a situation like the others described on this thread where a server did something wrong (e.g. entered a tip that was higher than the tip you wrote on a cc slip).
As it stands, you left a nice tip and the server appreciates it. It's not like someone accidentially dropped a $100 bill under the table and the server is pretending it's there's when obviously it wasn't.
Your party left a believable amount of money ON THE TABLE.
There are many instances
where friends and family fight over a check, one person refuses to allow the other to contribute at all so the other person says- "well, I'm contributing no matter what so I'm leaving it on the table." And the other person says "go right ahead, I guess the server is going to get a great tip". And they all say "fine-
so be it".
I believe you will come off looking badly that you tried to make it seem like a hard working person did something wrong when the mistake was yours and that you did it over $20.
Sorry to ramble. I just feel very strongly about this one having been a waiter all through college and grad school and having managed a restaurant one summer.
Let it go.