Swim?
There is no way you are gonna swim in a cold, fast river, with or without your waders filled with water. The best you can do is to float on your back, feet first down the river and try to back paddle and steer your drift back to shore, or clear of anything you might get caught up in & drown. Good luck with that. Carry a knife that you can get to in the event you need to slice open your waders to let the air out & get your feet down, or drain the water out once you get close enough to crawl out on the bank.
More than likely if you take a dump, it will be the result of finding yourself in a current seam you misjudged and being swept off your feet in a stupid move while trying to get out. And it doesn't take much in a situation like that to lose your footing,,,on just one foot,,,for a few seconds,,, and down you go.
This time of year, you never know whats gonna come floating down the river at ya. Keep a sharp eye out between casts & don't be so far out you can't get out of the way before something knocks you off your feet.