Everything Riffle Hitch just said. If you come into Oregon from the south, you cross one famous river after another as you traverse north on I-5. The first time I made that drive, I couldn't believe it. The Rogue, the South, followed by the North Umpqua, the South Santiam, the McKenzie. The Scenic loop that follows the Rogue up to Crater lake & back down along the N. Umpqua is well worth the trip. The highway 1 loop around the Olympic Peninsula, the Bald Eagles, the big trees, the rivers, is like nowhere else. Vancouver Island has First Nation museums & totem poles as well as a few rivers.
And then, of course, there is the Haig Brown house on the Campbell River.
The Oregon coast, as nice as it is, IMHO cannot hold a candle to the OP. The Deschutes Canyon is kind of out of the way, off the beaten path. But that's what preserves the Deschutes. Not to completely rule out fishing, certainly pack a rod & take advantage of whenever time presents the opportunity but, were it me, knowing what I know now, rather than a fishing trip, (which I would do later as a dedicated fishing trip) I would try to book a white water day trip. The family will probably enjoy that more than sitting in a boat, not catching fish, all day.