Combine sugar, flour, and salt in top of a double boiler, stir in milk.
Bring water to a boil.
Reduce heat to low, cook, stirring constantly, 10 to 12 minutes or until slightly thickened.
Beat egg yolks until thick and lemon colored.
Gradually stir about one-fourth of hot mixture into yolks, add to remaining hot mixture, stirring constantly.
Cook, stirring constantly, until mixture thickens.
Remove from heat, and gently stir in peanut butter and vanilla.
Layer one-fourth of vanilla wafers in a 3-quart casserole, top with one-third of bananas.
Pour one-third of filling over bananas.
Repeat layers twice, ending with remaining vanilla wafers.
Spread top with whipped topping, if desired.
Chill.