Prepare banana pudding according to package directions. Place first five ingredients into a bowl; add buttermilk, oil, and egg. With a spoon mix all ingredients and beat until smooth.
Heat a frying pan or a griddle to 375 degrees. Spray cooking surface with non-stick spray. Pour 4 ounces of batter for each pancake, cook until hot and bubbly one side, flip and then cook for about another 30 seconds or so and remove from the cooking surface.
Place pancakes on a plate until they are ready to be used. Turning pancakes is made much easier if you spray your pancake turner with a non-stick spray. The turner does not stick to the uncooked dough of pancake when you are turning them. If batter seems to be too thick, add a little more buttermilk and mix well.
To assemble, lay one pancake on a plate, add about 1/4 cup of banana cream and spread on a pancake. Place about 4 or 5 banana slices onto banana pudding. Top banana pudding layer with another pancake. Add 4 or 5 slices of banana to the top of the pancake. Drizzle with caramel sauce. Top with whipped cream if desired.