Lay bread slices onto a large sheet pan in a single layer. Set aside.
Mix 5 whole eggs, 3 yolks, 1 tsp vanilla, 1 tsp kosher salt, heavy cream, and half and half together in a bowl; whisk until combined. Pour over the bread slices. Flip the slices to ensure they are fully saturated. Allow bread to soak for at least 30 minutes (or up to 1 hour if your bread slices are especially thick) in the refrigerator.
With butter or cooking spray, grease a round 12’’ (or similar size) ceramic dish or baking pan.
Preheat oven to 375 degrees F.
With about 10 minutes of soaking left, begin to make the brown sugar syrup.
In a medium skillet, melt the butter and brown sugar together. Once melted add 1 tsp of vanilla and 1 tsp salt, cinnamon and nutmeg and stir until smooth. Remove from heat and pour into the bottom of the greased dish.
On top of the brown sugar mixture, in a single layer, spread out the banana slices and sprinkle with walnuts. Set aside.
Using kitchen shears, cut the soaked brioche slices into thirds, then layer them in a circular fashion, starting on the outer rim and working your way towards the center, on top of the bananas, until they are covered. Sprinkle the 3 Tbsp of brown sugar on top, then place in the oven on the center rack.
Bake for 25 minutes, or until the bread is golden on top.