Preheat oven to 350 degrees F. Grease an 8 or 9 inch baking dish, square or round and set aside.
Combine the brown sugar with the peaches. Mix with your hands, adding pressure to release some of the juices (but don’t mash them into a pulp). Set aside.
In a mixing bowl, combine the flour, baking powder, and salt. Whisk to combine.
In a separate bowl, combine the white sugar, melted butter, and the buttermilk. Whisk until smooth, then stir in the egg, egg yolk, and vanilla.
Incorporate the flour mixture into the wet ingredients, adding about ¼ cup at a time, until a smooth batter forms. Pour into the prepared pan and smooth the top using a spatula. Spread the peaches over the top evenly, juices included, then place in the oven for 20-25 minutes, or until a cake tester inserted into the center of the cake comes out clean.
Allow cake to cool slightly before serving with any additional summer fruits, or with whipped cream, vanilla ice-cream, or a sweetened crème fraîche.