Preheat oven to 350 degrees and line a large baking sheet with foil. Lightly grease and set aside.

Remove the stems from the caps. Place the caps on the baking sheet and lightly brush with olive oil. Finely chop the stems and set aside.

In a large skillet, heat the olive oil over medium heat and turn on your vent fan (the next step might cause you to cough). Once the oil is hot, add the jalapeño, chopped mushroom stems, a pinch of salt, and a few cracks of pepper, and stir until softened, about 4 minutes. Add the chopped bacon and stir until fragrant and slightly crisp—an additional 2-3 minutes. Remove from the heat, set aside, and allow to cool for about 5 minutes.

In a medium mixing bowl, combine the cream cheese with the cooled jalapeño mushroom mixture, tasting for additional salt and pepper if needed. Stir until combined. Reserve 2 tablespoons of Parmesan and stir in the remaining cheese. Spoon about 1 tablespoon of the mixture into the mushroom caps, then sprinkle the tops with the remaining Parmesan.

Bake for 15-20 minutes, or until the filling is warm and tops are golden. If desired, broil for 1-2 minutes to brown the tops. Serve warm.