Cream together the softened unsalted butter, packed brown sugar, and granulated sugar in a large mixing bowl using an electric mixer until light and fluffy (about 3-4 minutes). Beat in the large eggs and vanilla extract until smooth.

In a separate bowl, whisk together the all-purpose flour, baking powder, baking soda, and salt. Gradually add to the butter mixture, mixing on low speed until dough forms.

Combine the brown sugar and cinnamon in a small bowl, then mix in milk until the filling is moist and spreadable.

Roll out the dough to about 1/4-inch thickness on a floured surface. Cut into rectangles approximately 3x4 inches.

Place a teaspoon of filling onto the center of half of the dough rectangles and top with remaining rectangles, sealing edges.

Preheat oven to 350°F (175°C) and bake cookies on a parchment-lined baking sheet for 12-15 minutes until lightly golden.

Let cookies cool for 5 minutes on baking sheet before transferring to a wire rack. Prepare icing by mixing powdered sugar and milk to desired consistency, drizzle over cooled cookies, and add sprinkles if desired.