Creamy Caramel Milksicles

1 1/2 cups low-fat milk
1/3 cup granulated sugar
2 tablespoons cornstarch
1 1/2 teaspoons vanilla extract
1/3 cup caramel ice cream topping at room temperature
6 (3 ounce) plastic or paper cups or 6 (2.5 ounce) freezer pop molds
6 popsicle sticks

Combine sugar and cornstarch in a medium saucepan and gradually whisk in milk. Simmer over medium heat until thickened, about 2 minutes, whisking frequently. Remove from heat; stir in vanilla extract. Transfer to a large shallow bowl; place in refrigerator for about 40 minutes, stirring occasionally. Drop heaping teaspoonsful of caramel topping randomly over pudding. Gently swirl into pudding. Spoon mixture into plastic cups; place cups in an 8-inch baking pan or pie pan. Place popsicle sticks in centers. Freeze until firm.