In a medium pot, combine milk and sugar. Set over medium heat and stir until sugar dissolves. Remove from heat and whisk in chocolate peanut butter. Whisk until smooth and homogenous. Transfer to an airtight container and store in the fridge until cold.
Using a hand held mixer, whip chilled heavy cream to almost stiff peaks. Add 1/3 of chilled peanut butter-milk mixture. Using a large sturdy spatula fold into whipped cream. Add another 1/3, fold, followed by remaining 1/3. Fold until there are no longer any streaks of whipped cream.
Immediately transfer mixture to popsicles molds, filling molds all the way. Place in freezer for 15 minutes and then remove. Insert wooden sticks into popsicle mold cavities. Return mold to freezer and freeze for at least 4 hours to fully set.
To make chocolate shell: combine chopped chocolate and coconut oil in a small heat-proof bowl. Set over a pot of simmer water, creating a bain-marie (double boiler). Melt chocolate over low heat. Stir to combine. Once chocolate has melted, remove from heat.
Dip frozen bars into chocolate shell mixture and immediately sprinkle with crushed candy canes. The chocolate will quickly set, so work quickly! Consume and enjoy immediately.