Is there a penalty to me if I do not enroll in both Medicare Part B and a Medicare D prescription drug plan at retirement?

Medicare does assess a penalty to a retiree who enrolls later than their “initial retirement period,” which refers to the period when they are first eligible for Medicare retirement benefits. Any penalty assessed by Medicare for late enrollment is a lifetime penalty on the monthly cost for Medicare Part B or on the monthly cost for Medicare D prescription drug plan. You pay this penalty to social security.