A design system that nobody contributes to is a graveyard with a build pipeline.
We have learned to treat the design system as a product with its own roadmap, owners, and feedback loop. That changes the cost calculus completely.
The biggest win is not pixel consistency. It is the speed at which a new feature can be sketched in code because the primitives are already there, audited, and trusted.
If the system slows you down, that is a design system bug. Fix it the way you would fix any other product bug.