There were layers on the Plan table buttons that prevented updates from Microsoft surfacing in the app. The layers were removed, and the Delete and New Plan button hidden again.

Now, when you open a project, the Open in Planner button works.

Layers that where removed: