Found the issue. No matter what the settings are, it is also required to change the default privileges on the /admin/manage/privileges global setting for all type of users.
I was expecting that changing the setting will update that list, or at least a notification on the setting to clarify how it works.