If the Closet icon is greyed out or does not appear next to the Settings button, it means the Closet is locked in the scene you are currently on. Keep playing! When the Closet icon is visible and colored in, you will be able to access the Closet.
 
Please note: Some Books do not have a Closet feature, and specific outfits can only be accessed at select points in the Book.