As you progress through The Crypt, you will come to this staircase:



Attack the blocks to open the way:

This one is found in The Field, on the way to The Clocktower. Attack the blocks to descend:





The lower level with the epitaph will lead you into The Cave if you head right, so you may be able to enter this area from that direction, but I haven't tested it.

Located in The Town, you can get this one after obtaining the double jump:



Bounce off the combat dummy, then progress to the top of the left-most building. You can then go left to enter the area with epitaph 3:

On the main path in The Underworld you will descend and have the option to head left; head that way and collect epitaph 4:

Found in The Clocktower; the entrance is directly above the dash upgrade. You will need to jump on the large cog to reach the top left-hand corner of the room. I don't have a screenshot of this one, and I found out too late you can't go back after you fight the King.

Soon after entering The Ramparts you will find a large staircase with an invisible floor extending behind it: