This is the first and foremost, the cage stockpile. Here all the animals/creatures that get in cage traps are kept. Items that you get in trades, traps, webbed cages, are kept here. From here on you get to decide what to do with them. You can tame them, or just simply create your FUN with it.
Now this is where the vermin workshop is located.
Here we can see that there's only two things to do here, catch vermin or tame it. So before taming a vermin, we simply catch it.
Upon choosing the " catch a vermin option ", a dwarf will take a cage trap and go to catch a vermin. Now, you better go for a metal cage, cheap metal cage.
Now a dwarf has caught a vermin. They will haul it back to the cage stockpile. Here it isn't entirely visible but in ASCII it will be seen like little blinking dots in cages. Here we have a grasshopper, wren and a rat. All three can be tamed.

Here we begin the taming procedure.
Go to vermin catcher and select the tame a vermin option. A dwarf will come to the workshop with a vermin and bring something to tame it with. In our case it brings a bed and a cabinet for the mice. I had seen food before but this is new for me, perhaps it wanted to help tame the mice by letting it nibble on wood. Either ways it tamed it in one go.
Here we see the dwarf hauling the bed.
And now the creature has been tamed successfully.
In the second instance, we see that the dwarf brought a cabinet for taming.
And here the second creature has been tamed.
Now we have tamed 2 creatures. Having more workshops of the same type help in doing the same task at a higher speed and helps sure the dwarves " need excitement in life " need. So rather than one dwarf hunting vermin, we can have 5 dwarves do the same job. 5 dwarves do it more efficienty.

Now we release the tamed / untamed vermin, as per your wish and need really. There isn't a reason to release the tamed vermin but you can.
Make a cage from here on the floor.
Now you have a cage on the floor.. not animal cage, the big cage you use in trap making.
Now in this cage we assign creatures. Just press the indicated button and click what creatures you'd like in that cage.
Now here we can see what is assigned. I assigned the tamed things. This also the same procedure for assigning any animals to cages to be traded off to traders. Just that you have to remove the placed cage so that it is moved to the cage stockpile to be traded off to traders.
Make a lever somewhere and link the lever to this cage. Now the lever is linked. Now pull the lever, before linking be sure to have like 2 - 3 mechanisms. Now when a dwarf pulls the lever, everything in the cage will be fully released in the space. Tamed creatures can be easily recaptured and placed in a cage when that cage is placed and the creatures as assigned again. Easy recapture.
This is how released vermin look like.

Assignment view. I want to catch them silk spiders and try some stuff out to see if we can get silk but that's much harder as saying and obtaining it.
This is the thumbnail GIF.