Once you have all the voxels (including the latch) selected, press the Delete or Del key on you keyboard to remove them.