In order to avoid ambiguity between simulation and thresholding it is advised to use right the same mask for map estimation/generation and map thresholding.For instance, if you run a GLM, you should specify the mask BEFORE running this GLM (you can do that in the options of both the SS and the MS-GLM dialog, upon specification of the design matrix).
If you use a different mask for GLM (or don't use a mask at all) and then you specify a mask "post-hoc" for "restricting" simulations you get the result that:
1) the mask is not effective for smoothness estimation;
2) the mask applies to simulated maps but not to the final map (the plugin is not allowed to exclude voxels outside the mask because these were not excluded from the analysis).
In conclusion, I would use the mask (or create a mask beforehands, e.g. a brain mask from the VTC or VMR (use the segmentation tool -> define VOI -> create MASK)) and use this mask for both GLM and CTE. You will see that the smoothness estimation and the final threshold will be effective within this orignial mask.