I think you can. See the attached example, created using Plot Excluded and Unhidden data points in Graph Builder.
The excluded points are shaded in the table script I've added to the table.
Ressel_0-1698229866890.png
Comparing side by side, the width of the boxes in the box plot are not influenced by the shaded, i.e., excluded & unhidden data.
Ressel_1-1698230097188.png
It looks to me that this is what you want?