Florent, Here are my thoughts. I agree with Victor, your best bet is to get the actual distributions for each treatment. From those you can model the median (this is usually the case as particle distributions are typically not normally distributed), and some measure of dispersion (e.g., standard deviation, fairly robust to distributional issues, see Shewhart) as separate response variables. Make sure you plot the distributions for each treatment. Sometimes you can "create" a response variable that better describes each distribution and therefore what factors are influencing the particle size.
Interestingly, the metric you are trying to use is not really 3 independent response variables. As you note, there is a distribution of particle size for each treatment. Your categorization into 3 categories to describe the distributions is creative, and calculating a proportion for each category also creative, but unfortunately categorical Y's are not very effective as they often lack discrimination (especially when you only have 3 categories).
On the other hand, It might be more interesting to see how the proportions change associated with the treatments. I would go ahead and analyze each category as a separate Y. You would be trying to answer the questions: Do any of the model effects impact the proportion of particle sizes that are <1mm? or 1-5mm? or >5mm?
Should those response proportions correlate? If they don't why?
I did notice a strange value in row 20 for factor X1 (0.31)?
I ran Fit Model (Least Square Fit) and Multivariate analysis and added the scripts to your table.
"All models are wrong, some are useful" G.E.P. Box