You say, "suborders would be the random effect because the fixed effect depends on it." Are you talking about nested levels. For example, in this case
If the levels of Factor B are such that level 1 in the first row is the first and level 1 in the third row is first, but they are not the same level, then Factor B is nested in Factor A. Think of a case where two parts are test at both levels of Factor A, but they are not the same parts. There are then actually 4 parts tested. On the other hand, if Factor B = 1 is the same for Factor A = 1 or 2, then Factor A and Factor B are crossed. Using the same example, there are only two parts that are each tested under both conditions of Factor A.
I need to know which case (crossed or nested) to be sure that you specified the fixed effects correctly.
I would still try to include Sub-order as one of the fixed effects. If that works, then you do not need GLMM. You can use GLM personality instead.