This following instruction does not assemble. Instead, jwasm complains with Error A2049: Invalid instruction operands.
vpsrldq ymm3, ymm3, 4
If I use XMM registers for the same instruction, jwasm generates code, but that is not what I need. The above instruction is supposed to shift right the contents of ymm3 register by four bytes. The generated code should be 0C5h, 0E5h, 73h, 0DBh, 04h. When I coded the XMM registers, and halted my program with a hardware debugger, I could change the E1 of the generated code to E5 as needed, and the CPU works properly.
Similarly, other SSE2/AVX instructions are not properly supported with YMM registers; the code supports XMM registers, not YMM registers, for the same instructions. vpxor is one in particular that does not assembly.
vpxor ymm3, ymm3, ymm5 <-- does not assemble
vpxor xmm3, xmm3, xmm5 <-- does.