Question: Matching Problem

Let say that you have two matricies:

X := Matrix(Statistics[Shuffle]([seq(x[i], i = 1 .. 100)]));
Y := Matrix(Statistics[Shuffle]([seq(y[i], i = 1 .. 100)]));

For example the matrix x could be:    x[45]    x[12]     x[2]     x[10]      etc
For example the matrix y could be:    x[12]    x[1]       x[20]     x[5]      etc

I now want to construct a third matrix Z which must look like this:

x[45]    x[12]     x[2]     x[10]      etc
y[45]    y[12]      y[2]     y[10]      etc

Hence, all 45 are located in column 1 and all 12 are located in column 2 etc
Which is the fastest way of doing this?  The matricies will have approx 10 000 entries each.

I tried:

rtable_scanblock(q, [], proc (val, ind, res) options operator, arrow; `if`(val = z5[1, j], ind[2], res) end proc, [])

but it is very slow....