Confusion between C++ and OpenGL matrix order (row-major vs column-major)

To summarize the answers by SigTerm and dsharlet: The usual way to transform a vector in GLSL is to right-multiply the transformation matrix by the vector:

mat4 T; vec4 v; vec4 v_transformed; v_transformed = T*v; 

In order for that to work, OpenGL expects the memory layout of T to be, as described by SigTerm,

{1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, transX, transY, transZ, 1 } 

which is also called 'column major'. In your shader code (as indicated by your comments), however, you left-multiplied the transformation matrix by the vector:

v_transformed = v*T; 

which only yields the correct result if T is transposed, i.e. has the layout

{ 1, 0, 0, transX, 0, 1, 0, transY, 0, 0, 1, transZ, 0, 0, 0, 1 } 

(i.e. 'row major'). Since you already provided the correct layout to your shader, namely row major, it was not necessary to set the transpose flag of glUniform4v.

You are dealing with two separate issues.

First, your examples are dealing with the memory layout. Your [4][4] array is row major because you've used the convention established by C multi-dimensional arrays to match your linear array.

The second issue is a matter of convention for how you interpret matrices in your program. glUniformMatrix4fv is used to set a shader parameter. Whether your transform is computed for a row vector or column vector transform is a matter of how you use the matrix in your shader code. Because you say you need to use column vectors, I assume your shader code is using the matrix A and a column vector x to compute x' = A x.

I would argue that the documentation of glUniformMatrix is confusing. The description of the transpose parameter is a really roundabout way of just saying that the matrix is transposed or it isn't. OpenGL itself is just transporting that data to your shader, whether you want to transpose it or not is a matter of convention you should establish for your program.

This link has some good further discussion: http://steve.hollasch.net/cgindex/math/matrix/column-vec.html

I think that the existing answers here are very unhelpful, and I can see from the comments that people are left feeling confused after reading them, so here is another way of looking at this situation.

As a programmer, if I want to store an array in memory, I cannot store a rectangular grid of numbers, because computer memory doesn't work like that, I have to store the numbers in a linear sequence.

Lets say I have a 2x2 matrix and I initialize it in my code like this:

const matrix = [a, b, c, d]; 

I can successfully use this matrix in other parts of my code provided I know what each of the array elements represents.

The OpenGL specification defines what each index position represents, and this is all you need to know to construct an array and pass it to OpenGL and have it do what you expect.

The row or column major issue only comes into play when I want to write my matrix in a document that describes my code, because mathematicians write matrixes as rectangular grids of numbers. However this is just a convention, a way of writing things down, and has no impact on the code I write or the arrangement of numbers in memory on my computer. You could easily re-write these mathematics papers using some other notation, and it would work just as well.

For the array above, I have two options for writing this array in my documentation as a rectangular grid:

|a b| OR |a c| |c d| |b d| 

Whichever way I choose to write my documentation, this will have no impact on my code or the order of the numbers in memory on my computer, it's just documentation.

In order for people reading my documentation to know the order that I stored the values in the linear array in my program, I can specify that this is a column major or row major representation of the array as a matrix. If it is in column major order then I should traverse the columns to get the linear arrangement of numbers. If this is a row major representation then I should traverse the rows to get the linear arrangement of numbers.

In general, writing documentation in row major order makes life easier for programmers, because if I want to translate this matrix

|a b c| |d e f| |g h i| 

into code, I can write it like this:

const matrix = [ a, b, c d, e, f g, h, i ]; 

For example:

GLM stores matrix values as m[4][4]. But it treats matrices as if they have a column major order. Even though for 2 dimensional array m[x][y] in C x represents a row and y represents a column, which means that matrix represented by this array has in fact row major order. The trick is to treat m[x][y] as if x represents a column and y represents a row. It is like you transposing the matrix without performing any additional operations to achieve that.

I will also add my two cents to this, since a lot of the answers contain confusing or simply wrong information or do not go into detail on where the confusion is coming from.

Looking at memory

Let's start by ignoring row-major and column-major (and any mentions of rows/columns in the docs) and we will focus on how the GPU interprets a mat4 when it comes to multiplication by default in GLSL. We will use standard mathematical conventions.

First, in memory, the 16 entries of the matrix are layed out linearly:

[x0 x1 x2 x3 y0 y1 y2 y3 z0 z1 z2 z3 w0 w1 w2 w3]

Conceptually, this is an array of vec4 i.e.

[[x0 x1 x2 x3][y0 y1 y2 y3][z0 z1 z2 z3][w0 w1 w2 w3]]

We know that this is the case if we read the respective sections about memory layout, alignment etc.. This means that if we index the matrix, we get as a result these vectors, i.e.

m[0]=[x0 x1 x2 x3]

The GLSL 4.6 documentation states in chapter 5.10.:

vec3 v, u;
mat3 m;
u = v * m;

will be equivalent to

u.x = dot(v, m[0]); // m[0] is the left column of m
u.y = dot(v, m[1]); // dot(a,b) is the inner (dot) product of a and b
u.z = dot(v, m[2]);

(left-multiply)

and

u = m * v;

is equivalent to

u.x = m[0].x * v.x + m[1].x * v.y + m[2].x * v.z;
u.y = m[0].y * v.x + m[1].y * v.y + m[2].y * v.z;
u.z = m[0].z * v.x + m[1].z * v.y + m[2].z * v.z;

The only possible conclusion from this is that in typical mathematical notation, the matrix looks like this:

x0 y0 z0 w0 x1 y1 z1 w1 x2 y2 z2 w2 x3 y3 z3 w3 

With standard matrix multiplication ("row by column"). Of course, the documentation constantly mentions that the matrix is indeed stored column-by-column and that m[i] accesses the columns of a matrix, so we can conclude that the OpenGL docs follow standard mathematical notation even if we mistrust them.

Thus, GLSL treats every single matrix as a column-major matrix by default. As far as I am aware, HLSL uses row-major by default. New versions of GLSL support layout-qualifiers, allowing you to explicitly specify a matrix as row-major, which will cause it to be interpreted accordingly.

Concrete example: Translation matrix

Now to the example using a translation matrix. Mathematically it looks like this:

1 0 0 Tx 0 1 0 Ty 0 0 1 Tz 0 0 0 1 

Which means that if you want to perform a translation using a 4x4 matrix in a GLSL shader the matrix has to have the layout

[1 0 0 0 0 1 0 0 0 0 1 0 Tx Ty Tz 1]

I think so far, this is pretty obvious. We understand how everything works now in the column-major world.

In column-major we usually see M * v whereas row-major often uses v * M.

GLSL implicitly treats v as a row-vector in the second case and we've previously established that the result consists of the dot-product with the columns of M. This means that M has to be transposed for this calculation to be correct which is probably obvious to anyone familiar with matrix multiplication. Note that "reading" a row-major matrix as column-major (and vice-versa) IS a transposition, because what you treat as columns are actually the rows.

Thus, as far as OpenGL is concerned, if you have row-major matrices client-side, you can

• Transpose them before upload and use M * v (i.e. via transpose argument of glUniformMatrix)

• Transpose them in shader as transpose(M) * v

• Mark them using layout(row_major) and use M * v

• Upload them as-is and use v * M (and reverse multiplication order of matrices in the shader). It'll be read column-major, which transposes it and makes this calculation correct.

The case in the question

OP was using row-major matrices client-side and was confused about why the transpose argument needed to be false for the transformations to work correctly. Assuming that OP stated the memory layout faithfully, my conclusion is that OP used v * M in the shader, which would make this correct. Alternatively it is quite possible that OP was just confused about the layout in the library.

Nowadays, the docs for the transpose argument read

Specifies whether to transpose the matrix as the values are loaded into the uniform variable. Must be GL_FALSE.

Which makes no statements about row-major or column-major. This is most likely due to the addition of layout(row_major), where the old definition as stated in the old post by OP would make no sense anymore.

Why everyone is confused

In the "row-major" world (i.e. HLSL), people have become used to the convention of using v * M instead of M * v. If the shader would read the row-major matrix as column-major, the fourth bullet point from above would apply and everything would be fine.

However, HLSL assumes row-major layout by default. If we want to be calculating v * M using row-major matrices that are also read row-major in the shader then this can only work if M is already transposed.

For this reason, you'll see a translation-matrix on the client-side defined as:

1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, Tx, Ty, Tz, 1 Linear: [1 0 0 0 0 1 0 0 0 0 1 0 Tx Ty Tz 1] 

Thus the matrix has the same linear layout as a column-major translation matrix, because in row-major, this is a transposed translation matrix (the translation bit is in the last row).

And that is why this topic is so confusing. For some reason, some people thought this would be a good convention to have, where you are basically hiding the fact that behind all of those variable names for your fancy transformation matrices, you actually have a transposed matrix. That is of course also why in this type of "framework", matrix multiplication has to be in the reverse order in shader AND client-side.

TL;DR

• What matters is how the linear memory is interpreted. GLSL default is column-major, HLSL default is row-major. Modern shader languages let you specify it.

• If the client-side layout does not match what the shader thinks the layout is, your matrix will behave as a transposed matrix

• If you have client-side row-major matrices in GLSL, your options are

  • Transpose them before upload and use M * v (i.e. via transpose argument of glUniformMatrix)

  • Transpose them manually in shader as transpose(M) * v

  • Mark them using layout(row_major) and use M * v

  • Upload them as-is and use v * M (and reverse multiplication order of matrices in the shader). It'll be read column-major, which transposes it and makes this calculation correct.

• In HLSL, people for some reason like to use v * M with row-major matrices on client-side and shader-side, which means that these people have to create all their matrices in transposed form implicitly. They have to reverse multiplication order due to this. Another consequence is that these row-major matrices have the same memory layout as a non-transposed column-major matrix, which is why some people made that claim in their answers.