2
\$\begingroup\$

This question is a continuation of What causes this InvalidOperationException when drawing primitives?.

I've been dealing with HLSL shaders in Monogame over the past few days. It has been extremely time-consuming and frustrating trying to duplicate shader functionality I was able to achieve easily with vanilla XNA. As an example, rendering a fullscreen quad using VertexPositionColor elements fails with the following error.

Vertex declaration error caused by missing elements.

Up until this morning, I had been using the BasicEffect class with as many configurations as I could think of without success. After it became clear that BasicEffect couldn't achieve what I wanted, I wrote a simple custom shader that looks like this.

struct VertexShaderInput { float4 Color : COLOR0; float4 Position : SV_POSITION0; }; struct VertexShaderOutput { float4 Color : COLOR0; float4 Position : SV_POSITION0; }; VertexShaderOutput VertexShaderFunction(VertexShaderInput input) { VertexShaderOutput output; output.Color = input.Color; output.Position = input.Position; return output; } float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0 { return input.Color; } technique Technique0 { pass Pass0 { VertexShader = compile vs_4_0 VertexShaderFunction(); PixelShader = compile ps_4_0 PixelShaderFunction(); } }

The shader above seems reasonable because, as the name VertexPositionColor implies, I only need position and color. But using that effect in place of BasicEffect didn't change anything either. I finally started digging more deeply into vertex declarations (something I had no knowledge of until today) and came across the following source code for VertexPositionColor.

VertexElement[] elements = new VertexElement[] { new VertexElement (0, VertexElementFormat.Vector3, VertexElementUsage.Position, 0), new VertexElement (12, VertexElementFormat.Color, VertexElementUsage.Color, 0) };

Based on those lines, the vertex declaration is expecting 1) a Vector3 position and 2) a color. That seemed like the answer I needed, since in my shader, position is float4 rather than float3. But changing position to float3 produced the following error.

SV_Position0 must be float4, not float3, as shown by this error.

So, to recap and describe what I can see so far:

  1. I'm trying to render a quad using four VertexPositionColor elements.
  2. Rendering fails because the vertex shader input (defined in my shader code above) represents position using float4 rather than float3.
  3. I can't change position to use float3 because the shader compiler version (vs_4_0) doesn't allow it (I think the shader version is relevant).

This whole this is just very confusing, especially since I've used shaders like this successfully in the past with XNA rather than Monogame. Is there a solution to this problem apart from creating my own custom VertexPositionColor replacement class with its own custom vertex declaration? That all seems overkill considering the simplicity of what I'm trying to do.

This issue echoes a general feeling I've gotten that HLSL shaders in Monogame are, for lack of better phrasing, messed up. Can anyone help clarify all this?

Thank you.

\$\endgroup\$

2 Answers 2

Reset to default
1
\$\begingroup\$

The actual root cause of this problem was identified by Felsir in this question. It turns out that an exception was thrown because no shader was applied when drawing primitives using GraphicsDevice. In other words, there was effectively nothing wrong with the shader itself (either using BasicEffect or a custom one). The exception message was misleading.

I'm not sure about StackExchange conventions on self-answering questions, but I wanted this question to have a correct accepted answer. If this kind of post doesn't fit conventions, please let me know.

\$\endgroup\$
1
  • \$\begingroup\$ Self-answering is okay, and often encouraged (unless your own answer is basically just a combination of existing answers / a code dump). \$\endgroup\$ Commented Sep 23, 2016 at 13:28
0
\$\begingroup\$ 
struct VertexShaderInput { float3 Position : POSITION0; float4 Color : COLOR0; }; struct VertexShaderOutput { float4 Position : SV_POSITION; float4 Color : COLOR0; }; VertexShaderOutput VertexShaderFunction(VertexShaderInput input) { VertexShaderOutput output; output.Color = input.Color; output.Position = float4(input.Position,1); return output; }

That is how it should be.

Input does not take SV_POSITION it has to be POSITION0 and must be first in struct same with output, SV_POSITION not SV_POSITION0 and it must be first in the struct.

If u look at the vertex struct in C# it has to match the one in HLSL.

I'm not sure if this will work in mono game but if u are talking full screen quad then u can do it like this:

don't set a vb or ib or inputlayout, call context.draw(3) and have the vertex shader like this...

// outputs a full screen triangle with screen-space coordinates // input: three empty vertices VertexShaderOutput VSQuad(uint vertexID : SV_VertexID) { VertexShaderOutput result; result.TexCoord = float2((vertexID << 1) & 2, vertexID & 2); result.Position = float4(result.TexCoord * float2(2.0f, -2.0f) + float2(-1.0f, 1.0f), 0.0f, 1.0f); return result; }

I would have color set as a const so u can set it from C# per object

Edit: The name of the vertex element does not match the vertex input because in mono game on the back end the elements would look something like this: 

 Public Shared VertexElements As InputElement() = New InputElement() {New InputElement("POSITION", 0, DXGI.Format.R32G32B32_Float, 0, 0, InputClassification.PerVertexData, 0), New InputElement("NORMAL", 0, DXGI.Format.R32G32B32_Float, InputElement.AppendAligned, 0, InputClassification.PerVertexData, 0), New InputElement("TEXCOORD", 0, DXGI.Format.R32G32_Float, InputElement.AppendAligned, 0, InputClassification.PerVertexData, 0) }

Notice how the name of POSITION not SV_POSITION in XNA it didn't matter how u did ur vertex structs the runtime would sort it out but now all names must match so if you put SV_ in your shader it want match if u decompile the backend I'm sure it will look something like what I posted because that is how u do it in sharp dx.

But I don't even do it like that anymore because input layout building sucks by hand so I wrote this:

 Private Function InputLayoutBuilder(_shader As ShaderBytecode) As InputLayout Dim blah As New ShaderReflection(_shader) Dim Elements As New List(Of InputElement) For i As Integer = 0 To blah.Description.InputParameters - 1 Dim nnn As ShaderParameterDescription = blah.GetInputParameterDescription(i) If nnn.SemanticName.Contains("SV_") Then Continue For End If Dim _InputElement As New InputElement Dim format As New DXGI.Format _InputElement.SemanticName = nnn.SemanticName _InputElement.SemanticIndex = nnn.SemanticIndex _InputElement.Slot = nnn.Stream _InputElement.InstanceDataStepRate = 0 If i = 0 Then _InputElement.AlignedByteOffset = 0 Else _InputElement.AlignedByteOffset = InputElement.AppendAligned End If If nnn.SemanticName = "WORLD" Then _InputElement.Slot = 1 If nnn.SemanticIndex = 0 Then _InputElement.AlignedByteOffset = 0 Else _InputElement.AlignedByteOffset = InputElement.AppendAligned End If _InputElement.InstanceDataStepRate = 1 _InputElement.Classification = InputClassification.PerInstanceData Else _InputElement.InstanceDataStepRate = 0 _InputElement.Classification = InputClassification.PerVertexData End If If nnn.SemanticName = "SHID" Then _InputElement.Slot = 1 _InputElement.AlignedByteOffset = InputElement.AppendAligned _InputElement.InstanceDataStepRate = 1 _InputElement.Classification = InputClassification.PerInstanceData End If If nnn.UsageMask = 1 Then If nnn.ComponentType = RegisterComponentType.UInt32 Then _InputElement.Format = Format.R32_UInt ElseIf nnn.ComponentType = RegisterComponentType.SInt32 Then _InputElement.Format = Format.R32_SInt ElseIf nnn.ComponentType = RegisterComponentType.Float32 Then _InputElement.Format = Format.R32_Float End If ElseIf nnn.UsageMask <= 3 Then If nnn.ComponentType = RegisterComponentType.UInt32 Then _InputElement.Format = Format.R32G32_UInt ElseIf nnn.ComponentType = RegisterComponentType.SInt32 Then _InputElement.Format = Format.R32G32_SInt ElseIf nnn.ComponentType = RegisterComponentType.Float32 Then _InputElement.Format = Format.R32G32_Float End If ElseIf nnn.UsageMask <= 7 Then If nnn.ComponentType = RegisterComponentType.UInt32 Then _InputElement.Format = Format.R32G32B32_UInt ElseIf nnn.ComponentType = RegisterComponentType.SInt32 Then _InputElement.Format = Format.R32G32B32_SInt ElseIf nnn.ComponentType = RegisterComponentType.Float32 Then _InputElement.Format = Format.R32G32B32_Float End If ElseIf nnn.UsageMask <= 15 Then If nnn.ComponentType = RegisterComponentType.UInt32 Then _InputElement.Format = Format.R32G32B32A32_UInt ElseIf nnn.ComponentType = RegisterComponentType.SInt32 Then _InputElement.Format = Format.R32G32B32A32_SInt ElseIf nnn.ComponentType = RegisterComponentType.Float32 Then _InputElement.Format = Format.R32G32B32A32_Float End If End If Elements.Add(_InputElement) Next If Elements.Count > 0 Then Dim newinputlayout As New InputLayout(MainDevice, _shader, Elements.ToArray) blah.Dispose() Return newinputlayout Else Return Nothing End If End Function

That will pull it all out of the shader and spit out an input layout no need to do it by hand just set in your shader and let this do the rest it also checks for a WORLD tag so u can pass a world matrix to the vertex shader for instancing. Mono game may let you get at all that stuff but it may be hidden under all there abstraction but that bit of code saves all this tweaking for different input strucs.

Try this:

Texture2D colorMap; sampler colorSampler = sampler_state { AddressU = Clamp; AddressV = Clamp; Filter = MIN_MAG_MIP_POINT; }; struct VertexShaderOutput { float4 Position : SV_Position; float2 TexCoord : TEXCOORD0; }; struct PixelShaderOutput { float4 Color : SV_Target0; }; // outputs a full screen triangle with screen-space coordinates // input: three empty vertices VertexShaderOutput VSQuad( uint vertexID : SV_VertexID ) { VertexShaderOutput result; result.TexCoord = float2((vertexID << 1) & 2, vertexID & 2); result.Position = float4(result.TexCoord * float2(2.0f, -2.0f) + float2 -1.0f, 1.0f), 0.0f, 1.0f); return result; } PixelShaderOutput PShader(VertexShaderOutput input) { PixelShaderOutput output = (PixelShaderOutput)1; float4 Color = colorMap.Sample(colorSampler, input.TexCoord); output.Color =Color; output.Color.a=1; return output; } technique11 Render { pass P0 { SetVertexShader( CompileShader( vs_5_0, VSQuad() ) ); SetPixelShader( CompileShader( ps_5_0, PShader() ) ); } }
\$\endgroup\$
5
  • \$\begingroup\$ Thank you very much for the answer, but the effect changes you suggested didn't work. The error I'm seeing just doesn't make sense. It's claiming that the current vertex declaration doesn't include all elements required by the vertex shader, then goes to list the elements that the declaration DOES have. But those elements (SV_Position0 and COLOR0) are exactly what I defined as required inputs into my custom vertex shader. I'm absolutely stumped. \$\endgroup\$ Commented Sep 23, 2016 at 6:53
  • \$\begingroup\$ Its because u have SV in the name of the input struct, the sharder compiler will reads it wrong and give u that error. SV_Position0 is invalid for vertex input. Go and have a look at the docs on the MSDN site. [link(msdn.microsoft.com/en-us/library/windows/desktop/…) POSITION[n] Vertex position in object space. n is an optional integer between 0 and the number of resources supported. For example, POSITION0, TEXCOOR1, etc. \$\endgroup\$ Commented Sep 23, 2016 at 7:10
  • \$\begingroup\$ SV_Position When SV_Position is declared for input to a shader, it can have one of two interpolation modes specified: linearNoPerspective or linearNoPerspectiveCentroid, where the latter causes centroid-snapped xyzw values to be provided when multisample antialiasing. When used in a shader, SV_Position describes the pixel location. Available in all shaders to get the pixel center with a 0.5 offset - From the MSDN docs \$\endgroup\$ Commented Sep 23, 2016 at 7:14
  • \$\begingroup\$ DX9 let you do lots of messy stuff, things changed in DX10 and even more so in DX11. I moved form XNA to Sharp DX because mono is really bad, lots of things are locked down. I was able to use most of my XNA code just the GFX side of things were different. I think mono game is using SharpDX as there DirectX wrapper so I'm 99% sure I'm right and that u need to remove SV_ from the input struct \$\endgroup\$ Commented Sep 23, 2016 at 7:18
  • \$\begingroup\$ Thanks for the detailed answers :) Turns out the problem was that no shader was being applied at all due to the order in which I called SpriteBatch/GraphicsDevice functions. I'll post another answer linking to the other question and quickly describing the fix. \$\endgroup\$ Commented Sep 23, 2016 at 13:12

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.