Testing/Conformance

Conformance tests

In order for an implementation to claim it supports WebGL it must pass a set of conformance tests. Many of the tests are there to hopefully find incompatibilites between implementations. This is to promote the goal that a WebGL program created in a particular browser on a particular OS will run on any browser and any OS without modification.

The biggest issue is that WebGL is based on OpenGL ES 2.0 which has significant differences from desktop OpenGL. Those differences need to be tested against.

WebGL Conformance Test Suite

https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/sdk/tests/webgl-conformance-tests.html

Source code

The currently available tests are hosted at

   https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/sdk/tests/

Their source code can be checked out with

   svn checkout https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl

Notes

Below is a non-exhaustive list of tests an implementation must pass.

• Test that glDrawElements fails if an index is out of range for the currently bound buffers.
• Test that glDrawElements succeeds if asked to draw a subset of some buffer that is in range for the subset of indices but would be out of range if the entire index buffer is used.
  • Example: An index buffer has the following indices [20, 30, 0, 1, 2, 3, 40, 50]. The bound buffers only have 4 vertices. If glDrawElements is called with count = 8, offset = 0 it should fail. If called with count = 4, offset = 2 it should succeed.
• Test that glDrawElements and glDrawArrays fails if any vertex attributes needed by the current program are enabled but not supplied.
• Test that glUniformMatrixfv fails if the transpose parameter is not GL_FALSE.
• Test that trying to set LEVEL > 0 on NPOT textures fails. GLES2 only supports NPOT textures with no mips.
• Test that drawing an NPOT with incorrect parameters sends 0,0,0,1 to the fragment shader. GLES2
• Test that in general, any GLenum that is allowed on Deskop GL but not in GLES2 fails. There are lots of these.
  • Test that glDrawElements fails with GL_INT.
  • Test that glDrawArrays and glDrawElements fails with GL_QUAD_STRIP, GL_QUADS, and GL_POLYGON.
  • Test that glTexImage2D fails if any Desktop GL enums are passed in, especially desktop only texture formats and source format enums including but not limited to
    • 1, 2, 3, 4
    • GL_BGR, GL_BGRA
    • GL_DEPTH_COMPONENT32
    • etc...
  • Test that glEnable and glDisable fail with GL_TEXTURE_2D, GL_ALPHA_TEST, GL_CLIP_PLANE0, etc..
• Test that calling glVertexAttribPointer when glBindBuffer's bound array buffer is 0 fails.
• Test that changing the size of the canvas does not change the size of the viewport.
• Test that in general, any thing allowed in GLSL on Desktop GL but not in GLES2 fails to compile.
  • Test that shaders that use identifiers allowed in Desktop GL but not in GLES2 fail to compile.
    • for example, a shader with "uniform vec4 gl_world;" must return -1 when GetUniformLocation(prg, "gl_world") is called. (GLES forbids uniforms with the prefix "gl_")
  • Test that shaders that use the "precision" keyword compile. (That keyword is not supported by desktop GL which means WebGL implementations will have to parse it out)
  • Test that shaders with something other than "#version 100" fail.
  • Test that shaders with "#ifdef GL_ES" compile as though GL_ES is true. DesktopGL will not do this meaning conforming WebGL implementation will need to do their something to make this work.
  • Test that shaders with "#extension" fail.
• Test that calling TexImage2D with border != 0 generates INVALID_VALUE
• Test that calling TexImage2D with null succeeds
• Test that calling TexImage2D with null the created texture's content is is 0,0,0,0. GL does not do this so WebGL implementations will need to otherwise it will be possible for WebGL to get the contents of vram from previously run apps.
• Test that there is no implementation state exposed to WebGL.
  • for example, the underlying implementation might create a texture, bind it and use it as an render target. If the WebGL user calls gl.getParameter(GL_TEXTURE_BINDING_2D) they should not get the name of the internal texture.
• Test that no state is lost by the internal implementation
  • WebGL implementations may have to do things like clear the backbuffer when a canvas is resized and/or change states (bind buffers, textures, etc) when compositing on the webpage. Those state changes must not effect the WebGL state. For example if clearing the backbuffer calls glClearColor() the implementation might lose the user's color if it does not save it.
• Test framebuffer object attachments
  • DEPTH_ATTACHMENT, STENCIL_ATTACHMENT, and DEPTH_STENCIL_ATTACHMENT must be (individually) supported
  • Attaching a DEPTH renderbuffer with an internal format different than DEPTH_COMPONENT16 must raise INVALID_OPERATION from framebufferRenderbuffer
  • Attaching a STENCIL renderbuffer with an internal format different than STENCIL_INDEX8 must raise INVALID_OPERATION from framebufferRenderbuffer
  • Attempting to attach non-zero renderbuffers to the following combinations of attachment points must raise INVALID_OPERATION from framebufferRenderbuffer:
    • DEPTH_ATTACHMENT and DEPTH_STENCIL_ATTACHMENT
    • STENCIL_ATTACHMENT and DEPTH_STENCIL_ATTACHMENT
    • DEPTH_ATTACHMENT and STENCIL_ATTACHMENT
• Test that getUniformLocation works for array uniforms