Testing/Conformance: Difference between revisions

From WebGL Public Wiki
Jump to navigation Jump to search
m Fixing stale link.
 
(11 intermediate revisions by 3 users not shown)
Line 5: Line 5:
The biggest issue is that WebGL is based on OpenGL ES 2.0 which has significant differences from desktop OpenGL. Those differences need to be tested against.
The biggest issue is that WebGL is based on OpenGL ES 2.0 which has significant differences from desktop OpenGL. Those differences need to be tested against.


Note: As for this writing these tests have not been implemented. If you are interested in implementing them please speak up.
== WebGL Conformance Test Suite ==


= Current test suite =
The latest version of the conformance suite can be run by visiting
 
https://www.khronos.org/registry/webgl/sdk/tests/webgl-conformance-tests.html
 
Various options may be passed in via the URL. See the [https://www.khronos.org/registry/webgl/sdk/tests/README.md README.md] for details.
 
Previous versions can be found at
 
https://www.khronos.org/registry/webgl/conformance-suites/
 
== Source code ==


The currently available tests are hosted at
The currently available tests are hosted at


     https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/sdk/tests/
     https://www.khronos.org/registry/webgl/sdk/tests/
    https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/sdk/tests/fast/


Their source code can be checked out with
Their source code can be checked out with


     svn checkout https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl
     git clone https://github.com/KhronosGroup/WebGL.git
 
== Submitting new tests ==
 
New submissions to the conformance test suite are welcome. Please see [[Using_Github_To_Contribute|Using Github to Contribute]] for information on adding tests and submitting a pull request on Github.


You can run the the current test harness here.
== Filing bugs ==
https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/sdk/tests/webgl-conformance-tests.html


== Tests ==
Please file bugs against the conformance suite using [http://www.khronos.org/bugzilla/ Khronos' Bugzilla] against product WebGL, component Conformance Tests.


Below is a non-exhaustive list of tests an implementation must pass.
== For Implementers of the WebGL API ==


* Test that glDrawElements fails if an index is out of range for the currently bound buffers.
If you or your company is implementing the WebGL API in a web browser and you desire to have your implementation certified as compliant, please visit the [http://www.khronos.org/implementers/webgl/ WebGL Implementers' Resources] for information on producing a submission of the conformance suite.
* Test that glDrawElements succeeds if asked to draw a subset of some buffer that is in range for the subset of indices but would be out of range if the entire index buffer is used. 
** Example: An index buffer has the following indices [20, 30, 0, 1, 2, 3, 40, 50].  The bound buffers only have 4 vertices. If glDrawElements is called with count = 8, offset = 0 it should fail. If called with count = 4, offset = 2 it should succeed.
* Test that glDrawElements and glDrawArrays fails if any vertex attributes needed by the current program are enabled but not supplied.
* Test that glUniformMatrixfv fails if the transpose parameter is not GL_FALSE.
* Test that trying to set LEVEL > 0 on NPOT textures fails. GLES2 only supports NPOT textures with no mips.
* Test that drawing an NPOT with incorrect parameters sends 0,0,0,1 to the fragment shader. GLES2
* Test that in general, any GLenum that is allowed on Deskop GL but not in GLES2 fails. There are lots of these.
** Test that glDrawElements fails with GL_INT.
** Test that glDrawArrays and glDrawElements fails with GL_QUAD_STRIP, GL_QUADS, and GL_POLYGON.
** Test that glTexImage2D fails if any Desktop GL enums are passed in, especially desktop only texture formats and source format enums including but not limited to
*** 1, 2, 3, 4
*** GL_BGR, GL_BGRA
*** GL_DEPTH_COMPONENT32
*** etc...
** Test that glEnable and glDisable fail with GL_TEXTURE_2D, GL_ALPHA_TEST, GL_CLIP_PLANE0, etc..
* Test that calling glVertexAttribPointer when glBindBuffer's bound array buffer is 0 fails.
* Test that changing the size of the canvas does not change the size of the viewport.
* Test that in general, any thing allowed in GLSL on Desktop GL but not in GLES2 fails to compile.
** Test that shaders that use identifiers allowed in Desktop GL but not in GLES2 fail to compile.
*** for example, a shader with "uniform vec4 gl_world;" must return -1 when GetUniformLocation(prg, "gl_world") is called. (GLES forbids uniforms with the prefix "gl_")
** Test that shaders that use the "precision" keyword compile. (That keyword is not supported by desktop GL which means WebGL implementations will have to parse it out)
** Test that shaders with something other than "#version 100" fail.
** Test that shaders with "#ifdef GL_ES" compile as though GL_ES is true. DesktopGL will not do this meaning conforming WebGL implementation will need to do their something to make this work.
** Test that shaders with "#extension" fail.
* Test that calling TexImage2D with border != 0 generates INVALID_VALUE
* Test that calling TexImage2D with no WebGLArray succeeds
* Test that calling TexImage2D with no WebGLArray that created texture's content is is 0,0,0,0. GL does not do this so WebGL implementations will need to otherwise it will be possible for WebGL to get the contents of vram from previously run apps.
* Test that there is no implementation state exposed to WebGL.
** for example, the underlying implementation might create a texture, bind it and use it as an render target. If the WebGL user calls gl.getParameter(GL_TEXTURE_BINDING_2D) they should not get the name of the internal texture.
* Test that no state is lost by the internal implementation
** WebGL implementations may have to do things like clear the backbuffer when a canvas is resized and/or change states (bind buffers, textures, etc) when compositing on the webpage. Those state changes must not effect the WebGL state. For example if clearing the backbuffer calls glClearColor() the implementation might lose the user's color if it does not save it.
* Test framebuffer object attachments
** DEPTH_ATTACHMENT, STENCIL_ATTACHMENT, and DEPTH_STENCIL_ATTACHMENT must be (individually) supported
** Attaching a DEPTH renderbuffer with an internal format different than DEPTH_COMPONENT16 must raise INVALID_OPERATION from framebufferRenderbuffer
** Attaching a STENCIL renderbuffer with an internal format different than STENCIL_INDEX8 must raise INVALID_OPERATION from framebufferRenderbuffer
** Attempting to attach non-zero renderbuffers to the following combinations of attachment points must raise INVALID_OPERATION from framebufferRenderbuffer:
*** DEPTH_ATTACHMENT and DEPTH_STENCIL_ATTACHMENT
*** STENCIL_ATTACHMENT and DEPTH_STENCIL_ATTACHMENT
*** DEPTH_ATTACHMENT and STENCIL_ATTACHMENT

Latest revision as of 14:04, 2 August 2013

Conformance tests

In order for an implementation to claim it supports WebGL it must pass a set of conformance tests. Many of the tests are there to hopefully find incompatibilites between implementations. This is to promote the goal that a WebGL program created in a particular browser on a particular OS will run on any browser and any OS without modification.

The biggest issue is that WebGL is based on OpenGL ES 2.0 which has significant differences from desktop OpenGL. Those differences need to be tested against.

WebGL Conformance Test Suite

The latest version of the conformance suite can be run by visiting

https://www.khronos.org/registry/webgl/sdk/tests/webgl-conformance-tests.html

Various options may be passed in via the URL. See the README.md for details.

Previous versions can be found at

https://www.khronos.org/registry/webgl/conformance-suites/

Source code

The currently available tests are hosted at

   https://www.khronos.org/registry/webgl/sdk/tests/

Their source code can be checked out with

   git clone https://github.com/KhronosGroup/WebGL.git

Submitting new tests

New submissions to the conformance test suite are welcome. Please see Using Github to Contribute for information on adding tests and submitting a pull request on Github.

Filing bugs

Please file bugs against the conformance suite using Khronos' Bugzilla against product WebGL, component Conformance Tests.

For Implementers of the WebGL API

If you or your company is implementing the WebGL API in a web browser and you desire to have your implementation certified as compliant, please visit the WebGL Implementers' Resources for information on producing a submission of the conformance suite.