1# dEQP Support 2 3ANGLE integrates dEQP (i.e. the OpenGL CTS) for conformance testing. It uses gtest to run tests, 4and provides the means for dEQP to use ANGLE. 5 6## Overriding dEQP files 7 8Occasionally, ANGLE overrides certain dEQP files by copying them to this directory, adding the 9`_override` suffix, and modifying them. `deqp.gni` is used to select these override files to be 10built with dEQP instead of the original files. 11 12This is primarily done to fix tests until they are fixed upstream. 13 14## Expectation files format 15 16For every set of dEQP tests, for example GLES3 tests on the Vulkan backend, an expectations file 17exists to let the test harness know which tests it should skip (as they are known to crash), or 18expect to see failed. Warnings are generated if a test unexpectedly passes, but an unexpected 19failure is an error. This let's ANGLE ensure there are no regressions. 20 21If multiple test expectations in a file match a specific test due to 22wildcards, the test harness picks the first match in the file as the 23overriding expectation. 24 25While developing a feature, or testing on a new platform, the expectations files can be modified to 26reflect the reality of the situation. The expected format for every line in these files is: 27 28 {BUG#} {MODIFIERS} : {TEST_NAME} = {PASS,FAIL,FLAKY,TIMEOUT,SKIP} 29 30`MODIFIERS` can be a combination of the below list, combined with a logical AND: 31 32 WIN XP VISTA WIN7 WIN8 WIN10 33 MAC LEOPARD SNOWLEOPARD LION MOUNTAINLION MAVERICKS YOSEMITE ELCAPITAN SIERRA HIGHSIERRA MOJAVE 34 LINUX CHROMEOS ANDROID 35 NVIDIA AMD INTEL 36 DEBUG RELEASE 37 D3D9 D3D11 OPENGL GLES VULKAN WGPU 38 NEXUS5X PIXEL2ORXL PIXEL4ORXL PIXEL6 39 QUADROP400 40 SWIFTSHADER 41 PREROTATION PREROTATION90 PREROTATION180 PREROTATION270 42 NOSAN ASAN TSAN UBSAN 43 44`TEST_NAME` can be a specific test name, or set of test names using `'*'` as wildcard anywhere in 45the name. Examples: 46 47 // Disabled everywhere as is too slow: 48 3445 : dEQP-GLES31.functional.ssbo.layout.random.all_shared_buffer.48 = SKIP 49 50 // Crashes on both D3D11 and OPENGL: 51 1442 OPENGL : dEQP-GLES31.functional.separate_shader.* = SKIP 52 1442 D3D11 : dEQP-GLES31.functional.separate_shader.* = SKIP 53 54 // Unsupported feature: 55 3726 VULKAN ANDROID : dEQP-GLES31.functional.synchronization.inter_call.without_memory_barrier.*atomic_counter* = FAIL 56 57 // Failing test in Nvidia's OpenGL implementation on windows: 58 1665 WIN NVIDIA OPENGL : dEQP-GLES31.functional.draw_indirect.negative.command_offset_not_in_buffer_unsigned32_wrap = FAIL 59 60 // Failing when emulated pre-rotation is enabled, no matter which angle: 61 1234 PREROTATION : dEQP-GLES3.*blit* = FAIL 62 63 // Failing when emulated pre-rotation is enabled with 270 degree angle: 64 1234 PREROTATION270 : dEQP-GLES3.*blit* = FAIL 65 66 // Flaky when run with thread-sanitizer (TSan) 67 6678 TSAN : dEQP-EGL.functional.sharing.gles2.multithread.random* = FLAKY 68