1*7e63c127SSorin Basca<?xml version="1.0" encoding="UTF-8" ?> 2*7e63c127SSorin Basca<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> 3*7e63c127SSorin Basca<html xmlns="http://www.w3.org/1999/xhtml" lang="en"> 4*7e63c127SSorin Basca<head> 5*7e63c127SSorin Basca <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6*7e63c127SSorin Basca <link rel="stylesheet" href="resources/doc.css" charset="UTF-8" type="text/css" /> 7*7e63c127SSorin Basca <link rel="stylesheet" href="../coverage/jacoco-resources/prettify.css" charset="UTF-8" type="text/css" /> 8*7e63c127SSorin Basca <link rel="shortcut icon" href="resources/report.gif" type="image/gif" /> 9*7e63c127SSorin Basca <script type="text/javascript" src="../coverage/jacoco-resources/prettify.js"></script> 10*7e63c127SSorin Basca <title>JaCoCo - Implementation Design</title> 11*7e63c127SSorin Basca</head> 12*7e63c127SSorin Basca<body onload="prettyPrint()"> 13*7e63c127SSorin Basca 14*7e63c127SSorin Basca<div class="breadcrumb"> 15*7e63c127SSorin Basca <a href="../index.html" class="el_report">JaCoCo</a> > 16*7e63c127SSorin Basca <a href="index.html" class="el_group">Documentation</a> > 17*7e63c127SSorin Basca <span class="el_source">Implementation Design</span> 18*7e63c127SSorin Basca</div> 19*7e63c127SSorin Basca<div id="content"> 20*7e63c127SSorin Basca 21*7e63c127SSorin Basca<h1>Implementation Design</h1> 22*7e63c127SSorin Basca 23*7e63c127SSorin Basca<p> 24*7e63c127SSorin Basca This is a unordered list of implementation design decisions. Each topic tries 25*7e63c127SSorin Basca to follow this structure: 26*7e63c127SSorin Basca</p> 27*7e63c127SSorin Basca 28*7e63c127SSorin Basca<ul> 29*7e63c127SSorin Basca <li>Problem statement</li> 30*7e63c127SSorin Basca <li>Proposed Solution</li> 31*7e63c127SSorin Basca <li>Alternatives and Discussion</li> 32*7e63c127SSorin Basca</ul> 33*7e63c127SSorin Basca 34*7e63c127SSorin Basca 35*7e63c127SSorin Basca<h2>Coverage Analysis Mechanism</h2> 36*7e63c127SSorin Basca 37*7e63c127SSorin Basca<p class="intro"> 38*7e63c127SSorin Basca Coverage information has to be collected at runtime. For this purpose JaCoCo 39*7e63c127SSorin Basca creates instrumented versions of the original class definitions. The 40*7e63c127SSorin Basca instrumentation process happens on-the-fly during class loading using so 41*7e63c127SSorin Basca called Java agents. 42*7e63c127SSorin Basca</p> 43*7e63c127SSorin Basca 44*7e63c127SSorin Basca<p> 45*7e63c127SSorin Basca There are several different approaches to collect coverage information. For 46*7e63c127SSorin Basca each approach different implementation techniques are known. The following 47*7e63c127SSorin Basca diagram gives an overview with the techniques used by JaCoCo highlighted: 48*7e63c127SSorin Basca</p> 49*7e63c127SSorin Basca 50*7e63c127SSorin Basca<img src="resources/implementation.png" alt="Coverage Implementation Techniques"/> 51*7e63c127SSorin Basca 52*7e63c127SSorin Basca<p> 53*7e63c127SSorin Basca Byte code instrumentation is very fast, can be implemented in pure Java and 54*7e63c127SSorin Basca works with every Java VM. On-the-fly instrumentation with the Java agent 55*7e63c127SSorin Basca hook can be added to the JVM without any modification of the target 56*7e63c127SSorin Basca application. 57*7e63c127SSorin Basca</p> 58*7e63c127SSorin Basca 59*7e63c127SSorin Basca<p> 60*7e63c127SSorin Basca The Java agent hook requires at least 1.5 JVMs. Class files compiled with 61*7e63c127SSorin Basca debug information (line numbers) allow for source code highlighting. Unluckily 62*7e63c127SSorin Basca some Java language constructs get compiled to byte code that produces 63*7e63c127SSorin Basca unexpected highlighting results, especially in case of implicitly generated 64*7e63c127SSorin Basca code like default constructors or control structures for finally statements. 65*7e63c127SSorin Basca</p> 66*7e63c127SSorin Basca 67*7e63c127SSorin Basca 68*7e63c127SSorin Basca<h2>Coverage Agent Isolation</h2> 69*7e63c127SSorin Basca 70*7e63c127SSorin Basca<p class="intro"> 71*7e63c127SSorin Basca The Java agent is loaded by the application class loader. Therefore the 72*7e63c127SSorin Basca classes of the agent live in the same name space like the application classes 73*7e63c127SSorin Basca which can result in clashes especially with the third party library ASM. The 74*7e63c127SSorin Basca JoCoCo build therefore moves all agent classes into a unique package. 75*7e63c127SSorin Basca</p> 76*7e63c127SSorin Basca 77*7e63c127SSorin Basca<p> 78*7e63c127SSorin Basca The JaCoCo build renames all classes contained in the 79*7e63c127SSorin Basca <code>jacocoagent.jar</code> into classes with a 80*7e63c127SSorin Basca <code>org.jacoco.agent.rt_<randomid></code> prefix, including the 81*7e63c127SSorin Basca required ASM library classes. The identifier is created from a random number. 82*7e63c127SSorin Basca As the agent does not provide any API, no one should be affected by this 83*7e63c127SSorin Basca renaming. This trick also allows that JaCoCo tests can be verified with 84*7e63c127SSorin Basca JaCoCo. 85*7e63c127SSorin Basca</p> 86*7e63c127SSorin Basca 87*7e63c127SSorin Basca 88*7e63c127SSorin Basca<h2>Minimal Java Version</h2> 89*7e63c127SSorin Basca 90*7e63c127SSorin Basca<p class="intro"> 91*7e63c127SSorin Basca JaCoCo requires Java 1.5. 92*7e63c127SSorin Basca</p> 93*7e63c127SSorin Basca 94*7e63c127SSorin Basca<p> 95*7e63c127SSorin Basca The Java agent mechanism used for on-the-fly instrumentation became available 96*7e63c127SSorin Basca with Java 1.5 VMs. Coding and testing with Java 1.5 language level is more 97*7e63c127SSorin Basca efficient, less error-prone – and more fun than with older versions. 98*7e63c127SSorin Basca JaCoCo will still allow to run against Java code compiled for these. 99*7e63c127SSorin Basca</p> 100*7e63c127SSorin Basca 101*7e63c127SSorin Basca 102*7e63c127SSorin Basca<h2>Byte Code Manipulation</h2> 103*7e63c127SSorin Basca 104*7e63c127SSorin Basca<p class="intro"> 105*7e63c127SSorin Basca Instrumentation requires mechanisms to modify and generate Java byte code. 106*7e63c127SSorin Basca JaCoCo uses the ASM library for this purpose internally. 107*7e63c127SSorin Basca</p> 108*7e63c127SSorin Basca 109*7e63c127SSorin Basca<p> 110*7e63c127SSorin Basca Implementing the Java byte code specification would be an extensive and 111*7e63c127SSorin Basca error-prone task. Therefore an existing library should be used. The 112*7e63c127SSorin Basca <a href="http://asm.objectweb.org/">ASM</a> library is lightweight, easy to 113*7e63c127SSorin Basca use and very efficient in terms of memory and CPU usage. It is actively 114*7e63c127SSorin Basca maintained and includes as huge regression test suite. Its simplified BSD 115*7e63c127SSorin Basca license is approved by the Eclipse Foundation for usage with EPL products. 116*7e63c127SSorin Basca</p> 117*7e63c127SSorin Basca 118*7e63c127SSorin Basca<h2>Java Class Identity</h2> 119*7e63c127SSorin Basca 120*7e63c127SSorin Basca<p class="intro"> 121*7e63c127SSorin Basca Each class loaded at runtime needs a unique identity to associate coverage data with. 122*7e63c127SSorin Basca JaCoCo creates such identities by a CRC64 hash code of the raw class definition. 123*7e63c127SSorin Basca</p> 124*7e63c127SSorin Basca 125*7e63c127SSorin Basca<p> 126*7e63c127SSorin Basca In multi-classloader environments the plain name of a class does not 127*7e63c127SSorin Basca unambiguously identify a class. For example OSGi allows to use different 128*7e63c127SSorin Basca versions of the same class to be loaded within the same VM. In complex 129*7e63c127SSorin Basca deployment scenarios the actual version of the test target might be different 130*7e63c127SSorin Basca from current development version. A code coverage report should guarantee that 131*7e63c127SSorin Basca the presented figures are extracted from a valid test target. A hash code of 132*7e63c127SSorin Basca the class definitions allows to differentiate between classes and versions of 133*7e63c127SSorin Basca classes. The CRC64 hash computation is simple and fast resulting in a small 64 134*7e63c127SSorin Basca bit identifier. 135*7e63c127SSorin Basca</p> 136*7e63c127SSorin Basca 137*7e63c127SSorin Basca<p> 138*7e63c127SSorin Basca The same class definition might be loaded by class loaders which will result 139*7e63c127SSorin Basca in different classes for the Java runtime system. For coverage analysis this 140*7e63c127SSorin Basca distinction should be irrelevant. Class definitions might be altered by other 141*7e63c127SSorin Basca instrumentation based technologies (e.g. AspectJ). In this case the hash code 142*7e63c127SSorin Basca will change and identity gets lost. On the other hand code coverage analysis 143*7e63c127SSorin Basca based on classes that have been somehow altered will produce unexpected 144*7e63c127SSorin Basca results. The CRC64 code might produce so called <i>collisions</i>, i.e. 145*7e63c127SSorin Basca creating the same hash code for two different classes. Although CRC64 is not 146*7e63c127SSorin Basca cryptographically strong and collision examples can be easily computed, for 147*7e63c127SSorin Basca regular class files the collision probability is very low. 148*7e63c127SSorin Basca</p> 149*7e63c127SSorin Basca 150*7e63c127SSorin Basca<h2>Coverage Runtime Dependency</h2> 151*7e63c127SSorin Basca 152*7e63c127SSorin Basca<p class="intro"> 153*7e63c127SSorin Basca Instrumented code typically gets a dependency to a coverage runtime which is 154*7e63c127SSorin Basca responsible for collecting and storing execution data. JaCoCo uses JRE types 155*7e63c127SSorin Basca only in generated instrumentation code. 156*7e63c127SSorin Basca</p> 157*7e63c127SSorin Basca 158*7e63c127SSorin Basca<p> 159*7e63c127SSorin Basca Making a runtime library available to all instrumented classes can be a 160*7e63c127SSorin Basca painful or impossible task in frameworks that use their own class loading 161*7e63c127SSorin Basca mechanisms. Since Java 1.6 <code>java.lang.instrument.Instrumentation</code> 162*7e63c127SSorin Basca has an API to extends the bootsstrap loader. As our minimum target is Java 1.5 163*7e63c127SSorin Basca JaCoCo decouples the instrumented classes and the coverage runtime through 164*7e63c127SSorin Basca official JRE API types only. The instrumented classes communicate through the 165*7e63c127SSorin Basca <code>Object.equals(Object)</code> method with the runtime. A instrumented 166*7e63c127SSorin Basca class can retrieve its probe array instance with the following code. Note 167*7e63c127SSorin Basca that only JRE APIs are used: 168*7e63c127SSorin Basca</p> 169*7e63c127SSorin Basca 170*7e63c127SSorin Basca 171*7e63c127SSorin Basca<pre class="source lang-java linenums"> 172*7e63c127SSorin BascaObject access = ... // Retrieve instance 173*7e63c127SSorin Basca 174*7e63c127SSorin BascaObject[] args = new Object[3]; 175*7e63c127SSorin Bascaargs[0] = Long.valueOf(8060044182221863588); // class id 176*7e63c127SSorin Bascaargs[1] = "com/example/MyClass"; // class name 177*7e63c127SSorin Bascaargs[2] = Integer.valueOf(24); // probe count 178*7e63c127SSorin Basca 179*7e63c127SSorin Bascaaccess.equals(args); 180*7e63c127SSorin Basca 181*7e63c127SSorin Bascaboolean[] probes = (boolean[]) args[0]; 182*7e63c127SSorin Basca</pre> 183*7e63c127SSorin Basca 184*7e63c127SSorin Basca<p> 185*7e63c127SSorin Basca The most tricky part takes place in line 1 and is not shown in the snippet 186*7e63c127SSorin Basca above. The object instance providing access to the coverage runtime through 187*7e63c127SSorin Basca its <code>equals()</code> method has to be obtained. Different approaches have 188*7e63c127SSorin Basca been implemented and tested so far: 189*7e63c127SSorin Basca</p> 190*7e63c127SSorin Basca 191*7e63c127SSorin Basca<ul> 192*7e63c127SSorin Basca <li><b><code>SystemPropertiesRuntime</code></b>: This approach stores the 193*7e63c127SSorin Basca object instance under a system property. This solution breaks the contract 194*7e63c127SSorin Basca that system properties must only contain <code>java.lang.String</code> 195*7e63c127SSorin Basca values and therefore causes trouble in applications that rely on this 196*7e63c127SSorin Basca definition (e.g. Ant).</li> 197*7e63c127SSorin Basca <li><b><code>LoggerRuntime</code></b>: Here we use a shared 198*7e63c127SSorin Basca <code>java.util.logging.Logger</code> and communicate through the logging 199*7e63c127SSorin Basca parameter array instead of a <code>equals()</code> method. The coverage 200*7e63c127SSorin Basca runtime registers a custom <code>Handler</code> to receive the parameter 201*7e63c127SSorin Basca array. This approach might break environments that install their own log 202*7e63c127SSorin Basca managers (e.g. Glassfish).</li> 203*7e63c127SSorin Basca <li><b><code>ModifiedSystemClassRuntime</code></b>: This approach adds a 204*7e63c127SSorin Basca public static field to an existing JRE class through instrumentation. Unlike 205*7e63c127SSorin Basca the other methods above this is only possible for environments where a Java 206*7e63c127SSorin Basca agent is active.</li> 207*7e63c127SSorin Basca <li><b><code>InjectedClassRuntime</code></b>: This approach defines a new class 208*7e63c127SSorin Basca using <code>java.lang.invoke.MethodHandles.Lookup.defineClass</code> 209*7e63c127SSorin Basca introduced in Java 9.</li> 210*7e63c127SSorin Basca</ul> 211*7e63c127SSorin Basca 212*7e63c127SSorin Basca<p> 213*7e63c127SSorin Basca Starting from version 0.8.3 JaCoCo Java agent implementation uses the 214*7e63c127SSorin Basca <code>InjectedClassRuntime</code> to define new class in bootstrap class 215*7e63c127SSorin Basca loader when running on JRE 9 and higher, otherwise uses 216*7e63c127SSorin Basca <code>ModifiedSystemClassRuntime</code> to add field to an existing JRE class. 217*7e63c127SSorin Basca Starting from version 0.8.0 field is added to the class 218*7e63c127SSorin Basca <code>java.lang.UnknownError</code>, versions 0.5.0 - 0.7.9 were adding field 219*7e63c127SSorin Basca to the class <code>java.util.UUID</code>, having bigger chance of conflict 220*7e63c127SSorin Basca with other agents. 221*7e63c127SSorin Basca</p> 222*7e63c127SSorin Basca 223*7e63c127SSorin Basca 224*7e63c127SSorin Basca<h2>Memory Usage</h2> 225*7e63c127SSorin Basca 226*7e63c127SSorin Basca<p class="intro"> 227*7e63c127SSorin Basca Coverage analysis for huge projects with several thousand classes or hundred 228*7e63c127SSorin Basca thousand lines of code should be possible. To allow this with reasonable 229*7e63c127SSorin Basca memory usage the coverage analysis is based on streaming patterns and 230*7e63c127SSorin Basca "depth first" traversals. 231*7e63c127SSorin Basca</p> 232*7e63c127SSorin Basca 233*7e63c127SSorin Basca<p> 234*7e63c127SSorin Basca The complete data tree of a huge coverage report is too big to fit into a 235*7e63c127SSorin Basca reasonable heap memory configuration. Therefore the coverage analysis and 236*7e63c127SSorin Basca report generation is implemented as "depth first" traversals. Which means that 237*7e63c127SSorin Basca at any point in time only the following data has to be held in working memory: 238*7e63c127SSorin Basca</p> 239*7e63c127SSorin Basca 240*7e63c127SSorin Basca<ul> 241*7e63c127SSorin Basca <li>A single class which is currently processed.</li> 242*7e63c127SSorin Basca <li>The summary information of all parents of this class (package, groups).</li> 243*7e63c127SSorin Basca</ul> 244*7e63c127SSorin Basca 245*7e63c127SSorin Basca<h2>Java Element Identifiers</h2> 246*7e63c127SSorin Basca 247*7e63c127SSorin Basca<p class="intro"> 248*7e63c127SSorin Basca The Java language and the Java VM use different String representation formats 249*7e63c127SSorin Basca for Java elements. For example while a type reference in Java reads like 250*7e63c127SSorin Basca <code>java.lang.Object</code>, the VM references the same type as 251*7e63c127SSorin Basca <code>Ljava/lang/Object;</code>. The JaCoCo API is based on VM identifiers only. 252*7e63c127SSorin Basca</p> 253*7e63c127SSorin Basca 254*7e63c127SSorin Basca<p> 255*7e63c127SSorin Basca Using VM identifiers directly does not cause any transformation overhead at 256*7e63c127SSorin Basca runtime. There are several programming languages based on the Java VM that 257*7e63c127SSorin Basca might use different notations. Specific transformations should therefore only 258*7e63c127SSorin Basca happen at the user interface level, for example during report generation. 259*7e63c127SSorin Basca</p> 260*7e63c127SSorin Basca 261*7e63c127SSorin Basca<h2>Modularization of the JaCoCo implementation</h2> 262*7e63c127SSorin Basca 263*7e63c127SSorin Basca<p class="intro"> 264*7e63c127SSorin Basca JaCoCo is implemented in several modules providing different functionality. 265*7e63c127SSorin Basca These modules are provided as OSGi bundles with proper manifest files. But 266*7e63c127SSorin Basca there are no dependencies on OSGi itself. 267*7e63c127SSorin Basca</p> 268*7e63c127SSorin Basca 269*7e63c127SSorin Basca<p> 270*7e63c127SSorin Basca Using OSGi bundles allows well defined dependencies at development time and 271*7e63c127SSorin Basca at runtime in OSGi containers. As there are no dependencies on OSGi, the 272*7e63c127SSorin Basca bundles can also be used like regular JAR files. 273*7e63c127SSorin Basca</p> 274*7e63c127SSorin Basca 275*7e63c127SSorin Basca</div> 276*7e63c127SSorin Basca<div class="footer"> 277*7e63c127SSorin Basca <span class="right"><a href="${jacoco.home.url}">JaCoCo</a> ${qualified.bundle.version}</span> 278*7e63c127SSorin Basca <a href="license.html">Copyright</a> © ${copyright.years} Mountainminds GmbH & Co. KG and Contributors 279*7e63c127SSorin Basca</div> 280*7e63c127SSorin Basca 281*7e63c127SSorin Basca</body> 282*7e63c127SSorin Basca</html> 283
