1--- 2title: 'Text Properties API' 3linkTitle: 'Text Properties API' 4--- 5 6A variety of (internationally correct) text processing requires know the *properties* of unicode characters. 7For example, where in a string are the word boundaries (needed for line-breaking), which need to be ordered 8right-to-left or left-to-right? 9 10We propose a batch call that **characterizes** the code-points in a string. The method will return an array 11of bitfields packed in a 32bit unsigned long, containing the results of all of the Options. 12 13## Functional Requirement 14 15One measure of the value/completeness of the API is the following: 16 17*For sophisticated apps or frameworks (e.g. Flutter or Lottie) that need... 18- text shaping 19- line breaking 20- word and grapheme boundaries 21 22Certainly this API could include **more** than is strictly required for those use cases, but it is important that it include **at least** enough to allow them to function without increasing their (WASM) download size 23by having to include an copy of ICU (or its equivalent). 24 25## Ergonomics 26 27Associated with the above Function Requirements, another driver for the shape of the API is efficiency, esp. when called by **WASM** clients. There is a real cost for each JS <--> WASM call, more than the equivalent 28sequence between JS and the Browser. 29- Minimize # calls needed for a block of text 30- Homogenous arrays rather than sequence of objects 31 32Given this, implementations are encouraged to use **Uint32Array** typed array buffer for the result. 33 34```WebIDL 35// Bulk call to characterize the code-points in a string. 36// This can return a number of different properties per code-point, so to maximize performance, 37// it will only compute the requested properties requested (see optional boolean request fields). 38// 39interface TextProperties { 40 const unsigned long BidiLevelMask = 31, // 0..31 bidi level 41 42 const unsigned long GraphemeBreak = 1 << 5, 43 const unsigned long IntraWordBreak = 1 << 6, 44 const unsigned long WordBreak = 1 << 7, 45 const unsigned long SoftLineBreak = 1 << 8, 46 const unsigned long HardLineBreak = 1 << 9, 47 48 const unsigned long IsControl = 1 << 10, 49 const unsigned long IsSpace = 1 << 11, 50 const unsigned long IsWhiteSpace = 1 << 12, 51 52 attribute boolean bidiLevel?; 53 attribute boolean graphemeBreak?; 54 attribute boolean wordBreak?; // returns Word and IntraWord break properties 55 attribute boolean lineBreak?; // returns Soft and Hard linebreak properties 56 57 attribute boolean isControl?; 58 attribute boolean isSpace?; 59 attribute boolean isWhiteSpace?; 60 61 // Returns an array the same length as the input string. Each returned value contains the 62 // bitfield results for the corresponding code-point in the string. For surrogate pairs 63 // in the input, the results will be in the first output value, and the 2nd output value 64 // will be zero. 65 // 66 // Bitfields that are currently unused, or which correspond to an Option attribute that 67 // was not requested, will be set to zero. 68 // 69 sequence<unsigned long> characterize(DOMString inputString, 70 DOMString bcp47?); 71} 72``` 73 74## Example 75 76```js 77const properties = { 78 isWhiteSpace: true, 79 lineBreak: true, 80}; 81 82const text = "Because I could not stop for Death\nHe kindly stopped for me"; 83 84const results = properties.characterize(text); 85 86// expected results 87 88results[7,9,15,19,24,28,37,44,52,65] --> IsWhiteSpace | SoftLineBreak 89results[34] --> HardLineBreak 90``` 91 92## Related 93 94Some facilities for characterizing Unicode already exist, either as part of EcmaScript or the Web api. See [intl segmenter](https://github.com/tc39/proposal-intl-segmenter). This 95proposal acknowledges these, but suggests that any potential overlap in functionality is OK, 96given the design constraint spelled out in the [Ergonomics](#ergonomics) section. 97 98Similar to the contrast between canvas2d and webgl, this proposal seeks to provide very efficient, 99lower level access to unicode properties, specifically for sophisticated (possibly native ported to wasm) 100frameworks and apps. It is not intended to replace existing facilities (i.e. Segmenter), but rather 101to offer an alternative interface more suited to high-performance clients. 102 103We also propose a higher level interface specfically aimed at [Text Shaping](/docs/dev/design/text_overview). 104 105## Contributors: 106 [mikerreed](https://github.com/mikerreed), 107