1# The Scene Framework 2 3Known internally as "Flexiglass", this framework defines a graph where each node 4is a "scene" and each edge between the scenes is a transition. The scenes are 5the main components of System UI, on phones these are: the lockscreen, bouncer, 6shade, and quick settings panels/views/screens). Each scene is a standalone 7experience. 8 9The **main goal** of the framework is to increase code health by applying 10[Separation of concerns](https://en.wikipedia.org/wiki/Separation_of_concerns) 11over several dimensions: 12 131. Each scene is a standalone piece of UI; their code doesn't need to concern 14 itself with either transition animations or anything in other scenes. This 15 frees the developer to be able to focus only on the content of the UI for 16 that scene. 172. Transition definitions (which scene leads to which other scene following 18 which user action) are pulled out and separated from the content of the UI. 193. Transition animations (the effects that happen alongside the gradual change 20 from one scene to another) are also pulled out and separated from the 21 content of the UI. 22 23In addition to the above, some of the **secondary goals** are: 24 254. Make **customization easier**: by separating scenes to standalone pieces, it 26becomes possible for variant owners and OEMs to exclude or replace certain scenes 27or to add brand-new scenes. 285. **Enable modularization**: by separating scenes to standalone pieces, it 29becomes possible to break down System UI into smaller codebases, each one of 30which could be built on its own. Note: this isn't part of the scene framework 31itself but is something that can be done more easily once the scene framework 32is in place. 33 34## Terminology 35 36* **Scene** a collection of UI elements in a layout that, together, make up a 37 "screen" or "page" that is as large as the container. Scenes can be 38 navigated between / transition to/from. To learn more, please see 39 [this section](#Defining-a-scene). 40* **Element** (or "UI element") a single unit of UI within a scene. One scene 41 can arrange multiple elements within a layout structure. 42* **Transition** the gradual switching from one scene to another scene. There 43 are two kinds: [user-driven](Scene-navigation) and 44 [automatic](Automatic-scene-transitions) scene transitions. 45* **Transition animation** the set of UI effects that occurs while/during a 46 transition. These can apply to the entire scene or to specific elements in 47 the scene. To learn more, please see 48 [this section](#Scene-transition-animations). 49* **Scene container** (or just "container") the root piece of UI (typically a 50 `@Composable` function) that sets up all the scenes, their transitions, etc. 51 To learn more, please see [this section](#Scene-container). 52* **Container configuration** (or just "configuration") the collection of 53 scenes and some added information about the desired behaviour of a 54 container. To learn more, please see 55 [this section](#Scene-container-configuration). 56 57## Enabling the framework 58 59As of the end of 2023, the scene framework is under development; as such, it is 60disabled by default. For those who are interested in a preview, please follow 61the instructions below to turn it on. 62 63NOTE: in case these instructions become stale and don't actually enable the 64framework, please make sure `SceneContainerFlag.isEnabled` in the 65[`SceneContainerFlag.kt`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/src/com/android/systemui/scene/shared/flag/SceneContainerFlag.kt) 66file evaluates to `true`. 67 681. Set a collection of **aconfig flags** to `true` by running the following 69 commands: 70 ```console 71 $ adb shell device_config override systemui com.android.systemui.keyguard_bottom_area_refactor true 72 $ adb shell device_config override systemui com.android.systemui.keyguard_wm_state_refactor true 73 $ adb shell device_config override systemui com.android.systemui.migrate_clocks_to_blueprint true 74 $ adb shell device_config override systemui com.android.systemui.notification_avalanche_throttle_hun true 75 $ adb shell device_config override systemui com.android.systemui.predictive_back_sysui true 76 $ adb shell device_config override systemui com.android.systemui.scene_container true 77 ``` 782. **Restart** System UI by issuing the following command: 79 ```console 80 $ adb shell am crash com.android.systemui 81 ``` 823. **Verify** that the scene framework was turned on. There are two ways to do 83 this: 84 85 *(a)* look for the sash/ribbon UI at the bottom-right corner of the display: 86  87 88 NOTE: this will be removed proper to the actual release of the framework. 89 90 *(b)* Turn on logging and look for the logging statements in `logcat`: 91 ```console 92 93 # Turn on logging from the framework: 94 95 $ adb shell cmd statusbar echo -b SceneFramework:verbose 96 97### Checking if the framework is enabled 98 99Look for the log statements from the framework: 100 101```console 102$ adb logcat -v time SceneFramework:* *:S 103``` 104 105### Disabling the framework 106 107To **disable** the framework, simply turn off the main aconfig flag: 108 109```console 110$ adb shell device_config put systemui com.android.systemui.scene_container false 111``` 112 113## Defining a scene 114 115By default, the framework ships with fully functional scenes as enumarated 116[here](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/src/com/android/systemui/scene/shared/model/SceneKey.kt). 117Should a variant owner or OEM want to replace or add a new scene, they could 118do so by defining their own scene. This section describes how to do that. 119 120Each scene is defined as an implementation of the 121[`Scene`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/features/src/com/android/systemui/scene/ui/composable/Scene.kt) 122interface, which has three parts: 1. The `key` property returns the 123[`SceneKey`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/src/com/android/systemui/scene/shared/model/SceneKey.kt) 124that uniquely identifies that scene 2. The `userActions` `Flow` returns 125the (potentially ever-changing) set of navigation edges to other content, based 126on user-actions, which is how the navigation graph is defined (see 127[the Scene navigation](#Scene-navigation) section for more) 3. The `Content` 128function which uses 129[Jetpack Compose](https://developer.android.com/jetpack/compose) to declare of 130the UI itself. This is the UI "at rest", e.g. once there is no transition 131between any two scenes. The Scene Framework has other ways to define how the 132content of your UI changes with and throughout a transition to learn more please 133see the [Scene transition animations](#Scene-transition-animations) section 134 135For example: 136 137```kotlin 138@SysUISingleton class YourScene @Inject constructor( /* your dependencies here */ ) : Scene { 139 override val key = SceneKey.YourScene 140 141 override val userActions: StateFlow<Map<UserAction, SceneModel>> = 142 MutableStateFlow<Map<UserAction, SceneModel>>( 143 mapOf( 144 // This is where scene navigation is defined, more on that below. 145 ) 146 ).asStateFlow() 147 148 @Composable 149 override fun SceneScope.Content( 150 modifier: Modifier, 151 ) { 152 // This is where the UI is defined using Jetpack Compose. 153 } 154} 155``` 156 157### Injecting scenes 158 159Scenes are injected into the Dagger dependency graph from the 160[`SceneModule`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/facade/enabled/src/com/android/systemui/scene/ui/composable/SceneModule.kt;l=35-50;drc=564f233d5b597aedf06961c76e582464eebe8ba6). 161 162## Scene navigation 163 164As seen above, each scene is responsible for providing an observable `Flow` of a 165`Map` that connects `UserAction` (for example: swipe down, swipe up, back 166button/gesture, etc.) keys to `SceneModel` destinations. This is how the scene 167navigation graph is defined. 168 169NOTE: this controls *only* user-input based navigation. To learn about the other 170type of scene navigation, please see the 171[Automatic scene transitions](#Automatic-scene-transitions) section. 172 173Because this is a `Flow`, scene implemetations should feel free to emit new 174values over time. For example, the `Lockscreen` scene ties the "swipe up" user 175action to go to the `Bouncer` scene if the device is still locked or to go to 176the `Gone` scene if the device is unlocked, allowing the user to dismiss the 177lockscreen UI when not locked. 178 179## Scene transition animations 180 181The Scene Framework separates transition animations from content UI declaration 182by placing the definition of the former in a different location. This way, 183there's no longer a need to contaminate the content UI declaration with 184animation logic, a practice that becomes unscalable over time. 185 186Under the hood, the Scene Framework uses 187[`SceneTransitionLayout`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/core/src/com/android/compose/animation/scene/SceneTransitionLayout.kt), 188a `@Composable` function designed with scene graph and transitions in mind. In 189fact, the Scene Framework is merely a shallow wrapper around 190`SceneTransitionLayout`. 191 192The `SceneTransitionLayout` API requires the transitions to be passed-in 193separately from the scenes themselves. In System UI, the transitions can be 194found in 195[`SceneContainerTransitions`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/features/src/com/android/systemui/scene/ui/composable/SceneContainerTransitions.kt). 196As you can see, each possible scene-to-scene transition has its own builder, 197here's one example: 198 199```kotlin 200fun TransitionBuilder.lockscreenToShadeTransition() { 201 spec = tween(durationMillis = 500) 202 203 punchHole(Shade.Elements.QuickSettings, bounds = Shade.Elements.Scrim, Shade.Shapes.Scrim) 204 translate(Shade.Elements.Scrim, Edge.Top, startsOutsideLayoutBounds = false) 205 fractionRange(end = 0.5f) { 206 fade(Shade.Elements.ScrimBackground) 207 translate( 208 QuickSettings.Elements.CollapsedGrid, 209 Edge.Top, 210 startsOutsideLayoutBounds = false, 211 ) 212 } 213 fractionRange(start = 0.5f) { fade(Notifications.Elements.Notifications) } 214} 215``` 216 217Going through the example code: 218 219* The `spec` is the animation that should be invoked, in the example above, we use a `tween` 220animation with a duration of 500 milliseconds 221* Then there's a series of function calls: `punchHole` applies a clip mask to the `Scrim` 222element in the destination scene (in this case it's the `Shade` scene) which has the 223position and size determined by the `bounds` parameter and the shape passed into the `shape` 224parameter. This lets the `Lockscreen` scene render "through" the `Shade` scene 225* The `translate` call shifts the `Scrim` element to/from the `Top` edge of the scene container 226* The first `fractionRange` wrapper tells the system to apply its contained functions 227only during the first half of the transition. Inside of it, we see a `fade` of 228the `ScrimBackground` element and a `translate` o the `CollpasedGrid` element 229to/from the `Top` edge 230* The second `fractionRange` only starts at the second half of the transition (e.g. when 231the previous one ends) and applies a `fade` on the `Notifications` element 232 233You can find the actual documentation for this API 234[here](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/core/src/com/android/compose/animation/scene/TransitionDsl.kt). 235 236### Tagging elements 237 238As demonstrated above, elements within a scene can be addressed from transition 239defintions. In order to "tag" an element with a specific `ElementKey`, the 240[`element` modifier](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/core/src/com/android/compose/animation/scene/SceneTransitionLayout.kt) 241must be used on the composable that declared that element's UI: 242 243```kotlin 244Text( 245 text = "Some text", 246 modifier = Modifier.element(MyElements.SomeText), 247) 248``` 249 250In addition to the ability to refer to a tagged element in transition 251definitions, if the same `ElementKey` is used for one element in the current 252scene and another element in the destination scene, the element is considered to 253be a **shared element**. As such, the framework automatically translates and 254scales the bounds of the shared element from its current bounds in the source 255scene to its final bounds in the destination scene. 256 257## Scene container 258 259To set up a scene framework instance, a scene container must be declared. This 260is the root of an entire scene graph that puts together the scenes, their 261transitions, and the configuration. The container is then added to a parent 262`@Composable` or `View` so it can be displayed. 263 264The default scene container in System UI is defined in the 265[`SceneContainer.kt` file](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/features/src/com/android/systemui/scene/ui/composable/SceneContainer.kt). 266 267### Scene container configuration 268 269The `SceneContainer` function is passed a few parameters including a view-model 270and a set of scenes. The exact details of what gets passed in depends on the 271[`SceneContainerConfig` object](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/src/com/android/systemui/scene/shared/model/SceneContainerConfig.kt) 272which is injected into the Dagger dependency graph 273[here](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/src/com/android/systemui/scene/shared/model/SceneContainerConfigModule.kt). 274 275## Automatic scene transitions 276 277The scene framework supports the ability for scenes to change automatically 278based on device state or events other than direct user input. For example: when 279the device is locked, there's an automatic scene transition to the `Lockscreen` 280scene. 281 282This logic is contained within the 283[`SceneContainerStartable`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/src/com/android/systemui/scene/domain/startable/SceneContainerStartable.kt) 284class. 285 286## Side-effects 287 288Similarly to [the above](#Automatic-scene-transitions), the 289`SceneContainerStartable` also handles side-effects by updating other parts of 290the System UI codebase whenever internal scene framework state changes. As an 291example: the visibility of the `View` that contains our 292[scene container](#Scene-container) is updated every time there's a transition 293to or from the `Gone` scene. 294 295## Observing scene transition state 296 297There are a couple of ways to observe the transition state: 298 2991. [Easiest] using the `SceneScope` of the scene container, simply use the 300 `animateSharedXAsState` API, the full list is 301 [here](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/core/src/com/android/compose/animation/scene/AnimateSharedAsState.kt). 3022. [Harder] if outside the `SceneScope` of the scene container, observe 303 [`SceneInteractor.transitionState`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/src/com/android/systemui/scene/domain/interactor/SceneInteractor.kt;l=88;drc=af57d5e49431c6728e7cf192bada88e0541ebf0c). 304 305## Dependency Injection 306 307The entire framework is provided into the Dagger dependency graph from the 308top-level Dagger module at 309[`SceneContainerFrameworkModule`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/src/com/android/systemui/scene/SceneContainerFrameworkModule.kt) 310this puts together the scenes from `SceneModule`, the configuration from 311`SceneContainerConfigModule`, and the startable from 312`SceneContainerStartableModule`. 313 314## Integration Notes 315 316### Relationship to Jetpack Compose 317 318The scene framework depends on Jetpack Compose; therefore, compiling System UI with 319Jetpack Compose is required. However, because Jetpack Compose and Android Views 320[interoperate](https://developer.android.com/jetpack/compose/migrate/interoperability-apis/views-in-compose), 321the UI in each scene doesn't necessarily need to be a pure hierarchy of `@Composable` 322functions; instead, it's acceptable to use an `AndroidView` somewhere in the 323hierarchy of composable functions to include a `View` or `ViewGroup` subtree. 324 325#### Interoperability with Views 326The scene framework comes with built-in functionality to animate the entire scene and/or 327elements within the scene in-tandem with the actual scene transition progress. 328 329For example, as the user drags their finger down rom the top of the lockscreen, 330the shade scene becomes visible and gradually expands, the amount of expansion tracks 331the movement of the finger. 332 333That feature of the framework uses a custom `element(ElementKey)` Jetpack Compose 334`Modifier` to refer to elements within a scene. 335The transition builders then use the same `ElementKey` objects to refer to those elements 336and describe how they animate in-tandem with scene transitions. Because this is a 337Jetpack Compose `Modifier`, it means that, in order for an element in a scene to be 338animated automatically by the framework, that element must be nested within a pure 339`@Composable` hierarchy. The element itself is allowed to be a classic Android `View` 340(nested within a Jetpack Compose `AndroidView`) but all ancestors must be `@Composable` 341functions. 342 343### Notifications 344 345As of January 2024, the integration of notifications and heads-up notifications (HUNs) 346into the scene framework follows an unusual pattern. We chose this pattern due to migration 347risk and performance concerns but will eventually replace it with the more common element 348placement pattern that all other elements are following. 349 350The special pattern for notifications is that, instead of the notification list 351(`NotificationStackScrollLayout` or "NSSL", which also displays HUNs) being placed in the element 352hierarchy within the scenes that display notifications, the NSSL (which continues to be an Android View) 353"floats" above the scene container, rendering on top of everything. This is very similar to 354how NSSL is integrated with the legacy shade, prior to the scene framework. 355 356In order to render the NSSL as if it's part of the organic hierarchy of elements within its 357scenes, we control the NSSL's self-imposed effective bounds (e.g. position offsets, clip path, 358size) from `@Composable` elements within the normal scene hierarchy. These special 359"placeholder" elements can be found 360[here](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/features/src/com/android/systemui/notifications/ui/composable/Notifications.kt). 361 362