xref: /aosp_15_r20/frameworks/base/packages/SystemUI/docs/scene.md (revision d57664e9bc4670b3ecf6748a746a57c557b6bc9e)
1# The Scene Framework
2
3Known internally as "Flexiglass", this framework defines a graph where each node
4is a "scene" and each edge between the scenes is a transition. The scenes are
5the main components of System UI, on phones these are: the lockscreen, bouncer,
6shade, and quick settings panels/views/screens). Each scene is a standalone
7experience.
8
9The **main goal** of the framework is to increase code health by applying
10[Separation of concerns](https://en.wikipedia.org/wiki/Separation_of_concerns)
11over several dimensions:
12
131.  Each scene is a standalone piece of UI; their code doesn't need to concern
14    itself with either transition animations or anything in other scenes. This
15    frees the developer to be able to focus only on the content of the UI for
16    that scene.
172.  Transition definitions (which scene leads to which other scene following
18    which user action) are pulled out and separated from the content of the UI.
193.  Transition animations (the effects that happen alongside the gradual change
20    from one scene to another) are also pulled out and separated from the
21    content of the UI.
22
23In addition to the above, some of the **secondary goals** are:
24
254. Make **customization easier**: by separating scenes to standalone pieces, it
26becomes possible for variant owners and OEMs to exclude or replace certain scenes
27or to add brand-new scenes.
285. **Enable modularization**: by separating scenes to standalone pieces, it
29becomes possible to break down System UI into smaller codebases, each one of
30which could be built on its own. Note: this isn't part of the scene framework
31itself but is something that can be done more easily once the scene framework
32is in place.
33
34## Terminology
35
36*   **Scene** a collection of UI elements in a layout that, together, make up a
37    "screen" or "page" that is as large as the container. Scenes can be
38    navigated between / transition to/from. To learn more, please see
39    [this section](#Defining-a-scene).
40*   **Element** (or "UI element") a single unit of UI within a scene. One scene
41    can arrange multiple elements within a layout structure.
42*   **Transition** the gradual switching from one scene to another scene. There
43    are two kinds: [user-driven](Scene-navigation) and
44    [automatic](Automatic-scene-transitions) scene transitions.
45*   **Transition animation** the set of UI effects that occurs while/during a
46    transition. These can apply to the entire scene or to specific elements in
47    the scene. To learn more, please see
48    [this section](#Scene-transition-animations).
49*   **Scene container** (or just "container") the root piece of UI (typically a
50    `@Composable` function) that sets up all the scenes, their transitions, etc.
51    To learn more, please see [this section](#Scene-container).
52*   **Container configuration** (or just "configuration") the collection of
53    scenes and some added information about the desired behaviour of a
54    container. To learn more, please see
55    [this section](#Scene-container-configuration).
56
57## Enabling the framework
58
59As of the end of 2023, the scene framework is under development; as such, it is
60disabled by default. For those who are interested in a preview, please follow
61the instructions below to turn it on.
62
63NOTE: in case these instructions become stale and don't actually enable the
64framework, please make sure `SceneContainerFlag.isEnabled` in the
65[`SceneContainerFlag.kt`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/src/com/android/systemui/scene/shared/flag/SceneContainerFlag.kt)
66file evaluates to `true`.
67
681.  Set a collection of **aconfig flags** to `true` by running the following
69    commands:
70    ```console
71    $ adb shell device_config override systemui com.android.systemui.keyguard_bottom_area_refactor true
72    $ adb shell device_config override systemui com.android.systemui.keyguard_wm_state_refactor true
73    $ adb shell device_config override systemui com.android.systemui.migrate_clocks_to_blueprint true
74    $ adb shell device_config override systemui com.android.systemui.notification_avalanche_throttle_hun true
75    $ adb shell device_config override systemui com.android.systemui.predictive_back_sysui true
76    $ adb shell device_config override systemui com.android.systemui.scene_container true
77    ```
782.  **Restart** System UI by issuing the following command:
79    ```console
80    $ adb shell am crash com.android.systemui
81    ```
823.  **Verify** that the scene framework was turned on. There are two ways to do
83    this:
84
85    *(a)* look for the sash/ribbon UI at the bottom-right corner of the display:
86    ![ribbon](imgs/ribbon.png)
87
88    NOTE: this will be removed proper to the actual release of the framework.
89
90    *(b)* Turn on logging and look for the logging statements in `logcat`:
91    ```console
92
93    # Turn on logging from the framework:
94
95    $ adb shell cmd statusbar echo -b SceneFramework:verbose
96
97### Checking if the framework is enabled
98
99Look for the log statements from the framework:
100
101```console
102$ adb logcat -v time SceneFramework:* *:S
103```
104
105### Disabling the framework
106
107To **disable** the framework, simply turn off the main aconfig flag:
108
109```console
110$ adb shell device_config put systemui com.android.systemui.scene_container false
111```
112
113## Defining a scene
114
115By default, the framework ships with fully functional scenes as enumarated
116[here](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/src/com/android/systemui/scene/shared/model/SceneKey.kt).
117Should a variant owner or OEM want to replace or add a new scene, they could
118do so by defining their own scene. This section describes how to do that.
119
120Each scene is defined as an implementation of the
121[`Scene`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/features/src/com/android/systemui/scene/ui/composable/Scene.kt)
122interface, which has three parts: 1. The `key` property returns the
123[`SceneKey`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/src/com/android/systemui/scene/shared/model/SceneKey.kt)
124that uniquely identifies that scene 2. The `userActions` `Flow` returns
125the (potentially ever-changing) set of navigation edges to other content, based
126on user-actions, which is how the navigation graph is defined (see
127[the Scene navigation](#Scene-navigation) section for more) 3. The `Content`
128function which uses
129[Jetpack Compose](https://developer.android.com/jetpack/compose) to declare of
130the UI itself. This is the UI "at rest", e.g. once there is no transition
131between any two scenes. The Scene Framework has other ways to define how the
132content of your UI changes with and throughout a transition to learn more please
133see the [Scene transition animations](#Scene-transition-animations) section
134
135For example:
136
137```kotlin
138@SysUISingleton class YourScene @Inject constructor( /* your dependencies here */ ) : Scene {
139    override val key = SceneKey.YourScene
140
141    override val userActions: StateFlow<Map<UserAction, SceneModel>> =
142        MutableStateFlow<Map<UserAction, SceneModel>>(
143            mapOf(
144                // This is where scene navigation is defined, more on that below.
145            )
146        ).asStateFlow()
147
148    @Composable
149    override fun SceneScope.Content(
150        modifier: Modifier,
151    ) {
152        // This is where the UI is defined using Jetpack Compose.
153    }
154}
155```
156
157### Injecting scenes
158
159Scenes are injected into the Dagger dependency graph from the
160[`SceneModule`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/facade/enabled/src/com/android/systemui/scene/ui/composable/SceneModule.kt;l=35-50;drc=564f233d5b597aedf06961c76e582464eebe8ba6).
161
162## Scene navigation
163
164As seen above, each scene is responsible for providing an observable `Flow` of a
165`Map` that connects `UserAction` (for example: swipe down, swipe up, back
166button/gesture, etc.) keys to `SceneModel` destinations. This is how the scene
167navigation graph is defined.
168
169NOTE: this controls *only* user-input based navigation. To learn about the other
170type of scene navigation, please see the
171[Automatic scene transitions](#Automatic-scene-transitions) section.
172
173Because this is a `Flow`, scene implemetations should feel free to emit new
174values over time. For example, the `Lockscreen` scene ties the "swipe up" user
175action to go to the `Bouncer` scene if the device is still locked or to go to
176the `Gone` scene if the device is unlocked, allowing the user to dismiss the
177lockscreen UI when not locked.
178
179## Scene transition animations
180
181The Scene Framework separates transition animations from content UI declaration
182by placing the definition of the former in a different location. This way,
183there's no longer a need to contaminate the content UI declaration with
184animation logic, a practice that becomes unscalable over time.
185
186Under the hood, the Scene Framework uses
187[`SceneTransitionLayout`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/core/src/com/android/compose/animation/scene/SceneTransitionLayout.kt),
188a `@Composable` function designed with scene graph and transitions in mind. In
189fact, the Scene Framework is merely a shallow wrapper around
190`SceneTransitionLayout`.
191
192The `SceneTransitionLayout` API requires the transitions to be passed-in
193separately from the scenes themselves. In System UI, the transitions can be
194found in
195[`SceneContainerTransitions`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/features/src/com/android/systemui/scene/ui/composable/SceneContainerTransitions.kt).
196As you can see, each possible scene-to-scene transition has its own builder,
197here's one example:
198
199```kotlin
200fun TransitionBuilder.lockscreenToShadeTransition() {
201    spec = tween(durationMillis = 500)
202
203    punchHole(Shade.Elements.QuickSettings, bounds = Shade.Elements.Scrim, Shade.Shapes.Scrim)
204    translate(Shade.Elements.Scrim, Edge.Top, startsOutsideLayoutBounds = false)
205    fractionRange(end = 0.5f) {
206        fade(Shade.Elements.ScrimBackground)
207        translate(
208            QuickSettings.Elements.CollapsedGrid,
209            Edge.Top,
210            startsOutsideLayoutBounds = false,
211        )
212    }
213    fractionRange(start = 0.5f) { fade(Notifications.Elements.Notifications) }
214}
215```
216
217Going through the example code:
218
219* The `spec` is the animation that should be invoked, in the example above, we use a `tween`
220animation with a duration of 500 milliseconds
221* Then there's a series of function calls: `punchHole` applies a clip mask to the `Scrim`
222element in the destination scene (in this case it's the `Shade` scene) which has the
223position and size determined by the `bounds` parameter and the shape passed into the `shape`
224parameter. This lets the `Lockscreen` scene render "through" the `Shade` scene
225* The `translate` call shifts the `Scrim` element to/from the `Top` edge of the scene container
226* The first `fractionRange` wrapper tells the system to apply its contained functions
227only during the first half of the transition. Inside of it, we see a `fade` of
228the `ScrimBackground` element and a `translate` o the `CollpasedGrid` element
229to/from the `Top` edge
230* The second `fractionRange` only starts at the second half of the transition (e.g. when
231the previous one ends) and applies a `fade` on the `Notifications` element
232
233You can find the actual documentation for this API
234[here](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/core/src/com/android/compose/animation/scene/TransitionDsl.kt).
235
236### Tagging elements
237
238As demonstrated above, elements within a scene can be addressed from transition
239defintions. In order to "tag" an element with a specific `ElementKey`, the
240[`element` modifier](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/core/src/com/android/compose/animation/scene/SceneTransitionLayout.kt)
241must be used on the composable that declared that element's UI:
242
243```kotlin
244Text(
245    text = "Some text",
246    modifier = Modifier.element(MyElements.SomeText),
247)
248```
249
250In addition to the ability to refer to a tagged element in transition
251definitions, if the same `ElementKey` is used for one element in the current
252scene and another element in the destination scene, the element is considered to
253be a **shared element**. As such, the framework automatically translates and
254scales the bounds of the shared element from its current bounds in the source
255scene to its final bounds in the destination scene.
256
257## Scene container
258
259To set up a scene framework instance, a scene container must be declared. This
260is the root of an entire scene graph that puts together the scenes, their
261transitions, and the configuration. The container is then added to a parent
262`@Composable` or `View` so it can be displayed.
263
264The default scene container in System UI is defined in the
265[`SceneContainer.kt` file](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/features/src/com/android/systemui/scene/ui/composable/SceneContainer.kt).
266
267### Scene container configuration
268
269The `SceneContainer` function is passed a few parameters including a view-model
270and a set of scenes. The exact details of what gets passed in depends on the
271[`SceneContainerConfig` object](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/src/com/android/systemui/scene/shared/model/SceneContainerConfig.kt)
272which is injected into the Dagger dependency graph
273[here](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/src/com/android/systemui/scene/shared/model/SceneContainerConfigModule.kt).
274
275## Automatic scene transitions
276
277The scene framework supports the ability for scenes to change automatically
278based on device state or events other than direct user input. For example: when
279the device is locked, there's an automatic scene transition to the `Lockscreen`
280scene.
281
282This logic is contained within the
283[`SceneContainerStartable`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/src/com/android/systemui/scene/domain/startable/SceneContainerStartable.kt)
284class.
285
286## Side-effects
287
288Similarly to [the above](#Automatic-scene-transitions), the
289`SceneContainerStartable` also handles side-effects by updating other parts of
290the System UI codebase whenever internal scene framework state changes. As an
291example: the visibility of the `View` that contains our
292[scene container](#Scene-container) is updated every time there's a transition
293to or from the `Gone` scene.
294
295## Observing scene transition state
296
297There are a couple of ways to observe the transition state:
298
2991.  [Easiest] using the `SceneScope` of the scene container, simply use the
300    `animateSharedXAsState` API, the full list is
301    [here](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/core/src/com/android/compose/animation/scene/AnimateSharedAsState.kt).
3022.  [Harder] if outside the `SceneScope` of the scene container, observe
303    [`SceneInteractor.transitionState`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/src/com/android/systemui/scene/domain/interactor/SceneInteractor.kt;l=88;drc=af57d5e49431c6728e7cf192bada88e0541ebf0c).
304
305## Dependency Injection
306
307The entire framework is provided into the Dagger dependency graph from the
308top-level Dagger module at
309[`SceneContainerFrameworkModule`](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/src/com/android/systemui/scene/SceneContainerFrameworkModule.kt)
310this puts together the scenes from `SceneModule`, the configuration from
311`SceneContainerConfigModule`, and the startable from
312`SceneContainerStartableModule`.
313
314## Integration Notes
315
316### Relationship to Jetpack Compose
317
318The scene framework depends on Jetpack Compose; therefore, compiling System UI with
319Jetpack Compose is required. However, because Jetpack Compose and Android Views
320[interoperate](https://developer.android.com/jetpack/compose/migrate/interoperability-apis/views-in-compose),
321the UI in each scene doesn't necessarily need to be a pure hierarchy of `@Composable`
322functions; instead, it's acceptable to use an `AndroidView` somewhere in the
323hierarchy of composable functions to include a `View` or `ViewGroup` subtree.
324
325#### Interoperability with Views
326The scene framework comes with built-in functionality to animate the entire scene and/or
327elements within the scene in-tandem with the actual scene transition progress.
328
329For example, as the user drags their finger down rom the top of the lockscreen,
330the shade scene becomes visible and gradually expands, the amount of expansion tracks
331the movement of the finger.
332
333That feature of the framework uses a custom `element(ElementKey)` Jetpack Compose
334`Modifier` to refer to elements within a scene.
335The transition builders then use the same `ElementKey` objects to refer to those elements
336and describe how they animate in-tandem with scene transitions. Because this is a
337Jetpack Compose `Modifier`, it means that, in order for an element in a scene to be
338animated automatically by the framework, that element must be nested within a pure
339`@Composable` hierarchy. The element itself is allowed to be a classic Android `View`
340(nested within a Jetpack Compose `AndroidView`) but all ancestors must be `@Composable`
341functions.
342
343### Notifications
344
345As of January 2024, the integration of notifications and heads-up notifications (HUNs)
346into the scene framework follows an unusual pattern. We chose this pattern due to migration
347risk and performance concerns but will eventually replace it with the more common element
348placement pattern that all other elements are following.
349
350The special pattern for notifications is that, instead of the notification list
351(`NotificationStackScrollLayout` or "NSSL", which also displays HUNs) being placed in the element
352hierarchy within the scenes that display notifications, the NSSL (which continues to be an Android View)
353"floats" above the scene container, rendering on top of everything. This is very similar to
354how NSSL is integrated with the legacy shade, prior to the scene framework.
355
356In order to render the NSSL as if it's part of the organic hierarchy of elements within its
357scenes, we control the NSSL's self-imposed effective bounds (e.g. position offsets, clip path,
358size) from `@Composable` elements within the normal scene hierarchy. These special
359"placeholder" elements can be found
360[here](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/packages/SystemUI/compose/features/src/com/android/systemui/notifications/ui/composable/Notifications.kt).
361
362