1# Scripts 2 3This directory contains scripts for maintaining bpftool's GitHub mirror. 4 5## gh-label-release-assets.sh 6 7This script can **add labels to GitHub release assets**. Labels are used in the 8GitHub interface instead of the file name in the list of assets, but they do 9not alter the download URL. 10 11The script takes the tag reference for the release as single parameter. The 12repository to use, and the names and labels to set for assets are currently 13defined in the script itself. 14 15It requires the [GitHub command line][gh] (`gh`). 16 17Note that only users with push access to the repository can update release 18assets and set labels. 19 20[gh]: https://cli.github.com/ 21 22## sync-kernel.sh 23 24### Synchronize Linux and bpftool mirror 25 26This script synchronizes the bpftool mirror with the bpftool sources (and 27related files) from the Linux kernel repository. 28 29Synchronization is usually performed against the `bpf-next` and `bpf` trees, 30because this is where most bpftool updates are merged. 31 32By default, the script does not pick the very latest commits in these trees, 33but instead it uses the commits referenced in the libbpf submodule. This is 34because bpftool strongly relies on libbpf, and the libbpf GitHub mirror is used 35here as a submodule dependency. This libbpf mirror is also periodically updated 36to the latest `bpf-next` and `bpf` tree, and records to what kernel commits it 37was brought up-to-date. To ensure optimal compatibility between the bpftool 38sources and the libbpf dependency, we want to update them to the same point of 39reference. 40 41### Prerequisites 42 43There is no particular tool required for running the script, except `git` of 44course. 45 46However, you need a local copy of the Linux Git repository on your system 47in order to successfully run the script. You can set it up as follows: 48 49```console 50$ git clone 'https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git' linux 51$ cd linux 52$ git remote rename origin bpf-next 53$ git branch --move master bpf-next 54$ git remote add bpf 'https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git' 55$ git fetch bpf 56$ git checkout bpf/master 57$ git switch --create bpf 58$ git branch --set-upstream-to bpf/master 59$ git switch bpf-next 60``` 61 62At the end of the process, the repository should contain two branches 63`bpf-next` and `bpf`, pointing to the `master` branches of the `bpf-next` and 64`bpf` remote repositories, respectively. These two branches are required to 65synchronize the mirror correctly. 66 67You can later update this repository with the following commands: 68 69```console 70$ git switch bpf 71$ git pull --set-upstream bpf master 72$ git switch bpf-next 73$ git pull --set-upstream bpf-next master 74``` 75 76Also make sure you have cloned the bpftool mirror recursively, to check out the 77libbpf submodule. If you have not, run the following: 78 79```console 80$ git submodule update --init 81``` 82 83### Usage 84 85As preliminary steps: 86 87- Make sure that you have a Linux Git repository installed (see above), with 88 two branches `bpf-next` and `bpf` up-to-date. 89- Make sure your mirror repository is clean. 90 91Then run the script: 92 93```console 94$ ./sync-kernel.sh <bpftool-repo> <kernel-repo> 95``` 96 97If working from within the bpftool repository, the path to the `<bpftool-repo>` 98is typically the current working directory (`.`). The second argument, 99`<kernel-repo>`, is the path to the Linux Git repository mentioned earlier. 100 101Several environment variables can modify some internal parameters of the 102script: 103 104- Set `BPF_NEXT_BASELINE `to override the `bpf-next` tree commit to use (the 105 commit from the `bpf-next` branch with which the bpftool repository is 106 currently synchronized, prior to running the script). If unset, use the hash 107 from `<bpftool-repo>/CHECKPOINT-COMMIT` is used. 108- Set `BPF_BASELINE `to override the `bpf` tree commit to use (the commit from 109 the `bpf` branch with which the bpftool repository is currently synchronized, 110 prior to running the script). If unset, use the hash from 111 `<bpftool-repo>/BPF-CHECKPOINT-COMMIT` is used. 112- Set `BPF_NEXT_TIP_COMMIT` to override the `bpf-next` tree target commit (the 113 commit in `bpf-next` up to which the bpftool mirror should be synchronized). 114 If unset, use the hash from `<bpftool-repo>/libbpf/CHECKPOINT-COMMIT`, after 115 the libbpf repository update that takes place at the beginning of the update 116 process. 117- Set `BPF_TIP_COMMIT` to override the `bpf` tree target commit (the commit in 118 `bpf` up to which the bpftool mirror should be synchronized). If unset, use 119 the hash from `<bpftool-repo>/libbpf/BPF-CHECKPOINT-COMMIT`, after the libbpf 120 repository update that takes place at the beginning of the update process. 121- Set `SKIP_LIBBPF_UPDATE` to `1` to avoid updating libbpf automatically. 122- Set `MANUAL_MODE` to `1` to manually control every cherry-picked commit. 123 124### How it works 125 126This script synchronizes the bpftool mirror with upstream bpftool sources from 127the Linux kernel repository. 128 129It performs the following steps: 130 131- First, the script updates the libbpf submodule, commits the change, and (by 132 default) picks up libbpf's latest checkpoints to use them as target commits 133 for the bpftool mirror. 134 135- In the Linux repository, from the `bpf-next` branch, it creates new branches, 136 filters out all non-bpftool-related files, and reworks the layout to 137 replicate the layout from the bpftool mirror. 138 139- It generates patches for each commit touching `bpftool` or the required UAPI 140 files, up to the target commit, and exports these patches to a temporary 141 repository. 142 143- In a new branch in the bpftool mirror, the script applies (`git am`) each of 144 these patches to the mirror. 145 146- Then the script checks out the `bpf` branch in the Linux repository, and 147 repeats the same operations. 148 149- On top of the new patches applied to the mirror, the script creates a last 150 commit with the updated checkpoints, using a cover letter summarizing the 151 changes as the commit description. 152 153- The next step is verification. The script applies to the Linux repository 154 (`bpf-next` branch) a patch containing known differences between the Linux 155 repository and the bpftool mirror. Then it looks for remaining differences 156 between the two repositories, and warn the user if it finds any. Patches 157 picked up from the `bpf` tree are usually a source of differences at this 158 step. If the patch containing the known differences is to be updated after 159 the synchronization in progress, the user should do it at this time, before 160 the temporary files from the script are deleted. 161 162- At last, the script cleans up the temporary files and branches in the Linux 163 repository. Note that these temporary files and branches are not cleaned up 164 if the script fails during execution. 165