Raspbian trixie - #1106
Raspbian trixie#1106SteveMicroNova wants to merge 76 commits into
Conversation
39a13a4 to
42880db
Compare
|
This is a WIP that will take a while to finish This will change the baseline partition scheme of amplipi from basic boot+root to a 7 partition schema:
Currently, I have the ability to update from images via the Once in the other boot+root slot, the update verification service checks to see if that |
| the stored default AmpliPi password.""" | ||
|
|
||
| # Password config location | ||
| PASS_DIR = os.path.join(os.path.expanduser('~'), '.config', 'amplipi') |
There was a problem hiding this comment.
The whole .config folder and a few services got moved to /data to ensure they persist post-update, many lines like this had to be changed as a result
| [all] | ||
| tryboot_a_b=1 | ||
| boot_partition=2 | ||
|
|
||
| [tryboot] | ||
| boot_partition=3 | ||
|
|
||
| if exist mmc 0:1 /boot_partition.txt then | ||
| load mmc 0:1 ${loadaddr} /boot_partition.txt | ||
| env import -t ${loadaddr} ${filesize} | ||
| fi | ||
|
|
||
| boot mmc 0:${boot_partition} |
There was a problem hiding this comment.
This is the default autoboot.txt, which is what the update verification service edits when an update is successful. Specifically what is changed is swapping the [all] boot_partition value with the [tryboot] boot_partition value, that's all it takes to swap the default boot slot
| * A micro USB cable | ||
| * Your AmpliPi | ||
|
|
||
| ## Optional Step: Preserve your config |
There was a problem hiding this comment.
This file likely needs more reworking as realistically you shouldn't need to flash ever again, though also I'd say that makes more sense to leave until this code is in the hands of users and has a few support cases under its belt to verify if that's necessary
| # Run device-specific customization scripts from p7. | ||
| # These survive OTA updates and are re-applied on each new slot's first boot. | ||
| if [ -d "${SCRIPTS_DIR}" ]; then | ||
| for script in "${SCRIPTS_DIR}"/*.sh; do | ||
| [ -f "${script}" ] || continue | ||
| log "Running ${script}..." | ||
| bash "${script}" || log "Warning: ${script} exited non-zero" | ||
| done | ||
| fi |
There was a problem hiding this comment.
Some users have specific setups on their devices that we don't want to destroy on a full-image update, so I've given them a directory that they can fill with bash scripts to set up their services automatically post-update
I need to test what happens if a user provides scripts that fail for those cases
| @@ -9,10 +9,10 @@ Bootstrap a Raspberry Pi Compute Module for first-time AmpliPi setup. | |||
| --hostname NAME: The hostname to set on the pi, so once this script | |||
| successfully completes connect via {hostname}.local. | |||
| Default is amplipi | |||
| --cli-only: Use the lite version of Raspberry Pi OS which does not have | |||
There was a problem hiding this comment.
At some point, CLI only became a default case for our units. I decided to flip the default case in this script to reflect that
There was a problem hiding this comment.
This file either needs full reworking or deletion. The new 7 partition setup is complex enough that updating this is a bit of a lift when we hardly need it and can also just distribute a full 7 partition image for those who need that sort of thing
There was a problem hiding this comment.
The answer is reworking. It won't be a simple portion, but creating a clean image that isn't mucked up with any dev happenings is an important goal and this script is likely one of the better ways to achieve that
There was a problem hiding this comment.
Unlike bootstrap.py, I'd say configure.py is still very useful despite not being our update path anymore. manual deploys with the deploy+configure script are likely to be the primary ways we configure the root partitions that later get snapshotted into an image we distribute as an update
| vol, vol_f, vol_f_delta, vol_min, or vol_max. | ||
| """ | ||
| # Field precedence: vol (db) > vol_delta > vol (float) | ||
| # vol (db) is first in precedence yet last in the stack to cover the default case of no volume change |
There was a problem hiding this comment.
This looks like a rebase error. This branch is long lived and had some merge conflicts on rebase and so may need to be combed for reversions like this before merging
| try: | ||
| self.bus.close() | ||
| except Exception: | ||
| pass |
There was a problem hiding this comment.
This is an additional fix for the issue fixed by #1098 which was merged in #1091
My unit actually got into a bad state during testing, which led to me having the ability to test those changes and add a little more to the fix
My own issue was downstream of some configuration file providing an environment that the I2C didn't like
| 5. (Optional) Run the cleanup script to make the system 'factory fresh' with a randomgen ssh password and empty logs | ||
| " | ||
|
|
||
|
|
||
| echo " | ||
| NOTE: a modern AmpliPi setup is more complicated than simple bash scripting will allow, to match what a normal Pi would contain please follow the above directions, and then: | ||
| 1. use `sudo dd if=/dev/sdx1 of=amplipi-boot.img status=progress bs=4MiB` to make an image of your boot directory, changing sdx into whatever sd your device mounted to | ||
| 2. use `sudo dd if=/dev/sdx2 of=amplipi-root.img status=progress bs=4MiB` to make an image of your root directory, changing sdx into whatever sd your device mounted to | ||
| 3. use parted or gparted to wipe all partitions and make the following partitions: | ||
|
|
||
| p1: FAT32, 516 MB, flash with amplipi-boot.img | ||
| p2: FAT32, 516 MB, flash with amplipi-boot.img | ||
| p3: FAT32, 516 MB, flash with amplipi-boot.img | ||
| p4: Extended partition containing p5-7 containing remainder of disk | ||
| p5: EXT4, 10939 MB, flash with amplipi-root.img | ||
| p6: EXT4, 10939 MB, flash with amplipi-root.img | ||
| p7: EXT4, sized to fill the remaining space | ||
|
|
||
| 4. Edit the contents of the partitions as so: | ||
|
|
||
| p1: add autoboot.txt as seen in /config/autoboot.txt | ||
| p2: adjust cmdline.txt to point to partition 5 | ||
| p3: adjust cmdline.txt to point to partition 6 | ||
|
|
||
| p5: Add p7 to fstab as /data | ||
| p6: Add p7 to fstab as /data | ||
| p7: create a .config folder | ||
| " |
There was a problem hiding this comment.
This is useful documentation for someone who might be trying to get their bearings on this PR
There was a problem hiding this comment.
Also worth noting that this section isn't useful as a guide to get a unit up and running anymore
As I've continued working I've found more specific details that simply will not be captured here due to being things like "go deep into this directory and change this line in a random config file that I don't remember anymore"
There was a problem hiding this comment.
I've just considered that this file needs a greater explanation
The official docs are a bit sparse (or were when I first read them months ago), so I'll also explain myself:
tryboot_a_b=1 activates the tryboot mechanism, something supported in the raspbian kernel that allows for partition swapping using the command sudo reboot "0 tryboot". Using specifically that "0 tryboot" arg is how you boot into the partition listed under the [tryboot] boot_partition section.
That "0 tryboot" arg simply loads the boot partition listed under the [tryboot] header, it does not toggle anything in this file to make that partition the new default (which is handled by the amplipi-postflash service I've made), so doing a simple sudo reboot from the tryboot partition will send you back to whatever partition is listed under the [all] header
All the actual lifting is done by the individual boot partitions, which have their own kernels and mappings to their own root partitions
| <!-- | ||
| Progress bar, dynamically sized depending on if the update needs to be downloaded first as well as if the boot image is present | ||
| this translates to there being a minimum of two segments if the update is already downloaded and is only for root, or 5 if it isn't downloaded and contains boot | ||
| --> | ||
| <div class="p-3 pb-0"> | ||
| <div class="progress" style="position: relative;"> | ||
| <div id="update-progress-bar" class="progress-bar progress-bar-striped progress-bar-animated bg-info" role="progressbar" style="width: 0%" aria-valuenow="0" aria-valuemin="0" aria-valuemax="100">0%</div> | ||
| </div> | ||
| <div class="d-flex text-center mt-1"> | ||
| <small class="update-progress-label d-none" data-label="Downloading root image" style="width: 0%;">Download Root</small> | ||
| <small class="update-progress-label d-none" data-label="Downloading boot image" style="width: 0%;">Download Boot</small> | ||
| <small class="update-progress-label d-none" data-label="Verifying root image" style="width: 0%;">Verify Root</small> | ||
| <small class="update-progress-label d-none" data-label="Verifying boot image" style="width: 0%;">Verify Boot</small> | ||
| <small class="update-progress-label d-none" data-label="Flashing root" style="width: 0%;">Flash Root</small> | ||
| <small class="update-progress-label d-none" data-label="Flashing boot" style="width: 0%;">Flash Boot</small> | ||
| <small class="update-progress-label d-none" data-label="Downloading manifest" style="width: 0%;">Manifest</small> | ||
| <small class="update-progress-label d-none" data-label="Downloading release" style="width: 0%;">Download</small> | ||
| <small class="update-progress-label d-none" data-label="Applying update" style="width: 0%;">Apply</small> | ||
| </div> |
There was a problem hiding this comment.
I added a weighted progress bar during these changes to help make the updater feel more responsive and less like a black box of sorts
This is possible now that we're dealing with files of known size, and the system knows how far along the flashing process it is.
Here's an example of what it looks like:
| class SSEChannel: | ||
| """ Bundles the queue/in-progress-flag/latest-status trio a long-running background job reports | ||
| its progress through, plus the polling generator that serves it as SSE to a native EventSource. | ||
| /update/flash and /update/download/images each run their own job in a background thread, | ||
| separate from the SSE connection itself, so a dropped/reconnecting browser can't interrupt or | ||
| duplicate the actual work - progress is watched via a paired GET .../progress endpoint instead. """ |
There was a problem hiding this comment.
I needed to ensure that there could only be one update running at a time (a problem with our current updater, but even more important to handle here) and thus I had to make a progress tracking channel.
While doing so, I also had the idea of having a visible progress bar on the frontend that this now also feeds
There was a problem hiding this comment.
This is probably going to be the most useful file during PR review, keeping this up to date will prove easier than keeping the actual PR description up to date especially since I can easily link to specific files from this
3bbb544 to
e4d9a70
Compare
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1106 +/- ##
==========================================
- Coverage 50.67% 48.06% -2.61%
==========================================
Files 40 41 +1
Lines 7154 8004 +850
==========================================
+ Hits 3625 3847 +222
- Misses 3529 4157 +628
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…ure.py to better handle installs from point 0 as well as better handle updates with this new OS
Update check_pass to explicitly use python 3.8 instead of python 3.13 add with_alsa flag to configure.py to optionally forcefully override
…k in configure script
…hanges in files referencing moved scripts
…scripts to ensure proper routing
…hat always pass the assertion that the tests are worried about
…visible ways on failed service installs
… a good unit's identity, add an autopass flag for use with automated runs
… something that should've been a full image update
73f3700 to
0ffc994
Compare
1. better enforcing update manifest contents 2. ensuring that the update_validation trigger is only created at the end of a successful update rather than partway through
…nd handle reference migrations across the repo
…i-firstboot-password service rename the amplipi-firstboot-password service to just amplipi-firstboot service to reflect this multitooling
…progenitor scripts
What does this change intend to accomplish?
This is a reworking of AmpliPi to function in a 64 bit raspbian trixie A:B tryboot setup with the goal of having a more robust system overall and a cleaner update flow with full image updates
Keep an eye on the docs/a_b_update_design.md file for a good rundown of the high level design and purpose of the new components within this PR
A few things that need to be worked on yet:
Checklist
./scripts/test