Skip to content

Commit 8e3257b

Browse files
steveseguinactions-user
authored andcommitted
```
fix(jetson): Re-enable GStreamer `gtksink` preview in desktop sessions This commit resolves issues with desktop video previews failing or rendering incorrectly on NVIDIA Jetson devices when `publish.py` is run within a GNOME/X11 terminal session. - Introduces `installers/nvidia_jetson/gtk_gtksink_patch_installer.sh`. This script rebuilds the Gtk introspection stack, `glib`, `gobject-introspection`, and the custom GStreamer tree, ensuring `gtksink` is properly enabled and typelibs are correctly linked for desktop GUI use. - Updates `publish.py` to: - Prioritize `gtksink` in `select_display_sink` for Jetson X11. - Modify `ximagesink` to use `handle-events=false`. - Implement specific `videoconvert` format handling (`I420`, `BGRx`, `RGBA`) for `xvimagesink`, `gtksink`/`ximagesink`, and `glimagesink` respectively, improving pipeline compatibility and rendering. - Adds instructions to `README.md` and `installers/nvidia_jetson/README.md` guiding users to apply the patch for restoring desktop preview functionality. ``` [auto-enhanced]
1 parent 544a4bc commit 8e3257b

4 files changed

Lines changed: 288 additions & 9 deletions

File tree

README.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -176,6 +176,16 @@ cd ~/raspberry_ninja/installers/nvidia_jetson
176176
./quick_update.sh
177177
```
178178

179+
If you plan to launch the desktop preview from GNOME (for example when running `publish.py` from a terminal window in the GUI) on the Jetson Nano 16GB image, run the Gtk/GStreamer patch helper once to restore the `gtksink` preview support:
180+
181+
```
182+
cd ~/raspberry_ninja/installers/nvidia_jetson
183+
chmod +x gtk_gtksink_patch_installer.sh # first run only
184+
./gtk_gtksink_patch_installer.sh
185+
```
186+
187+
It rebuilds the Gtk introspection stack and the custom GStreamer tree so playback works inside the desktop session, not just when autostarted on boot.
188+
179189
Only use `installer.sh` when building a new image from scratch; it performs
180190
heavy package clean-up and distro upgrades that are unnecessary on top of the
181191
ready-made images.

installers/nvidia_jetson/README.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,22 @@ major cleanups, removes large desktop packages, and attempts a distro upgrade,
2323
all of which are unnecessary—and potentially disruptive—on top of the provided
2424
pre-built images.
2525

26+
### Restore Gtk previews on GNOME
27+
28+
If you upgraded an older image with `installer.sh` and desktop previews fail to
29+
open under GNOME, run the Gtk/GStreamer patch helper to rebuild the Gtk-enabled
30+
pipeline and resync the typelib files:
31+
32+
```
33+
cd ~/raspberry_ninja/installers/nvidia_jetson
34+
chmod +x gtk_gtksink_patch_installer.sh # first run only
35+
./gtk_gtksink_patch_installer.sh
36+
```
37+
38+
It rebuilds the Gtk introspection stack, refreshes the custom GStreamer tree,
39+
and exposes the resulting typelibs to `/usr/local` so the `gtksink` preview
40+
window works correctly from a Jetson desktop session.
41+
2642
### Disable screen blanking
2743

2844
When you run `sudo ./setup_autostart.sh` it now asks whether to disable the Jetson
Lines changed: 241 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,241 @@
1+
#!/usr/bin/env bash
2+
# Targeted Jetson patch installer that rebuilds the Gtk introspection stack
3+
# and the custom GStreamer tree so that gtksink works with the Python viewer.
4+
set -euo pipefail
5+
6+
log() {
7+
printf '[jetson-gtk-patch] %s\n' "$*"
8+
}
9+
10+
export PATH="$HOME/.local/bin:/usr/local/bin:${PATH}"
11+
export PKG_CONFIG_PATH="/usr/local/lib/pkgconfig:/usr/local/lib/aarch64-linux-gnu/pkgconfig:${PKG_CONFIG_PATH-}"
12+
13+
if [[ "$(id -u)" -eq 0 ]]; then
14+
SUDO=""
15+
else
16+
if command -v sudo >/dev/null 2>&1; then
17+
SUDO="sudo"
18+
else
19+
log "This script requires sudo privileges."
20+
exit 1
21+
fi
22+
fi
23+
24+
APT_PACKAGES=(
25+
build-essential
26+
ninja-build
27+
git
28+
pkg-config
29+
python3
30+
python3-pip
31+
python3-gi
32+
gir1.2-gtk-3.0
33+
libgtk-3-dev
34+
libgirepository1.0-dev
35+
libglib2.0-dev
36+
libffi-dev
37+
libmount-dev
38+
libselinux1-dev
39+
libpcre3-dev
40+
libxml2-dev
41+
flex
42+
bison
43+
libcairo2-dev
44+
libdrm-dev
45+
libegl1
46+
libgles2
47+
libwayland-dev
48+
libxkbcommon-dev
49+
libepoxy-dev
50+
)
51+
52+
SRC_ROOT=${SRC_ROOT:-"$HOME/src/jetson-gtk-patch"}
53+
GLIB_VERSION=${GLIB_VERSION:-"2.76.1"}
54+
GI_VERSION=${GI_VERSION:-"1.76.1"}
55+
GSTREAMER_REF=${GSTREAMER_REF:-"1.23.0"}
56+
GSTREAMER_REPO=${GSTREAMER_REPO:-"https://gitlab.freedesktop.org/gstreamer/gstreamer.git"}
57+
GSTREAMER_DIR=${GSTREAMER_DIR:-"$SRC_ROOT/gstreamer"}
58+
TYPELIB_TARGET=${TYPELIB_TARGET:-"/usr/local/lib/girepository-1.0"}
59+
60+
mkdir -p "${SRC_ROOT}"
61+
62+
ensure_prereqs() {
63+
if [[ -z "${RN_SKIP_APT:-}" ]]; then
64+
log "Installing build prerequisites via apt-get…"
65+
$SUDO apt-get update
66+
$SUDO apt-get install -y "${APT_PACKAGES[@]}"
67+
else
68+
log "Skipping apt-get because RN_SKIP_APT is set."
69+
fi
70+
71+
log "Ensuring recent meson and ninja via pip…"
72+
python3 -m pip install --user --upgrade pip
73+
python3 -m pip install --user --upgrade meson ninja
74+
}
75+
76+
already_on_version() {
77+
local pkg=$1
78+
local want=$2
79+
80+
if ! command -v pkg-config >/dev/null 2>&1; then
81+
return 1
82+
fi
83+
84+
if ! pkg-config --exists "${pkg}"; then
85+
return 1
86+
fi
87+
88+
local have
89+
have=$(pkg-config --modversion "${pkg}")
90+
[[ "${have}" == "${want}" ]]
91+
}
92+
93+
build_glib() {
94+
if already_on_version "glib-2.0" "${GLIB_VERSION}"; then
95+
log "glib-2.0 ${GLIB_VERSION} already present; skipping rebuild."
96+
return
97+
fi
98+
99+
local tarball="glib-${GLIB_VERSION}.tar.xz"
100+
local src_dir="${SRC_ROOT}/glib-${GLIB_VERSION}"
101+
102+
log "Building glib-${GLIB_VERSION} from source…"
103+
cd "${SRC_ROOT}"
104+
rm -rf "${src_dir}"
105+
wget -q "https://download.gnome.org/sources/glib/${GLIB_VERSION%.*}/${tarball}" -O "${tarball}"
106+
tar -xf "${tarball}"
107+
cd "${src_dir}"
108+
rm -rf build
109+
meson setup build \
110+
--prefix=/usr/local \
111+
--buildtype=release \
112+
-Dman=false
113+
ninja -C build
114+
$SUDO ninja -C build install
115+
$SUDO ldconfig
116+
}
117+
118+
build_gobject_introspection() {
119+
if already_on_version "gobject-introspection-1.0" "${GI_VERSION}"; then
120+
log "gobject-introspection ${GI_VERSION} already present; skipping rebuild."
121+
return
122+
fi
123+
124+
local tarball="gobject-introspection-${GI_VERSION}.tar.xz"
125+
local src_dir="${SRC_ROOT}/gobject-introspection-${GI_VERSION}"
126+
127+
log "Building gobject-introspection-${GI_VERSION} from source…"
128+
cd "${SRC_ROOT}"
129+
rm -rf "${src_dir}"
130+
wget -q "https://download.gnome.org/sources/gobject-introspection/${GI_VERSION%.*}/${tarball}" -O "${tarball}"
131+
tar -xf "${tarball}"
132+
cd "${src_dir}"
133+
rm -rf build
134+
meson setup build \
135+
--prefix=/usr/local \
136+
--buildtype=release
137+
ninja -C build
138+
$SUDO ninja -C build install
139+
$SUDO ldconfig
140+
}
141+
142+
sync_typelibs() {
143+
log "Linking Gtk typelibs into ${TYPELIB_TARGET}"
144+
$SUDO mkdir -p "${TYPELIB_TARGET}"
145+
146+
local sources=(
147+
"/usr/local/lib/girepository-1.0"
148+
"/usr/lib/aarch64-linux-gnu/girepository-1.0"
149+
"/usr/lib/girepository-1.0"
150+
)
151+
152+
for src in "${sources[@]}"; do
153+
if [[ -d "${src}" ]]; then
154+
while IFS= read -r -d '' typelib; do
155+
local base
156+
base=$(basename "${typelib}")
157+
$SUDO ln -sf "${typelib}" "${TYPELIB_TARGET}/${base}"
158+
done < <(find "${src}" -maxdepth 1 -name '*.typelib' -print0)
159+
fi
160+
done
161+
}
162+
163+
checkout_gstreamer() {
164+
if [[ ! -d "${GSTREAMER_DIR}/.git" ]]; then
165+
log "Cloning GStreamer (${GSTREAMER_REF}) into ${GSTREAMER_DIR}"
166+
rm -rf "${GSTREAMER_DIR}"
167+
if ! git clone --depth 1 --branch "${GSTREAMER_REF}" "${GSTREAMER_REPO}" "${GSTREAMER_DIR}"; then
168+
log "Requested ref ${GSTREAMER_REF} not found; falling back to main."
169+
git clone --depth 1 "${GSTREAMER_REPO}" "${GSTREAMER_DIR}"
170+
fi
171+
else
172+
log "Updating existing GStreamer checkout…"
173+
git -C "${GSTREAMER_DIR}" remote set-url origin "${GSTREAMER_REPO}"
174+
git -C "${GSTREAMER_DIR}" fetch --depth 1 origin "${GSTREAMER_REF}" || git -C "${GSTREAMER_DIR}" fetch --depth 1 origin main
175+
if ! git -C "${GSTREAMER_DIR}" checkout "${GSTREAMER_REF}"; then
176+
git -C "${GSTREAMER_DIR}" checkout origin/main
177+
fi
178+
git -C "${GSTREAMER_DIR}" reset --hard HEAD
179+
fi
180+
}
181+
182+
build_gstreamer() {
183+
checkout_gstreamer
184+
185+
log "Configuring GStreamer with Gtk sink enabled…"
186+
cd "${GSTREAMER_DIR}"
187+
local build_dir="${GSTREAMER_DIR}/build"
188+
local setup_args=(
189+
--prefix=/usr/local
190+
--buildtype=release
191+
-Ddoc=disabled
192+
-Dtests=disabled
193+
-Dexamples=disabled
194+
-Ddevtools=disabled
195+
-Dgst-plugins-base:gl_winsys=egl
196+
-Dgst-plugins-base:gl=enabled
197+
-Dgst-plugins-bad:gtk=enabled
198+
)
199+
200+
if [[ -d "${build_dir}" ]]; then
201+
meson setup "${build_dir}" "${setup_args[@]}" --reconfigure
202+
else
203+
meson setup "${build_dir}" "${setup_args[@]}"
204+
fi
205+
206+
log "Building and installing GStreamer…"
207+
ninja -C "${build_dir}"
208+
$SUDO ninja -C "${build_dir}" install
209+
$SUDO ldconfig
210+
}
211+
212+
verify_runtime() {
213+
log "Verifying Gtk availability from Python…"
214+
GI_TYPELIB_PATH="${TYPELIB_TARGET}:${GI_TYPELIB_PATH-}" python3 - <<'PY'
215+
import gi
216+
gi.require_version("Gtk", "3.0")
217+
from gi.repository import Gtk, Gdk
218+
print("Gtk bindings OK:", Gtk.MAJOR_VERSION, Gtk.MINOR_VERSION, Gtk.MICRO_VERSION)
219+
print("Gdk display:", bool(Gdk.Display.get_default()))
220+
PY
221+
222+
log "Checking gtksink plugin…"
223+
if gst-inspect-1.0 gtksink >/dev/null 2>&1; then
224+
gst-inspect-1.0 gtksink | awk 'NR<=10 {print}'
225+
else
226+
log "gtksink still missing; investigate build logs above."
227+
return 1
228+
fi
229+
}
230+
231+
main() {
232+
ensure_prereqs
233+
build_glib
234+
build_gobject_introspection
235+
sync_typelibs
236+
build_gstreamer
237+
verify_runtime
238+
log "Patch workflow completed."
239+
}
240+
241+
main "$@"

publish.py

Lines changed: 21 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -471,15 +471,18 @@ def select_display_sink(default_sink: str = "autovideosink") -> str:
471471
using_wayland = bool(os.environ.get("WAYLAND_DISPLAY"))
472472
using_x11 = bool(os.environ.get("DISPLAY")) and not using_wayland
473473
if using_x11:
474-
if gst_element_available("xvimagesink"):
475-
printc(" ! Jetson desktop (X11) detected. Using xvimagesink to avoid EGL issues.", "0AF")
476-
return "xvimagesink sync=false"
477474
if gst_element_available("ximagesink"):
478-
printc(" ! Jetson desktop (X11) detected. Using ximagesink to avoid EGL issues.", "0AF")
479-
return "ximagesink sync=false"
475+
printc(" ! Jetson desktop (X11) detected. Using ximagesink with XInput disabled.", "0AF")
476+
return "ximagesink handle-events=false sync=false"
477+
if gst_element_available("gtksink"):
478+
printc(" ! Jetson desktop (X11) detected. Using gtksink to embed in a GTK window.", "0AF")
479+
return "gtksink sync=false"
480480
if gst_element_available("glimagesink"):
481-
printc(" ! Jetson desktop (X11) detected. Using glimagesink to avoid EGL issues.", "0AF")
482-
return "glimagesink sync=true"
481+
printc(" ! Jetson desktop (X11) detected. Using glimagesink for compositor compatibility.", "0AF")
482+
return "glimagesink sync=false"
483+
if gst_element_available("xvimagesink"):
484+
printc(" ! Jetson desktop (X11) detected. Using xvimagesink as fallback.", "0AF")
485+
return "xvimagesink sync=false"
483486
return default_sink
484487

485488
if is_jetson_device():
@@ -3168,7 +3171,7 @@ def on_incoming_stream(self, webrtc, pad):
31683171
if clear_display_surfaces():
31693172
printc("🧹 Cleared display surface before viewer output", "66F")
31703173
self._display_surface_cleared = True
3171-
3174+
31723175
outsink = select_display_sink("autovideosink")
31733176
print(f"Selected display sink pipeline: {outsink}")
31743177
sink_base = outsink.split()[0]
@@ -3204,7 +3207,16 @@ def build_conversion_chain(using_hw_decoder: bool) -> str:
32043207
"video/x-raw,format=NV12 ! "
32053208
"videoconvert ! video/x-raw,format=RGB"
32063209
)
3207-
return "videoconvert ! video/x-raw,format=RGB"
3210+
if sink_base == "xvimagesink":
3211+
# xvimagesink relies on the XVideo extension which expects YUV surfaces.
3212+
# Forcing RGB caps causes negotiation to fail under GNOME/X11.
3213+
return "videoconvert ! video/x-raw,format=I420"
3214+
if sink_base in {"gtksink", "ximagesink"}:
3215+
return "videoconvert ! video/x-raw,format=BGRx"
3216+
if sink_base == "glimagesink":
3217+
# glimagesink prefers RGBA in system memory before uploading to GL.
3218+
return "videoconvert ! video/x-raw,format=RGBA"
3219+
return "videoconvert ! video/x-raw,format=BGRx"
32083220

32093221
if "VP8" in name:
32103222
fallback_decoder = "vp8dec"

0 commit comments

Comments
 (0)