Skip to content

Conversation

@JoeZiminski
Copy link
Member

@JoeZiminski JoeZiminski commented Sep 2, 2025

This PR introduces cross-platform packaging of datashuttle. This requires both packaging datashuttle itself as well as vendoring a terminal emulator to deal with some rendering issues.

The packaging process is threefold:

  1. package datashuttle
  2. package a lightweight script that launches the vendored terminal emulated and runs the packaged datashutlte within it.
  3. wrap in an installer (inno setup on Windows, bundle on macOS, unsure on Linux, we might not wrap in an installer there).

Currently, this PR:

  1. Works on Windows
  2. Packaging works on macOS intel and silicon. However, the silicon version does not work on intel, and vice versa. A little more work should be done to investigate why this is (a single distribution is possible in some cases). If it's not possible, we will have two separate distributions. Still need to wrap in a installer bundle.
  3. Packaging is working on ubuntu, need to look into packaging on manylinux and whether to wrap in an installer.

Currently the architecture is:

  1. datashuttle.spec is shared between operating systems, and is the pyinstaller config file to control packaging of datashuttle itself. It will package datashuttle itself, by packaging the datashuttle_launcher.py script. This will then create a executable to be called in the vendored terminal emulator.
  2. There are three top-level packaging scripts, one for each OS (package_windows.py, package_macos.py, package_linux.py). These coordinate the download of the terminal emulator to vendor, packaging of datashuttle, then triggers the running of terminal_launcher_xxx.spec (one for macos, 'windows' is currently used by Windows and Linux) which in turn packages terminal_launcher.py. This handles how the vendored terminal should call the datashuttle executable.

So essentially, the idea is to package datashuttle, and then package a script that opens the vendored terminal and runs the datashuttle executable within it.

One the packaging is complete, we will need to extend the CI to run these installers, generate the artifact to distribute, and add it to the website.

Note that this PR is still in the prototype phase and requires some refactoring and tidying up.

@JoeZiminski JoeZiminski force-pushed the cross-platform-packaging branch from 8e52ff3 to 6685bf7 Compare September 2, 2025 20:21
@JoeZiminski JoeZiminski force-pushed the cross-platform-packaging branch from 79049d6 to 5b6a91c Compare September 2, 2025 20:25
@JoeZiminski JoeZiminski force-pushed the cross-platform-packaging branch from 5660e9e to a941e13 Compare September 2, 2025 20:28
@JoeZiminski JoeZiminski force-pushed the cross-platform-packaging branch from c5b5c6a to 9c95587 Compare September 2, 2025 20:43
@JoeZiminski JoeZiminski force-pushed the cross-platform-packaging branch from 00cf5ae to acdaada Compare September 2, 2025 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants