Skip to content

Conversation

@ReeceHumphreys
Copy link
Contributor

@ReeceHumphreys ReeceHumphreys commented Nov 24, 2025

  • Tickets addressed: N/A
  • Review: By commit
  • Merge strategy: Merge (no squash)

Description

In an effort to reduce the size of basilisks wheels and shift to a more sustainable long term architecture it is desirable to not ship support data files inside the wheels. As such, we need a mechanism to fetch and cache files on demand as they are requested.

If you clone the repo most of the data is still provided directly. As before, some data such as JPL Spice data was downloaded during the build. With this branch such Spice data will now also be downloaded on demand.

To accomplish this, the way data files are loaded and used in bsk has changed drastically and now leverages Pooch to handle fetching the files. This is a library that has been battle tested by many other popular Python scientific libraries for similar reasons.

As such, this PR does the following:

  • Removes supportData files from sdist and wheels
  • Adds a Pooch-based fetcher for all supportData categories
  • Introduces a generated registry with MD5 hashes for versioned files
  • Implements fallbacks for external (non-versioned) NAIF (JPL Spice) kernels
  • Updates Denton GEO model and related modules to use the new fetch API
  • Adds local-repo detection so editable installs use on-disk supportData
  • Updates documentation to explain the new fetch/caching workflow
  • Regenerates supportData registry and updates data enums accordingly
  • update bskLargeData command to prefetch all dynamic data components

Verification

CI/CD and local testing of newly built wheels verify that they do not contain the supportData files and can fetch the files using the new API.

Documentation

  • bskLargeData docs minorly updated.
  • New documentation page explaining how support data is now handled
  • updated release notes

Future work

N/A

@ReeceHumphreys ReeceHumphreys moved this to 🏗 In progress in Basilisk Nov 24, 2025
@ReeceHumphreys ReeceHumphreys self-assigned this Nov 24, 2025
@ReeceHumphreys ReeceHumphreys force-pushed the feature/pooch branch 3 times, most recently from 7914eec to ad9ee1d Compare December 1, 2025 01:00
@ReeceHumphreys ReeceHumphreys changed the title Load supportData files from GitHub using Pooch Migrate supportData to on-demand fetching with Pooch Dec 1, 2025
@ReeceHumphreys ReeceHumphreys moved this from 🏗 In progress to 👀 In review in Basilisk Dec 1, 2025
@ReeceHumphreys ReeceHumphreys marked this pull request as ready for review December 1, 2025 15:48
@ReeceHumphreys ReeceHumphreys requested a review from a team as a code owner December 1, 2025 15:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants