Skip to content

LOO refactoring and adding pred_measure feature#363

Draft
florence-bockting wants to merge 9 commits into
masterfrom
pred_measure
Draft

LOO refactoring and adding pred_measure feature#363
florence-bockting wants to merge 9 commits into
masterfrom
pred_measure

Conversation

@florence-bockting

@florence-bockting florence-bockting commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Fixes #281

This PR is work in progress and the description will be refined over time.

Description

This PR is a larger project with the aim to refactor the loo package and to add new features related to a new pred_measure family.

In the course of refactoring this PR aims to address also the following PRs:

How to navigate this PR

As this is a large, long-running PR it might help to first read through the supporting documents that explain the design, scope, and current status:

  • developer-notes.md: The central reference for this project: current status, open tasks, outstanding design decisions, and known blockers. Start here.
  • pred_measure-formulas.Rmd: An overview of all supported scores and metrics, including their mathematical definitions.
  • pred_measure-family.Rmd: A conceptual introduction to the new pred_measure family and its intended usage.

Acknowledgements

This work builds on an initial prototype of the new pred_measure feature implemented by @VisruthSK under guidance from @jgabry (see VisruthSK/loo-sandbox).

@github-actions

github-actions Bot commented Jun 2, 2026

Copy link
Copy Markdown

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 404656a is merged into master:

  • ✔️loo_function: 1.49s -> 1.49s [-0.21%, +0.74%]
  • ✔️loo_matrix: 1.43s -> 1.43s [-0.57%, +0.52%]
    Further explanation regarding interpretation and methodology can be found in the documentation.

@florence-bockting

florence-bockting commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

Linking here to Issue #179 as it seems to be about loo scores. We should see whether it is still relevant and can be implemented within loo refactoring.

@florence-bockting

florence-bockting commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

There is an open issue requesting a subset() method for psis objects #343 and an (unrelated) open PR #110. Might be worth considering whether we can address this as well in this refactoring PR. If yes, we should

@VisruthSK VisruthSK added this to the v3.0.0 milestone Jun 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment