Replace all in-memory cores with InMemoryCore

[mkeeter]

This PR unifies the disparate DumpAgentCore, DryCore*, ArchiveCore, and DumpCore into a single InMemoryCore type. Once this PR lands, we will be down to the irreducible set of ProbeCore, InMemoryCore, and NetCore.

Each of these cores stored data in memory, all in slightly different ways, and with different degrees of rigor – for example, some of them could not handle reads across multiple contiguous memory spans. The new InMemoryCore uses the newly-published datamap crate, which is a rigorous wrapper around a BTreeMap<u32, Vec<u8>> (non-overlapping, efficient reads, etc).

This change adds an extra memcpy in a few places. Previously, some of the cores stored data in a single linear array and then indexed into it with a BTreeMap<u32, (usize, usize)> (address → (offset, size)), instead of owning the data in the map. I think the consistency of a single implementation is worth it; this is not going to be a limiting factor in typical Humility usage.

*the DryCore has been mostly forgotten, but is a way to store dumps without a matching archive then rehydrate them later

[labbott]

mostly nits

[labbott]

Is this better modeled as an enum vs Option explosion?

[mkeeter]

I don't think so – we end up using many possible combinations, so it would be awkward to have InMemoryCore::FlashAndMemButNoRegisters, etc.

Two things that we could change if folks feel strongly:

• We could merge mem and flash, since they should be non-overlapping. Some of the previous cores kept them separate to give finer-grained error messages, but that also could (theoretically) lead to confusion if the same data range is present in both; combining them would bail with an error when inserting overlapping data. On the other hand, there are a few places which use Core::is_archive, so we'd have to revisit those call sites.
• The fields also don't need to be Options, since an empty map is equivalent; again, the Option is for better error messages, to distinguish "we don't have any flash data" versus "we don't have that flash data". I guess we could treat empty maps as a special case, though, which would make the Option unnecessary.

[labbott]

All these is_X functions make me wonder if we're missing an enum for a core type somewhere (or maybe it's not that bad?)

[mkeeter]

Agreed – later down the line, we may want to turn the dyn Core on its head, replacing it with an enum Core with our three core types (net, probe, in-memory). This would be better from a type-safety perspective, because stuff like send would only exist on the NetCore, but would be a bunch of work.

[Copilot]

Pull request overview

This PR consolidates multiple “in-memory” core implementations into a single InMemoryCore backed by the datamap crate, and updates dump/network tooling to use this unified core representation across the Humility workspace.

Changes:

• Introduces humility-core::mem::InMemoryCore and removes legacy archive/dump core variants.
• Replaces HubrisFlashMap and DumpAgentCore usage with HubrisDataMap + InMemoryCore across dump/net/CLI commands.
• Updates CLI arg types and golden test outputs to match new error text and APIs.

Reviewed changes

Copilot reviewed 46 out of 47 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
humility-net/src/lib.rs	Switches flash backing store to HubrisDataMap and uses InMemoryCore when interacting with the dump agent.
humility-dump/src/lib.rs	Uses InMemoryCore as the dump extraction target and updates error variants accordingly.
humility-dump-agent/src/lib.rs	Removes DumpAgentCore and writes dump contents/registers into InMemoryCore.
humility-core/src/mem.rs	Adds InMemoryCore implementation (flash/mem/register maps) and dump/archive constructors.
humility-core/src/lib.rs	Exposes the new mem module and removes old module exports.
humility-core/src/hubris.rs	Replaces HubrisFlashMap with HubrisDataMap alias and adds HubrisArchive::flash_map().
humility-core/src/core.rs	Updates Core trait “dump-ness” signal to is_memory_core() and returns InMemoryCore from attach helpers.
humility-core/src/archive.rs	Removes the standalone ArchiveCore implementation.
humility-core/Cargo.toml	Adds datamap dependency.
humility-cli/src/lib.rs	Updates dump handling to use PathBuf and updated dump-loading signatures.
humility-bin/src/main.rs	Updates HUMILITY_DUMP env parsing to PathBuf.
cmd/spd/src/lib.rs	Updates dump detection to is_memory_core().
cmd/sensors/src/lib.rs	Updates dump detection to is_memory_core() for backend selection/constraints.
cmd/registers/src/lib.rs	Updates dump detection to is_memory_core() for register-reading constraints.
cmd/hydrate/src/lib.rs	Replaces DryCore with InMemoryCore + RAM region insertion.
cmd/dump/src/lib.rs	Uses InMemoryCore as dump output core for agent read/simulate paths.
cmd/diagnose/src/lib.rs	Updates dump detection to is_memory_core().
Cargo.toml	Adds workspace dependency entry for datamap.
Cargo.lock	Adds the resolved datamap crate.
humility-bin/tests/cmd/tasks-slvr/tasks-slvr.u16-ringbuf.stdout	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/tasks-slvr/tasks-slvr.task.power.stdout	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/tasks-slvr/tasks-slvr.control_plane_agent.overflow.0.stdout	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/spd/spd.u16-ringbuf.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/spd/spd.task.power.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/spd/spd.task.net.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/spd/spd.extern-regions.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/spd/spd.control_plane_agent.overflow.0.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/host-panic/host-panic.u16-ringbuf.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/host-panic/host-panic.task.power.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/host-panic/host-panic.task.net.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/host-panic/host-panic.extern-regions.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/host-panic/host-panic.control_plane_agent.overflow.0.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/counters-ipc/counters-ipc.u16-ringbuf.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/counters-ipc/counters-ipc.task.power.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/counters-ipc/counters-ipc.task.net.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/counters-ipc/counters-ipc.extern-regions.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/counters-ipc/counters-ipc.control_plane_agent.overflow.0.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/counters-ipc-full/counters-ipc-full.u16-ringbuf.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/counters-ipc-full/counters-ipc-full.task.power.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/counters-ipc-full/counters-ipc-full.task.net.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/counters-ipc-full/counters-ipc-full.extern-regions.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/counters-ipc-full/counters-ipc-full.control_plane_agent.overflow.0.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/counters-ipc-filtered/counters-ipc-filtered.u16-ringbuf.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/counters-ipc-filtered/counters-ipc-filtered.task.power.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/counters-ipc-filtered/counters-ipc-filtered.task.net.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/counters-ipc-filtered/counters-ipc-filtered.extern-regions.stderr	Updates expected error output for new in-memory read failures.
humility-bin/tests/cmd/counters-ipc-filtered/counters-ipc-filtered.control_plane_agent.overflow.0.stderr	Updates expected error output for new in-memory read failures.

Comments suppressed due to low confidence (1)

humility-core/src/mem.rs:126

• from_dump slices the dump file using p_memsz, which can exceed p_filesz (e.g. BSS) or extend past the end of the file in truncated/corrupt dumps, causing a panic during slice creation. This should copy only p_filesz bytes (and zero-pad up to p_memsz) with an explicit bounds check so invalid dumps fail gracefully.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[mkeeter]

I tested this with humility dump using both a probe and network connection, and humility tasks over a network connection. All of the dumps look fine. Unit tests also exercise both DumpCore and ArchiveCore, so I'm not too worried here.

[evan-oxide]

Was this added by accident?

[mkeeter]

nothing to see here