Skip to content

[JFMGC-54] Report pooled health topology via health_instances - #10

Merged
shackmann merged 4 commits into
mainfrom
yahor/JFMGC-54/add_health_instances
Aug 27, 2026
Merged

[JFMGC-54] Report pooled health topology via health_instances#10
shackmann merged 4 commits into
mainfrom
yahor/JFMGC-54/add_health_instances

Conversation

@yakavaliou

@yakavaliou yakavaliou commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator
  • Add JointFMClient.health_instances() to probe each configured endpoint independently and report availability, per-instance max_sample_count, summed parallel capacity, and a compact topology (e.g. 1x7000, 1x3000).
  • Keep health() consensus metadata using the minimum reachable cap as the sample-batch size; endpoints remain unaware of each other.
  • Surface topology in the CLI health command and document the sum-vs-min distinction in README / API reference.
>>> from jointfm_client.client import JointFMClient
>>> from dataclasses import asdict
>>> 
>>> client = JointFMClient.from_env()
>>> asdict(client.health_instances(refresh=True))
{'instances': ({'deployment_id': '6a7f76b43a393b6b724e0819', 'metadata': {'status': 'ok', 'schema_version': 'v1', 'image_version': '0.2.0', 'model_version': 'jointfm-inference:0.2.0+ckpt.chevron_i576_o288_f16_t8_h16l16_mam7_af_t3r2_cnn_k3l4_hpst_h16l2_studentt_m2cr5df8', 'checkpoint_version': 'chevron_i576_o288_f16_t8_h16l16_mam7_af_t3r2_cnn_k3l4_hpst_h16l2_studentt_m2cr5df8', 'checkpoint_path': '/models/jointfm.pt', 'device': 'cuda', 'head': 'studentt', 'decoding_strategy': 'parallel_dense', 'supported_query_modes': ('forecast',), 'supported_return_modes': ('log_prob', 'mean', 'quantiles', 'samples'), 'supported_time_index_modes': ('absolute_datetime', 'continuous_float', 'ordinal'), 'time_index_encoding': 'legacy_discrete_grid', 'max_sample_count': 5000, 'data_generation': {'sampler_type': 'fin', 'min_features': 0, 'max_features': 16, 'min_targets': 1, 'max_targets': 8, 't_input': 4.0, 't_output': 2.0, 'n_input': 576, 'n_output': 288}}, 'error': None}, {'deployment_id': '6a80af033a75b0a67a4df8ab', 'metadata': {'status': 'ok', 'schema_version': 'v1', 'image_version': '0.2.0', 'model_version': 'jointfm-inference:0.2.0+ckpt.chevron_i576_o288_f16_t8_h16l16_mam7_af_t3r2_cnn_k3l4_hpst_h16l2_studentt_m2cr5df8', 'checkpoint_version': 'chevron_i576_o288_f16_t8_h16l16_mam7_af_t3r2_cnn_k3l4_hpst_h16l2_studentt_m2cr5df8', 'checkpoint_path': '/models/jointfm.pt', 'device': 'cuda', 'head': 'studentt', 'decoding_strategy': 'parallel_dense', 'supported_query_modes': ('forecast',), 'supported_return_modes': ('log_prob', 'mean', 'quantiles', 'samples'), 'supported_time_index_modes': ('absolute_datetime', 'continuous_float', 'ordinal'), 'time_index_encoding': 'legacy_discrete_grid', 'max_sample_count': 5000, 'data_generation': {'sampler_type': 'fin', 'min_features': 0, 'max_features': 16, 'min_targets': 1, 'max_targets': 8, 't_input': 4.0, 't_output': 2.0, 'n_input': 576, 'n_output': 288}}, 'error': None}), 'max_sample_count': 10000, 'topology': ((2, 5000),), 'topology_label': '2x5000'}

New parameters in the response

  • topology --> compact summary of reachable caps only, grouped by size — e.g. 1x7000, 1x3000 or 2x5000. Unavailable peers are omitted.
  • max_sample_count (top-level) --> sum of reachable peers’ caps = overall parallel sample capacity (7000 + 3000 → 10000). Not the same as service.max_sample_count.

Note

Low Risk
Additive public API and CLI output; existing health() batch-cap and forecast batching behavior is preserved.

Overview
Adds JointFMClient.health_instances() and public InstanceHealth / HealthInstances types so callers can see per-deployment probe results when using JOINTFM_DEPLOYMENT_IDS (or a single endpoint).

Health probing is unified: health() still returns consensus HealthMetadata with max_sample_count = minimum reachable cap (sample-batch size for forecasts); health_instances() lists every configured peer (including failures), sums reachable caps as parallel capacity, and exposes topology / topology_label (e.g. 2x5000). The pool’s PoolHealthGate and probe_health() now attach per-peer outcomes; caching stores both consensus and instance rows.

The jointfm-client health CLI prints instances, topology, and top-level max_sample_count (sum, distinct from service.max_sample_count). README and API reference document the min-vs-sum distinction.

Reviewed by Cursor Bugbot for commit 085951d. Bugbot is set up for automated code reviews on this repo. Configure here.

@shackmann shackmann left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice. Thank you!

@shackmann
shackmann merged commit b1bc19f into main Aug 27, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants