Add FXMacroData tool to the Atomic Forge

[roberttidball]

I run FXMacroData, the API this tool calls. This contribution and its test updates were prepared with AI assistance.

Summary

Adds a native Atomic Forge fxmacrodata tool for release history, latest macroeconomic values, release calendars, the indicator catalogue, FX history, COT positioning and commodity prices. USD macro data, the USD calendar and the catalogue support keyless access; protected endpoints use a caller-supplied key in the X-API-Key header.

The tool provides typed inputs, synchronous and asynchronous execution, and an error field for invalid input, HTTP failures and malformed response bodies. It keeps returned rows, pagination and response metadata in separate output fields.

Review fixes

• Validate the original response before redacting it, so an echoed key cannot hide an invalid pagination field.
• Preserve values when redacted object keys collide, including nested responses. Unique suffixes advance independently of previously redacted names, and replacement text cannot recreate the key.
• Reject malformed and out-of-range URL ports when configuration is created. HTTPS origin checks and disabled redirects remain in place.
• Merge current main, retaining its tool listings and external lock entries, and add the FXMacroData workspace member and local tool metadata.

Validation

• 85 offline tests passed on Python 3.12 using the released atomic-agents==2.10.3, with actual Pydantic models and mocked HTTP responses.
• The original PR source fails 11 new validation/redaction regressions. A bounded test also reproduces the former collision-loop failure.
• Black 26.5.1 (py312, line length 127) and Flake8 7.3.0 pass on the tool and test modules. Black differs from the repository's existing 24.x development constraint.
• All 27 production functions have full measured line coverage; the highest function CRAP score is 5.

These checks cover this tool. The full monorepo test suite, original locked environment, documentation build, and live API calls were not run for this update. No credentials are included.

[Eigenwise]

Historical review of 79f02c32.

The macro history/calendar tool has useful distinct coverage, and your affiliation and free/paid split are disclosed. I reproduced four issues offline at 79f02c32 using actual Requests/Pydantic and dummy keys:

• Credential forwarding, tool/fxmacrodata.py:174: a 302 from the HTTPS API to an unrelated HTTP host carries X-API-Key to that host. Keep authenticated requests on the intended HTTPS origin.
• Secret in returned errors, lines106/173/190: a key with leading whitespace triggers InvalidHeader, and stringifying that exception returns the complete key. Error messages must exclude credential values.
• Malformed responses, lines192-200: HTTP200 with {detail: 'upstream failure'} or invalid data values becomes error=None, data=[]. Other malformed bodies raise output ValidationError outside the advertised error-return contract. Validate the response shape and return a useful error.
• Date inputs, lines142-148: impossible dates and reversed ranges reach the request; an empty start date silently disappears. Use real date validation and range checks.

The author suite passes: 23 tests. The independent failure probes still reproduce these cases. Canonical Black127 and repository Flake8 also pass, so import spacing isn't a blocker.

On this historical head, native production complexity was CC14 for build_request and CC8 for run. The earlier score-only refactor request is withdrawn.

Please remove the UTM referral parameters from the README links, including the subscription link, and use neutral service URLs. No live API calls, real credentials or purchases were used in this review.

The later offline review of 184709f3 confirms redirect/header/error/date repairs and neutral links. At 0ed527b4, 63 author tests and 132 independent observations also confirm pagination, config masking and ordinary nested-response redaction repairs. The findings above describe the historical head; the current review instead identifies redaction collisions/validation bypass and malformed-port error handling.

[roberttidball]

Thanks for the careful review and the reproductions. All four are fixed in 184709f:

• Credential forwarding: requests go out with allow_redirects=False and a 3xx comes back as an error, and a non-HTTPS base_url is refused before any request. Tests: test_redirect_to_other_http_host_does_not_forward_api_key (real Requests, 302 to an HTTP host; only the HTTPS URL is ever sent) and test_non_https_base_url_is_refused.
• Secret in errors: the key is stripped and checked for header-safe characters up front, request exceptions are reported by type name only, and every error passes through a redaction step. Tests: test_key_with_leading_whitespace_is_stripped_and_never_echoed, test_key_that_cannot_be_a_header_is_reported_without_its_value, test_request_exception_text_never_reaches_the_output, test_upstream_detail_echoing_the_key_is_redacted.
• Malformed responses: row endpoints require an object whose data is a list of objects and whose pagination is an object; the catalogue requires a non-empty mapping of objects. Anything else, including 200 with {detail: ...} or a non-JSON body, returns error (with the detail when present) instead of data=[] or a ValidationError. Tests: test_malformed_200_responses_return_an_error (9 cases), test_non_json_200_response_returns_an_error.
• Dates: dates are parsed with date.fromisoformat behind a strict YYYY-MM-DD pattern, an empty string is an error rather than dropped, and start_date after end_date is rejected. Test: test_impossible_empty_and_reversed_dates_are_rejected.

Also from your notes: request building and response parsing are split into small functions (radon max CC is now 5, build_request and run are 1 and 2), the new Any annotations are replaced with Pydantic's JsonValue, abbreviated names are spelled out, and the README links no longer carry referral parameters. 44 tests pass with 100% line coverage, and Flake8 and Black (line length 127) are clean.

[Eigenwise]

Historical review of 184709f3.

Rechecked 184709f3. The redirect, unsafe-header, returned-error and date repairs pass the offline probes. Twenty redirect variants send only the intended HTTPS request, invalid dates send zero requests, and the referral parameters are gone. All 24 production functions now have genuine measured CRAP scores below 6.

Two remaining blockers in tool/fxmacrodata.py:

• Malformed pagination, lines209-212: HTTP200 {data: [], pagination: []} still returns error=None, pagination=None. false, 0 and an empty string also pass. Validate a present pagination value before or {} normalizes it.
• Credential exposure, lines114-117 and278-282: the complete dummy key appears in the actual config repr/str. A successful response with {data: [], detail: 'echo ' + key} also copies it into metadata and output repr. Hide credentials in config representations and protect successful output paths as well as error strings.

A lower-priority configuration concern: malformed HTTPS base URLs such as https:// raise InvalidURL/ValueError outside the advertised error-return path, before any request. Validate config early or make that exception contract explicit.

The author suite passes all 44 tests. Seventy-seven independent offline Requests/Pydantic observations verify the repairs and reproduce the remaining cases. Black127 and canonical Flake8 pass. Keep the useful assertions. No live API, real credentials, vendor installation or purchases were used.

The later offline review of 0ed527b4 confirms pagination validation, config masking and ordinary nested-response redaction repairs. It still reproduces redaction collisions/validation bypass and the malformed-port error path. The 44-test result and 77 observations above belong to 184709f3.

[roberttidball]

Thanks for rechecking. All three are addressed in 0ed527b:

• Malformed pagination: a present pagination must be an object whose documented fields have their documented types (counts are int and not bool, next_offset int or null, flags bool); only absent or null counts as no pagination, and nothing is normalised before the check. Tests: test_present_malformed_pagination_is_an_error ([], false, 0, "" and five wrong-type objects) and test_null_pagination_is_treated_as_absent.
• Credential exposure: api_key is now a SecretStr, so the config repr, str and JSON dump show it masked, and every successful response is redacted recursively (values and keys) before rows, metadata or pagination are built, as well as the error strings. Tests: test_config_repr_and_str_hide_the_key, test_key_echoed_in_a_successful_response_is_redacted (your {data: [], detail: 'echo ' + key} case plus a row and a field name), test_key_echoed_in_a_catalogue_response_is_redacted.
• Base URL: base_url is validated when the config is built (https scheme and a non-empty host, trailing slash stripped), so https:// and similar values raise a ValidationError up front and can no longer reach a request. The earlier runtime HTTPS check moved there, so test_non_https_base_url_is_refused became test_invalid_base_url_is_rejected_by_config (six cases) with test_base_url_is_normalised_by_config.

63 tests pass with 100% line coverage, every function is at or below complexity 5, and Black (127) and Flake8 are clean.

[Eigenwise]

Rechecked 0ed527b4 offline: all 63 author tests pass, and 132 independent Requests/Pydantic observations confirm the pagination, config masking and ordinary nested-response redaction repairs. All 27 production functions have genuine measured native CRAP scores below 6.

Two remaining findings in tool/fxmacrodata.py:

• Redaction loses data and bypasses validation, lines278-279 and313: a successful row with two keys that sanitize to [redacted] silently loses one value. Catalogue, metadata and pagination collisions reproduce too. With the dummy key total_count, pagination.total_count=True is renamed before validation and returns error=None. Validate the original structure first and preserve colliding values when sanitizing output.
• Malformed port escapes the error-output path, lines140-143 and312: https://api.fxmacrodata.com:bad/v1 passes config validation, then run raises Requests.InvalidURL before any send. Validate the port at config creation or handle that failure consistently.

Production coverage is 167/167 statements and 32/32 measured branches, with 13 unchanged main-example exclusions. Current-head style was not verified; keep the useful assertions. No live API, real credentials, vendor installation or source changes.

[roberttidball]

Could you re-review a95fb9e when convenient? It validates the original pagination before redaction, preserves values when redacted keys collide, and rejects malformed ports during configuration. The complete tool suite passes 85 offline tests on Python 3.12 with atomic-agents 2.10.3. The full monorepo suite and live API were not run. Thanks.

[Eigenwise]

Rechecked a95fb9e238524aa3f7a72386e2cf34b3e6533294. The collision fix retains distinct dictionary values with sanitized unique keys, and pagination validates the original response before redaction. Malformed and out-of-range ports now fail config validation. The new regressions cover these cases, nested value retention and marker/suffix overlap. Both earlier findings are resolved; no new concrete regression found in this delta.

Exact-head normal CI passed: 408 core tests, 3 MiniMax skips, formatting and lint passing. That CI does not run the Forge suite. The author reports 85 offline tool tests passing; I did not independently execute that run or repeat the earlier tests. Live API behavior remains unverified.