Skip to content

fix(sdk/c): find JSON keys, not matching values; decode string escapes - #65

Open
Greeva48 wants to merge 1 commit into
embeddedos-org:masterfrom
Greeva48:fix/c-sdk-json-unescape
Open

Greeva48 wants to merge 1 commit into
embeddedos-org:masterfrom
Greeva48:fix/c-sdk-json-unescape

Conversation

@Greeva48

@Greeva48 Greeva48 commented Nov 12, 2026 •

Copy link
Copy Markdown

Summary

The C SDK's *_from_json decoders read string values wrongly in two ways:

  • Keys matched values. The key lookup was strstr() on the quoted name, so eipc_ack_from_json() found the status value "error" before the "error" key and returned an empty error message for every failed ack.
  • Escapes stayed in. String values were copied with their escapes still in them, so \", \n and Go's \u003c reached callers as literal backslash sequences.

A key is now a quoted name followed by a colon, and string values are decoded as they are copied.

Type of Change

  • feat — New feature
  • fix — Bug fix
  • docs — Documentation only
  • style — Formatting, no code change
  • refactor — Code restructuring without behavior change
  • test — Add or fix tests
  • build — Build system or dependency changes
  • ci — CI/CD pipeline changes
  • perf — Performance improvement

Changes

  • sdk/c/src/eipc_json.c:
    • New json_find_value(): it walks the strstr() matches of "key" and accepts the first one followed (after optional whitespace) by :. json_find_string, json_find_uint64 and json_find_float all use it, so the three copies of the lookup are now one.
    • json_find_string decodes \" \\ \/ \b \f \n \r \t and \uXXXX while copying. \uXXXX becomes UTF-8, surrogate pairs are combined, and a malformed or unpaired escape becomes U+FFFD. \u0000 also becomes U+FFFD, because a NUL would cut the C string short. Values longer than the output buffer are still truncated, but a decoded \uXXXX is never cut in half.
    • Two small helpers, json_hex4() and json_utf8().
    • The file keeps its UTF-8 BOM and LF line endings.
  • sdk/c/tests/test_chat_json.c: two new cases.
    • test_json_key_not_matched_in_value: an ack with status "error", and a header whose service_id is "priority".
    • test_json_string_escapes_decoded: ack and intent round trips with quotes, a backslash, a newline and a tab, plus a chat response containing \u003c, \u0026, \u00e9, \/ and a surrogate pair, and an intent containing \u0000, which must decode to U+FFFD.

Testing

  • Unit tests pass (ctest --test-dir sdk/c/build --output-on-failure): 5/5
  • Integration tests pass (none exercise the C SDK)
  • Manual testing performed: the new tests against the unfixed and the fixed eipc_json.c
  • New tests added for new functionality

macOS arm64, Apple clang 21.0.0, CMake 4.4.2, go1.26.10. Base 741690e, head cacadc3.

Check Command Result
Fail before git checkout origin/master -- sdk/c/src/eipc_json.c (keeping the new tests), cmake --build sdk/c/build && ./sdk/c/build/tests/test_chat_json FAIL (expected): test_json_key_not_matched_in_value ... FAIL: error message after an "error" status, test_json_string_escapes_decoded ... FAIL: ack error round-trips, 3/5 passed, 2 failed, exit 1
Pass after same at cacadc3 PASS: 5/5 passed, 0 failed, exit 0
C SDK suite cmake -S sdk/c -B sdk/c/build -DEIPC_BUILD_TESTS=ON && cmake --build sdk/c/build && ctest --test-dir sdk/c/build --output-on-failure PASS: 100% tests passed out of 5
Strict warnings (not a CI step) same build with -DCMAKE_C_FLAGS="-Wall -Wextra -Werror" PASS: exit 0, no warnings
CI Build go build ./... PASS: exit 0
CI Vet go vet ./... PASS: exit 0
CI Gofmt check gofmt -l . PASS: no files
CI Test go test -race -count=1 -p 1 ./... (CI has no -p 1; it is used here to run one package at a time) PASS: exit 0, 18 packages ok, 6 with no test files
CI Cross-compile make build-all PASS: exit 0
Windows MSVC build of the C SDK — NOT RUN: no Windows host, and CI doesn't build the C SDK. The change is plain C11 (the SDK's standard) with no platform calls

Pre-Submission Checklist

  • Code compiles without warnings (-Wall -Wextra -Werror for C)
  • All existing tests pass
  • New tests added for new functionality
  • Documentation updated if API changed (no API change)
  • Commit messages follow <type>(<scope>): <description> convention
  • Branch is rebased on latest master (741690e)

Closing issue

Fixes #64

Screenshots / Logs

Before and after

Raw logs: raw-fail-before.log, raw-pass-after.log, raw-ctest.log, raw-ci.log.

Additional Notes

  • Not in CI: the C SDK (sdk/c) is not built by any workflow, so its tests run only locally. The table above shows that run.

Summary by Sourcery

Fix C SDK JSON parsing to reliably locate object fields and return decoded string values.

Bug Fixes:

  • Correct C SDK JSON decoders so field names are distinguished from matching string values during key lookup.
  • Decode JSON string escapes, including Unicode sequences and surrogate pairs, into UTF-8 when parsing values.

Enhancements:

  • Consolidate JSON value lookup across string, integer, and floating-point parsers while preserving safe truncation behavior.

Tests:

  • Add coverage for key/value name collisions, escaped string round trips, Unicode decoding, surrogate pairs, and NUL replacement.

The C SDK's *_from_json decoders read string values wrongly in two ways.

The key lookup was strstr() on the quoted key name, which also matches
a string value with the same text. eipc_ack_from_json() looks up
"error" and found the status value "error" first, so every failed ack
came back with an empty error message. A header whose service_id is
"priority" lost its priority the same way. A key is now a quoted name
followed by a colon.

String values were copied verbatim between the quotes, so escapes were
never decoded. A value written by eipc_ack_to_json() or
eipc_intent_to_json() came back with its backslashes, and text from
Go's encoding/json, which writes < > & as \u003c \u003e \u0026, came
back as those six-character codes. The decoder now handles \" \\ \/
\b \f \n \r \t and \uXXXX (as UTF-8, with surrogate pairs); an
over-long value is still truncated, but never inside a decoded
\uXXXX. \u0000 becomes U+FFFD, since a NUL would cut the C string
short.

test_chat_json gains two cases that fail before this change and pass
after it.
@sourcery-ai

sourcery-ai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Reviewer's Guide

Implementation

Add key-aware value lookup and decode JSON string escapes into UTF-8, then use the shared lookup for string, integer, and float values.

  • sdk/c/src/eipc_json.c L18-95, L101-146 (+5 small edits)

Tests

Add regression coverage for keys that also appear in string values and for escaped strings, including Unicode and NUL handling.

  • sdk/c/tests/test_chat_json.c L88-161, L168-169

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@qodo-code-review

Copy link
Copy Markdown

ⓘ Qodo reviews are paused because your trial has ended. Ask your workspace admin to add credits to resume reviews. Manage billing

@kody-ai

kody-ai Bot commented Nov 12, 2026

Copy link
Copy Markdown

You've used all your free Kodus-paid PR reviews 🎁

Your trial is still active — this just means the PR reviews we cover during the trial are used up.

Connect your own AI key to keep Kody reviewing — unlimited reviews, on any plan (Free included).

Want more trial reviews to finish evaluating before adding a key? Talk to our founders. 😎

@coderabbitai

coderabbitai Bot commented Nov 12, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 59 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 686162a8-7cd1-438c-82de-8ea33b5e13dd
Commits

Reviewing files that changed from the base of the PR and between 741690e and cacadc3.

Files selected for processing: 2

  • sdk/c/src/eipc_json.c
  • sdk/c/tests/test_chat_json.c
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@sourcery-ai

sourcery-ai Bot commented Nov 12, 2026

Copy link
Copy Markdown

Linked issues

Checked at commit cacadc3

#64 C SDK: *_from_json returns the wrong string values (a key matches an equal value; escapes are not decoded) · 2 of 2 met

# Requirement Status Notes
1 Distinguish keys from matching string values during lookup, including ack error and header priority. ✅ Met json_find_value() accepts a quoted name only when followed by a colon and is shared by the string, integer, and float lookups; tests cover both collisions.
2 Decode JSON string escapes so escaped text round-trips and Unicode escapes become UTF-8. ✅ Met json_find_string() decodes the listed escapes and \uXXXX sequences, combines surrogate pairs, and the new tests cover round trips and Unicode decoding.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 4 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="sdk/c/src/eipc_json.c" line_range="25" />
<code_context>
+    uint32_t v = 0;
+    int i;
+    for (i = 0; i < 4; i++) {
+        char c = p[i];
+        v <<= 4;
+        if (c >= '0' && c <= '9')
</code_context>
<issue_to_address>
**Truncated escapes read past input**

When a JSON string ends with a truncated `\u` escape, `json_hex4` reads four bytes without checking that they remain before the input terminator, causing an out-of-bounds read and undefined behavior.

Check that four hex digits remain before reading them, and handle truncated escapes without reading past the input.
</issue_to_address>

### Comment 2
<location path="sdk/c/src/eipc_json.c" line_range="74-78" />
<code_context>
-        out[0] = '\0';
-        return -1;
+    plen = strlen(pattern);
+    while ((p = strstr(p, pattern)) != NULL) {
+        const char *v = p + plen;
+        while (*v == ' ' || *v == '\t' || *v == '\n' || *v == '\r')
+            v++;
+        if (*v == ':') {
+            v++;
+            while (*v == ' ' || *v == '\t' || *v == '\n' || *v == '\r')
</code_context>
<issue_to_address>
**Decoders return embedded field values**

When an earlier string contains escaped field-like text, or a nested object contains the requested field before the root-level field, `json_find_value` scans raw text and accepts the first matching `"key":` without checking whether it is inside a string or at the requested object depth, so decoders return the embedded or nested value instead of the root-level field.

Parse the JSON while tracking strings, escapes, and object depth, and return only a matching root-level member.
</issue_to_address>

### Comment 3
<location path="sdk/c/src/eipc_json.c" line_range="116" />
<code_context>
+            enc[0] = *end++;
+        } else if (end[1] == 'u') {
+            uint32_t cp = json_hex4(end + 2);
+            end += (cp == JSON_BAD_HEX) ? 2 : 6;
+            if (cp >= 0xD800 && cp <= 0xDBFF && end[0] == '\\' && end[1] == 'u') {
+                uint32_t lo = json_hex4(end + 2);
</code_context>
<issue_to_address>
**Malformed escapes alter decoded text**

When input contains an unknown escape such as `\q` or an invalid `\u` sequence, `json_find_string` copies the escaped character and drops the backslash for unknown escapes, while invalid `\u` sequences can leave following characters to be copied; callers receive altered text instead of the documented U+FFFD replacement.

Replace each malformed escape with U+FFFD and consume the malformed escape so its remaining characters are not copied as ordinary text.

Also at `sdk/c/src/eipc_json.c:136-137`.
</issue_to_address>

### Comment 4
<location path="sdk/c/src/eipc_json.c" line_range="127" />
<code_context>
+            /* A NUL would cut the C string short, so \u0000 becomes
+             * U+FFFD, like a malformed escape or an unpaired surrogate. */
+            if (cp == 0 || cp == JSON_BAD_HEX || (cp >= 0xD800 && cp <= 0xDFFF))
+                cp = 0xFFFD;
+            n = json_utf8(cp, enc);
+        } else if (end[1]) {
</code_context>
<issue_to_address>
**Malformed escape regressions go unnoticed**

When an input contains malformed `\u` hex, an unpaired surrogate, or a trailing backslash, the U+FFFD handling and trailing-backslash branches have no regression assertions, so changes that break these decoder paths still pass the C tests and callers receive incorrectly decoded strings.

Add public-decoder regression tests asserting the specified output for malformed Unicode escapes, unpaired surrogates, and a trailing backslash.

Also at `sdk/c/src/eipc_json.c:140`.
</issue_to_address>

Sourcery assessment

Approval pending. 4 findings to address first.

Blocking findings: sdk/c/src/eipc_json.c:25, sdk/c/src/eipc_json.c:78, sdk/c/src/eipc_json.c:116, sdk/c/src/eipc_json.c:127


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread sdk/c/src/eipc_json.c
uint32_t v = 0;
int i;
for (i = 0; i < 4; i++) {
char c = p[i];

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Truncated escapes read past input

When a JSON string ends with a truncated \u escape, json_hex4 reads four bytes without checking that they remain before the input terminator, causing an out-of-bounds read and undefined behavior.

Check that four hex digits remain before reading them, and handle truncated escapes without reading past the input.

Prompt for AI agents
In `sdk/c/src/eipc_json.c` at line 25:

**Truncated escapes read past input**

When a JSON string ends with a truncated `\u` escape, `json_hex4` reads four bytes without checking that they remain before the input terminator, causing an out-of-bounds read and undefined behavior.

Check that four hex digits remain before reading them, and handle truncated escapes without reading past the input.

Comment thread sdk/c/src/eipc_json.c
Comment on lines +74 to +78
while ((p = strstr(p, pattern)) != NULL) {
const char *v = p + plen;
while (*v == ' ' || *v == '\t' || *v == '\n' || *v == '\r')
v++;
if (*v == ':') {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Decoders return embedded field values

When an earlier string contains escaped field-like text, or a nested object contains the requested field before the root-level field, json_find_value scans raw text and accepts the first matching "key": without checking whether it is inside a string or at the requested object depth, so decoders return the embedded or nested value instead of the root-level field.

Parse the JSON while tracking strings, escapes, and object depth, and return only a matching root-level member.

Prompt for AI agents
In `sdk/c/src/eipc_json.c` at lines 74-78:

**Decoders return embedded field values**

When an earlier string contains escaped field-like text, or a nested object contains the requested field before the root-level field, `json_find_value` scans raw text and accepts the first matching `"key":` without checking whether it is inside a string or at the requested object depth, so decoders return the embedded or nested value instead of the root-level field.

Parse the JSON while tracking strings, escapes, and object depth, and return only a matching root-level member.

Comment thread sdk/c/src/eipc_json.c
enc[0] = *end++;
} else if (end[1] == 'u') {
uint32_t cp = json_hex4(end + 2);
end += (cp == JSON_BAD_HEX) ? 2 : 6;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Malformed escapes alter decoded text

When input contains an unknown escape such as \q or an invalid \u sequence, json_find_string copies the escaped character and drops the backslash for unknown escapes, while invalid \u sequences can leave following characters to be copied; callers receive altered text instead of the documented U+FFFD replacement.

Replace each malformed escape with U+FFFD and consume the malformed escape so its remaining characters are not copied as ordinary text.

Also at sdk/c/src/eipc_json.c:136-137.

Prompt for AI agents
In `sdk/c/src/eipc_json.c` at line 116:

**Malformed escapes alter decoded text**

When input contains an unknown escape such as `\q` or an invalid `\u` sequence, `json_find_string` copies the escaped character and drops the backslash for unknown escapes, while invalid `\u` sequences can leave following characters to be copied; callers receive altered text instead of the documented U+FFFD replacement.

Replace each malformed escape with U+FFFD and consume the malformed escape so its remaining characters are not copied as ordinary text.

Also at `sdk/c/src/eipc_json.c:136-137`.

From SECURITY-STANDARDS.md L12-14:

Validate every input at the boundary it enters: type, range, length, format and
allowed values. Reject what does not match rather than coercing it. Validation
performed only in the client is not validation.

Comment thread sdk/c/src/eipc_json.c
/* A NUL would cut the C string short, so \u0000 becomes
* U+FFFD, like a malformed escape or an unpaired surrogate. */
if (cp == 0 || cp == JSON_BAD_HEX || (cp >= 0xD800 && cp <= 0xDFFF))
cp = 0xFFFD;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Malformed escape regressions go unnoticed

When an input contains malformed \u hex, an unpaired surrogate, or a trailing backslash, the U+FFFD handling and trailing-backslash branches have no regression assertions, so changes that break these decoder paths still pass the C tests and callers receive incorrectly decoded strings.

Add public-decoder regression tests asserting the specified output for malformed Unicode escapes, unpaired surrogates, and a trailing backslash.

Also at sdk/c/src/eipc_json.c:140.

Prompt for AI agents
In `sdk/c/src/eipc_json.c` at line 127:

**Malformed escape regressions go unnoticed**

When an input contains malformed `\u` hex, an unpaired surrogate, or a trailing backslash, the U+FFFD handling and trailing-backslash branches have no regression assertions, so changes that break these decoder paths still pass the C tests and callers receive incorrectly decoded strings.

Add public-decoder regression tests asserting the specified output for malformed Unicode escapes, unpaired surrogates, and a trailing backslash.

Also at `sdk/c/src/eipc_json.c:140`.

From TESTING.md L10:

- **Unit** — logic and its branches, including edge cases.

@Aswin-V

Aswin-V commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Thanks — the premise checks out against master 741690e (the strstr() key match and the verbatim escape copy are both there as described), and the new tests cover the reported cases. One issue in the new \u decoding in sdk/c/src/eipc_json.c (json_find_string): json_hex4(end + 2) reads four bytes unconditionally, so a truncated \u escape at the end of the input (e.g. {"intent":"a\u1 with no closing quote) reads past the NUL terminator — up to 3 bytes past the end of the C string. The same applies to the surrogate-pair lookahead json_hex4(end + 2). Please bound the reads, e.g. only call json_hex4 when end[2] && end[3] && end[4] && end[5], treating a short escape as malformed (U+FFFD) instead. Noting for the record: no CI workflow builds sdk/c (verified — ci.yml/codeql.yml contain no sdk/c), so this C change is covered only by the local test logs in the PR body.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

C SDK: *_from_json returns the wrong string values (a key matches an equal value; escapes are not decoded)

2 participants