How to Add Resiliency to Errors in a `no_std` and `no_alloc` Context ?

[joboudreault]

Goal

Being able to completely ignore a field in a JSON string in a no_std and no_alloc environment with the serde_json_core crate. The JSON processed may be of two forms :

Successful response :

{
    "status": "success",
    "message": "",
    "data": {"some_data": 42}
}

Unsuccessful response :

{
    "status": "error",
    "message": "error message",
    "data": {"unknown_field": "some_string"}
}

The JSON is then converted into this Rust structure :

pub struct ApiResponse<'a, T> {
    pub status: ApiStatus,
    pub message: &'a str,
    pub data: Option<T>,
}

If the status field is "error", then the data field must be completely ignored even if there are unknown fields of any kind (string, number, object, null, ...).

Problem

Currently : there is no way to perform this deserialization without the serde_json_core::from_slice() function returning Result::Err(_). Thus, the message field is not accessible.

Expected : there exists a way to successfully deserialize the JSON string whether it is an error or not, and access the error message.

Note : All attempts have a Minimal Reproducible Example (MRE) in this GitHub repository (here).

Attempt 1 (normal)

Here, we tried simply using the #[derive(Deserialize)] macro.

Deserialization error :

Error: CustomError;; JSON does not match deserializer’s expected format.

Attempt 2

Here, we tried to deserialize with a custom deserializer thinking that serde will return None if it cannot successfully convert the data field into T.

Deserialization error :

Error: CustomError;; JSON does not match deserializer’s expected format.

Attempt 3

Here, we tried to deserialize with a custom deserializer thinking that the Result returned by the <T as Deserialize>::deserialize() function may be intercepted and mapped to Ok(None)

Deserialization error :

Error: TrailingCharacters;; JSON has non-whitespace trailing characters after the value.

Attempt 4

Here, we tried to deserialize with a custom deserializer and custom Visitor emptying all fields in the data field's object. In this attempt, we realized that all the functions in the Deserializer<'de> trait are taking ownership which disallow us to perform an operation trying to convert T or to None depending on the status field.

Deserialization panic message :

Body is a 'Data' structure: ExpectedObjectCommaOrEnd

Note: All attempts have a MRE on this GitHub repository (here).

Possible Solution

There exists a solution when using serde_json and allocating a HashMap as described in issue #1583. However, we cannot use this method because there must not be any allocator, thus HashMap cannot be used.

Environment

All compilation is run with the following Rust version :

$ cargo -Vv
cargo 1.79.0 (ffa9cf99a 2024-06-03)
release: 1.79.0
commit-hash: ffa9cf99a594e59032757403d4c780b46dc2c43a
commit-date: 2024-06-03
host: x86_64-unknown-linux-gnu
libgit2: 1.7.2 (sys:0.18.3 vendored)
libcurl: 8.6.0-DEV (sys:0.4.72+curl-8.6.0 vendored ssl:OpenSSL/1.1.1w)
ssl: OpenSSL 1.1.1w  11 Sep 2023
os: Arch Linux [64-bit]

$ rustc -Vv
rustc 1.79.0 (129f3b996 2024-06-10)
binary: rustc
commit-hash: 129f3b9964af4d4a709d1383930ade12dfe7c081
commit-date: 2024-06-10
host: x86_64-unknown-linux-gnu
release: 1.79.0
LLVM version: 18.1.7

[ryan-summers]

I suspect #86 may resolve this issue, but in general, serde-json-core is not really intended to be handling dynamic types.

As a workaround, I'd recommend that you do a two step deserialization process, where you ignore the data field (serde attribute skip) for the first deserialization round. Then, check the status for success and only deserialize the full structure if the request was successful.