Supported hook for substituting serializers during a (de)serialization operation

[kamalnayan04]

What is your use-case and why do you need this feature?

I work on an Android app whose entire UI is driven by a JSON payload from the backend. A single screen response is a few hundred KB, deeply nested, and touches dozens of distinct @Serializable types. Parsing it is one of the more expensive things the app does on a cold start.

Two things I keep wanting and cannot get without a lot of ceremony:

The first is per-type parse cost. I want to know which types actually dominate a decodeFromString call — not the total, which I already have, but the breakdown. If a Component holds a GradientData which holds a List<Color>, I want a timing around each of those, attributed to the type, so I can go and fix the expensive one. Today the only honest way to get that is to hand-write a wrapping serializer for every type I am curious about, which is exactly the set of types I don't know yet.

The second is failure attribution. When one of our own serializers throws, I want the type and the path to land in our metrics pipeline. 1.11.0 making JsonDecodingException public with path and offset on it helps a lot here, and I'm grateful for it, but it only covers exceptions Json itself raises — a NumberFormatException out of a custom serializer still arrives with no context (which I realise is #2165).

Both of these are the same shape of problem: I want to do something uniform to every serializer used in one decode, including generated ones and ones from libraries I don't own, without changing what any of them produce.

The constraint that rules out the obvious answer is that this has to stay on the streaming path. Going through JsonElement first is precisely the cost I am trying to measure, and at our payload sizes it isn't affordable anyway.

What I've tried, and where each one stops

I don't want to claim this is impossible today, because it isn't. It's just expensive in a way I think the library could make cheap.

@KeepGeneratedSerializer with @Serializable(with = ...) — this is the resolution of #793 / #1169 and it does the job, on the streaming path, with no JsonElement. But it's per type. It needs source access to every class and an annotation on each one, and a decent fraction of the types in our payload live in modules we don't own.

SerializersModule.contextual — doesn't override plugin-generated serializers, which is by design and documented. Making it apply means marking every use site @Contextual, or listing types in @file:UseContextualSerialization, so it lands back in the same place: I have to enumerate the types up front.

JsonTransformingSerializer — materialises a JsonElement, so it's out for the reason above.

A format that delegates to Json and substitutes serializers on the way through — this is the approach @pdvrieze sketched in #2699, and it's the one that actually does what I want. Because generated serializers route every nested value through decodeSerializableElement / decodeSerializableValue on whatever decoder they're handed, wrapping the decoder gets you recursion for free and never leaves the streaming path.

What puts me off shipping it is the cost of the wrapper rather than the idea. It means hand-delegating the whole Decoder and CompositeDecoder surface and remembering to re-wrap on beginStructure and decodeInline; it means also implementing JsonDecoder, or every serializer that does decoder as JsonDecoder breaks underneath me; and Decoder isn't guaranteed stable for implementation by user code, so I'd be signing up to re-check that wrapper on every library upgrade for the lifetime of the app.

So what I'm asking for isn't a new capability so much as a supported way to do something the library evidently already permits in principle.

Describe the solution you'd like

A small opt-in hook at the point where a serializer is about to be used — the decodeSerializableValue / decodeSerializableElement boundary — so that interception is recursive by construction and the streaming path is left alone. Something along the lines of:

interface SerializerInterceptor {
    fun <T> intercept(serializer: KSerializer<T>): KSerializer<T>
}

registered on the format:

val json = Json {
    interceptors += TimingInterceptor()
}

The properties that matter to me: it applies to generated serializers without any change to generated code or to the classes themselves; it applies recursively to properties, collection elements and map keys/values; it composes if more than one is registered; it costs nothing when none is; the wrapped result is cacheable per serializer rather than allocated per value; and it doesn't touch serializer resolution, only what the resolved serializer gets wrapped in.

I've deliberately written that as a sketch rather than a proposal, because where the hook belongs is the part I'd most like your opinion on.

One thing I am explicitly not asking for

An interceptor is an obvious place to catch an exception and substitute a fallback, and I want to be clear that I'm not asking for that. Decoder's own documentation says catching SerializationException from a decode* method is not allowed and leaves the decoder in an arbitrary state, and #2238 was closed on exactly that ground. Recovery needs a separate primitive — some way to skip or restore the current value — and that's a different discussion with different risks. Observation-only interceptors are useful on their own and don't need any exception-safety guarantee, so I'd rather keep the two apart.

Related issues

• Provide a way to inject/intercept generated serializer #793 → Keep access to plugin generated serializer when mark class with @Serializable(with = MySerializer) #1169, resolved with @KeepGeneratedSerializer. That covers per-type wrapping; this is about wrapping scoped to an operation, applied recursively, without per-type opt-in.
• Wrapping a type's serializer with another without changing the actual type #2699 is the closest open issue. It's about the ergonomics of substituting the serializer for a specific type, where this is about a substitution point that applies to everything in a decode. If you'd rather keep the discussion in one place I'm happy for this to be folded in there.
• Provide error details in case of [de]serialization failure in a custom serializer #2165 — error details from failures inside custom serializers. An interceptor would let users add that themselves, which isn't the same as the library doing it properly.
• Fallback value for unknown enum values #3057 — enum fallbacks; one narrow instance of what people build when there's no general hook.
• We can't restore deserialize after error #2238 — decoder state after a failure, and the reason recovery is out of scope above.

Questions before I write anything

1. Should substitution sit at serializer resolution, on Decoder/CompositeDecoder, or purely as format-level configuration?
2. Is there appetite for this in core at all? The alternative I'd be just as happy with is a stable, extendable DelegatingDecoder / DelegatingEncoder in the library, so the Wrapping a type's serializer with another without changing the actual type #2699 approach stops depending on API that isn't meant to be implemented downstream. That's a much smaller change and it solves my problem too.
3. If it does go in core, where should caching of wrapped serializers live?

Happy to implement whichever direction you prefer — I'd just rather agree the shape first than send a PR you'd have to redesign.

[fzhinkin]

@kamalnayan04, regarding the first thing, the one about measuring deserialization duration per type - is it something you want to do continuously for a released application, or it's only about profiling a dev version of an app "locally"?

[kamalnayan04]

@fzhinkin Production, continuously, sampled — not local profiling.

It's a server-driven UI, so which types show up and how deeply they nest is decided per response rather than per release: total decode time is a single number over roughly 2000 dispatches across ~50 types, which tells me a screen got slower but not whether the backend started sending more of a type or larger instances of one.

Per-type timing and per-type failure attribution are what make that answerable, and the only supported way to get them today is @KeepGeneratedSerializer + @Serializable(with = ...) on each type, which takes the one with slot, can't be attached to the builtin List/Map serializers or to types I don't own, and quietly loses coverage as models get added.

The failure-attribution half of this we run today on Gson, through TypeAdapterFactory + getDelegateAdapter; Moshi has the same shape with JsonAdapter.Factory + nextAdapter, and Jackson with BeanDeserializerModifier — this hook is the one piece that doesn't port when I migrate.

To be clear, @Serializable(with = ...) itself is stable. The alternative that avoids per-declaration wrapping — a format that delegates to Json, as suggested in #2699 — means implementing Decoder/CompositeDecoder myself, which the KDoc explicitly calls out as not stable for inheritance in third-party code. Either a supported interception point or a DelegatingDecoder/DelegatingEncoder I could extend would remove that — I don't have a preference between them, only that whichever it is, it's public API rather than something I have to re-verify on every upgrade.

[pdvrieze]

The override can work. Xmlutil has this functionality in it policy: https://pdvrieze.github.io/xmlutil/serialization/nl.adaptivity.xmlutil.serialization/-xml-serialization-policy/override-serializer-or-null.html

The biggest question is what overhead it generates (and what approach is best in terms of optimization – I suspect an overridable function is at it allows constant folding and inlining)

[pdvrieze]

A challenge with performance monitoring (beyond micro-benchmark issues) is that the cost would be shared between parsing an object initialisation. Serialization blends these operations making it hard to distinguish.

[kamalnayan04]

@pdvrieze overrideSerializerOrNull is the shape I'd lean toward too, the nullable return keeps the unconfigured path a cheap check the caller can fold away, and the substituted serializer is cacheable per serializer so the lookup cost lands once at resolution rather than per value. On the monitoring side, you're right that a serializer-scoped timer blends parsing and construction, but that blend is actually what I want here: the signal is total decode cost attributed to a type, not pure self-time, and the one thing that does distort attribution, nested timings rolling up into the parent. I can handle by subtracting child costs in the instrumentation.

One detail worth flagging from a prototype I built: the wrapper has to delegate via decoder.decodeSerializableValue(delegate) rather than delegate.deserialize(decoder), otherwise StreamingJsonDecoder's polymorphic class-discriminator handling gets bypassed
I saw polymorphic cases fail until I switched to decodeSerializableValue. And on the overhead question, I have real server-driven payloads and a benchmark harness, so I'm happy to benchmark a prototype override against them if that helps quantify it.

[sandwwraith]

#2165 will be fixed in the next release, see #3235 .

Regarding measuring the time — this is what profilers are for. I don't know all the nuances, but Android Studio comes with a great profiler, which instruments the code offline IIRC. Perhaps you can retain this instrumentation in your production build and then send these metrics to your server somehow. In any case, this seems to be very out of scope for a serialization library. Interceptors idea generalize this, but I don't know any other use cases that would benefit from it.

[kamalnayan04]

Thanks — and thanks for #3235, I read through it. Wrapping decodeSerializableValue with path and cause is the seam I was hoping the library would own, and it covers the NumberFormatException case from my first post.

To clarify the framing from my first post: timing was one example of the shape, not the ask. The ask was always a uniform hook over every serializer in a decode, and timing and failure attribution were two things I'd build on it. The one I run in production today is failure attribution by type — what Gson (TypeAdapterFactory + getDelegateAdapter), Moshi (JsonAdapter.Factory + nextAdapter) and Jackson (BeanDeserializerModifier) give me, and the piece that doesn't port when I migrate. On the profiler: it's a fair tool for a dev build, but it can't run sampled in a release build, and it attributes to stack frames rather than to types in a response whose type graph changes per request, so it doesn't replace the hook for that case either.

#3235 is a good illustration of why I'd rather have the hook than more message logic: the encoder side attaches serializer.descriptor.serialName, the decoder side attaches only the path, and path is per-instance ($.widgets[7].gradient) so it can't be aggregated by type. That's not a complaint about #3235 — no message format can anticipate every metrics pipeline. An interception point lets each user attach whatever they need, and the library keeps its messages simple.

On other use cases: #1205 (skip malformed list items) is the clearest one. Every workaround in that thread is per-declaration — a custom list serializer on each property, or replacing List<T> with SafeList<T> in every model — and the reference solution @qwwdfsad linked there is a Moshi JsonAdapter.Factory, which is exactly one interception point over every adapter. #2699 and #3057 are the same shape, and xmlutil ships it as overrideSerializerOrNull on its serialization policy — format-level configuration, which answers my own Q1. I'm not asking for recovery semantics here; that's #1205's discussion. What all of these have in common is that they're solvable centrally and cleanly once a wrapper can be applied to every serializer, and not otherwise.

So the ask is the interceptor itself, the same primitive Gson and Moshi expose. I'm happy to put up a PR — would you be open to considering it?

[kamalnayan04]

@sandwwraith just following up on this issue.