DoX

[nvx]

singleflightx has a nice pattern where DoX can request multiple ids at once, it would be super handy if this pattern was extended to batchify to do multiple lookups at once from a single goroutine.

[samber]

I agree. It would also reduce locks.

Want to work on it ?

[nvx]

I'm keen to have a crack at it.

One gotcha I noticed though is while singleflightx's Do just returns the value directly, the callback passed in is only a single value too, while DoX where the callback returns a map of values the result of DoX wraps the result in a Result type so it can be signalled if the callback func did not return a key for that type. This differs from batchify where the callback always accepts a slice, but Do doesn't return the result wrapped in a Result type, so it's not possible to easily tell if a given result was returned or not (for comparable types you could compare against the zero value for that type, but this goes out the window if the type is not comparable).

I guess the edge case might exist in singleflightx if in-flight requests for the same key passed between Do and DoX are deduplicated as well but it's not completely clear if this is the case.

Do you think batchify's Do should have a Result wrapper on the result always? Or not worth breaking the API and instead just adding a comment describing the behaviour and rely on DoX even for a single variable if you need to handle a value not being returned by the callback?

[samber]

I suppose that in singleflight, the error might be different between values (or no error), but in batchify, a single error is expected from a batch ? So we don't have to add an error to each value. Returning a map without missing input values would be used this way:

values, ... := b.DoX(...)

if output, ok := values[input]; ok {
   println(output)
}

I think the input should be comparable, because it allows us to return a map. Slices of values are very slow to access.

So I would vote for a global error, with only non-missing values:

// map, error
values, err := b.DoX(1, 2, 3, 4)

In hot, I added a third variable listing missing values:

// map, slice, error
values, missingValues, err := b.DoX(1, 2, 3, 4)

WDYT?

[nvx]

For errors that's fine, I was thinking more of "Lookup these 5 keys" but one of those keys weren't found, so not really an error, but you'd only have 4 returned results.

I guess if you're returning a map you can check if your key exists in the map easily enough and you should know the values you passed, but the single case of Do() is where it's hard to detect since it'd just be returning the zero value (like calling Do() for 5 different keys which all get batched together, but only 4 of those exist)

The input being comparable is fine, I was meaning the result for comparable-ness which is where it gets trickier determining if the zero value was returned or not (assuming that the zero value isn't a valid result separate from "no result returned" too!)