One other layer we should think about is output formats. There are quite a few primary ones: JSON API, JSON HAL, JSON-LD, etc.
There's also possibly transformers, though this is less likely necessary in our case, and might be mitigated by hooks in resource handlers (though I could see using the default handlers, and just tweaking the format slightly). One source of inspiration here is Fractal.