Merge RWKV back to GGML?

[m1chae1bx]

https://github.com/saharNooby/rwkv.cpp is a forked GGML version that implements the RNN-Transformer model RWKV.

[saharNooby]

Hi! Can you elaborate on what value it will bring to merge the rwkv.cpp repo into ggml?

As I see it myself: ggml is fundametnally a library/framework, and merging downstream application into it is like merging transformers into PyTorch, or merging stable-diffusion-webui into gradio.

Furthermore, maybe ggml can be extended with a simple example of RWKV inference; but rwkv.cpp has outgrown its "prototype & example" phase, and we have plans about extending it even more. I think being in ggml repo will constrain rwkv.cpp development heavily.

[m1chae1bx]

Apologies. I used the wrong word. Merge is not really what I meant but as what you have said, implement a sample inference based on rwkv.cpp. It’ll be beneficial to downstream libraries built on top of GGML like ctransformers.

[ggerganov]

The rkwv.cpp project is doing a great job at maintaining and supporting, so there is no great value in adding the code back to this repo. The only potential benefit from having a minimalistic rwkv example here is that in the future, when we update ggml we can easily adapt the example to the new changes and then the rwkv.cpp will have an easier time into upgrading to the new ggml version. But again, I think it's fine the way it is

[m1chae1bx]

Okay, noted.