-
Notifications
You must be signed in to change notification settings - Fork 533
Description
Prerequisites
Before submitting your question, please ensure the following:
- [No ] I am running the latest version of PowerInfer. Development is rapid, and as of now, there are no tagged versions.
- [yes ] I have carefully read and followed the instructions in the README.md.
- [Yes] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
Question Details
-
Can the Tiiny hardware 'pool' its computing power with my host computer? For example, if I have a powerful Mac M4 with 16 GB RAM, can PowerInfer utilise both the Mac's NPU and the Tiiny hardware simultaneously to give the whole setup a massive performance boost, or is the Tiiny hardware designed to work entirely on its own? If it can, its computing power can reach new heights. BTW, it should, otherwise it would be a wasted opportunity.
-
If the system does not leverage the host’s power, what are the minimum requirements for the device I use to control it? Can the Tiiny hardware do 100% of the heavy lifting so that I can run high-end LLMs using an entry-level Chromebook as my interface? Is ChromeOS compatible with Tiiny? If I am spending all my money on the 'Tiiny' at least let me use it with the cheapest range of Chromebook. It will be a very cost-effective partnership for many. Chromebooks may be cheap but they are very lightweight and have good battery life and can pair well with 'tiiny' and a good power bank.
-
Is it possible to run the 'Tiiny' purely over a local network (like Wi-Fi) without being physically plugged into the host computer? Since the hardware is doing the heavy work, can I just connect to it wirelessly from anywhere in my house?
-
Can the system combine local Tiiny computing with cloud API power? Much like the HazyResearch/minions project, I’m interested in using the local hardware for drafting and retrieval while only calling out to an API for the most complex reasoning steps. Just thinking. this is not important.
Additional Context
Questions 1 and 2 are main questions. Others are just for my curiosity.