Multithreaded request processing

[GlebBatykov]

Hi, creating a web framework for Dart is a great idea!

Do you plan to add work with multithreaded request processing under the hood to the framework, for example, as it was in Aqueduct framework?

[Mehmetyaz]

Hi. Thank you! Dart makes multithreading very easy with Isolate. Your suggestion is very good, and I will definitely consider it. I will write an article about it, but I should briefly state that my performance policy is:

Do not care about the amount of ram consumed by the components and the space occupied by the application on the disk. Use customized components for operations as much as possible.

Considering this with my policy, I should test the optimization with multithreading as well.
This will take some time.

[GlebBatykov]

I once wrote my own mini web framework for Dart (I did not publish it, and now it is no longer relevant due to its use dart:mirros. AOT compilation cannot be used together with dart:mirros, what is unpleasant).

In general, why am I saying this.

You can use isolates to, for example, use the "shared" property of HttpServer parameter to distribute the requests coming to the server between several isolates.

This allowed me on my computer to increase the rps of my Dart server 3x (9000 rps to 26000 rps).

You can also use isolates to, for example, handle heavy routes on the server (I measured the execution time when using a route for the first time and then on subsequent calls to this route either processed it immediately in the isolate that received the request, or sent it to be processed in the pool of isolate workers).

Your framework looks interesting and I just suggest thinking about using isolates under the hood when processing messages.

However, I understand that perhaps this approach to using multithreading does not coincide with your vision of your framework.

[GlebBatykov]

I just noticed that all the Dart web frameworks I've seen except Aqueduct don't use multithreading, which surprised me.

Of course, the increase in productivity may not always be large, it depends on the task and on the target machine.

[Mehmetyaz]

It's too early to determine if they overlap. I need to try your recommendations, test their ram consumption and CPU usage effects. Because you're talking about a serious speed difference. I also request that the issue remains open until I share the results on here.

[GlebBatykov]

It seems to me that it is also worth considering the fact that the next update of Dart (you can read about this in the beta branch on the git hub - CHANGELOG.md) will significantly increase the performance of isolates, as well as greatly reduce the amount of memory that they consume, reduce delay start time.

Which, in my opinion, should further expand the possible options for their use.

[Mehmetyaz]

In this case, there are 3 different approach.

1. Same component tree, multiple httpserver. In this approach, is not available for our. Because incoming requests need to be converted and sent to the main isolate and re-transformed and answered. This approach is very easy for me to develop (just a new HttpService implementation). But it doesn't seem to be of much use.

2. Create multiple component tree. This approach can cause too much ram consumption. Because the data stored in the states will be kept multiple times (for example cached data). In addition, it can cause problems such as establishing more than one bridge to the database. Not everyone will want this. But we can save a lot of processing power. Again, it's pretty easy to develop.

3. The solution that makes the most sense to me is this: in addition to the second approach, some states and services (eg logger) can be shared between isolates, depending on the user's preference. This is an approach that will take a relatively long time to develop. Also, ram consumption can be optimized according to the second approach. In addition, it may not provide a performance increase in all cases. Especially if access to shared services is required for transaction completion.

Which method would you recommend in this case? Or are there any other approaches you would suggest?

[Mehmetyaz]

can therefore share various VM-internal data structures. in CHANGELOG.md. So this innovation will work in service sharing. dart-lang/sdk#46754