Add flags option to nanomsg port msg calls. Make port_remove non-blocking.

[fruffy]

A little change to make sure clean up is somewhat non-blocking. There are some cases where cleanup can take a really long time. This is because __del__ tries to remove every single port synchronously, which gets slow for several hundred ports. Ideally, cleanup should be done in bulk by deleting all ports at once, but the current setup does not seem to support this.

[antoninbas]

Doesn't that increase the probability of failure though?
If the message cannot be sent right away (maybe the peer is still consuming the previous message?), you will get an exception. It will interrupt clean up and the port won't be removed properly at the peer. Am I missing something?
Maybe you should use a thread pool instead to handle port removal in parallel.

[fruffy]

True, not happy about this approach, but it seems important to make the delete call non-blocking.

Interesting, did not think about a thread pool. The problem is where would that thread pool be? The messages are sent every time __del__ is called, which can happen at any point (exceptions, exits, normal execution). So I guess it would need to be tied to whatever manages these objects, which is the DataPlane class?

[antoninbas]

If you think making the call non-blocking will work well for your use case, then the less contentious and simplest option may be to add a command-line argument.
I believe that platform_args were meant for something like this originally:

ptf/ptf

Lines 285 to 287 in bb28a88

	group.add_argument(
	"-a", "--platform-args", help="Custom arguments for the platform"
	)

You could consider reviving platform_args and adding one for the nn platform to toggle non-blocking port remove.