Improve logging infrastructure when used with xUnit

[suvamM]

In its current form, the logging capabilities of Coyote testing when used from within xUnit is quite limited. To get the logging to work, custom solutions need to be implemented (example: make a custom Coyote logger which accepts the xUnit logger, pass this customer logger to the Coyote testing engine, etc), which makes it both time-consuming and is bit of a trial-and-error process (some solutions may not work at all). Setting the WithVerbosityEnabled flag does not generate any logs at all.

Could you kindly provide a clean API that allows retrieving the Coyote logs, with the appropriate verbosity level, from within xUnit?

[pdeligia]

We are not able to support custom logging for xUnit or an API specific for xUnit from inside Coyote, else we need to do the same for every single framework. (We also do not want to be too opinionated, different xUnit users might want to use variations of a logger. We want to be able to support the most generic scenario and users can specialize from there.) Unless you are thinking something else here, in which case could you elaborate on what that API would look like?

The stance Coyote takes is to provide a generic ILogger interface that users can implement for their own needs and register it to the testing engine. If you have an existing logger in your application, it is easy to convert it to ILogger as it's based on the System.IO.TextWriter class of C# and just need to implement this interface.

Coyote itself uses xUnit for its regression test suite, and the logging there works without issues today. See base test class here.

var logger = new TestOutputLogger(this.TestOutput);
var engine = new TestingEngine(configuration, test)
{
  Logger = logger
};

As I mentioned offline, an updated ILogger is scheduled for an upcoming release soon that polishes this experience more, introduces levels of verbosity (including Debug, which currently Coyote does not forward to a custom logger) and introduces a new flag for writing logs to console (currently WithVerbosityEnabled is forwarding to console if enabled). But that should not affect your current situation.

Saying the above, I am not sure why your xUnit logger is not printing messages as I am not familiar with your code. Can you elaborate on what kind of messages are you trying to print with an example and how you are printing them?

Note that the logger you install to the testing engine of Coyote is only used to capture output that the Coyote engine logs (for task-based programs that means only debug logs on scheduling decisions, no application logs; for actor-based programs it also logs all the actor related events, like sending events and creating actors). If your application has custom log output, then Coyote does not capture that unless you redirect it manually to the same logger that you are installing to the TestingEngine. From inside the application you then need to use that logger (instead of e.g. writing to Console with Console.WriteLine(...)). Like in this sample here. Let me know if this helps.

[pdeligia]

Btw, if curious, the new logging infra is available in this PR #377, but currently requires building Coyote from source.

[suvamM]

@pdeligia Thanks for the comment! The scenario I am looking for is a clean way to emit Coyote debug logs only, and not application specific logs. I'll wait for the upcoming version to try out the functionality.

[pdeligia]

Thanks for confirming @suvamM. What you are asking will indeed be available as soon as the new logging infrastructure gets released soon. Capturing debug logs from Coyote to a custom logger was not available today (without hacks that intercept console output), that is why you were unable to see them.