Here is an example of a simple experiment:
{
"version": "1.0.0",
"title": "test",
"description": "test",
"configuration": {
"myarg": "exp.json",
"another_arg": "another.json"
},
"tags": [
"network",
"lala"
],
"controls": [
{
"name": "Safeguards",
"provider": {
"type": "python",
"module": "chaosaddons.controls.safeguards",
"arguments": {
"probes": [
{
"name": "Non existing probe",
"description": "Probe that doesn't exist",
"type": "probe",
"provider": {
"type": "python",
"module": "blablabla",
"func": "bla",
"arguments": {}
},
"tolerance": 2
}
]
}
}
}
],
"steady-state-hypothesis": {
"title": "Check something",
"probes": [
{
"name": "My probe",
"description": "My best probe",
"tolerance": {
"type": "regex",
"pattern": "exp.json",
"target": "stdout"
},
"type": "probe",
"provider": {
"type": "process",
"path": "echo",
"arguments": "${myarg}"
}
}
]
},
"method": [
{
"name": "My probe",
"description": "My best probe",
"tolerance": {
"type": "regex",
"pattern": "exp.json",
"target": "stdout"
},
"type": "probe",
"provider": {
"type": "process",
"path": "echo",
"arguments": "${myarg}"
}
}
],
"rollbacks": [
{
"name": "My probe",
"description": "My best probe",
"tolerance": {
"type": "regex",
"pattern": "exp.json",
"target": "stdout"
},
"type": "probe",
"provider": {
"type": "process",
"path": "echo",
"arguments": "${myarg}"
}
}
]
}```
Notice the probe with module and function as "blabla"
Obviously it doesn't exists. But if you run the experiment, it just runs normally by ignoring that probe.
If I put the real probe there, it works fine.
I think the expected result would be some kind of an exception and abort everything.
We've been running experiments for a long time without knowing that we've had a non working probe.
Here is an example of a simple experiment:
{ "version": "1.0.0", "title": "test", "description": "test", "configuration": { "myarg": "exp.json", "another_arg": "another.json" }, "tags": [ "network", "lala" ], "controls": [ { "name": "Safeguards", "provider": { "type": "python", "module": "chaosaddons.controls.safeguards", "arguments": { "probes": [ { "name": "Non existing probe", "description": "Probe that doesn't exist", "type": "probe", "provider": { "type": "python", "module": "blablabla", "func": "bla", "arguments": {} }, "tolerance": 2 } ] } } } ], "steady-state-hypothesis": { "title": "Check something", "probes": [ { "name": "My probe", "description": "My best probe", "tolerance": { "type": "regex", "pattern": "exp.json", "target": "stdout" }, "type": "probe", "provider": { "type": "process", "path": "echo", "arguments": "${myarg}" } } ] }, "method": [ { "name": "My probe", "description": "My best probe", "tolerance": { "type": "regex", "pattern": "exp.json", "target": "stdout" }, "type": "probe", "provider": { "type": "process", "path": "echo", "arguments": "${myarg}" } } ], "rollbacks": [ { "name": "My probe", "description": "My best probe", "tolerance": { "type": "regex", "pattern": "exp.json", "target": "stdout" }, "type": "probe", "provider": { "type": "process", "path": "echo", "arguments": "${myarg}" } } ] }``` Notice the probe with module and function as "blabla" Obviously it doesn't exists. But if you run the experiment, it just runs normally by ignoring that probe. If I put the real probe there, it works fine. I think the expected result would be some kind of an exception and abort everything. We've been running experiments for a long time without knowing that we've had a non working probe.