I just found 2 critical bugs with pypiper result reporting. Actually it may just be 2 outcomes from the same critical bug.
When you report a result it doesn't overwrite the result if there's already one there. This means there's no way to update something if an incorrect result was entered. In the past, results would be reported as many times as you ran the pipeline. All of these results are recorded. Now, the pipeline only reports the first result. If you re-report that result, it's simply lost.
When I'm re-running a pipeline to correct something, I get these:
These results exist for 'demo4': seqcol_digest
These results exist for 'demo4': Time
These results exist for 'demo4': Success
And they're not updated. So, that's the first problem: reporting of results is limited to a single value now, and multiple reports is not recorded.
The second issue is related to it:
The result is not only not reported to the results.yaml file, it's also not reported to screen! So it doesn't even show up in the log file anymore. Results should be reported to screen every time a result report is requested.
Solutions
The second issue should be easy to solve, just make sure pypiper is always printing the result. I'm not sure if this should happen in pypiper or in pipestat, but somewhere it has to happen.
The first problem is harder to solve, because it's a consequence of the switch from a simple tab-delimited stats.tsv file pypiper used to use, to the pipestat file. In pipestat, results are recorded per sample, not per pipeline run, which means there's going to be issues when you re-run a sample. The best I can think of is this:
- In the short term, at the very least, the last reported result should be the one used, not the first reported result, as is currently happening. I guess this is as simple as forcing updates whenever you report an existing result.
- In the longer term, pipestat should offer the option to include a history of results, and these should be stored somehow in the file (and database). This may not actually be too hard to implement; just add a 'history' function, and when something is overwritten, just move the old values into the history in a way that is an array, rather than a single value. Then, pipestat could offer a
clear history function to remove old stuff, if desired, but otherwise, repeated reports of the same result will simply add to the history.
I just found 2 critical bugs with pypiper result reporting. Actually it may just be 2 outcomes from the same critical bug.
When you report a result it doesn't overwrite the result if there's already one there. This means there's no way to update something if an incorrect result was entered. In the past, results would be reported as many times as you ran the pipeline. All of these results are recorded. Now, the pipeline only reports the first result. If you re-report that result, it's simply lost.
When I'm re-running a pipeline to correct something, I get these:
And they're not updated. So, that's the first problem: reporting of results is limited to a single value now, and multiple reports is not recorded.
The second issue is related to it:
The result is not only not reported to the
results.yamlfile, it's also not reported to screen! So it doesn't even show up in the log file anymore. Results should be reported to screen every time a result report is requested.Solutions
The second issue should be easy to solve, just make sure pypiper is always printing the result. I'm not sure if this should happen in pypiper or in pipestat, but somewhere it has to happen.
The first problem is harder to solve, because it's a consequence of the switch from a simple tab-delimited stats.tsv file pypiper used to use, to the pipestat file. In pipestat, results are recorded per sample, not per pipeline run, which means there's going to be issues when you re-run a sample. The best I can think of is this:
clear historyfunction to remove old stuff, if desired, but otherwise, repeated reports of the same result will simply add to the history.