More information on wrong results

Why is there so little information when the uploaded solutions end up wrong? Shouldn’t we be able to get a bit of input on what went wrong by, for example, receiving the test-case in which it failed?

Otherwise, we end up blindly trying to fix a problem that we have no idea what it is.

This would make sense if we had a system that rewarded users for having a near-correct solution. An example of this is an exam in some class. The student doesn’t know the solution when he is doing the exam, but he’s score is given based on how good is solution was, even if it ended up not being completely correct.

This scoring system, however, does not provide any input on what you did wrong and does not provide any score unless you successively completed the problem.

because it would then be easy to hardcode the solutions where your algorithm is failing.


Hardcode? This would only work if the tests are static and not dynamic. Is this the case?

If they are static and it would not make sense to provide the testcase in which it failed, it would make sense to update the scoring system to also give points based on your solution overall, not only on OK or WRONG results.

this is the point : i think they are static, but maybe i’m wrong.

as for now, the scoring system is binary : OK, or WRONG.

but you suggestion will probably be read by an admin. :slight_smile:

problem with partial scoring is that typically there is small number of corner cases comparing to huge number of standard cases, to be fair, program should be tested for all i <= N from statement…

Instead of test case I’d like to see the number of test cases that my solution passed, so I know is my actual submission is corrected (when the number increas) or I added just a new bug…

the big problem with that, is that you can “read” the input files data by guessing the values and raising an error to proceed by dichotomy. it’s the same technique when you do blind SQL injection. as soon as you get some information from the online judge, you can virtually obtain all the test data. and i think this behaviour is not wished by our admins.