It would be very good if CodeChef could provide a detailed set of test case for a given problem and at what particular test case did a particular code fail. This would be a really good initiative on your part to help people in improving the tiny small errors that are really-really hard to spot and are often committed and never spotted.
35 Likes
We do not release test data for various reasons. However, we firmly believe that giving away the test data is a spoiler. It kills the motivation to try that extra bit harder and kills the fun and the challenge of the unknown. We do not want our contests to take that short cut. We make all submissions visible as soon as the contest ends to let you learn from other people’s submissions. We also upload editorials for each problem with detailed explanation, so that you can learn from them. We encourage you to go through the editorials at the end of the contest and ask your questions if you have, on the page itself. Your questions will be answered directly by our problem setting panel. However, we are planning to show judge information for each submission like the time taken for run each test case, at what test case did the submission fail, etc. in the foreseeable future.
53 Likes
I agree with you, that learning is easier with test cases, unfortunately almost no contest page shows test cases, thre are 2 exceptions that I know TopCoder and CodeForces.
3 Likes
Maybe there’s a way our experiences can help you help us 
For example, in Serbia, binaries are tested completely different. There’s always 10 test cases, but each test case contains one set of input data. So, that way you get 10 points for each test case, and the max you can get is 100… And I still can’t see why we can’t have test cases for the tasks that are not in Compete section, that are in Practice section… 
4 Likes
Where is the motivation to try harder when it is not even revealed why particular code produces the wrong answer? If my code compiles on ten different compilers before I submit, figuring in every possible test case on a simple solution, and displaying the exact output specified and taking the exact input specified, but I get a wrong answer with your tester - I have a feeling this situation requires modifying code to figure out how your site even WORKS over and over and over again, with no idea of where to go next because no frame of reference is provided.
17 Likes
@bhampton: There is a frame of reference. We reveal all the solutions of all the problems after the contest and the problems are also moved into the practice section for the users to try at leisure. And you can see other’s solutions too. Check this: http://www.codechef.com/submissions. Revealing test data is a spoiler and short cut to correcting your solution. Its the pursuit of finding the unknown that makes it exciting.
3 Likes
We can also have partial scoring test data on CodeChef. The online judge has the capability. But that does not yet convince us to reveal test data. Ad mentioned before, even in the practice section, we believe that revealing test data is a spoiler.
1 Like
Another useful idea for me is to writing in which test-case we have the problem: e.g. TLE in test-case 8. This will let us know how close we are in the correct answer!
1 Like
I have to say that although it might seem frustrating during a live contest to see WA or TLE and being totally “blind”, it is also the ONLY way to help contestants to improve their skills and I speak from experience, as one time during a Codeforces problem, I was able to immediately see a pattern by looking at a test case where I was getting WA, which revealed the whole problem to me…
It can seem unfair, but trying harder, reading tutorials on a particular data structure or implementation idea along with the editorials/accepted codes once the contest ends, it’s the only true way of improving 
5 Likes
Even if we dont get partial scoring, providing just the detail like what was the status for each test file would be great help. It gives a hint if we are going totally wrong or somewhat in right direction.
If revealing test data is a spoiler then certainly revealing hundreds of examples of correct code is a spoiler!
It’s so frustrating to as few as one input example from the problem definition. What is one to do when one’s code seems correct but a simple “Wrong Answer” message is returned?
Google Code Jam gives you more test data for the “small” version of each problem. At least you get to try your code with some data.
This is really a dealbreaker for me. I like the CodeChef site and the amount of problems available, but it’s just too frustrating to get such small input sets. I’m moving on.
1 Like
Of course i agree that providing test vectors will give short cut to solution. But for practice problems it is better to provide test vectors. It will help in knowing the boundary conditions and exceptions that we miss during writing the code. we just wanted a preparatory standards to go for a challenge.
Rather than blindly telling that your answer is wrong, it is always good to know which case i missed causing the issue.
yea…this system is followed in codeforces
I remember, in my childhood I used to solve math problems but when I used to get stuck on a problem I tend to look at the answer/hint and then solve again removing that mistake. My father used to tell me, “Son, this is a wrong way of solving problems. You must think a lot of times before getting the actual answer. Think over and over again until you get the right answer. Dig every corner of your mind. Then it will help you a lot.” I remember his words, and it really helped. I do not have to say anything more for this topic, but I know that this is a good feature of NOT providing the test cases.
Disclaimer: The author of this post has put his personal opinion about his thinking on the topic. And people might still hate it or dislike it, for which author would not argue over the same.
11 Likes
Not revealing failing test cases during contests is justifiable, However illogical if i’m solving problems from problem banks . Spoiler? How? After repeated failing if i’m not able to figure out theres no learning its just time spent to waste. As for editorials,LOOK-i have a completely different algorithm of my own and i want to know here i went wrong if i do-THAT’S LEARNING.“where you went wrong”-thats learning. Editorials are just a way to see the problem through somebody else’s mind and follow their algorithm and techniques. I don’t see it any different from copying others solutions.
32 Likes
Most of the time, we are not sure whether the problem has been correctly understood. One possible solution could be that we may provide with our test cases along with solutions to be checked whether they are correct or not. if this is not a spoiler then proper checks and balances would have to be put in place.
If by chance there is a bug in the solver then this system can very easily point that out instead of relying on persons whose wrong submissions have been accepted.
BUT ATLEAST AT THE END OF THE CONTEST ,YOU CAN PROVIDE PROVIDE THE PARTICULAR TEST CASE WHERE A PARTICULAR CODE FAILS.THIS WILL HELP A LOT OF PROGRAMMER TO IDENTIFY THEIR MISTAKES INSTEAD OF LEARNING IT FROM EDITORIALS.
9 Likes
The best solution to this is to provide the number of test cases out of total in which our code fails and the number of test cases in which our soln passes!!! So that if we get W.A , a person may think that his algorithm or approach is wrong but the truth may be far from it!! May be the logic fails in one or two test cases so that the person can work on finding the corner test cases!!! Just like how hackerearth does!! That indeed is a way of saving hours of frustation and wasting time to understand the editorials which are generally cubersome and hard to understand
1 Like
Bye bye codechef 
1 Like
Hackerrank also
1 Like