Auto Tester - Find testcases where your code fails

general
,
1

Hi,

I have made a script by which you can find testcases where your code fails. To run script, you need to give 4 paramters:

  1. Generator code. Code which generates random testcases.
  2. Correct code. This brute force code which will always give correct results.
  3. Wrong code. This is code with better complexity but is giving wrong answers.
  4. Number of test cases. Enter how many testcases you want to genrate where your correct code and wrong code gives different results.

You can find script and other details here.

Note : This script is made for C++ but you can make it run for other languages by making desired changes in line 5 to 12.

12 Likes
2

I would like to share my PoV regarding this script.

This is a great script, appreciable work. But i would like to tell users how i learnt to debug.

I make corner cases, random generator for test cases, and for certain problems, worst case making (like fully connected graph for floyd warshall becomes O(N^3).)

Though at beginner level, its okay to rely on this tool, but in the long run, U should learn to rely on urself.

I have no intention to offend the author, but i felt it neccessary to mention.

Hope u dont mind, else delete comment option is there, and i wont mind at all @dushsingh1995

3 Likes
3

@taran_1407 -I understand. In this case also, user has to make the generator himself.

4

Nice Work Bro It Just Eases a lot of work of manually checking through the output file for bug.

1 Like
5

What I usually do is to use python for generating random tests - it is so much easier than C++/Java. Furthermore, it is easy enough to write a method that uses brute force in the same source code file as the solution (and asserting that two different solutions produce the same answer). Thereafter, I would press the up arrow and enter key in terminal repeatedly for multiple tests (I will also print the test case in console in case I accidentally skip a failing test case). Yes, this is a lazy method. But it suffices for my needs as it is fast and convenient without needing to resort to third party tester scripts.

Nevertheless, I appreciate your efforts to help others!

6

Can anyone specify how to generate large test cases ?

7

Suppose you have to generate large test case for https://www.codechef.com/JAN18/problems/MONSTER.

Here is the generator > https://ideone.com/r8TlDZ

But for above script to work, you need to generate smaller test cases because brute force will time out for larger test cases.

8

What I used to do in long challenges(sometimes in short too) if I was getting wrong answer for an optimal solution.

  1. Write another code which will be having higher complexity than the current one but it is 100% correct.

  2. Write random test case generator. I didn’t knew it is easier in python. How?

  3. Open 3 terminal. In first, generate random test case from generator then run other two code on this test case. Then compare the output files.

That’s exactly what the script is doing. :slight_smile:

9

you can also use spoj testcase generator in competitive programming to generate test cases in specific format,length and number etc…even in matrix,tree,graph form.it will give u testcases but u have to manually test using testing code or just checking all answers.here is
link

10

That is helpful. Thanks for putting effort in it.