loved the problems this time around 
Even I loved quite many of them. My favourite was CUTPLANT. AVGPR adn WEIGHTNUM had good concepts involved as well wheich I felt are good for beginners
2 Likes
One of the high-scoring solutions to CHEFPAR worked by reverse-engineering some of the parameters for the hidden problems and individually optimising different random seeds for each hidden problem. This was done by making use of the fact that you get told if your program aborts on a hidden problem, so some hidden information leaks back. The program would not work so well (or might not run at all) on a new random problem instance.
I donât know if this is allowed or not, but I donât think it is a desirable feature (and I wouldnât use this method) as it seems to be against the spirit of the competition, in my opinion anyway. I suggest that the challenge problems be adjusted in future to reduce the effectiveness of this method. One thing that could be done is reduce the number of submissions allowed, since 200 is rather a lot: 50 should be plenty since you can try your method offline on test data. (Maybe even less than 50 is better.)
Another problem is that there are potentially even larger information leaks from the non-hidden problem instances (four instances in this case), because you get to see about 20 digits of output and there is execution time and memory usage that can be manipulated. Perhaps it would be a good idea to exclude the non-hidden problem instances entirely from the final judgement (in this case judging on 16 instances rather than 20). At least, that would fix this subproblem (information leak from non-hidden problem).
I suppose the nuclear option is not to give any information back at all (even TLE, RE etc) about what your program did on the hidden instances. I would personally be in favour of this, though I realise this could lead to disappointments if none of your submitted solutions run properly on the hidden set of instances.
1 Like
@vijju123 CHEF VIJJUâS CORNER Nice initiative. Increases the effectiveness of editorial. And helps a lot in overcoming mistakes.
1 Like
Thank you 
I started this when I wrote ICPC editorials for Amritapuri- to put basically any content which I feel might be useful to some, but couldnt be put in formal section. (I got too many complaints of my editorials being too long and too explanative xD). Chef Vijjuâs corner, or the unofficial part is a nice place for some light hearted humor, and other approaches and âwhat ifâ things. Glad you liked it . (After all, I believe my editorials should stand out from rest, shouldn;t they? xD)
2 Likes
We are aware of that. Just to reduce that they reduced it from 500 to 200. If any further reductions are needed, they will do so as well :). Do you think it is direly needed right now?
Yes, they should stand out from rest.
A simple way of helping the issue is to do something like what is done at the Yandex optimization track, just consider the LAST submission for the final score. This doesnât give the incentive of submitting a lot of times to randomly get a good outcome (since you donât know the full test suite). Rate limiting, while a decent idea, does not alleviate the issue that submitting a random algo a lot of times is beneficial. @admin @aryanc403 @vijju123
3 Likes
Yes because I think it is better if the competition is to find the program that works best on a random problem instance, rather than being a competition to reverse-engineer the hidden instances. I think that will make CodeChef a more attractive environment to compete in. (At the moment the top entry in division 1 is engineered to work on the specific hidden instances and wonât work properly on random instances. I donât blame the author for doing this, but in my opinion itâs a more healthy competition if this kind of answer isnât possible.)
My personal Opinion-
@alexthelemon One thing which is being neglected here is that challenge problems require strategy which is different for different users. And this is also not so trivial. What you did is I see as anther strategy. Which probably nobody tried. And doing this also not an easy task. And I wish you All the Best for next question where you can again apply this strategy. There is no harm in Doing this. For other problems also we try to find edge cases and sometimes hard code the edge case. Others might not agree with me.
@admin will collect all feedback from the thread today or latest by tomorrow. This will be considered upon. Thank you, both of you
1 Like
aryanc403: I appreciate that this is allowed strategy and is not trivial to carry out, but I donât think it is a good one to encourage because it deflects from the primary purpose of making a good algorithm. I know (as you suggest) that I could do the same thing myself, but I am not going to do this: if that is the only way to win then Iâd prefer to compete elsewhere instead. I donât think Iâm the only one to think this, because vijju said that the maximum number of submissions has been reduced to try to prevent this. (I have a suggestion as how to modify the rules in a separate post below.)
1 Like
Suggestion for how to prevent reverse-engineering of challenge problems.
Taken from the thread on CHEFPAR and reverse-engineering instance parameters. In my opinion it is better if the competition is to find the program that works best on a random problem instance, rather than being a competition to reverse-engineer the hidden (or visible) instances. I think that would make CodeChef a more attractive environment to compete in (it is already very good, by the way - I hope this suggestion might help make it a bit better).
Definitions: by âfeedback-instancesâ I mean those which contribute to your provisional rank, and for which you get back a score (four of these in the CHEFPAR contest, one for each type). By âhidden instancesâ I mean those for which you donât get a score back during the contest (sixteen of these in the CHEFPAR contest, four for each type).
As far as I can make out, there are two competing needs:
(i) It is nice if people can be confident during the competition that their program will work (not abort, time-out etc) on hidden instances, so they need some kind of feedback during the competition that it works on hidden instances. Itâs also nice if people feel able to try out lots of solutions over the 10 days.
(ii) It is good (in my opinion, anyway) if you canât use the information from the result code (AC, TLE etc) to reverse-engineer parameters of the hidden instances. Itâs also good if you canât use the result information (score, time, memory usage) from the feedback-instances to reverse-engineer their parameters.
I think you can satisfy both of these needs by the following modifications to the rules:
(i) You are still allowed to submit a lot of answers (say 200), but in only (say) 20 of these will you get result code feedback from the hidden instances. 200 should be plenty to try out lots of ideas, and 20 should be plenty to make sure your program (that you already know runs successfully on feedback-instances) doesnât crash on hidden instances. The user would get to choose which (up to 20) submissions count for the extra result code information. (The displayed time and memory usage would be taken from the feedback-instances, so you canât get extra information about the hidden instances that way.)
(ii) feedback-instances are excluded entirely from the final score (after competition ends). In the case of the April 2018 challenge, that would mean you would be judged on 16 hidden instances rather than 20 = 4 + 16 instances. The reason for this is that it is very easy to leak information back from the feedback-instances, much easier than from the hidden instances. For example in CHEFPAR, during the competition you got back a 16-digit final score: you could encode information in some of these digits, so you get back much more than 1 bit of information per submission.
I think these changes would keep most of the necessary feedback so that users know their code is working and preserve the fun of seeing a live scoreboard, but would mostly prevent the problem being âhackedâ.
1 Like
@alexthelemon I strongly agree with you that reverse-engineering isnât fair. Unfortunately, it hasnât prohibited till now, hence itâs allowed. Probably, it was the main reason why some contestants were too good at challenges for years. Weâll discuss it and I hope we find some solution(hidden time/memory and few submissions sounds good). Thanks for your feedback!! Also, congrats on winning Div2!! Good luck in Div1
3 Likes
For Documentation. Today as soon as editorial were released. Our @vijju123 were quite active in forum to answer queries of users. This is also a Nice initiative. If we have someone from problem setting panel to resolve our queries. =
1 Like
@aryanc403- Why? Thanks mate for the compliments!
We intend to do that as a regular practice to be honest, but problem setters are really busy at times. Like, some are having end-semester exams &etc. Even I have those, but comparatively I am a bit free right now, so I am trying to resolve as much as I can
This time, I also tried to answer as many comments as I can xD. Just to assist setters. You wont believe, but we had over 150 comments for weightnum alone, all saying one thing âWhy is W upto 300?â
Well, there is only 1 @vijju123 in entire world. And I have the motto that âIf its me, things should change for good .â I tried my best to act on lines I expect the panel to act- eg- answering all comments, even if its a trivial âComment deniedâ or âAsking for hints isnt allowed.â Its really important in my opinion, that, setting panel should respond to comments else there hangs a feeling of âAloofnessâ. Like, in CF rounds, all queries get answered, even if it takes 5-10minutes. If it happens there, why not here?
Though, I must agree on other side too, answering comments is tiring. For AVGPR and WEIGHNUM, there were like, 10 new comments every hour. Dont believe me? See the tab below-
Click to view
.png)
And that brings me to one of my key points. People, especially Div2, should NOT share ideone links etc. in comments, not ask for hints, or ask setter to debug his/her code. Thats simply outrageous. Because of that, we keep on missing some valid comments 
1 Like
In my opinion it is better if the competition is to find the program that works best on a random problem instance
Oh! Are you suggesting that the TC at which program runs should be dynamically generated rather than being a fixed case?
We fear that some contestants may get unlucky (or too lucky- both are bad ). Like, once in the last problem of long (Something on squarefree numbers ) the TC were dynamically generated. My solution which TLE on cases, got accepted on 5th try. We will need to find a way to minimize- or even prevent these instances from happening if we are to implement it
I think we can implement hiding the time and memory taken for challenge problem- merely telling if its ACâd or not. That can help a lot.
I think we can do away with telling the âScoreâ of problem- merely telling how many points it fetched you out of 100 seems good.
The suggestion to âsubmit upto 200 solutions, out of which at most 20 (which ran on hidden TC) will be considered for leaderboardâ seems nice. 20 submissions limits reverse engineering by a lot.
Already pinged @admin to collect feedback by tomorrow or day after, so feel free to suggest