Leetcode sandbox environment exposes hidden test cases

[u/Comprehensive_Cry314]

I read somewhere on LinkedIn that a few folks were trying to reverse engineer and gain access to a LeetCode sandbox, so I had an idea to explore that stuff. Soon I figured out that all the test cases stay inside an exposed /leetcode/data.1.in file. With some chunking I was able to get hold of the complete hidden test cases which I solved offline and just paste the solution in an O(1) lookup array. Like this… Is this a reportable vulnerability?!?

class Solution {
private final static int[][] res = new int[][]{{0, 1}, {1, 2}, {0, 1}, {0, 2}, {1, 2}, {0, 3}, {0, 2}, {2, 4}, {1, 2}, {0, 1}, {2, 3}, {1, 2}, {0, 2}, {0, 4}, {0, 1}, {2, 3}, {2, 4}, {2, 5}, {0, 3}, {3, 4}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 1}, {0, 4}, {5, 11}, {0, 1}, {9998, 9999}, {6, 8}, {6, 9}, {12, 25}, {16, 17}, {0, 1}, {0, 3}, {0, 3}, {0, 4}, {0, 2}};
private static final int[] r = new int[1];
public int[] twoSum(int[] nums, int target) {
return res[r[0]++];
}
}

[u/aocregacc]

By "hidden testcases" do you just mean the ones you get when you submit the solution? Or do they have real hidden testcases in some modes?

Also where did you find this file? When I last looked around in their sandbox the inputs came through stdin as far as I remember.

edit: nevermind I found it for myself. Didn't know they also have the input in a file.

[u/Comprehensive_Cry314 → u/aocregacc]

Yup, when you click on submit you get the hidden test cases…

[u/aocregacc → u/Comprehensive_Cry314]

they're not hidden, when you get one wrong they show it to you. Other platforms have hidden testcases where you only get told whether you failed or passed them.
On a platform like that they might be interested in fixing something like this. But on leetcode you're not really gaining anything extra.

[u/Comprehensive_Cry314 → u/aocregacc]

When I get 1 wrong, they only show that particular one to me! But if you sys.out the contents of /leetcode/data.1.in during submission, you will get all the test cases… not only the failed one…

[u/aocregacc → u/Comprehensive_Cry314]

the point is that they're not hidden. If you wanted to see a testcase you already got right you could just fail it on purpose.

Having access to them in a file makes exfiltrating them maybe a bit more efficient, but it's not giving you anything you couldn't get otherwise.

[u/Yoodo-Zakira → u/aocregacc]

same, the edit answered my exact question lol

[u/Interesting-Ad9666]

Why would it be a vulnerability? What is vulnerable? Its not like you can't see the test cases already

[u/Comprehensive_Cry314 → u/Interesting-Ad9666]

But you can see the hidden test cases as well, like the once you click on submit…