Toptal created the JavaScript coding challenge web app to attract attendees to their conference booths. Its success led them to launch a pilot version on the web, accessible to their community and beyond.

From the outset, Toptal challenged developers to find innovative ways to achieve high scores in the JavaScript coding challenge. This emphasis on creative problem-solving within constraints aligns with the skills of successful independent freelancers.

JavaScript Coding Challenge Questions

The challenge presented a series of JavaScript questions, mirroring those encountered in interviews. These ranged in difficulty from basic:

1
2
3
4
5

box.double = function double (x) {
    //return x doubled
    
    
};

…to more intermediate:

1
2
3
4

box.dateRank = function dateRank (x) {
    // x is a date in 2019 as string (example: "06/30/2019")
    // return the rank of the day in 2019 (i.e., "09/01/2019" translates to 244)
};

Toptal aimed to engage both novice and experienced developers, encouraging friendly competition on a level playing field. The point system allowed even junior developers to participate, and the randomized order of equally-weighted questions provided a slightly different experience with each attempt.

Participants aimed to complete as many tasks as possible within a three-minute time limit. Completing all tasks before the time ran out yielded bonus points: 10 points for each remaining second. Multiple attempts were permitted.

Toptal anticipated that participants would initially take their time to solve the questions carefully, later copying and pasting the answers into the web application.

Just one hour and 40 minutes after launch, the first participant employed this strategy, achieving the maximum score of 1445 points, along with bonus points for the remaining time.

A Turning Point in the JavaScript Coding Challenge

With the copy-and-paste method, top contenders shifted their focus from coding to automation speed.

A simple approach involved writing JavaScript code to solve each task while continuously checking for button availability before pasting the solutions into the browser console:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

const solutions = {
  'double': 'return x*2',
  'numberToString': '...',
  'square': '...',
  'floatToInt': '...',
  'isEven': '...',
  'squareroot': '...',
  'removeFirstFive': '...',
  // ...
  'dateRank': '...',
  // ...
};
const get_submit_button = () => document.querySelector('.task-buttons > .col > .btn');
const solve = () => {
  const ace_editor = ace.edit(document.querySelector('.ace_editor'))
  const submission = ace_editor.getValue()
  for (const key in solutions) {
    if (submission.includes('box.' + key + ' ')) {
      ace_editor.insert(solutions[key])
      get_submit_button().click()
      setTimeout(() => {
        get_submit_button().click()
        setTimeout(() => {
          solve()
        }, 400)
      }, 900)
      return
    }
  }
}
solve()

This process could be further automated using tools like Selenium. However, a faster method involved automating API interaction to directly submit task solutions:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

const request = require('request');
const runTask = (data, entryId, callback) => {
  const tests = data.nextTask.tests_json;
  const results = Object.fromEntries(
    Object.entries(tests).map(([key, value]) => [key, value.result])
  );
  request.post(`https://speedcoding.toptal.com/webappApi/entry/${entryId}/attemptTask`, {
    form: {
      attempt_id: data.attemptId,
      tests_json: JSON.stringify(results),
    },
  }, (error, res, body) => {
    if (error) throw error;
    const next = JSON.parse(body).data
    if (next.isChallengeEntryFinished) {
      callback(next)
      return
    }
    runTask(next, entryId, callback)
  });
}
const runEntry = (callback) => {
  request.post('https://speedcoding.toptal.com/webappApi/entry', {
    form: {
      challengeSlug: 'toptal-speedcoding',
      email: ...,
      leaderboardName: ...,
      isConfirmedToBeContacted: ...,
      dateStop: ...
    },
  }, (error, res, body) => {
    if (error) throw error;
    const {
      data
    } = JSON.parse(body);
    const entryId = data.entry.id
    runTask(data, entryId, callback)
  });
}
runEntry(console.log)

Importantly, this version of the challenge only validated code on the client side. This loophole allowed participants to submit answers directly instead of the actual code, saving valuable milliseconds.

For three days, scores remained stagnant. Some participants focused on micro-optimizations, while others repeatedly submitted solutions, hoping for a less congested server to gain an advantage. However, a significant surprise was in store.

Breaking the JavaScript Coding Challenge’s Maximum Score

Let’s consider the scoring system: 1445 points for completing all tasks and 180 seconds with a 10-point bonus per remaining second. This means a theoretical maximum score of 3245 points for instant submission of all answers.

One user shattered this limit, achieving a score exceeding 6000 points, which continued to climb steadily.

How was this possible?

A closer look revealed the strategy. The top contestant, a seasoned full-stack developer and Toptaler with over 15 years of competitive programming experience, exploited a loophole. He deployed multiple bots to overload the server, enabling him to repeatedly complete the highest-scoring task and attribute the points to a single entry.

While not against the rules, which encouraged creativity, this approach significantly impacted server performance, affecting other users. Toptal responded by boosting server capacity, but this only pushed the score higher, from 56,000 to 70,000 points.

A Flourish bar chart race

Despite their commitment to open-ended participation, the impact on server stability and fairness for other users prompted Toptal to address the loophole.

The implemented fix prevented others from replicating the exploit on the final day. Consequently, Toptal expanded the prize pool, awarding AirPods to the top six contestants instead of just the winner.

Humble Beginnings and Ferocious Ends: Some JavaScript Coding Challenge Stats

Even top performers had modest beginnings. The highest first attempt score among those with five or more attempts was 645, with a median score of 25 points.

By the contest’s end, the number of attempts offered insights into the strategies employed. While efficiency varied, the top contestant stood out with a significantly higher attempt count.

Moving Forward

What lies ahead?

This was merely the first iteration of the JS coding challenge. Toptal plans to host more challenges, incorporating lessons learned and enhancing the experience. Future iterations will feature attempt throttling and expansion beyond JavaScript to encompass diverse programming languages.

While bolstering security measures, Toptal remains committed to fostering creativity, welcoming the use of bots and unconventional approaches in future challenges.

Special thanks to Pavel Vydra, Anton Andriievskyi, Tiago Chilanti, and Matei Copot for their contributions to the challenge and this article, and to @Zirak for the open-source project that formed the basis of the contest app. Likewise, thanks to everyone who participated in—and ran—the contest.