Using a fully featured code editor, have candidates demonstrate their coding abilities with project-based tasks—whether to build APIs, query databases, or architect a full-stack application.
Create Project challenges
Navigate to your Custom library, then select Projects.
Upon clicking New project a modal will appear with two options:
-
Native IDE (recommended): Pre-installed packages and frameworks with built-in AI grading and cheating detection.
- VS Code IDE (advanced): Set up your project from scratch, or clone from GitHub, and install required libraries.
Each option allows you to give the project a title, difficulty level, a short description, and access to the IDE.
Native IDE
Once you have entered the simulated environment, there are various features that can be used and enabled to customize a project.
1. Languages and frameworks: Toggle to coding languages, front-end languages, back-end frameworks such as Jupyter Notebook, Django, Express, and more.
2. Filetree: Add and hide files from candidates or make them read-only. Then, when candidates take this challenge, the passing/failing logs can appear on the right side as feedback.
3. Add candidate instructions for the task.
4. Allow candidates to access a database.
5. Allow candidates to access ChatGPT and agent mode when working on the project.
6. Enable or disable AI Grading.
Back-end Frameworks & Jupyter Notebooks
Create challenges for back-end frameworks like Django, Rails, Spring, .NET, Jupyter Notebooks, and more. Select one of those options from the language dropdown, and the environment will initialize so that you can then edit it. Below is a screenshot of the Jupyter Notebook environment.
Grading Take-home Projects
Take-home projects can be either ungraded or graded,depending on whether AI grading is disabled or enabled.
Ungraded Projects
If AI grading is disabled, admins can manually assign a point value grade on a scale of 0-10 within the candidate report.
AI-Graded Projects
If AI grading is enabled, you'll be prompted to modify the grading criteria based on your expectations for the project. Candidates will be automatically assessed against your custom rubric for both best practices and technical coding abilities.
In the candidate's assessment report, an AI-generated analysis will be displayed along with the prompts submitted by the candidate throughout the project if ChatGPT is enabled. This provides additional insight into their problem-solving process and use of AI tools.
In the Challenge solutions section of the candidate's report, you can "Expand all" to view a detailed breakdown of the candidate's performance against each of your custom grading criteria.
These rubric scores are normalized to a 0-10 scale to generate the project score displayed in the report summary. The normalized project scores also contribute to the candidate's overall assessment score shown at the top of the report.
Loading Take-home Projects in Interviews
You can easily load your take-home projects and VS Code projects in live interviews. More info on this here.