fde-gym

What is fde-gym

fde-gym is an online evaluation platform for the LLM era. Each evaluation case comes from real client delivery scenarios. Users must use a coding Agent to complete the task, with final judgement given only after the system goes live.

Key features

• Each evaluation case comes from real client delivery scenarios
• Users complete tasks by guiding a coding Agent
• Provides a running system, documentation, history and personnel that can be consulted
• After the system goes live, final judgement is based on real traffic and a score of 0-100 is given
• One submission opportunity; points are deducted for consuming client personnel time

Who it is for

Suitable for developers, researchers and teams who need to train or evaluate LLM coding Agent capabilities. Used to simulate end-to-end problem solving in real client delivery scenarios.

How to get started

1. Visit https://github.com/deeplethe/fde-gym
2. Visit the website to view the list of cases
3. Select a case and read the materials provided by the client
4. Guide the coding Agent to make changes and submit for live evaluation

Compiled from public descriptions on the tool's official website / README / launch page. Pricing and user numbers have not been verified; please check the official website. Some sections were drafted with AI assistance. Written in Chinese and English; other languages are machine-translated.