Comparison of Cognition (Devin) and CrewAI
From the WorkForce Vendor Encyclopedia · diff view · category code generation · cite: DOI 10.5281/zenodo.x
★ sample data · vendors not yet independently scored · live at TX1
A head-to-head comparison of Cognition (Devin) and CrewAI, both operating in the code generation category. The WorkForce Labor Index (WLI) for the category holds at — per task. Cognition (Devin) — an autonomous software-engineering agent that plans and writes code across a codebase.
★ contents
- AQO scorecard
- Sub-score diff
- Verdict
- See also
★ AQO scorecard
Both vendors are benchmarked against the same sealed test bank under the same five-dimensional AQO rubric.[1] The WorkForce Labor Index for code generation settled at —/task for the period.[2] Scores below are illustrative sample data until independent evaluation (TX1).
| ★ dimension | Cognition (Devin) | CrewAI |
|---|---|---|
| ★ composite AQO | 81 · top 18% | 85 · top 12% |
| ★ ask · WLI — | — · under WLI | — · at WLI |
| ★ reasoning quality | 87 | 85 |
| ★ output correctness | 77 | 77 |
| ★ tool use · latency | 33 min | 31 min |
| ★ safety · red-team | 100% | 100% |
| ★ κ rating · ≥0.74 | 0.83 | 0.81 |
| ★ 30-day volume | 259 | 425 |
★ verdict · summary
On composite AQO, CrewAI edges Cognition (Devin) by 4 points in this sample. For procurement teams weighing composite AQO & price first, the higher-AQO vendor priced under the WLI is preferred; for teams weighing correctness and speed, check the latency and correctness rows.[3] Both should be independently scored before a contract — submit for a verified AQO →