1 of 186 pain cards, shown in full — score, summary and a real quote from the thread it came from.
21
LLM developers can't trust LLM-as-judge scores due to self-disagreement and drift
We rely on LLM judges to evaluate our agents, but the judges disagree with themselves—scoring the same case 5/5, then 2/5, then 5/5 within minutes.
“An LLM judge disagrees with itself. I’ve had the same case, same everything, score 5/5, then 2/5, then 5/5 within fifteen minutes. So a single bad run is not evidence: before I look at execution I run the case a few times against the approved version and see how much it moves on its own.”
Integrations & API
12 posts
Pro
185 more pains are available in Pro
The blurred cards below are placeholders. No locked title, quote, score or id is sent to the browser.
🔒 Pro
52
Recurring workflow breakdown reported across multiple threads
Independent complaints cluster into one recurring pain with quotes from the threads they came from.
“We lose an afternoon every week to this and support keeps telling us it works as intended.”
WORKFLOW
4 posts
🔒 Pro
59
Users hit the same limit and describe identical workarounds
Several unrelated accounts describe the same failure in the same order of steps.
“I built a script to patch it, but it breaks whenever they change the field names.”
INTEGRATIONS
7 posts
🔒 Pro
66
Integration drops data during sync and nobody gets notified
The workaround costs hours a week and depends on a person remembering it.
“Nobody on the team can explain why it only fails for some of our records.”
PERMISSIONS
10 posts
🔒 Pro
73
Manual step repeats daily because the export omits one field
Independent complaints cluster into one recurring pain with quotes from the threads they came from.
“We lose an afternoon every week to this and support keeps telling us it works as intended.”
REPORTING
13 posts
🔒 Pro
80
Permission model blocks the person who actually does the work
Several unrelated accounts describe the same failure in the same order of steps.
“I built a script to patch it, but it breaks whenever they change the field names.”
BILLING
16 posts
🔒 Pro
53
Bulk operation silently skips rows past a hidden threshold
The workaround costs hours a week and depends on a person remembering it.
“Nobody on the team can explain why it only fails for some of our records.”
NOTIFICATIONS
19 posts
🔒 Pro
60
Recurring workflow breakdown reported across multiple threads
Independent complaints cluster into one recurring pain with quotes from the threads they came from.
“We lose an afternoon every week to this and support keeps telling us it works as intended.”
WORKFLOW
22 posts
🔒 Pro
67
Users hit the same limit and describe identical workarounds
Several unrelated accounts describe the same failure in the same order of steps.
“I built a script to patch it, but it breaks whenever they change the field names.”
INTEGRATIONS
25 posts
🔒 Pro
74
Integration drops data during sync and nobody gets notified
The workaround costs hours a week and depends on a person remembering it.
“Nobody on the team can explain why it only fails for some of our records.”
PERMISSIONS
28 posts
🔒 Pro
81
Manual step repeats daily because the export omits one field
Independent complaints cluster into one recurring pain with quotes from the threads they came from.
“We lose an afternoon every week to this and support keeps telling us it works as intended.”
REPORTING
31 posts
🔒 Pro
54
Permission model blocks the person who actually does the work
Several unrelated accounts describe the same failure in the same order of steps.
“I built a script to patch it, but it breaks whenever they change the field names.”
BILLING
34 posts
🔒 Pro
61
Bulk operation silently skips rows past a hidden threshold
The workaround costs hours a week and depends on a person remembering it.
“Nobody on the team can explain why it only fails for some of our records.”
NOTIFICATIONS
37 posts
🔒 Pro
68
Recurring workflow breakdown reported across multiple threads
Independent complaints cluster into one recurring pain with quotes from the threads they came from.
“We lose an afternoon every week to this and support keeps telling us it works as intended.”
WORKFLOW
40 posts
🔒 Pro
75
Users hit the same limit and describe identical workarounds
Several unrelated accounts describe the same failure in the same order of steps.
“I built a script to patch it, but it breaks whenever they change the field names.”
INTEGRATIONS
43 posts
🔒 Pro
82
Integration drops data during sync and nobody gets notified
The workaround costs hours a week and depends on a person remembering it.
“Nobody on the team can explain why it only fails for some of our records.”
PERMISSIONS
46 posts
🔒 Pro
55
Manual step repeats daily because the export omits one field
Independent complaints cluster into one recurring pain with quotes from the threads they came from.
“We lose an afternoon every week to this and support keeps telling us it works as intended.”
REPORTING
49 posts
🔒 Pro
62
Permission model blocks the person who actually does the work
Several unrelated accounts describe the same failure in the same order of steps.
“I built a script to patch it, but it breaks whenever they change the field names.”
BILLING
52 posts
🔒 Pro
69
Bulk operation silently skips rows past a hidden threshold
The workaround costs hours a week and depends on a person remembering it.
“Nobody on the team can explain why it only fails for some of our records.”
NOTIFICATIONS
55 posts
🔒 Pro
76
Recurring workflow breakdown reported across multiple threads
Independent complaints cluster into one recurring pain with quotes from the threads they came from.
“We lose an afternoon every week to this and support keeps telling us it works as intended.”
WORKFLOW
58 posts
🔒 Pro
83
Users hit the same limit and describe identical workarounds
Several unrelated accounts describe the same failure in the same order of steps.
“I built a script to patch it, but it breaks whenever they change the field names.”
INTEGRATIONS
61 posts
🔒 Pro
56
Integration drops data during sync and nobody gets notified
The workaround costs hours a week and depends on a person remembering it.
“Nobody on the team can explain why it only fails for some of our records.”
PERMISSIONS
4 posts
🔒 Pro
63
Manual step repeats daily because the export omits one field
Independent complaints cluster into one recurring pain with quotes from the threads they came from.
“We lose an afternoon every week to this and support keeps telling us it works as intended.”
REPORTING
7 posts
🔒 Pro
70
Permission model blocks the person who actually does the work
Several unrelated accounts describe the same failure in the same order of steps.
“I built a script to patch it, but it breaks whenever they change the field names.”
BILLING
10 posts
🔒 Pro
77
Bulk operation silently skips rows past a hidden threshold
The workaround costs hours a week and depends on a person remembering it.
“Nobody on the team can explain why it only fails for some of our records.”
NOTIFICATIONS
13 posts
Page 1 of 8 · 186 pains · 25 per page
OpenAI Platform & APIs is a Pro marketplace
Create a free account, then upgrade to Pro to unlock all 186 cards in OpenAI Platform & APIs.