Finance
SubjECTive-QA
SubjECTive-QA: measuring subjectivity in earnings-call-transcript QA. Models rate each (question, answer) pair on a 3-class scale (0/1/2) for each of six subjective features (Assertive, Cautious, Clear, Optimistic, Relevant, Specific). Scored as classification correctness against human gold labels.
3,300items
5subjects
CC-BY-4.0license
financedomain
nlp_taskdomain
textmodality
item-level responses released
Saturation status: Unknown
Response matrix
Fit to width. Hover for subject & item; click a cell for details.
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect
Subjects
- 1Mixtral-8x22B-Instruct-v0.10.5879
- 2Llama-3-8b-chat-hf0.5642
- 3Llama-3-70b-chat-hf0.5571
- 4Mixtral-8x7B-Instruct-v0.10.5038
- 5gpt-4o0.4924