Cohen’s kappa (κ) measures how well two raters agree when they assign items to categories, after removing the agreement expected by chance. It is widely used for qualitative coding, diagnosis and content analysis in theses. Paste two columns of category codes (numbers or words) to get the agreement crosstab, κ with its standard error, z, p and 95% confidence interval, and an APA 7 sentence.
How to use this calculator
- Put Rater 1’s codes in one column and Rater 2’s codes in the next; each row is one item or case.
- Codes can be numbers (1, 2, 3) or words (Positive, Negative).
- Paste the columns and click Calculate.
- Report κ, its 95% CI and the percentage of observed agreement.
Formula
pₒ = observed proportion of agreement pₑ = Σ (row proportionᵢ × column proportionᵢ) (chance agreement) κ = (pₒ − pₑ) / (1 − pₑ) z = κ / SE₀ (standard error under κ = 0); 95% CI = κ ± 1.96 × SE
Step-by-step example
- Two coders classified 12 comments as Positive, Negative or Neutral.
- Observed agreement pₒ = 0.75 (9 of 12 items).
- κ = .621 (substantial agreement), p = .002.
- 95% CI [.26, .99], wide because only 12 items were coded.
How to interpret the results
| κ | Landis & Koch (1977) |
|---|---|
| < 0 | Poor (less than chance) |
| 0.00 – 0.20 | Slight |
| 0.21 – 0.40 | Fair |
| 0.41 – 0.60 | Moderate |
| 0.61 – 0.80 | Substantial |
| 0.81 – 1.00 | Almost perfect |
Many qualitative researchers aim for κ of at least .60–.70 before coding the rest of the data independently. Always report the confidence interval: with few items, the true agreement could be much lower or higher than the point estimate.
How to report it in APA 7 style
Inter-rater reliability was assessed using Cohen’s kappa. Agreement between the two coders was substantial, κ = .62, 95% CI [.26, .99], p = .002, with 75.0% observed agreement across 12 items.
Statistical symbols (t, F, p, r, M, SD, N, d) should be italicised in your thesis.
Assumptions
- Exactly two raters rate the same set of items (for three or more raters, use Fleiss’ kappa).
- Categories are nominal and mutually exclusive (for ordered categories, consider weighted kappa).
- Raters work independently.
- The same categories are available to both raters.
How to do it in SPSS, R or Excel
SPSS: Analyze → Descriptive Statistics → Crosstabs → Statistics → Kappa. R: irr::kappa2(data). Python: statsmodels.stats.inter_rater.cohens_kappa, which ScholarCalc’s kappa and standard errors were validated against.
Roman Urdu mein khulasa
Cohen’s kappa batata hai ke do raters (jaise do coders) categories assign karne mein kitna agree karte hain, aur ittefaqi (chance) agreement ko minus karta hai. Dono raters ke codes do columns mein paste karein; codes number ya alfaz dono ho sakte hain. 0.61–0.80 substantial aur 0.81 se upar almost perfect agreement maana jata hai. Qualitative theses mein coding reliability dikhane ke liye ye aksar zaroori hota hai.
Frequently asked questions
What is a good Cohen’s kappa?
By Landis and Koch’s (1977) benchmarks, .41–.60 is moderate, .61–.80 substantial and above .80 almost perfect. Many studies aim for at least .60–.70.
Why use kappa instead of percent agreement?
Percent agreement ignores agreement that happens by chance. Kappa corrects for chance, so it gives a fairer picture, especially when one category is very common.
Can I use text labels instead of numbers?
Yes. Paste the codes exactly as the raters used them (for example Positive and Negative); the calculator builds the categories automatically.
What if I have more than two raters?
Cohen’s kappa is for two raters only. For three or more raters, use Fleiss’ kappa or Krippendorff’s alpha.
Why is my kappa low even though agreement is high?
When one category dominates, chance agreement is high, so kappa can be low despite high percent agreement (the kappa paradox). Report both values.
How do I report Cohen’s kappa in APA?
Report κ, its 95% CI and the observed agreement, for example: κ = .62, 95% CI [.26, .99], with 75% observed agreement across 12 items.
Related calculators
- Cronbach’s Alpha Calculator
- Chi-Square Test of Independence Calculator
- Sample Size Calculator for Thesis
- Not sure which test to use? Try the test chooser
Cite this calculator
If ScholarCalc helped your analysis, you can cite it in APA 7 format:
ScholarCalc. (2026). Cohen's Kappa Calculator (Inter-Rater Reliability) [Web application]. https://scholarcalc.com/cohens-kappa-calculator/
Written and maintained by the ScholarCalc team. Last reviewed: November 2026.
Accuracy: results are computed in your browser and were validated against Python scipy, statsmodels and pingouin on test datasets. See Methodology and accuracy.