RERUN
Claim 2

Does the name on a CV change who gets a callback?

Independent rebuild

The study said CVs with white-sounding names got 50% more callbacks than similar CVs with Black-sounding names. The Lab will retrace it when forecasting closes.

Whose code is this?

Our own small program, run on a public copy of the experiment's data from OpenIntro. It isn't the authors' code. It got a ratio of 1.4968 against the paper's 1.50. The working paper's table has 2,445 CVs in each group and this data has 2,435, so the two percentages differ a little (9.65% and 6.45% here).

See the code

It's one of the best-known experiments on hiring, and it's still cited whenever people argue about anonymous CVs.

Cited by 5,025 studies

We test one thing: does this code and data give this number? That's not the same as the science being right. The forecast price never changes the verdict.

See the proof
1.50times the callbacks, the paper says. Counts if between 1.49 and 1.51.
Where the number comes fromAll sent resumes: 10.06%, 6.70%, ratio 1.50Table 1, first row, of the working paper (NBER 9873)

The exact claim as posted

Do CVs with white-sounding names get more callbacks? Bertrand and Mullainathan found 1.5 times as many

Bertrand and Mullainathan (American Economic Review, 2004) sent about 5,000 made-up CVs to real job ads in Boston and Chicago, giving them either white-sounding or Black-sounding names at random. CVs with white-sounding names got 1.50 times as many callbacks. This runs our own small program on a public copy of the experiment's data. The working paper's table has slightly more CVs than this data. The rebuild isn't the authors' code, so a mismatch ends as inconclusive.

Make a forecast
  1. People forecast
  2. Forecasting ends
  3. The Lab retraces it
  4. Verdict posted, open to challenge
  5. Winners collect

What we didn't check

  • Whether the sample was fair or large enough.
  • Whether test data leaked into training or tuning.
  • Whether the result holds on new data.
  • Whether the study's method was the right one for its question.
  • Whether the same gap exists today, or outside Boston and Chicago in 2001 and 2002.
  • Whether the names signal only race, and not also class.

Reading how the odds moved…

Watch the Lab work

Open for forecasts
Connecting to the Lab0 entries

The Lab starts when this market locks, in 2 days. Stay here to watch it live.

What the Lab retraces

Source
https://doi.org/10.1257/0002828042002561
M. Bertrand, S. Mullainathan; rebuild by the RERUN project, Rebuild: MIT. Data: GPL-3 (OpenIntro), fetched from OpenIntro, not redistributed.
Code, at this exact version
https://github.com/Maveri-Q/rerun-rebuilds
commit 175eddea88cc8adab8dd20f2100b5b453d359244
Commands
python3.11
$ python names-on-cvs-2004/rebuild.py resume.csv > rerun_output.txt && cat rerun_output.txt
Where the result is read
rerun_output.txt using regex callback_ratio (-?[0-9.]+)
Data files
resume.csv
fingerprint 3085bba2a81ac05dde852e697fdc35aa7f4333e0b9a28f535ab6353c7af5d438
Limits
2 CPU, 4 GB, 15 minutes
2a325a5acb04febe5efd5b805632cf8c44cab27f04b1a5cad7497defedaaf8cf
Show the claim exactly as stored on the blockchain
{"artifact":{"commit":"175eddea88cc8adab8dd20f2100b5b453d359244","data":[{"path":"resume.csv","sha256":"3085bba2a81ac05dde852e697fdc35aa7f4333e0b9a28f535ab6353c7af5d438","url":"https://www.openintro.org/data/csv/resume.csv"}],"entrypoint":"python names-on-cvs-2004/rebuild.py resume.csv > rerun_output.txt && cat rerun_output.txt","output":{"extractor":{"expr":"callback_ratio (-?[0-9.]+)","type":"regex"},"path":"rerun_output.txt"},"repo_url":"https://github.com/Maveri-Q/rerun-rebuilds","runtime":"python3.11","setup":[]},"metric":{"comparison":"within_tolerance","name":"callbacks for white-sounding names per callback for Black-sounding names","reported_value":"1.50","tolerance":{"type":"abs","value":"0.01"}},"private_data":null,"resources":{"cpu":2,"max_minutes":15,"mem_gb":4},"source":{"authors":"M. Bertrand, S. Mullainathan; rebuild by the RERUN project","doi":"10.1257/0002828042002561","license":"Rebuild: MIT. Data: GPL-3 (OpenIntro), fetched from OpenIntro, not redistributed.","type":"paper","url":"https://doi.org/10.1257/0002828042002561"},"statement":"Bertrand and Mullainathan (American Economic Review, 2004) sent about 5,000 made-up CVs to real job ads in Boston and Chicago, giving them either white-sounding or Black-sounding names at random. CVs with white-sounding names got 1.50 times as many callbacks. This runs our own small program on a public copy of the experiment's data. The working paper's table has slightly more CVs than this data. The rebuild isn't the authors' code, so a mismatch ends as inconclusive.","title":"Do CVs with white-sounding names get more callbacks? Bertrand and Mullainathan found 1.5 times as many","version":1}

Community notes

Notes are people's own words, kept on the blockchain. They never change a verdict, and the Lab's AI never reads them. Posting costs a little NEAR for storage, and what isn't used comes back.