Sign in Talk to AI Sales Start free

Email A/B testing

Email A/B testing is sending two or more versions of an email to random parts of the same list and comparing the results. In cold email the fair measure is the reply rate, because open counts include emails nobody read.

Updated October 2026 · 3 min read · GetSales team

How an email A/B test works

  1. Pick one change. The subject line, the first line, the ask or the wait before a follow-up. Change one thing per test, or you cannot tell which change made the difference.
  2. Split the list at random. Each person gets one version and keeps it for the whole sequence, so their follow-ups match the email they first got.
  3. Choose the measure first. Decide before sending what counts as a win: replies, positive replies or meetings booked.
  4. Test the gap. Compare the two reply rates with a test for two proportions: a z test on large groups, Fisher's exact test on small ones.1

What to test in a cold email

  • The first line. A line about their role against a line about something their company just did.
  • The ask. A call request against a question they can answer with yes or no.
  • The offer. Which problem you lead with.
  • The sequence. How many follow-ups, how far apart, and whether LinkedIn or email goes first.
  • The subject line. Judge it by replies too, since its usual measure, the open, is the least reliable one.

Why opens are a weak measure

Open tracking depends on remote content, usually a small image, loading when the email is opened. Apple Mail's Protect Mail Activity downloads remote content in the background by default, whether or not the person engages with the email, and keeps senders from learning when it was opened. So an open count includes emails nobody read.2 A reply is written by a person, which makes the reply rate the fairer score for a cold email test.

In a GetSales workflow an A/B test step splits people across 2 to 9 weighted arms, and each person keeps theirs. Each sending step shows what it sent, who replied and the rate.

Sources

  1. NIST/SEMATECH, e-Handbook of Statistical Methods, section 7.3.3: How can we determine whether two processes produce the same proportion of defectives?, undated. Read 28 October 2026.
  2. Apple, Mail Privacy Protection & Privacy, 12 December 2025. Read 28 October 2026.

Last updated: October 2026

CRM (customer relationship management)

A CRM (customer relationship management system) is the software a company uses to keep one shared record of its customers and prospects: who they are, every conversation with them and every deal, so the whole team works from the same history.

Updated

Email verification

Email verification is a check, before you send, that an email address exists and will take mail. A verifier finds the domain's mail server, asks it about the mailbox without sending anything, and marks the address valid, invalid, catch-all or unknown.

Updated

Cold email

A cold email is a first email to someone you have never been in touch with, sent to start a business conversation. It goes to one person's work address, says why you chose them, and asks only for a reply.

Updated