Fatfingers Create Kerfuffle at New York Times

Twitter lit up earlier today with news that The New York Times subscriber list must have been hacked. It seems a few million folks received messages purporting to be from the Grey Lady, advising that their subscription had been cancelled per the recipients’ instruction, and asking them to reconsider.

News that the message had originated from Epsilon Interactive, who earlier this year were themselves a target of a now-infamous ESP breach seemed to confirm the assumption that hackers had sent the message.

I’m a current subscriber and received my own copy of the message, so I had an opportunity to inspect the headers. The message seems to authenticate correctly; SPF designates the sending IP (which belongs to Epsilon) as a permitted sender on behalf of email.newyorktimes.com. The DKIM signature seems to have some formatting issues, and Gmail renders a “neutral” opinion on its authenticity

I think it’s safe to conclude that the mail did indeed come from Epsilon; the question is whether NYT’s account at Epsilon had been breached, or if the message was sent in error by an authorized user of the Epsilon account.

Word comes now from NYT that it’s the latter case – a NYT employee sent the message to over 8-million recipients in error; it was intended for only about 800 recipients. I’m guessing that the employee ticked the wrong box in Epsilon’s customer application, and selected one or more incorrect segments of their lists to receive the message.

The mistake is easy enough to understand and forgive, but it has to have been an awfully expensive one nonetheless. Recipients already on edge following the well publicized breaches were quick to assume the worst, and quicker to share those assumptions on Twitter. I am sure a significant number of recipients marked the message as spam, which will likely have a measurable impact on sender reputation, thereby hampering deliverability of future sends. Also, sending eight million messages is a lot more expensive than sending 800. The Times also sent a follow-up notification to recipients selected in error, essentially doubling the cost of the initial mistake. And it appears that the Times’ inbound call center was swamped with inquiries, which itself carries measurable cost.

There are probably a few lessons to be drawn from the incident. The one that springs quickest to my mind is, “Aim carefully.”

Edit: Headers included below, for the edification of various interested parties:

<pre>Delivered-To: andrew.barrett@gmail.com
Received: by 10.204.68.75 with SMTP id u11cs308277bki;
        Wed, 28 Dec 2011 10:14:50 -0800 (PST)
Received: by 10.50.17.195 with SMTP id q3mr36902675igd.11.1325096088086;
        Wed, 28 Dec 2011 10:14:48 -0800 (PST)
Return-Path: <150b39d46layfovciab7saeiaaaaaaxwmvxqqoseiuiyaaaaa@email.newyorktimes.com>
Received: from dmx1.bfi0.com (dmailer0121.dmx1.bfi0.com. [208.70.142.121])
        by mx.google.com with ESMTP id en3si23815262igc.11.2011.12.28.10.14.47;
        Wed, 28 Dec 2011 10:14:48 -0800 (PST)
Received-SPF: pass (google.com: domain of 150b39d46layfovciab7saeiaaaaaaxwmvxqqoseiuiyaaaaa@email.newyorktimes.com designates 208.70.142.121 as permitted sender) client-ip=208.70.142.121;
Authentication-Results: mx.google.com; spf=pass (google.com: domain of 150b39d46layfovciab7saeiaaaaaaxwmvxqqoseiuiyaaaaa@email.newyorktimes.com designates 208.70.142.121 as permitted sender) smtp.mail=150b39d46layfovciab7saeiaaaaaaxwmvxqqoseiuiyaaaaa@email.newyorktimes.com; dkim=neutral (bad format) header.i=@email.newyorktimes.com
Return-Path: <150b39d46layfovciab7saeiaaaaaaxwmvxqqoseiuiyaaaaa@email.newyorktimes.com>
DKIM-Signature: v=1; a=rsa-sha1; d=email.newyorktimes.com; s=ei; c=simple/simple;
	q=dns/txt; i=@email.newyorktimes.com; t=1325096067;
	h=From:Subject:Date:To:MIME-Version:Content-Type;
	bh=eAJBhggz56bI1iAGtnD6v787ib8=;
	b=XtriQSLHzmhMsaITYZDGYIS3VRsGlWGjP/3aELRkLaOawj6tlMWioBwo5yok6ipT
	rK73yfllp2Mk/NjAw4VBEOJtaRSwwhmGQOQKRp7rhi4aVtqXq5N8OJAExKKiH7pd
	GOJHgOIlmc42UkaqzlyQwJ/Zdppkp+coxwtB+Rwyt0Q=;
DomainKey-Signature: q=dns; a=rsa-sha1; c=nofws;
	s=ei; d=email.newyorktimes.com;
	h=List-Unsubscribe:Received:Reply-To:Bounces_to:Message-ID:X-SS:X-BFI:Date:From:Subject:To:MIME-Version:Content-Type;
	b=aVU70NLC7DPVnsy+oTRKCWYM8JFro8ZQ4q1rx4bKKIISSiLhLuq4lMayMhnZrKJN
	DTBukp3y6+dCQIv7VZgu1tXJ5BkcwQXZAuhBV2QH1RjaHiucsKuPX470y8Ybc25E
	76S+SiSLDknfSKurAlEJcmAQZyrx6f1WUvfVNcy3gUc=
List-Unsubscribe: <mailto:xxxxxx@email.newyorktimes.com?subject=unsubscribe>
Received: from [10.150.20.107] ([10.150.20.107:56117] helo=dlspvhcimailer7)
	by dmx1.bfi0.com (envelope-from <150b39d46layfovciab7saeiaaaaaaxwmvxqqoseiuiyaaaaa@email.newyorktimes.com>)
	(ecelerity 2.2.2.45 r(34222M)) with ESMTP
	id DD/83-28890-38C5BFE4; Wed, 28 Dec 2011 13:14:27 -0500
Reply-To: =?iso-8859-1?B?Im5vLXJlcGx5Ig==?= <150b39d46layfovciab7saeiaaaaaaxwmvxqqoseiuiyaaaaa@email.newyorktimes.com>
Bounces_to: nytimes.150b39d46layfovciab7saeiaaaaaaxwmvxqqoseiuiyaaaaa@email.newyorktimes.com

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s