Perpetual IT

Well-known US cybercrime journalist Brian Krebs recently published a warning about vishing attacks against business users.

The FBI promptly followed up on Krebs’s article with a warning of its own, dramatically entitled Cyber criminals take advantage of increased telework through vishing campaign.

So, what is vishing?

And how does it differ from phishing, something that most of us see far to much of?

The V in vishing stands for voice, and it’s a way of referring to scams that arrive by telephone in the form of voice calls, rather than as electronic messages.

Of course, many of us use voicemail systems that automatically answer and record messages when we aren’t able or willing to take a call in person, and many modern voicemail systems can be programmed to package up their recordings and deliver them as email attachments or as web links.

So the boundary between voice calls and electronic messages is rather blurred these days.

Nevertheless, many of still routinely pick up calls in person when we can – especially those of us who run a business, or who have family members we’re supporting through coronavirus lockdown or who aren’t well and might need urgent help.

We know several people who keep a landline especially as a contact point for family and friends.

They give out their landline number sparingly on what you might call a “need-to-know” basis, and use their mobile number – which is comparatively easy to change if needed, and easy to monitor and filter using a suitable app – for day-to-day purposes where giving out a working number can’t easily be avoided.

As you can imagine, however, the crooks only need to uncover your phone number once, perhaps via a data breach, and they can call it forever, especially if it’s a landline that you’re keeping because people who are important to you know it and rely on it.

Semi-targeted phone attacks

The crooks don’t even need to know any details behind your number to abuse it, in the same way that they don’t need to know your full name, where you live or what you do for a living in order to spam and scam you by email.

Obviously, the more an attacker knows about you, the more they can tailor their scams – or target them, in the military jargon that’s become trendy in the cybersecurity field.

Even being able to say “Hello Your Real Name” instead of “Dear Customer” makes a message more believable, and including personal information can make a spam or scam more convincing still.

That’s why porn scammers, also known as sextortionists, who email to demand money for “suppressing” a prurient video of you (one that they don’t have because it doesn’t exist), include personal data in the message, such as your phone number or an old password.

They do this as a way of “proving” that they really did hack your computer, even though they almost certainly acquired the data from an ancient data breach.

Vishing scams, however, just like smishing scams (phishing via SMS), can sound realistic even if the crooks can do no better than guess at your online life.

Unlike emails, SMSes and voice messages – especially automated ones that use a synthetic voice and don’t need to be interactive – can get away with being stripped to the basics.

SMSes are limited to 160 characters, while voice messages are limited by the fact that about 30 seconds is the longest that people are likely to listen with any sort of attention to a recorded warning – and that is enough time for just 60 words dictated with any clarity.

And by picking a popular and widely-used service as the theme of the scam – such as a well-known global home delivery brand, or email provider, or payment processor, the crooks have a good chance of guessing correctly for a significant minoirity, perhaps even an absolute majority, of recipients.

Vishing at home

60 words or so turns out to be more than enough to create a believable bait, especially when it’s a voice message that lacks the permanence of an email or an SMS.

And, in the UK at least, there seems to have been a recent surge in home delivery vishing campaigns.

We can’t tell whether this is just one group of crooks who are focusing on both vishing and the UK at the moment, or if it’s a broader global trend, but we (and people we know in the UK) are experiencing unwanted vishing calls at a much greater rate than any time in the past few years.

We’re not talking about interactive scams here, like those fake technical support calls where a crook with the gift of the gab call up out of the blue to pester, lie, cheats and frighten you about made-up malware on your computer in order to talk you into buying a fraudulent “cleanup service” that you didn’t need in the first place.

This new wave of calls are automated, using voice synthesis to “speak” with diction and an accent that is nearly, but not quite, as good as Siri, and they seem to follow a shorter and much crisper script than similar scams we’re aware of from he past couple of years.

Most older recordings we’ve heard have English text with poor wording and grammar that was either synthetically generated by poor-quality voice software or dictated by someone reading inexpertly from a printed script.

But this latest batch sounds much more believable, following scripts roughly along these lines (we don’t have recordings, so these are paraphrased from various Naked Security readers’ memory):

Your Amazon order for [several hundred pounds ending in -99] has now been processed. Your [phone product] will soon be dispatched and you should receive it in [a small number] of days. For further information or to cancel the order, press 1 now to speak to an operator.

Your Amazon Prime subscription will auto-renew. Your card will be billed for [several tens of pounds ending in -.99]. To cancel your subscription or to discuss this renewal, press 1 now.

One of our readers pressed 1 to see what would happen (we don’t recommend doing this, simply because the only thing you can be certain of is that you will be talking to an out-and-out criminal who knows your phone number and perhaps even where you live).

As you can probably imagine, the reader ended up talking to a real human in what sounded like a boiler-room call centre, just as you would if you were called directly by one of those technical support scammers claiming to be from Microsoft or your internet provider.

Why it works

The sad things about this sort of scam are:

• The crooks use internet telephony (VoiP), so they pay close to zero for the calls.
• The calls emerge into the landline or mobile network inside your country, so they often show up with a believable local number.
• Synthetic voice calls are widely used by legitimate businesses these days, so they are no longer a telltale sign that the call is suspicious.
• The call centre crooks only ever deal with “already active” callers who have pressed 1, making their scamming process more efficient.
• The calls are hard to avoid, especially if they arrive on a line that you keep primarily for family emergencies.
• The incoming call numbers change all the time, so that adding them to your phone’s blocklist, if it has one, doesn’t help much.
• Reporting them feels like a waste of time, because the callers are almost certainly outside the jurisdiction of your own telecommunications regulator.

What to do?

Unfortunately, this is one of those cybercrimes for which we don’t have a good set of “this will fix the problem” answers.

Some people find that running all their calls through voicemail acts as a filter and stops the calls being intrusive, but if it’s a landline you rely on for the timely report of family emergencies then you still need to let the phone ring aloud to alert you to the call, and you may not know what incoming numbers to expect anyway.

(If your emergencies include possible calls from healthcare workers or hospitals, you will often find that those people and organisations withold their numbers to cut down on nuisance replies or to protect the privacy of the workers involved.)

Reporting unwanted phone calls can be somewhere between impossible, if the number is witheld and very hard, depending on your country.

For example, in the UK there is – rather annoyingly – a different procedure for reporting scam calls, which is where someone calls you up and talks a load of lies or unwanted junk into your ear, and abandoned or silent calls (“hangups”), which is where the caller cuts the connection before a human comes on the line at their end.

Calls where the other end doesn’t say a word, either through an unnerving silence or by using an automated voice only, are understandably considered creepier and therefore criminally more serious than viva voce, in-your-ear dishonesty, and are therefore regulated differently.

In the former case, in our experience trying to report rogue callers in the UK in the past, you can make your report anonymously; in the latter, the process is more complicated and you have to say who you are, presumably because scam calls are a regulatory issue but abandoned and silent calls may be a criminal offence.

So, if you can recover the caller’s number and are willing to report it, we encourage you do to so.

But we accept that this may be too much effort, or require too much personal involvement, for some people in some countries, so we’re not going any further than encouragement here.

All we can advise as a matter of routine is the rythmic and easily-rememered ditty that the Australian cybersecurity industry came up with many years ago as a way of thinking about how you deal with spammers and online charlatans: Don’t try. Don’t buy. Don’t reply.

Don’t let yourself get sucked, surprised or seduced into taking any direct action – not even if you think it might be amusing to see who’s at the other end – after all, you’re talking to a crook, so the best thing that can happen to you is nothing.

If you are worried about a fraudulent transaction, whether it’s via Amazon or any other coronavirus-friendly online merchant, login to your account yourself, or call the company’s helpine yourself, using contact information you already have.

Never rely on information provided inside an email, or read out to you in a call, as a way of deciding whether to believe the email or the call.

After all, if the call or email is true, the reply you will receive will be truthful and will say, “It’s true.”

But if the call or email is false, the reply you will receive will be a lie, and will also say, “It’s true”!

---

Here’s a phishing email we received recently that ticks all the cybercriminal trick-to-click boxes.

From BEC, through cloud storage to an innocent-sounding One Note document, right into harm’s way.

Instead of simply spamming out a clickable link to as many people as possible, the crooks used more labyrinthine techniques, presumably in the hope of avoiding being just one more “unexpected email that goes directly to an unlikely login page” scam.

Ironically, while mainstream websites concentrate on what they call frictionlessness, aiming to get you from A to B as clicklessly as possible, some cybercrooks deliberately add extra complexity into their phishing campaigns.

The idea is to require a few extra steps, taking you on a more roundabout journey before you arrive at a website that demands your password, so that you don’t leap directly and suspiciously from an email link to a login page.

Here’s the phish unravelled so you can see how it works.

Stages of attack

First, we received an innocent looking email:

This one actually came from where it claimed – the proprietor of a perfectly legitimate UK engineering business, whose email account had evidently been hacked.

We didn’t know the sender personally, but we’re guessing he was a Naked Security reader and had corresponded with us in the past, so we appeared in his address book along with hundreds of other people.

We assume that many of the recipients corresponded with the sender regularly and would not only be inclined to trust his messages but also to expect attachments relating to business and projects they’d been discussing.

Taking over someone else’s email account for criminal purposes is often referred to as BEC, short for business email compromise, and it’s often assoicated with so-called CEO or CFO fraud, where the crooks deliberately target the CEO’s or the CFO’s account so they can issue fake payment instructions, apparently from the most senior level.

In this case, however, the crooks had clearly set out use one compromised account as a starting point to compromise as many more as they could, presumably intending either to use the new passwords for their own next wave of BEC crimes, or to sell them on for someone else to abuse.

Opening the attachment takes you to a One Drive file that looks legitimate enough at first sight, especially for recipients who communicate regularly with the sender:

The Sharepoint link you’re expected to click to access the One Note file does look suspicious because there’s no clear connection between the sender’s company and the location of the One Note lure.

But the sender’s business relates to construction, and the domain name in the Sharepoint link apparently refers to a building company, so the link is at plausible, at least.

The One Note file itself is very simple:

It’s only at this stage that the crooks present their call-to-action link – the click that they didn’t want to put directly ino the original email, where it would have stood out more obviously as a phishing scam.

You’d be forgiven for assuming that the Review Document button here simply opens up or jumps to a part of the One Note file that you’ve already got open…

…but, of course, there is no New Project PDF file, and the “link” that’s apparently there for you to review the document just takes you to the bogus login page that the criminals have been luring you towards all along.

The fake login page is hidden away (or was – the site is offline now [2020-09-02T14:00Z]) on a hacked WordPress site belonging to an events company.

Fortunately, the crooks gave themselves away doubly at this point.

Firstly, they got the name of the sender’s company wrong in this part of the scam (that’s the text redacted just before the word “Ltd”, which is the UK abbreviation for a limited liability company).

The sender’s company name ends in the word Structural, given that he’s in the construction business, but the criminals blundered and typed in the word Surgical – a small but obvious red flag to anyone who does business with the sender.

Secondly, the hacked events company where the crooks hid their phishing pages is in based Kyiv in Ukraine, and has a domain name that is neither related to the construction industry nor located in the UK, where the original email came from. (We redacted the site name in the image below.)

If you do click through, despite the unexpected link and the unlikely domain name, then you’ll finally reach a login form, three steps removed from the original email, complete with animated imagery suggestive of Office 365:

The login is apparently necessary in order to access what is supposed to be an Excel file.

However, the unexplained switch to Excel jars with the previous page, where you were promised a PDF file, and you will notice that the criminals have written Microsoft, Excel and Small Business incorrectly.

You also ought to be suspicious at a Microsoft login page that offers you so many alternative authentication choices.

That’s something smaller websites do in order to capitalise on the fact that you probably already have accounts with the big players, but you wouldn’t expect Microsoft to use any of its competitors as an authentication service.

Of course, if you do put in a password, it goes straight to the crooks, who then present you with a fake error message, perhaps in the hope you might try another account and give them a second password.

What to do?

• Don’t click login links that you reach from an email. That’s an extension to our usual advice never to click login links that appear directly in emails. Don’t let the crooks distract you by leading you away from your email client first to make their phishing page feel more believable. If you started from an email, stop if you hit a password demand. Find your own way to the site or service you’re supposed to use.
• Keep your eyes open for obvious giveaways. As we’ve said many times before, the only thing worse that being scammed is being scammed and then realising that the signs were there all along. Crooks don’t always make obvious mistakes, but if they do, make sure you don’t miss them.
• If you think you put in a password where you shouldn’t, change it as soon as you can. Find your own way to the official site of the service concerned, and login directly. The sooner you fix your mistake, the less chance the crooks have of getting there first.
• Use 2FA whenever you can. Accounts that are protected by two-factor authentication are harder for crooks to take over, because they can’t just harvest your password and use it on its own later. They need to trick you into revealing your 2FA code at the very moment that they’re phishing you.
• Consider phishing simulators like Sophos Phish Threat. If you are part of the IT security team, Phish Threat gives you a safe way to expose your staff to phishing-like attacks, so they can learn their lessons when it’s you at the other end, not the crooks.

---

Thanks to Michelle Farenci of the Sophos Security Team for her behind-the-scenes work on this article.

Here’s a phish that our own security team received themselves.

Apart from some slightly clumsy wording (but when was the last time you received an email about a technical matter that was plainly written in perfect English?) and a tiny error of grammar, we thought it was surprisingly believable and worth writing up on that account, to remind you how modern phishers are presenting themselves.

Out are the implied threats, the exclamation points (!!!) and the money ($$$) you might lose if you don’t act right now; in are the happy and unexceptionable “here’s a problem that you can fix all by yourself without waiting for IT to help you” messages of a sort that many companies are using these days to reduce support queuing times.

Yes, you ought to be suspicious of emails like this. No, you shouldn’t click through even out of interest. No, should never enter your email password in circumstances like this.

But the low-key style of this particular scam caught our eye, making it the sort of message that even a well-informed user might fall for, especially at the end of a busy day, or at the very start of the day after.

Here’s how it arrives – note that in the sample we examined here, the crooks had rigged up the email content so that it seemed to be an automated message from the recipient’s own account, which fits with the theme of an automatic delivery error:

I​n​c​o​m​i​n​g​ ​m​e​s​s​a​g​e​s​ ​f​o​r​ [REDACTED] c​o​u​l​d​n​’​t​ ​b​e d​e​l​i​v​e​r​e​d​.

This message was sent in response to multiple incoming messages being rejected consistently from 2:00 AM, Wednesday, August 19, 2020.

To fix, recover and prevent further rejection of emails by our server, connect to your Company-Assigned OWA portal securely below.

Only if you were to dig into the email headers would it be obvious that this message actually arrived from outside and was not generated automatically by your own email system at all.

The clickable link is perfectly believable, because the part we’ve redacted above (between the text https://portal and the trailing /owa, short for Outlook Web App) will be your company’s domain name.

But even though the blue text of the link itself looks like a URL, it isn’t actually the URL that you will visit if you click it.

Remember that a link in a web page consists of two parts: first, the text that is highlighted, usually in blue, and that is clickable; second, the destination, or HREF (short for hypertext reference), where you actually go if you click the blue text.

A link is denoted in HTML by an ANCHOR tag that appears between the markers <A> and </A> while the destination web address is denoted by an HREF attribute inside the opening anchor tag delimiter.

Like this:

This is a <A HREF='https://example.com'>clickable link</A> going to EXAMPLE.COM But the link <A HREF='https://example.com'>https://different.example</A> also
goes to EXAMPLE.COM, because the URL used is determined by the HREF setting, even if the text of the link itself looks like a URL. The domain DIFFERENT.EXAMPLE
here isn't actually a web address, it's just text that looks like a web address.

Why not just block links that look like other links?

If you’re thinking that “links that deliberately look as though they go somewhere else” sound suspicious, you’d be right.

You might wonder why browsers, operating systems and cybersecurity products don’t automatically detect and block this kind of trick, where there’s an obvious and deliberate mismatch between the clickable text and the link it takes you to.

Unfortunately, even mainstream sites use this approach, making it effectively impossible to rely up front on what a link looks like, or even where it claims to go in your browser, in order to work out exactly where your network traffic will go next.

For instance, here’s a Google search for here's an example:

You can see that if you ① search for here's an example, you’ll receive a answer in which ② an explicit domain name (here, english.stackexchange.com) is used as the visible text of a clickable link.

You can also see that when you hover over the domain name link, you’ll see ③ a full URL that apparently confirms that clicking the link will take you to the named site.

However, if you use Firefox’s Copy Link Location option to recover the ultimate link, you’ll see – thanks to the magic of JavaScript – that your web request actually goes to a URL of this sort:

https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=& cad=rja&uact=8&ved=[REDACTED]& url=https%3A%2F%2Fenglish.stackexchange.com%2Fquestions%2F225855%2Fheres-an-example[...]

Eventually, you will end up at the URL shown at position ③ in the screenshot above, but you’ll be redirected (quickly enough that you nmight not notice) via a Google track-and-redirect link first.

So you do end up where the browser told you, but not quite as explicitly and directly as you might have expected – you get there indirectly via Google’s own advertising network.

What happens next?

The good news is that in the case of this phish you will see the actual web page you’ll be taken to if you hover your cursor over the link-that-looks-like-a-different-link.

That’s because email clients and webmail systems generally don’t allow JavaScript to run, given that emails could have come from anywhere – even if they say they came from your own account, as this one does.

So you ought to spot this phish easily if you stop to check where the link-that-looks-internal really ends up.

In our case (note that the exact URL and server name may vary every time), the real link did not go to https://portal.[REDACTED]/owa, as suggested by the text of the link.

Instead, it went to a temporary Microsoft Azure cloud web storage URL, as shown below, which clearly isn’t the innocent-looking URL implied in the email:

[REDACTED].web.core.windows.net

A quick check of the domain name via the Sophos Intelix online threat detection service shows its true colours:

$ luax intelix-lookup.lua [REDACTED].web.core.windows.net Authenticating to Sophos Intelix: OK.
Items to check: 1 { productivityCategory "PROD_SPYWARE_AND_MALWARE" riskLevel "HIGH" securityCategory "SEC_MALWARE_REPOSITORY" ttl = 300
}

This server has nothing to do with your company’s email, and everything to do with putting you in harm’s way.

The phishing page

If you do click through, and your endpoint or firewall filter doesn’t block the request, you will see a phishing page that we must grudgingly admit is elegantly simple:

Your email address is embedded in the link in the email that you click on, so the phishing page can fill in the email field as you would probably expect.

When we tried this page, deliberately putting in fake data, we received an error message after the first attempt, as though we’d made a mistake typing in the password:

No matter what we did the second time, we achieved “success”, and moved onwards in the scam.

How it ends

One tricky problem for phishing crooks is what to do at the end, so you don’t belatedly realise it’s a scam and rush off to change your password (or cancel your credit card, or whatever it might be).

In theory, they could try using the credentials you just typed in to login for you and then dump you into your real account, but there’s a lot that could go wrong.

The crooks almost certainly will test out your newly-phished password pretty soon, but probably not right away while you are paying attention and might spot any anomalies that their attempted login might cause.

They could just put up a “thanks, you may now continue normally” page, and often that’s exactly what they do as a simple way to sign off their scam.

Or they find a page that’s related to the account they were phishing for, and redirect you there.

Thi leaves you on a web page that really does have a genuine URL in the address bar – what’s often called a decoy page because it leads you out at the end of the scam with your innocence intact.

That’s what happened here – it’s not perhaps exactly the page you might expect, but it’s believable enough because it leaves you on a genuine Outlook-related web page with a genuine Microsoft URL:

What to do?

• Always verify links in emails before you click them. You should check where you you end up after clicking (see the next tip), but don’t click through casually and think, “I’ll wait to check further down the line to see if things look bad.” Check before you click as well. The earlier you spot a phishing scam, the less likely it is you’ll be sucked in and the earlier you’ll be able to report it.
• Carefully check the URL of any login page. These days, most cybercriminals are using HTTPS websites, because everyone expects a padlock in the address bar. But the padlock doesn’t say you are on the correct site, merely that you are on a site with an HTTPS certificate. If you’re currently using your mobile phone, consider switching to your laptop if you can, and checking out the link from there. It’s worth the extra trouble because the address bar is easier to read and tells you more.
• Avoid logging in at all via links you received in an email. If it’s a service you already know how to use – whether it’s your email, your banking site, your blog pages or a social media account – learn how to reach the login page directly, and how to access the account’s status pages after you’re in. If you always find your own way to your account login pages and ignore email login links even if you think they are genuine, you’ll never fall for fake links by mistake.
• Turn on 2FA if you can. Two-factor authentication means that you need a one-time login code, usually texted to your phone or generated by a special app, that changes every time. 2FA doesn’t guarantee to keep the crooks out but it makes your password alone much less use to them.
• Never turn off or change security settings because an email tells you to. Many phishing emails include instructions that claim to help you improve your security, but the changes they demand are there to make you less secure and help the crooks to get further. If in doubt, leave it out!
• Change passwords at once if you think you just got phished. The sooner you change your current password after putting it into a site you subsequently suspect, the less time the crooks have to try it out. Similarly, if you get as far as a “pay page” where you enter payment card data and then realise it’s a scam, call your bank’s fraud reporting number at once. (Look on the back of your actual card so you get the right phone number.)

Two more suggestions…

If you’re a sysadmin looking to keep phishing attacks out, why not take a look at:

• Sophos Phish Threat. This is a phishing simulator that lets you test out your staff in a sympathetic way, using realistic but artificial scams, so your users can make their mistakes when it’s you at the other end, rather than when it’s a cybercriminal.
• Sophos Intelix. This is a live threat lookup service that you can use in your own system software and scripts to add high-speed threat detection for suspicious websites, URLs and files. A simple HTTPS-based web API that replies in JSON means you can use Sophos Intelix from just about any programming or scripting language you like. (Registration is free and you get a generous level of free submissions each month, after which you can pay-as-you-go if you want to do high volumes of queries.)

---

Twitter apologized on Tuesday for sticking business clients’ billing information into browser cache – a spot where the uninvited could have had a peek, regardless of not having the right to see it.

In an email to its clients, Twitter said it was “possible” that others could have accessed the sensitive information, which included email addresses, phone numbers and the last four digits of clients’ credit card numbers. Any and all of that data could leave businesses vulnerable to phishing campaigns and business email compromise (BEC) – a crime that the FBI says is getting pulled off by increasingly sophisticated operators who’ve grown fond of vacuuming out payrolls.

Mind you, Twitter hasn’t come across evidence that billing information was, in fact, compromised.

On 20 May, Twitter updated the instructions that Twitter sends to browser cache, thereby putting a stopper in the leak. The two affected platforms are ads.twitter.com or analytics.twitter.co. If you viewed your billing information on either platform before 20 May, your billing information may have gotten stuck in browser cache.

Browser-sharers take heed

Twitter said that if you used a shared computer during that time, someone who used the computer after you may have seen the billing information stored in the browser’s cache. The company notes that most browsers generally store data in their cache by default for a short period of time – say, 30 days.

What to do?

Twitter recommends that those who use a shared computer to access Twitter Ads or Analytics billing information should clear the browser cache when they log out.

Twitter’s mea culpa

Whoops, Twitter said:

We’re very sorry this happened. We recognize and appreciate the trust you place in us, and are committed to earning that trust every day.

The company didn’t say how many accounts were affected.

If you’ve got questions, Twitter says you can write to its Office of Data Protection, here.

Not the first flub

This isn’t the first time that Twitter’s stumbled with account security.

In May 2018, we got a warning from Twitter admitting that the company had made a serious security blunder: it had been storing unencrypted copies of passwords. That’s right: plaintext passwords, saved to disk.

You’re reading Naked Security, so there’s a good chance you already know that plaintext passwords are an acutely bad idea.

A few years prior to that, in June 2016, Twitter locked out some users after nearly 33 million logins went up for sale. The thievery was credited to a well-known hacker and dark-web seller: a Russian actor known by the handle Tessa88. Twitter said at the time that its systems hadn’t been breached and that the logins may have come from other password leaks.

That’s a whole lot of leaked passwords and about 33 million reasons to repeat the “use a unique, strong password” mantra. Need a real bruiser of a password? Here’s how to pick a strong password.

Ixnay on the password reuse, too, of course. That’s where a password manager comes in handy.

Do all that to protect your credentials, wipe browser cache if you’re potentially affected by this browser cache storage glitch, and stay safe!

Who doesn’t know their mother’s maiden name?!

A bot that’s trying to convince you it’s human but which hasn’t been programmed to answer that question or improvise very convincingly, that’s who. Or, as I said when I finished playing a new online Turing Test game called Bot or Not, NAILED IT!!

Bot or Not is an online game that pits people against either bots or humans. It’s up to players to figure out which they’re engaging with in the 3-minute game, in which they’re forced to question not only whether their opponent is human but exactly how human they themselves are.

The creators of Bot or Not – a Mozilla Creative Awards project that was conceived, designed, developed and written by the New York City-based design and research studio Foreign Objects – say that these days, bots are growing increasingly sophisticated and are proliferating both online and offline. It’s getting tougher to tell who’s human, which can come in handy in customer service situations but is a bit scary when you think about scam bots preying on us on Tinder and Instagram, or corporate bots that try to steal your data.

The friendly face of pervasive surveillance

In their explanation of Bot or Not’s purpose, the game’s creators point to a recent Gartner industry report that predicted that by 2020, the average person will engage in more conversations with bots than with their spouses.

Think about it: how often do you talk to voice assistants like Siri or OK Google? Chatbots have become seamlessly integrated into our lives, presenting what Foreign Objects calls “a massive risk to privacy” and will remain so for as long as collecting personal data remains the primary business model for major tech platforms.

Big tech knows that in order to get the most data out of our daily lives, they need us to invite bots into our homes, and to enjoy ourselves while we do so.

One example: smart speakers, those always-listening devices that are constantly surveilling our homes. As we’ve reported in the past, smart speakers mistakenly eavesdrop up to 19 times a day. They record conversations when they hear their trigger words… or by something that more or less sounds like one of their trigger words. Or by a burger advertisement. Or, say, by a little girl with a hankering for cookies and a dollhouse.

Last year, smart-speaker makers found themselves embroiled in backlash over privacy after news that smart speakers from both Apple and Google were capturing voice recordings that the companies were then letting their employees and contractors listen to and analyze. Both companies suspended their contractors’ access.

What does Bot or Not have to do with all that? Foreign Objects says that while government regulation is struggling to keep up with new technologies, there’s little public awareness or legal resistance to stop companies from developing a global surveillance network on an unprecedented scale – something that’s already been done on a massive scale with the plethora of devices with smart assistants.

Governments are not only lagging behind on policy, they are also part of the problem.

This is about more than these devices listening in on our private moments. It’s about big-tech corporations willingly handing over citizens’ private data to police without consent, Foreign Objects says.

As chatbots slide seamlessly into our personal and domestic lives, it has never been more important to demand transparency from companies and policy initiative from regulators.

Smart speakers running on artificial intelligence (AI) are one thing. Chatbots, however, are taking data interception to a whole new level, say the creators of Bot or Not:

In the hands of big platforms, chatbots with realistically human-like voices are openly manipulative attempts to gather our data and influence our behaviours.

They point to advanced “duplex” chatbots released in the last few years by Microsoft and Google, so-called because they can speak and listen at the same time, mimicking the experience of human conversation. If you’re wondering how that might feel, you can look to Google’s Duplex neural network AI, introduced last year and designed to sound and respond like a human being, down to all the “umms” and “aahs.”

It was too real. Google faced a backlash over its failure to disclose that the person on the other end of the line – a supposedly human hairdresser taking a customer booking was one such – was actually a bot.

Sociologist of technology Zeynep Tufecki’s response at the time:

[The lack of disclosure is] horrifying. Silicon Valley is ethically lost, rudderless and has not learned a thing.

Deception: “It’s a feature, not a bug”

Google later added a disclosure feature to Duplex’s interactions, but Bot or Not’s creators aren’t sure that a warning label is enough. They liken these human-like voice chatbots to deepfakes in their potential to give rise to entirely new forms of deception and abuse, particularly to those who are already vulnerable to bot-based scams, such as the elderly.

These things are meant to trick us into thinking they’re human, Foreign Objects points out. Google didn’t screw up with those “umms” and “aahs.” Deception is part of parcel of the design:

There is a fundamental contradiction in human-like service bots. On one hand, legally and ethically, they need to disclose their artificiality; on the other, they are designed to deceive users into thinking, and acting, as if they were also humans. Duplex stunned audiences because its ‘um’s and ‘ah’s’ mimic the affect and agency of a fellow human being.

I found Bot or Not pretty easy to nail as a bot. I mean, come on, it didn’t know its own mother’s maiden name.

But would I have the same ease with Google Duplex? … and what does it all matter?

It matters when bots/AI/voice assistants get pulled into court to provide evidence in trials, for one. It’s happened before, Foreign Objects points out: in 2017, Amazon had to fight to keep recordings from its Echo IoT device out of court in a murder case.

Amazon claimed that Alexa’s data was in fact part of Amazon’s protected speech. … which, some have argued, might in fact bestow First Amendment protections. And this is why that matters, according to Foreign Objects:

In the US, First Amendment protections would mean that the makers of bots, like Google, Amazon and countless others, could not be held responsible for the consequences of their creations, even if those bots act maliciously in the world. All the same, … insisting that expressions made by ‘bots’ are strictly the speech of their creators comes wrapped up in its own complications, especially when humans are conversing daily with bots as friends, therapists, or even lovers.

In light of AI advancement, it’s important to be on guard as we engage with these chatbots in ever more intimate contexts such as these. We should all bear in mind that no matter how “LOL,” “IDK” and “ahhh”-ish they come off as, they are, in fact, surveillance-gathering tools. Does it matter whether they’re corporations or crooks trying to get at our data?

Either way, Foreign Objects says, this is privacy invasion in the ever-growing web of pervasive surveillance.