Apple to start scanning US iPhones for images of child sexual abuse
Issued on: 06/08/2021
Apple will start scanning US iPhones for images of sexual abuse and report any such findings to authorities. © Loic Venance, AFP/ File picture
Text by:
Apple unveiled plans to scan U.S. iPhones for images of child sexual abuse, drawing applause from child protection groups but raising concern among some security researchers that the system could be misused, including by governments looking to surveil their citizens.
The tool designed to detected known images of child sexual abuse, called “neuralMatch,” will scan images before they are uploaded to iCloud. If it finds a match, the image will be reviewed by a human. If child pornography is confirmed, the user’s account will be disabled and the National Center for Missing and Exploited Children notified.
Separately, Apple plans to scan users’ encrypted messages for sexually explicit content as a child safety measure, which also alarmed privacy advocates.
The detection system will only flag images that are already in the center’s database of known child pornography. Parents snapping innocent photos of a child in the bath presumably need not worry. But researchers say the matching tool — which doesn’t “see” such images, just mathematical “fingerprints” that represent them — could be put to more nefarious purposes.
Matthew Green, a top cryptography researcher at Johns Hopkins University, warned that the system could be used to frame innocent people by sending them seemingly innocuous images designed to trigger matches for child pornography. That could fool Apple’s algorithm and alert law enforcement. “Researchers have been able to do this pretty easily,” he said of the ability to trick such systems.
Other abuses could include government surveillance of dissidents or protesters. “What happens when the Chinese government says, ‘Here is a list of files that we want you to scan for,’” Green asked. “Does Apple say no? I hope they say no, but their technology won’t say no.”
Tech companies including Microsoft, Google, Facebook and others have for years been sharing digital fingerprints of known child sexual abuse images. Apple has used those to scan user files stored in its iCloud service, which is not as securely encrypted as its on-device data, for child pornography.
Apple has been under government pressure for years to allow for increased surveillance of encrypted data. Coming up with the new security measures required Apple to perform a delicate balancing act between cracking down on the exploitation of children while keeping its high-profile commitment to protecting the privacy of its users.
But a dejected Electronic Frontier Foundation, the online civil liberties pioneer, called Apple’s compromise on privacy protections “a shocking about-face for users who have relied on the company’s leadership in privacy and security.”
Meanwhile, the computer scientist who more than a decade ago invented PhotoDNA, the technology used by law enforcement to identify child pornography online, acknowledged the potential for abuse of Apple’s system but said it was far outweighed by the imperative of battling child sexual abuse.
“Is it possible? Of course. But is it something that I’m concerned about? No,” said Hany Farid, a researcher at the University of California at Berkeley, who argues that plenty of other programs designed to secure devices from various threats haven’t seen “this type of mission creep.” For example, WhatsApp provides users with end-to-end encryption to protect their privacy, but also employs a system for detecting malware and warning users not to click on harmful links.
Apple was one of the first major companies to embrace “end-to-end” encryption, in which messages are scrambled so that only their senders and recipients can read them. Law enforcement, however, has long pressured the company for access to that information in order to investigate crimes such as terrorism or child sexual exploitation.
Apple said the latest changes will roll out this year as part of updates to its operating software for iPhones, Macs and Apple Watches.
“Apple’s expanded protection for children is a game changer,” John Clark, the president and CEO of the National Center for Missing and Exploited Children, said in a statement. “With so many people using Apple products, these new safety measures have lifesaving potential for children.”
Julia Cordua, the CEO of Thorn, said that Apple’s technology balances “the need for privacy with digital safety for children.” Thorn, a nonprofit founded by Demi Moore and Ashton Kutcher, uses technology to help protect children from sexual abuse by identifying victims and working with tech platforms.
(AP)
The tech giant is rolling out a new tool to identify child abuse stored on iPhones, sparking criticism from privacy campaigners.
BY OSCAR WILLIAMS
New Statesman
In 2016, the FBI asked Apple to provide technical support to unlock a dead terrorist's iPhone as part of an investigation. Tim Cook, the company's chief executive, however, refused and warned that a “skeleton key” would have given the FBI the opportunity to unlock hundreds more devices. The decision was a political risk for Apple, but it established the company's reputation as the US tech giant most committed to protecting its customers' privacy.
In the years since, Apple has doubled down on its pro-privacy principles. The company has entered into a public war of words with Facebook over the ethics of its advertising-funded business model, while also making it harder for advertisers to track users across different apps. Privacy has become a key selling point of the iPhone brand, distinguishing it from smartphones that run Google's Android software.
Given its record on privacy, many security experts were surprised to discover on 5 August that the company has agreed to start scanning US users' photo libraries for child abuse. The new tool, dubbed neuralMatch, cross-references codes of images set to be uploaded to iCloud against a library of child abuse material. If a match is found, the account is disabled and US authorities are informed.
[See also: NYU’s researchers are the latest victims of Big Tech’s war on scrutiny]
The announcement has sparked an uproar among privacy campaigners who had previously seen Apple as being on their side of the debate. “Apple can explain at length how its technical implementation will preserve privacy and security in its proposed backdoor, but at the end of the day, even a thoroughly documented, carefully thought-out, and narrowly-scoped backdoor is still a backdoor”, commented the Electronic Frontier Foundation.
What this doesn't do, however, is give Apple or law enforcement agencies access to a users' photos, and it doesn't apply to photos which aren't uploaded to iCloud. And some expect that the new policy is designed to appease law enforcement agencies ahead of a move to encrypt iCloud libraries.
What concerns privacy campaigners most is that such a system could be repurposed by law enforcement agencies in countries such as China to identify individuals who had shared anti-government literature. But some experts are unconvinced that this represents a significant enough risk to avoid rolling out a technology that could aid the fight against child abuse.
[See also: Are Apple and Google weaponising privacy?]
“Is it possible [that it would be abused]? Of course. But is it something that I’m concerned about? No,” Hany Farid, a researcher at the University of California-Berkeley, told the Associated Press, noting that other, similar, technology had not been repurposed in this way.
For many, the potential future privacy consequences of the technology are heavily outweighed by the benefits of combatting child abuse. “Apple's expanded protection for children is a game changer,” said John Clark, the CEO of the National Center for Missing and Exploited Children. “With so many people using Apple products, these new safety measures have life-saving potential for children.”
[See also: Apple vs Facebook: how the war between the Silicon Valley giants is changing tech]
Oscar Williams is a senior journalist at the New Statesman covering technology.
August 06, 2021Ravie Lakshmanan
Apple on Thursday said it's introducing new child safety features in iOS, iPadOS, watchOS, and macOS as part of its efforts to limit the spread of Child Sexual Abuse Material (CSAM) in the U.S.
To that effect, the iPhone maker said it intends to begin client-side scanning of images shared via every Apple device for known child abuse content as they are being uploaded into iCloud Photos, in addition to leveraging on-device machine learning to vet all iMessage images sent or received by minor accounts (aged under 13) to warn parents of sexually explicit photos in the messaging platform.
Furthermore, Apple also plans to update Siri and Search to stage an intervention when users try to perform searches for CSAM-related topics, alerting the "interest in this topic is harmful and problematic."
"Messages uses on-device machine learning to analyze image attachments and determine if a photo is sexually explicit," Apple noted. "The feature is designed so that Apple does not get access to the messages." The feature, called Communication Safety, is said to be an opt-in setting that must be enabled by parents through the Family Sharing feature.
How Child Sexual Abuse Material is Detected
Detection of known CSAM images involves carrying out on-device matching using a database of known CSAM image hashes provided by the National Center for Missing and Exploited Children (NCMEC) and other child safety organizations before the photos are uploaded to the cloud. "NeuralHash," as the system is called, is powered by a cryptographic technology known as private set intersection. However, it's worth noting that while the scanning happens automatically, the feature only works when iCloud photo sharing is turned on.
What's more, Apple is expected to use another cryptographic principle called threshold secret sharing that allows it to "interpret" the contents if an iCloud Photos account crosses a threshold of known child abuse imagery, following which the content is manually reviewed to confirm there is a match, and if so, disable the user's account, report the material to NCMEC, and pass it on to law enforcement.
Researchers Express Concern About Privacy
Apple's CSAM initiative has prompted security researchers to express anxieties that it could suffer from a mission creep and be expanded to detect other kinds of content that could have political and safety implications, or even frame innocent individuals by sending them harmless but malicious images designed to appear as matches for child porn.
U.S. whistle-blower Edward Snowden tweeted that, despite the project's good intentions, what Apple is rolling out is "mass surveillance," while Johns Hopkins University cryptography professor and security expert Matthew Green said, "the problem is that encryption is a powerful tool that provides privacy, and you can't really have strong privacy while also surveilling every image anyone sends."
Apple already checks iCloud files and images sent over email against known child abuse imagery, as do tech giants like Google, Twitter, Microsoft, Facebook, and Dropbox, who employ similar image hashing methods to look for and flag potential abuse material, but Apple's attempt to walk a privacy tightrope could renew debates about weakening encryption, escalating a long-running tug of war over privacy and policing in the digital age.
The New York Times, in a 2019 investigation, revealed that a record 45 million online photos and videos of children being sexually abused were reported in 2018, out of which Facebook Messenger accounted for nearly two-thirds, with Facebook as a whole responsible for 90% of the reports.
Apple, along with Facebook-owned WhatsApp, have continually resisted efforts to intentionally weaken encryption and backdoor their systems. That said, Reuters reported last year that the company abandoned plans to encrypt users' full backups to iCloud in 2018 after the U.S. Federal Bureau of Investigation (FBI) raised concerns that doing so would impede investigations.
"Child exploitation is a serious problem, and Apple isn't the first tech company to bend its privacy-protective stance in an attempt to combat it. But that choice will come at a high price for overall user privacy," the Electronic Frontier Foundation (EFF) said in a statement, noting that Apple's move could break encryption protections and open the door for broader abuses.
"All it would take to widen the narrow backdoor that Apple is building is an expansion of the machine learning parameters to look for additional types of content, or a tweak of the configuration flags to scan, not just children's, but anyone's accounts. That's not a slippery slope; that's a fully built system just waiting for external pressure to make the slightest change," it added.
The CSAM efforts are set to roll out in the U.S. in the coming months as part of iOS 15 and macOS Monterey, but it remains to be seen if, or when, it would be available internationally. In December 2020, Facebook was forced to switch off some of its child abuse detection tools in Europe in response to recent changes to the European commission's e-privacy directive that effectively ban automated systems scanning for child sexual abuse images and other illegal content without users' explicit consent.
Apple’s search for child abuse imagery raises serious privacy questions
Posted: August 6, 2021 by
The Internet has been on fire since the August 4 discovery (disclosed publicly by Mathew Green) that Apple will be monitoring photos uploaded to iCloud for child sexual abuse material (CSAM). Some see this as a great move by Apple that will protect children. Others view this as a potentially dangerous slide away from privacy that may not actually protect children—and, in fact, could actually cause some children to come to harm.
How does this work?
It’s important to understand that, contrary to what it sounds like, Apple will not be rifling through all your photos on iCloud. All scanning for CSAM material will be done on the device itself, by an artificial intelligence algorithm. That system, called neuralMatch, will perform two functions.
The first is to create a hash of any photos on the device before they are uploaded to iCloud. (A “hash” is a computed value that should be a unique representation of a file, but that cannot be reversed to obtain a copy of the file itself.) This hash will be compared to a database of hashes of known CSAM materials on the device. The result is recorded cryptographically and stored on iCloud alongside the photo. If the user passes some minimum threshold of photos that match known CSAM hashes, Apple will be able to access those photos and the iCloud account will be shut down.
The second function is to protect children under 13 against sending or receiving CSAM images. Apple’s AI will attempt to detect whether images sent or received via iMessage have such content. In such cases, the child will be warned, and if they choose to view or send the content anyway, their parents will be notified. For children between 13 and 18, the same warning will be shown, but parents will apparently not be notified. This all relies on the child’s Apple ID being managed under a family account.
Why should I worry about monitoring a child’s texts?
There are a lot of potential problems with this. This can be a serious violation of a child’s privacy, and the behavior of this feature is predicated on an assumption that may not be true: That a child’s relationship with their parents is a healthy one. This is not always the case. Parents or guardians could be complicit in the creation of CSAM content. Further, an abusive parent could see a warning about a legitimate photo that was falsely identified as CSAM content, and could harm the child based on false information. (Yes, the parent would have the option to view the photo, but it’s possible a parent may choose not to. I certainly wouldn’t want such an image of my child in my head.)
Also, consider the fact that this applies to being sent an image, not just sending an image. Imagine the trouble a bully or scammer could cause by sending CSAM material, or the damage that could be done if a child of an abusive parent were sent a CSAM image and viewed it without fully understanding why it was being blocked or what the consequences would be!
And finally, as the EFF’s Eva Galperin pointed out on Twitter there is also the danger that this well intentioned functionality “is going to out a lot of queer kids to their homophobic parents”.
What’s the problem with monitoring photos uploaded to iCloud?
Although a comparison of a hash to a file has a low chance of false positives, it can definitely happen. Apple claims that there should be a one in one trillion chance of false positives, but it remains to be seen if that is true in practice.
Apple is providing a process to appeal in cases where an account is wrongly closed because of false positives. However, anyone who has been involved in reviews and appeals with Apple knows they don’t always go your way, nor are they always speedy. Sometimes they are, sometimes not. Time will tell how big a problem this is.
What about the privacy issues?
For a company that has constantly talked about protecting users’ privacy, this seems like a reversal. However, Apple has clearly put a lot of thought into this, and is emphasizing the fact that none of this happens on their servers. Apple states that all the processing happens on the device, and that it does not see the images (unless it’s determined that abuse is happening).
Further, CSAM is a big problem. I don’t think there’s anyone—other than pedophiles—who wouldn’t want to see all production of and trafficking in CSAM brought to an end. So many will praise Apple for taking this action.
This doesn’t mean there aren’t issues, though. Many view this as a first step onto a slippery slope. Blocking CSAM is a good thing, but there’s nothing to prevent the tools that Apple has built from being used for other things. For example, suppose the US government puts pressure on Apple to start detecting terrorism-related content. What exactly would that look like, if Apple decided to—or was forced to—comply? What would happen if a law-abiding person’s iCloud account was flagged as being involved in terrorist activity due to false positives on their photos? And what about tracking more prosaic crimes, such as drug use?
I could go on, as there are lots of things that governments of the world—including the US government—might want Apple to track. Although I tend to be willing to extend trust to Apple, this may not be something that is entirely within Apple’s control. They are a US company, and it’s possible for future US law to force Apple to do things their leadership wouldn’t have wanted to do.
We’ve also seen Apple bend to the desires of governments before. For example, Apple has conceded to demands from the government of China that are counter to Apple’s philosophy. Although the cynical point to this as evidence that Apple is more interested in profits from China’s large market (and they’re not entirely wrong) there’s more to it than that. Most of Apple’s manufacturing is done in China, and they’d be in a huge pile of trouble if China decided to shut down Apple’s ability to do business there. This means China has leverage they it use to make Apple bend to its wishes, at least within China.
Why is Apple doing this?
I’m sure there will be a lot of debate and speculation on this topic. Part of it is undoubtedly a desire to protect children and prevent distribution of CSAM. Part of it may be marketing.
To me, though, this all boils down to a political move. Apple has been a fantastic advocate for encryption and privacy, even going to the extreme of refusing the FBI’s demands relating to gaining access to a suspected terrorist’s iPhone.
It’s a common request from law enforcement to tech companies to give them “backdoors.” Essentially, this boils down to some kind of private access to users’ data, in theory accessible only to law enforcement agents. The problem with such backdoors is that they don’t tend to remain secret. Hackers can find them and gain access, or rogue government agents can abuse or even sell their access. There is no such thing as a secure backdoor.
Apple’s refusal to create backdoors for government access has angered many who believe that Apple is preventing law enforcement from doing their jobs. A common refrain for people trying to push for backdoors is the old standby, “but think of the children!” CSAM is frequently brought up as a reason why access to messaging, file storage, etc, is needed. This is actually a somewhat clever argument, by making it seem (falsely) like arguing against backdoors is also an argument in support of pedophiles.
By taking specific action against CSAM, Apple has effectively neutered this argument. Politicians will no longer be able to (in essence) accuse Apple of protecting pedophiles as a means of pushing for legislation to require backdoors.
Conclusion
In the end, this is something that is going to cause a lot of controversy and differences of opinion. Some are in support of Apple’s actions, while others are adamantly in opposition. Apple seems to be trying to do the right thing here, and appears to have put a lot of effort into ensuring that the way this is done is most respectful of privacy, but there are some legitimate reasons to question whether this new feature is a good idea.
Those reasons should not be conflated with support for or opposition to CSAM, which we can all agree is a very bad thing. There’s a lot of discussion that should be had on this topic, but CSAM is a very emotional subject, and we should all try to prevent that from coloring our evaluation of the potential problems with Apple’s approach.
No comments:
Post a Comment