Apple WWDC 2024 | Apple announces ChatGPT integration, Apple Intelligence (AI) tools, iOS upgrades, and enhanced Siri

To recap, Apple kicked off its Worldwide Developers Conference (WWDC) with a keynote address where company executives announced a range of exciting releases, including ChatGPT integration, Apple Intelligence (AI) tools, an enhanced Siri, better customisation options across apps and the Homescreen, iPhone mirroring, and the iPad calculator.

OpenAI CEO Sam Altman also announced the AI startup’s previously anticipated partnership with Apple.

This live coverage has ended.

Tim Cook refers to the new AI announcements as “game-changers” and hails the coming days of WWDC, inviting developers to explore the freely available resources and the newly announced features.

Apple announces that Siri will be able to use OpenAI’s ChatGPT in order to answer user questions and queries across text and images. Siri will ask the user for permission before accessing ChatGPT, which can also work with a user’s documents or help them generate images.

Apple users will be able to access ChatGPT without creating an account with OpenAI. The ChatGPT integration will be coming later this year, and Apple plans to add support for other large language models as well.

Apple announces the creation of ‘Genmojis’ where users will be able to enter text prompts in order to make stickers or emojis when messaging others, as well as more complex and sophisticated images.

The ‘Image Wand’ will also improve existing sketches and descriptions to create an image for the user.

Siri will be enhanced with generative AI, and will be able to take new actions across multiple apps, with privacy controls in place.

For example, Siri will be able to cross-reference media content within the smartphone along with data available online, in order to answer a user’s personal questions.

Apple Intelligence will work with language, images, and text for deep, natural language understanding to speed up daily tasks such as lining up Apple device notifications, powering users’ writing/editing tools, creating original images for conversations, organising images in photo libraries or creating illustrations/sketches/animations. 

Users will be able to talk to Apple Intelligence in order to get the data they are looking for.

Apple stresses on privacy and maintaining control over your own data, through “on-device processing.”

Apple says the A17 Pro and M chip families will be able to power the intense energy needs of AI tasks across Apple devices.

Apple announces Private Cloud Compute for complex AI requests while protecting privacy.

Apple discusses the need for powerful, intuitive, easy-to-use, well-integrated, personalised, and private AI, leading towards “personal intelligence - the next big step for Apple.”

Apple acknowledges the work done with AI chatbots but criticises their lack of personal knowledge, and stresses on the need for AI that understands users.

‘Assassin’s Creed: Shadows’ will be coming to iPad and Mac later this year

Apple announces the Passwords app in order to keep a user’s passwords in one place and make them easier to retrieve.

Safari gets a new release with streamlined content and Highlights that show you important information while you are browsing through large volumes of media. This is powered by machine learning, and a Reader feature will provide distraction-free summaries. Meanwhile, the Viewer feature will help showcase videos without clutter.

The macOS Sequoia upgrade will help Apple apps work together more seamlessly and make use of iPhone mirroring through the Mac screen. iPhone notifications will be coming to Mac, and users can operate their iPhone through their MacBook screen. The iPhone will remain locked during this process, for security reasons.

For example, users can work on a video on their iPhone, through their MacBook window.

iPadOS 18 will bring more customisation options, a new floating tab bar that can turn into a side bar, easier document browsing controls, and refined animations.

The iPad will also be getting a calculator, with history and unit conversion features. Device owners can use their Apple Pencil to write equations and work with the calculator to solve the sum, while the system maintains the consistency of their handwriting. This feature is called ‘Math Notes.’ 

The Notes app will also be able to support this feature, apart from learning your handwriting to make it more legible while preserving its original style. This is called ‘Smart Script.’

Apple introduces the ‘Training Load’ metric to measure how workouts affect the wearer’s body, by using a combination of bodily data and algorithms. This is meant to help smartwatch wearers better track their efforts and calibrate their training to avoid injury and exhaustion.

Apple watch health data will also become easier to access and evaluate at just a glance through the Vitals app, in order to help users be aware of metrics outside their normal ranges.

There are also more health features for pregnant people.

Apple is using machine learning to curate the wearer’s best photos to help them set the image as their watch face without sacrificing elegance.

Apple users can interact with Siri in noisy places by shaking their head or nodding with their earbuds in, so as to maintain their privacy in public.

The AirPods Pro are also getting Voice Isolation to remove background noises and improve call quality. 

The Photos app is getting a fresh and cleaner look to filter out clutter, or media such as screenshots. Photos can be organised by the day, or viewed as a collage. Collections are also easier to share, along with finding favourite groups of people. A ‘Trips’ section is meant to commemorate the user’s vacations and holidays.

Users can also boost topics that are important to them within the Photos app. The new carousel highlights favourite photos and featured ones.

Apple announces the ‘Tap to Cash’ payment method so people can exchange Apple Cash without having to divulge their personal details or their number. There are also two new ways to pay with Apple online.

Owners will be able to lock their iPhone apps to prevent external users from accessing the device owner’s app data. Apps can also be hidden for privacy reasons.

In iOS 18, there are more tap-back options to speed up communication with emojis. Users can also choose to schedule the messages they want to send, and use text effects for emphasis.

Apple announced Messages via Satellite for iPhone 14 and later, to communicate even when there is patchy internet service. The messages sent over satellite will be end-to-end encrypted.

Emails will also be better organised by categories.

Apple announces more options to arrange apps and widgets for better Homescreen customisation. App icons also get a new revamped look in dark mode, as well as iOS-suggested colour tints that complement the phone’s wallpaper - or other colours that the user prefers.

Coming to the control centre, users can navigate to multiple new groups of controls - with a single continuous swipe.

Apple’s Tim Cook started by hailing the company’s Apple TV+ offerings and its “great originals,” before the VP of the Vision Products Group, Mike Rockwell, spoke about the potential of VisionOS. 

He then announced VisionOS 2 to “propel spatial computing forward.”

Apple executives introduced the concept pf “spatial photos” to experience a digital library’s photos in an immersive way with the Vision Pro headset. VisionOS 2 also supports easier hand gestures, and Mac Virtual Display at greater resolutions and sizes. VisionOS 2 will cater to travellers and users with long commutes, as well as those shooting spatial video.

Apple’s WWDC 2024 kicks off with a stylistic video showing Apple executives sky-diving from a plane, before CEO Tim Cook welcomes the gathering and promises “profound new intelligence capabilities.”

Apple this year laid off a little over 600 workers in California, reported media outlets in April, marking its first significant round of job cuts after the COVID-19 pandemic.

Apple’s May 7 event did not exactly end on a smooth note. CEO Tim Cook called the iPad Pro “the thinnest product we’ve ever created” at the time, and shared an ad titled ‘Crush.’ In it, a giant hydraulic press was shown destroying musical instruments, artists’ tools, books, cameras, and games, only to replace them with the iPad Pro. 

There was immediate outrage, with even celebrities like Hugh Grant mocking the ad and the message of shattering human creativity that it seemed to convey.

Last month, Apple hosted another event where it introduced its thinnest portable tablet: the iPad Pro, with the latest Apple M4 chip. The premium gadget-maker also announced the redesigned iPad Air in two sizes, and the new Apple Pencil Pro.

Apple’s WWDC comes at a sensitive time. There are many emotions surrounding GenAI and its increasing presence in both our daily software and our hardware. 

While many tech users are ready to embrace the surge in competing chatbots and now, AI personal computers (PCs), others are worried for their livelihoods and are looking to protect their creative works and personal information from Big Tech companies scraping data to train their large language models.

There likely won’t be any hardware additions, apart from hinting what is to come this September.

Apple’s Messages app is expected to receive RCS support and it might get an option to schedule messages. A colourful tap-back option is also on the cards.

Apple’s iPad may get the Calculator app.

The Health app could see vital updates in order to deliver better blood pressure monitoring data. Hearing test support with the Apple AirPods is also expected.

The watchOS might get some upgrades, along with Fitness features

Apple is expected to drop the iOS 18 update for iPhones, which should fix bugs, enhance security, and perhaps add more customisation for the Home screen and icons.

Similarly, iPadOS 18, macOS 15, watchOS 11, tvOS 18, and visionOS 2 could get updates as well, to make them more AI-ready and patch any security issues.

iPhone shutterbugs who use the Photos app could get features similar to Google’s Pixel and Samsung’s S series, with improved options like object eraser and the magic editor.

With the new OS updates, Apple devices are expected to get a new Password app to serve users’ autofill needs and do away with the iCloud Keychain feature.

Media outlets reported on a potential deal between Apple and ChatGPT-maker OpenAI. Today’s event could see Apple officially confirming or denying the reported deal. The integration of OpenAI’s ChatGPT into iPhone, Mac and iPad is a possible announcement. With this, Apple could challenge Google and Samsung on the AI turf.

AI integration could help users with automatic summaries in text, notes, emails, and messages. Apple’s Siri is also expected to get a major boost with the introduction of AI.

What are watchers most eagerly expecting today? Many hope generative artificial intelligence, or GenAI, will specifically be on the agenda.

Apple’s seemingly slow adoption of the technology has been a point of criticism while Big Tech rivals like Google and Microsoft are racing to add GenAI-enabled chatbots to their search engines and workplace offerings - or even physical devices.

During last year’s WWDC, Apple introduced its Vision Pro headset, the new 15‑inch MacBook Air with M2, Mac Studio with M2 Max and M2 Ultra, Mac Pro with M2 Ultra, and previews of iOS 17, iPadOS 17, macOS Sonoma, and watchOS 10.

In case you need a refresher while you wait, you can catch up with The Hindu’s coverage of Apple’s WWDC in 2023.

While most Apple fans, critics, and watchers are familiar with the company’s fall event where the gadget-maker announces its highly anticipated line-up of new iPhones and their prices, WWDC is also crucial as Apple introduces many of the software upgrades and new features its devices will support in the near future. 

The WWDC keynote is followed by the Platforms State of the Union, as well as sessions, labs, and community events that will go on until June 14.