10 questions with the Live Activities team – Discover


With Live Activities, your app can provide up-to-date, glanceable information — like weather updates, a plane’s departure time, or how long it’ll be until dinner is delivered — right on the Lock Screen. What’s more, thanks to lively features like the Dynamic Island on iPhone 14 Pro and iPhone 14 Pro Max, Live Activities can also be a lot of fun.

Apple evangelists, designers, and engineers came together at Ask Apple to answer your questions about Live Activities and the Dynamic Island. Here are a few highlights from those conversations, including guidance about sizing and styling, when to dismiss a Live Activity, and why widgets and Live Activities are different (except when they’re not).

How do I update a Live Activity without using Apple Push Notification service (APNs)?

Your app can use a pre-existing background runtime functionality, such as Location Services, to provide Live Activity updates as you see fit. You can also use BGProcessingTask and background pushes to provide less frequent updates to your Live Activity. Keep in mind that these background tasks aren’t processed immediately by the system. You can read more below:

Displaying live data with Live Activities

The 4-hour default to dismiss a Live Activity is too long for my use case. What are the guidelines for dismissing a Live Activity after it ends?

When ending a Live Activity, you can provide an ActivityUIDismissalPolicy to tell the system when to dismiss your UI. Alternatively, you can choose to dismiss the Live Activity immediately or after a certain time has passed.

How can my app detect when someone dismisses a Live Activity?

Your app should use the activityStateUpdates async sequence to observe state changes for each Live Activity.

When an app is force quit, is the associated Live Activity dismissed?

Live Activity life cycles aren’t tied to the host app’s process, so they’ll stay if the app is force quit. Your widget extension’s life cycle is also separate. It’s entirely possible that different instances of the widget extension are called to render views for the same Live Activity, so it’s important not to store any state locally in the widget extension.

How do Live Activities and widgets differ?

Live Activities and widgets both provide glanceable information at a moment’s notice. Live Activities are great for displaying situational information related to an ongoing task that someone initiated. Good examples include food deliveries, workouts, and flight departure times. Widgets can provide glanceable information that’s always relevant. Good examples include to-do lists, this week’s weather forecast, or how close someone is to closing their rings on Apple Watch.

While both Live Activities and widgets rely on WidgetKit to lay out their UI, they’re structured a bit differently. Live Activities are a single view that updates programmatically, while widgets consist of a timeline of preconstructed views.

Should my Live Activity attempt to change the background color of the Dynamic Island?

The Dynamic Island is most immersive when you don’t provide background color or imagery — think of it purely as a canvas of foreground view elements. More design guidance is provided in the Human Interface Guidelines.

Human Interface Guidelines – Live Activities

Do Live Activities support interactive buttons?

Live Activities on the Lock Screen and in the Dynamic Island don’t support interactive buttons or other controls. Including buttons in your Live Activity could confuse someone into thinking they’re able to interact with the view. For this reason, you should avoid displaying anything in your UI that resembles a button.

The best user experience exists within your app, which is why all interaction with a Live Activity results in opening your app. A Live Activity’s Lock Screen presentation and expanded presentation can include multiple links into your app, so you can provide different destinations, depending on the context of your Live Activity.

Are Live Activities the only way to support the Dynamic Island?

Your app can implement other system services, such as CallKit and Now Playing, that display system UI in the Dynamic Island. However, Live Activities are the only way for your app to provide its own UI in the Dynamic Island.

Is it possible to add animations to the Dynamic Island?

While there’s no support for arbitrary animations in your Live Activity views, your app can change how a Live Activity’s content updates from one state to the next. Read more in the “Animate content updates” section of the article below.

Displaying live data with Live Activities

Where can I find more documentation about Live Activities?

The ActivityKit documentationprovides a wealth of information about implementing Live Activities, including how to update and end a Live Activity using APNs. In addition, the Human Interface Guidelinesoffer design guidance and recommended sizes for the various presentations. You can also find some inspiration in the Food Truck sample project from WWDC22.

Human Interface Guidelines – Live Activities

Displaying live data with Live Activities

Updating and ending your Live Activity with ActivityKit push notifications

ActivityKit

WidgetKit



Spotlight on: Spatial Audio – Discover


When designing soundscapes for apps and games, the right notes can make all the difference. And when those notes are built to support multichannel audio, they might even turn someone’s head. (Literally.)

Endel and Odio are just two of the many apps and games taking advantage of Spatial Audio. They use multichannel mixes, Core Audio, and AVFoundation to add texture and dimensionality, creating resonating surround-sound experiences that further immerse listeners into the world within their apps.




Design for spatial interaction

Discover the principles for creating intuitive physical interactions between two or more devices, as demonstrated by Apple designers who worked on features for iPhone, HomePod mini, and AirTag. Explore how you can apply these patterns to your own app when designing features for Apple platforms, and…

Endel (pictured above) conjures up personalized and adaptive soundscapes based on biometrics and environment to help people focus and get better sleep. Its inaugural Spatial Audio soundscape — one with the satisfyingly otherworldly name of Spatial Orbit — brings the app’s remarkable mix of art and AI to a new dimension.

“It feels like you’re inside a vast, glittery space,” says Dmitry Evgrafov, Endel cofounder and chief sound officer. “It’s almost like the sonic equivalent of pointillism, where the small dots create a structure themselves and you kind of drown in the thing. It’s a very beautiful state, and it’s not something you can reproduce in stereo.”

Endel’s Spatial Orbit soundscape is “not something you can reproduce in stereo,” says cofounder Dmitry Evgrafov.

When bringing Spatial Audio into their ecosystem, the Endel team’s first task was determining if the technology was compatible with their ever-changing, generative soundscape. That job fell largely to Kyrylo Bulatsev, cofounder and chief technology officer. “[Spatial Audio] meant we had to add one more dimension to the non-static element,” he says. “Besides choosing what sound to play and when, we had to think about where the sound would be and how it would move around you.”

That soundscape also had to hit the “thin line between augmenting an experience and making it distracting,” Evgrafov says. That’s because while most apps (and games and movies and songs) are designed for active engagement, Endel aims to be a perfect background companion — enhancing your experience without pulling from your focus. “Our use case is different from other products that utilize the technology,” says Evgrafov (whom fellow cofounder Oleg Stavisky credits with “all the beautiful sounds in the app”).

It’s almost like the sonic equivalent of pointillism.

Dmitry Evgrafov, *Endel* cofounder and chief sound officer

A pianist and musician with 10 albums to his credit, Evgrafov certainly knows his way around stereo. “But randomization of the position of audio in the space? That’s a whole different beast,” he says.

The first serious prototype of Spatial Orbit was earthbound, set to a realistic jungle scene. “The idea was you’d walk around this magical Garden of Eden and exotic tropical animals would sing around you,” he says. “We had a harp playing by the water, a creek, birds that don’t exist in the real world, stuff like that.”

Similar ideas kept coming: a Gregorian choir that slowly shuffled past you while chanting, field recordings from inside a cave. Although the concepts were cool and the prototypes sounded great, the team kept running up against the same problem. “They weren’t Endel,” says Evgrafov. “They transported you to a place, but that meant people were using the app consciously. They didn’t match what we stood for.”

Like all of *Endel’s* soundscapes, Spatial Orbit soundscape changes with your biometrics, environment, and local time.

The final version of Spatial Orbit does match what Endel stands for — and achieves the synthesis of art and technology that Endel strives for. “The rain [in our soundscape] is almost metaphorical,” says Evgrafov. “It’s got this slightly augmented feel that allows you to just drown a little and be with your thoughts, focus on your book, or whatever you’re doing.”

Tweaking the soundscape was an adventure in itself. “Watching people test Endel is kind of a funny exercise,” laughs Stavitsky. That’s because there’s really not an established way to test an personalized auto-generated soundscape for a group of people all at once.

[The rain has] this slightly augmented feel that allows you to just drown a little and be with your thoughts, focus on your book, or whatever you’re doing.

Dmitry Evgrafov

“We invented the process and the toolset,” says Evgrafov. It involved a lot of people wandering Endel’s Berlin offices… and elsewhere. “It was also a lot of me in public spaces just staring at nothing, like a cat.”

In the end, Spatial Orbit captures that elusive mix of innovative technology and artistic resolve. “When we realized the science was there and that it still checked all the Endel boxes, it was a big relief,” says Evgrafov. “We thought, ‘OK, we can be non-intrusive and Spatial at the same time.’”

Download Endel from the App Store

Odio also focuses on creating great ambient soundscapes — but with a sci-fi twist. “I want our composers to imagine inventing planets and filling them up with sound,” says Joon Kwak, the app’s Seoul-based cofounder. “We want to walk people through these new planets.”

The app’s soundscapes, which can evoke anything from a crashing waterfall to a buzzy digital backdrop to the spooky calm of the deep sea, use head tracking and multichannel audio to create a truly mesmerizing mix. (The app is also a visual feast, with each soundscape accompanied by ever-shifting techno-tinged art.)

But you’re no passive listener in these audio realms. The individual elements that make up each soundscape can be manipulated through an imaginative, playful UI that lets you reposition each audio element (like that waterfall) anywhere you like.

*Odio*'s soundscapes evoke everything from nature sounds to buzzy digital environments — and each comes with gorgeous, ever-shifting art.

Befitting its futuristic feel, Odio’s backstory is one of serendipitous meetings, well-timed hardware and software releases, and a stroke of good fortune. Kwak conceived the app’s initial version as a graduation project at the Design Academy Eindhoven. Originally known as Virtual Sky, the prototype contained the bones of what would become Odio, but was largely grounded in real-world sounds. It also required a mess of hardware and special equipment — all of which was rendered pretty much irrelevant once AirPods with Spatial Audio arrived.

“I was depressed for a while,” laughs Kwak. “I was like, ‘I’ve been working on this for months, and now it’s pointless!’ But then I thought about it more deeply and realized, ‘Oh, this just means I don’t need to provide hardware,’ and it was actually great.”

Kwak partnered with Volst, a company that was interested in a 3D soundscape app. With the building blocks in place, Odio’s UI developer and designer, Rutger Schimmel, took on the challenge of bringing Kwak’s project to life — a process that went much faster than expected.

I want our composers to image inventing planets and filling them up with sound.

Joon Kwak, *Odio* cofounder

“We knew the AirPods had [surround sound] support, but we were skeptical,” he says. “We thought, ‘OK, they have head tracking, but it’s probably just for first-party stuff.’ But we were still excited, so we quickly set up an Xcode project to get the data from the AirPods to the device.”

They had a prototype up and running on the headphones within minutes. “We were blown away by how easy it was,” Schimmel says. “And in about an hour we decided on excellent 3D audio frameworks from Apple that were the perfect foundation for what we were working on.” Coding began in January. By April, the team had a Swift-built demo ready to go.

To build an Odio soundscape, composers like Kwak, Odio sound designer Max Frimout, and a team of outside musicians collaborate — generally in Logic Pro — by blending ambient sounds, synthetic bells and whistles, and music.

Odio’s composers create the soundscapes, but it’s up to you — and this clever UI — to place them where you like..

After the soundscapes are completed and duly field-tested in coffee shops, parks, and subways, the artists hand their files over to Schimmel. For a role that involves cutting-edge design, immersive audio, and incredible degrees of customization, Schimmel’s toolbox is surprisingly uncluttered: AVAudioEnvironmentNode (AVKit) for creating the 3D audio environment, CMHeadphoneMotionManager (Core Motion) to access headphone motion data, and Sentry for error tracking and QA.

“Everything else in Odio is created from scratch in Swift — from data management to interacting with soundscapes to real-time buffering the interactive sound files,” Schimmel says.

The result is a remarkable example of the power and simplicity of designing for Spatial Audio. “Honestly,” Schimmel says, “most of the hard work is done by the composer.”

Download Odio from the App Store




Discover geometry-aware audio with the Physical Audio Spatialization Engine (PHASE)

Explore how geometry-aware audio can help you build complex, interactive, and immersive audio scenes for your apps and games. Meet PHASE, Apple’s spatial audio API, and learn how the Physical Audio Spatialization Engine (PHASE) keeps the sound aligned with your experience at all times — helping…




Immerse your app in Spatial Audio

Discover how spatial audio can help you provide a theater-like experience for media in your apps and on the web. We’ll show you how you can easily bring immersive audio to those listening with compatible hardware, and how to automatically deliver different listening experiences depending on…



Explore Spatial Audio – Discover


Discover how you can bring a new dimension of sound to your apps and games with Spatial Audio. We’ll show you how you can easily bring immersive audio to listeners with compatible hardware, help you take advantage of the PHASE and Audio Engine APIs, and offer recommendations on tailoring your project’s experience to tell stories in new, exciting ways. We’ll also share how apps like Endel and Odio added Spatial Audio to deliver incredible sound.

Videos




Immerse your app in Spatial Audio

Discover how spatial audio can help you provide a theater-like experience for media in your apps and on the web. We’ll show you how you can easily bring immersive audio to those listening with compatible hardware, and how to automatically deliver different listening experiences depending on…




Discover geometry-aware audio with the Physical Audio Spatialization Engine (PHASE)

Explore how geometry-aware audio can help you build complex, interactive, and immersive audio scenes for your apps and games. Meet PHASE, Apple’s spatial audio API, and learn how the Physical Audio Spatialization Engine (PHASE) keeps the sound aligned with your experience at all times — helping…




Design for spatial interaction

Discover the principles for creating intuitive physical interactions between two or more devices, as demonstrated by Apple designers who worked on features for iPhone, HomePod mini, and AirTag. Explore how you can apply these patterns to your own app when designing features for Apple platforms, and…

Feature stories




Spotlight on: Spatial Audio

Learn how developers are creating immersive surround-sound experiences.




Behind the Design: Odio

Discover how this app conjures up its 3D soundscapes.

Resources

PHASE

Audio Engine



Developer Spotlight: Rootd – Discover


When Ania Wysocka started having panic attacks as a university student, she turned to the first resource she thought of. “My instinct was to look for an app that could explain what was happening to me,” she says.

But when the hypnosis and therapy apps she downloaded didn’t have what she was seeking, she decided to create Rootd, whose lo-fi vibe matches its simple mission: to demystify panic attacks and bring on-the-spot relief.

Rootd offers a primer in the biology of anxiety, as well as breathing tools, journal prompts, and guided visualizations narrated by Wysocka herself. “The whole concept of Rootd is to befriend your fear and face it head-on,” says the British Columbia–based creator.

We caught up with her to discuss her own history, the power of perseverance, and how she learned to build an app from scratch.

With robust resources and a cute blue monster, Rootd offers ways to help navigate anxiety.

With robust resources and a cute blue monster, Rootd offers ways to help navigate anxiety.

What’s your personal experience with panic attacks?

I was in my fourth year of university and hanging out at a friend’s house when I experienced my first one. My heart started racing and I felt overcome with feelings of doom. As a young, relatively healthy person, it was terrifying and confusing. My doctor said everything was OK with me. I said, “Then how do you explain what happened?” They were quick to move on to the next patient.

What did you do to track down the explanation?

I looked in academic journals, but they were all written in medical jargon and not very helpful. I ended up seeing a counselor and reading the work of Claire Weekes. She was one of the first doctors to say that panic disorder does exist, that in a panic attack you enter a cycle of fear in which you’re constantly expecting it to happen again. That resonated.

How did you figure out how to turn that insight into an app?

It was years after my first panic attack, when I’d collected enough information and started feeling much better, that I went for it. I’d thought through the wireframes, design, and marketing, but I definitely didn’t know the techie stuff. I worked with a developer I’d known from a former job; he knew the nuts and bolts of building an app. It took a lot of trial and error. It required a lot of perseverance. Things kept falling apart, and we needed to just rebuild.

After you launched Rootd, how did you get the word out?

You don’t actually need to spend millions of dollars putting ads on buses. My main marketing strategy early on was App Store optimization via keyword optimization, which I learned about by reading articles online. The keywords that go into the body of the product description are like gold. You want to make sure you’re using all the space you can as strategically as possible.

What was your intention behind the app’s relatively simple design?

When you’re in a state of distress, you don’t need flair. You need focused and simple. The big red button is called the Rooter. You know how a huge oak tree in a storm will barely move, while a plant in the ground can topple over? Our goal is to get you to be the oak tree, so you feel confident you can withstand everything coming at you.

Download Rootd from the App Store

Learn more about the App Store Small Business Program

Originally published on the App Store



Celebrate women in app development – Discover


This International Women’s Month, we’re celebrating women founders, creators, developers, and designers. Read on to learn more about their journeys and the stories behind their apps and games.

Behind the Design: Rebel Girls

The Rebel Girls app uses immersive audio experiences, gorgeous art, and clever interactive elements to spotlight its historic heroines. “We’re creating an omnichannel for girls,” says Jes Wolfe, CEO of Rebel Girls. “The app takes the best of our books, podcasts, and audio stories and puts them into a flagship destination.”




Behind the Design: Rebel Girls

Find out how the groundbreaking book became an ADA-winning app.

Download Rebel Girls from the App Store

Behind the Design: Wylde Flowers

This charming Apple Design Award-winning game is a cross-pollination of farming simulation, eerie mystery, optional love story, and exploration of tolerance and understanding. Also, you’re a witch who sometimes turns into a cat. “The Wylde Flowers experience is a bit different for everybody,” says Amanda Schofield, cofounder, creative director, and managing director of indie developer Studio Drydock. “It’s all about self-expression and self-exploration.”




Behind the Design: Wylde Flowers

Discover how Studio Drydock created this charming Apple Design Award-winning game.

Download Wylde Flowers from Apple Arcade

Developer Spotlight: Rootd

When she started having panic attacks as a university student, Ania Wysocka (pictured above) wanted “to look for an app that could explain what was happening to me,” she says. But when the hypnosis and therapy apps she downloaded didn’t have what she was seeking, she decided to create Rootd to demystify panic attacks and bring on-the-spot relief.




Developer Spotlight: Rootd

Talk about an impressive résumé.

Download Rootd from the App Store

Behind the Design: Overboard!

In the evocative murder mystery game Overboard!, you play not as the detective but the murderer most foul — Veronica Villensey, a fading 1930s starlet who’s tossed her husband off a cruise ship. To bring the story to life, artist and designer Anastasia Wyatt trawled into the rich potential of the game’s vintage setting, pulling designs from 1930s fashion, magazines, and even sewing pattern books.




Behind the Design: Overboard!

How Inkle built an upside-down whodunit in just 100 days.

Download Overboard! from the App Store

Behind the Design: Pok Pok Playroom

When the husband-and-wife team of Esther Huybreghts and Mathijs Demaeght first began dreaming up Pok Pok Playroom, they made a solemn vow: parents shouldn’t need to mute the app in a restaurant. “We didn’t want media and jingles and jangles that get stuck in your head,” Huybreghts laughs. “We wanted a quieter experience.”




Behind the Design: Pok Pok Playroom

Pok Pok Playroom is a quiet feast for little senses. There are switches to flip, gears to grind, blobs to plop together, and bells to ring — and those are just a handful of the animations designed to make the app feel like a tactile, handmade toy.

Download Pok Pok Playroom from the App Store

Developer Spotlight: Ground News

In 2017, Harleen Kaur launched Ground News, a news aggregator that helps you see how media outlets across the political spectrum are covering—or ignoring—a topic. Not only does it let you read coverage from thousands of publications worldwide, it also shows the political bent of an article or outlet (which is ranked by a third-party service and Ground News users themselves).




Developer Spotlight: Ground News

Talk about an impressive résumé.

Download Ground News from the App Store

Developer Spotlight: Prêt-à-Template

When Prêt-à-Template founder and CEO Roberta Weiand launched her app in 2014, it quickly became a darling among fashion designers around the world. With its library of templates, textures, and patterns, the app lets anyone sketch their dream outfit.




Developer Spotlight: Prêt-à-Template

Download Prêt-à-Template from the App Store

Developer Spotlight: The Dyrt

Sarah Smith, an avid camper and cofounder of The Dyrt, was frustrated by how hard it was to find details on a campsite before you booked. She wanted to know that, say, site 2 was next to a busy road, while site 7 was along a river. She wondered why nobody seemed to be solving the problem. Then she had a thought that changed everything: “Why can’t I do it?”




Developer Spotlight: The Dyrt

Download The Dyrt from the App Store

Read more

Discover more apps founded by women



UK police shocked to discover taser disguised as iPhone


iPhone-shaped tasers are easy to find in US markets



A taser designed to resemble Apple’s iPhone was taken from a boy, and UK police say it was capable of delivering 650,000 volts.

Non-lethal tasers are illegal in the UK, but that didn’t deter someone from smuggling one in. The taser in question was built to look like an iPhone, which can be purchased easily in the United States.

According to a report from Birmingham Live, police in Sutton Coldfield arrested a boy in possession of such a taser. It resembled an iPhone, but a button pressed on the side showed it was clearly a taser.

“Apparently they were developed in the States to combat muggings and can deliver shock of up to 650,000 volts of electricity,” Councillor Richard Parkin said after a ward meeting. “They are illegal in this country; possession of a non-lethal taser is a criminal offence which I understand carries a maximum prison sentence of ten years.”

An actual photo of the taser wasn’t released, but a quick search on Amazon US will produce plenty of results. Tasers are sold in a variety of forms, from key fobs to flashlights, in order to keep a potential target from suspecting it is a weapon.

This novelty iPhone taser would likely hurt and knock someone down but wouldn’t be as effective as something more purpose-built, like a taser gun. Remember that volts hurt, but amps kill.



Level up your apps and games – Discover


Explore the latest developer videos and learn about Metal, SharePlay, enterprise apps and more.

What’s new for enterprise developers

Discover how you can build compelling apps for your business on iOS, iPadOS, macOS, and watchOS. We’ll take you through a curated overview of the latest updates to Apple platforms and show you how to transform workflows, inform business decisions, and boost employee productivity.




What’s new for enterprise developers

Discover how you can build compelling apps for your business on iOS, iPadOS, macOS, and watchOS. We’ll take you through a curated overview of the latest updates to Apple platforms and explore relevant features that you can use to create engaging enterprise apps to transform workflows, inform…

Discover Metal Performance HUD

Get to know the new Metal Performance heads-up display panel built to help you analyze graphics performance in real time. Metal Performance HUD displays key graphics statistics so you can monitor, log, and identify tough-to-spot performance problems.




Discover Metal Performance HUD

Get to know the new heads-up display panel built to help you analyze graphics performance in real time. Metal Performance HUD displays key graphics statistics so you can monitor, log, and identify tough-to-spot performance problems.

Add SharePlay to your multiplayer game with Game Center

Learn how to let your players jump into games with friends they’re on FaceTime calls with, using SharePlay. We’ll show you how easy it is to turn on SharePlay support if you are already using the Game Center multiplayer UI. And if you’ve built a custom interface, we’ll give you the few lines of code you need to support SharePlay.




Add SharePlay to your multiplayer game with Game Center

Learn how to let your players jump into games with friends they’re on FaceTime calls with, using SharePlay. We’ll show you how easy it is to turn on SharePlay support if you are already using the Game Center multiplayer UI. And if you’ve built a custom interface, we’ll give you the few lines of…

Migrate custom intents to App Intents

Learn how you can easily convert your existing custom intents to App Intents. We’ll take you through the conversion of your intents to Swift and discuss how you can improve discoverability of your app features when you create App Shortcuts.




Migrate custom intents to App Intents

Learn how you can easily convert your existing custom intents to App Intents. We’ll take you through the conversion of your intents to Swift and discuss how you can improve discoverability of your app features when you create App Shortcuts.

To learn more about App Intents, watch “Implement App…

Implement Apple Pay and order management

Explore Apple Pay, an easy and secure way for people to make payments in your iOS, iPadOS, and watchOS apps and on the web. We’ll take you through the entire Apple Pay implementation workflow – including how you can signal support for Apple Pay, request payment and handling updates, and add order details at the end of a payment flow to help people track their purchases.




Implement Apple Pay and order management

Apple Pay provides an easy and secure way for people to make payments in your iOS, iPadOS, and watchOS apps as well as on the web. We’ll take you through the entire Apple Pay implementation workflow – including how you can signal support for Apple Pay, request payment and handling updates, and…



Spotlight on: Apple Pencil hover – Discover


When it comes to designing creative interactions, the Procreate team knows how to get the job done.

The illustration app kicked off in 2011 with touch-based controls — “just five fingers of input,” says CEO James Cuda — and won a rare pair of Apple Design Awards over the next decade for their innovative approaches to digital drawing, sketching, and painting.

While finger painting remains a core part of the app, Apple Pencil has become a significant part of Procreate’s story. Apple Pencil gives artists customization and control of their line width and opacity, stroke style, and quick-access controls. And with the introduction of Apple Pencil hover, the Procreate team is investing even more heavily in the stylus. “It’s truly made a profound impact in our design phase,” says Cuda.

Hover over your canvas, and X marks the spot.

Hover over your canvas, and X marks the spot.

As with pretty much any other digital or analog drawing tool, Apple Pencil operates on the X and Y axes of a canvas, requiring direct input from the stylus nib to draw a line or select a tool. The second-generation Apple Pencil also adds support for direct input along the side of the stylus — which gives developers an option to add shortcuts within their apps. Now, Apple Pencil hover is bringing tool and previewing shortcuts into an entirely new dimension. (The Z-axis, specifically.)

“[It’s] a whole new layer of interaction,” says Cuda. “Everything springs to life as your Apple Pencil comes near.”

With ColorDrop, you can precisely preview your colors before tapping your canvas.

With ColorDrop, you can precisely preview your colors before tapping your canvas.

Apple Pencil hover activates when the nib is up to 12 mm above the display on iPad Pro with the M2 chip. Developers can customize what the feature does within their app, including offering tool variations, menu selection, and even previewing lines themselves — so artists can draw, sketch, and color with even greater control. “The ability to not make a commitment or damage the artwork is transformative,” Cuda says.

And the feature’s functionality is only half the fun. “It makes everything feel so playful,” says Claire d’Este, Procreate’s chief product officer. “I find myself rolling up and down menus just to see it responding. There’s something so nice about everything lighting up as I’m thinking about what to do next.”

The ability to not make a commitment or damage the artwork is transformative.

James Cuda, Procreate CEO

The Procreate team has hidden these sorts of playful moments throughout the entire app. In the gallery view, hovering over thumbnails expands the image or previews animations. Tools like the color picker or menu buttons react as you move across them. And then, of course, there’s the canvas.

With your iPad on a desk or table, hover works in conjunction with Multi-Touch capabilities.

With your iPad on a desk or table, hover works in conjunction with Multi-Touch capabilities.

‘Your mind starts racing’

“There are two phases with something like this,” says Procreate chief technology officer Lloyd Bottomley. “The first is the initial, ‘Wow, this is cool.’ But then your mind starts racing because you’re trying to think of all the things you could do with it.”

With so many possibilities open to them, the Procreate team had to approach each idea with care and scrutiny to ensure they were aiding and improving design and creation workflows rather than hindering them. “We’re obsessed with keeping people focused on that point of interaction,” Bottomley says.

One area that proved surprisingly challenging: the brush cursor. “Honestly, we thought we’d have just one design through the entire brush library,” says Cuda. “The problem was there’s not one singular representation of that hover mark, because our brushes can do anything — you can have brushes inside brushes; you can have brushes that move across each other. To represent all that in a hover state was really challenging.”

We’re obsessed with keeping people focused on that point of interaction.

Lloyd Bottomley, Procreate chief technology officer

After a few weeks of back and forth, the team landed on a solution: customizable cursors that change with different brushes. “We needed to move away from that idea of ‘one thing to rule them all’ to a series of settings that could get us there,” says Cuda. “Now, all the brush-makers out there can customize what their hover state will look like.”

A second target was the app’s ColorDrop feature, which instantly fills a section of your artwork when you drag and drop a color on it, paint-by-numbers style. Using Apple Pencil hover, people can preview of what the artwork will look like before committing to a color, speeding up the process dramatically. “If you’re doing inking — comic book art, for instance — it’s huge,” says d’Este.

If you’re doing inking — comic book art, for instance — it’s huge.

Claire d’Este, Procreate chief product officer

It’s also a timely example of how a small change can make a massive improvement in an artistic workflow. “Those kinds of interactions take a surprising amount of time,” says Bottomley. “Even moving your arm across the screen takes time. Now you barely have to move.”

Hover over your color, then drop it on your canvas.

Hover over your color, then drop it on your canvas.

The saga of the sliders

While the Procreate team delighted in improving interactions for brush cursors and ColorDrop, they had a much bigger problem they hoped Apple Pencil hover would solve: a little piece of UI that had been bothering the team since the app’s very first days.

“Our goal is to put absolute focus on the artist’s content,” says Cuda. Procreate’s interface has long championed minimalistic tool windows and intuitive gestures like tap-to-undo to keep the canvas clear for the work. But they hit a proverbial artistic wall when trying to build UI for repetitive interactions like adjusting brush size or opacity.

Iterations came and went; the pair tried variations on pinching and zooming and tapping and holding, but nothing felt properly connected to the rest of the Procreate experience. “We ditched it all,” says Bottomley, “and went with that very conventional set of sliders you see on the left hand side of the screen.”

The sliders were functional. They were intuitive enough. But whenever the team thought about features they really liked about the app, the sliders were conspicuously absent — until Apple Pencil hover. “Once hover was announced, we realized we could work with Multi-Touch like we couldn’t before,” says Cuda.

With Apple Pencil hover, could they at last kill the sidebar? “We developed a gesture we thought would be just so ubiquitous and approachable,” says Cuda. “The idea was you would use two fingers to pinch and zoom while you’ve got hover up, so you could clearly see where your brush is and how it changes in size before you mark the screen. We were convinced it would be the way of the future.”

And then they began testing.

Place iPad on a surface to use hover with MultiTouch gestures…

Place iPad on a surface to use hover with MultiTouch gestures…

“We realized we were wrong as soon as we put it in practice,” says Cuda. The gesture worked brilliantly when iPad was sitting on a table or against a stand — a common-enough use case — but anyone using the tablet on a couch had a different experience. “You’re clutching the device with two hands,” he says. “And as soon as you pinch and hover, the device is no longer, uh, in your clutches.”

We had to put our egos aside and go, ‘OK, maybe we were wrong.’

James Cuda, Procreate CEO

The challenge was enough to send the team back to the drawing board. “The gesture is useful; it’s just not the singular interface methodology were hoping to create,” says Cuda. “We had to put our egos aside and go, ‘OK, maybe we were wrong.’ And we had to think about what was best for the customer.”

For customers holding the device, it meant the return of the sliders. “We’re keeping them for mobile drawing,” Cuda says. “On a desk or stand, when you’ve got both hands free, the sidebar goes away and we get exactly what we wanted. So it was a wild ride for a couple of weeks making those calls.”

… or use the sliders on the edge of the screen.

… or use the sliders on the edge of the screen.

‘It’s hard to go back’

People using iPad Pro with the M2 chip and the second-generation Apple Pencil, can explore Procreate’s Apple Pencil hover features now. But Cuda and the team are focused strongly on the future, regarding hover as an important new tool in the shed — enough so that Cuda says it already “solves a bunch of things” in regards to upcoming projects.

“It doesn’t feel like we’re tapping into a technology but creating a natural extension of what you could already do,” d’Este says. “Once you’ve experienced this, it’s hard to go back.”

Read Behind the Design with Procreate

Download Procreate from the App Store



App Store Awards 2022 – Discover


For over a decade, the App Store editorial team has taken a moment at the end of the year to celebrate the very best apps and games. We’ve heralded the work of individual self-taught developers and huge international teams.

With so many wonderful apps on the App Store all over the world, selecting just a few award winners is no easy task. As always, we focus on technical innovation, user experience, and design. Then we consider the impact these apps have had on our lives.

It’s our honor to celebrate this year’s winners. Enjoy!

iPhone App of the Year: BeReal

A social media sensation that gave us an authentic look into the lives of our friends and family.

Download BeReal from the App Store

iPhone Game of the Year: Apex Legends Mobile

This thoughtfully reengineered groundbreaking battle royale is its best version to date.

Download Apex Legends Mobile from the App Store

iPad App of the Year: GoodNotes 5

With best-in-class Apple Pencil support and powerful collaboration tools, the gold standard of digital notetaking got even better.

Download GoodNotes 5 from the App Store

iPad Game of the Year: Moncage

A stunningly fresh spin on mobile puzzling that expanded our minds and got us thinking in thrilling new ways.

Download Moncage from the App Store

Mac App of the Year: MacFamilyTree

We loved documenting and visualizing our family history with this comprehensive genealogy app.

Download MacFamilyTree from the Mac App Store

Mac Game of the Year: Inscryption

The surprises never stop in this deep deck-building adventure.

Download Inscryption from the Mac App Store

Apple Watch App of the Year: Gentler Streak

A fitness tracker that encouraged us to tune in to our bodies—and take a rest when we needed.

Download Gentler Streak from the App Store

Apple TV App of the Year: ViX

A Spanish-language streamer that brought together the drama we love and the sports that get our heart pumping.

Download ViX from the App Store

Apple TV Game of the Year: El Hijo

We loved how this spirited stealth adventure values brains over brawn—and looks stunning on the big screen.

Download El Hijo from the App Store

Arcade Game of the Year: Wylde Flowers

With remarkably authentic diversity, this life sim’s idyllic world makes free expression (and casting spells) incredibly fun.

Download Wylde Flowers from Apple Arcade

Cultural Impact Winners:

Dot’s Home

Shining a spotlight on historical injustices through a compelling time-travel tale.

Download Dot’s Home

How We Feel

Helping us engage more deeply with our emotions—and providing strategies for addressing them in the moment.

Download How We Feel from the App Store

Inua

Honoring the heritage of the Inuit, whose folklore gives a breathtaking tale its beating heart.

Download Inua from the App Store

Locket Widget

Bringing friends and family closer by helping us see the small moments we otherwise might miss.

Download Locket Widget from the App Store

Waterllama

Encouraging everyone to stay hydrated through a winning combination of gentle guidance and adorable characters.

Download Waterllama from the App Store

Originally published on November 28, 2022 on the App Store Today tab.



Q&A: 10 Questions with Design Evangelism – Discover


For the inaugural edition of Ask Apple, members of our Design Evangelism team got together to answer your Slack questions about design philosophy, color guidelines, keyboard shortcuts, and much more. Here are a few highlights from that conversation, including guidance about the HIG, tips on reducing clutter, and a very important message about the tab bar.

Do you ever feel like your design isn’t quite right, but you’re not sure why?

All the time! In fact, “Feeling like the design isn’t quite right” can sometimes seem like an everyday mood. When this happens, there are a few strategies we find helpful, and the first is: Phone a friend! Sometimes it takes another person to gut-check why we’re feeling uncertain about a design, and it’s always great to engage in a conversation and critique. Plus, this not only requires you to explain the problem (which alone can help you identify what’s not working), it also allows you to personally step away from it — at least for a moment.

How do you know when to start cutting features to make your app less cluttered and more user-friendly?

This is a great exercise for a whiteboard or sticky notes. First, write down all the features/areas of your app. Then, bucket them into what fulfills the goals your users will have. If something feels superfluous, consider whether you need that feature. There’s a balance between what you need visible all the time, what can be a few taps away, and what doesn’t need to be there at all. (It’s also a helpful way to prioritize the most important functionality of your app — which can help you better organize your app’s hierarchy!)

Is is considered best practice to limit device orientation on iPhone?

You should really leave device orientation up to users. We love when apps support both portrait and landscape and would only recommend limiting orientation in certain app scenarios, such as when movement or device mounting would make orientation-switching feel distracting.

What are some guidelines for colors and shades?

Using color for actions is a subtle way to brand the interface without being distracting or intrusive. Start by selecting a main tint color, establishing your workflows and actions, then sketching those out with the tint color representing actions. (You’ll notice our first-party apps all have one key tint color; for instance, Mail is blue and Podcasts is purple.) When you’re working on high-fidelity visual designs, use a palette that complements that color.

Is it necessary to include tab bar labels for common tabs like Home, Search, or Profile?

In many cases, labels are recommended for clarity and accessibility. Home, Search, and Profile are generally sufficient for communicating meaning on their own, but they’re exceptions to the rule. Many icons are not as widely understood. Tab bar labels create a stronger distinction from toolbars, which don’t have labels. Plus, removing the labels doesn’t provide much benefit to users. It doesn’t save any space, nor does it significantly reduce visual information from the interface.

How should I think about keyboard shortcuts that feel intuitive and don’t interfere with system shortcuts? When should I use one modifier over another (for instance, Option vs. Shift vs. Command)?

In general, the go-to modifier key is Command because it’s easiest to reach with your left thumb. And speaking of, here are a few more rules of thumb:

  • The fewer modifier keys, the better.
  • Using the first letter of the action name helps people remember the shortcut.
  • From an ergonomic standpoint, keys nearest modifiers and easily reachable using one’s index and middle fingers — Q, W, E, A, S, D, O, and P — tend to be more successful as shortcuts.

Want to dive in a little deeper? You’ll find lots of good information in the HIG.

Human Interface Guidelines – Keyboards

How large are the specified margins for the iOS and iPadOS safe areas?

On iOS and iPadOS, the layout margins are 16pt in the compact width size class and 20pt in the regular width size class. But we typically think of safe areas as distinct things that work in tandem with margins. Safe areas are highly dynamic and change with device orientation, screen size, and a variety of other factors (like whether navigation bars, toolbars, or tab bars are displayed). You’ll find lots of information in the HIG.

Human Interface Guidelines – Layout

When designing for lists, how can I stop rows and cells from feeling overcrowded?

Think about progressive disclosure and hierarchy. What information do people need at each level of your app? When cells feel overcrowded, question the purpose of each element. Maybe a photo or icon isn’t beneficial, or maybe the secondary text can be a description on the detail view.

Do you always try to adhere as much as possible to the HIG, or do you try to do something different with every design?

Great question! We do try to adhere to our foundations and follow our own design patterns for the benefit of consistency and understanding. (We also try to use system components because they’re so efficient to build with.) But that being said, we’re also willing to push on the HIG where it makes sense — if it means an overall benefit for users. We’re pretty adamant that the HIG shouldn’t be a set of rules, but very good suggestions. And we evolve it constantly based on what we’re seeing in the community and how we want to push our own aesthetics and interactions for the future.

Is it good practice to hide the tab bar when navigating to sub-pages?

Nope. 🙂

(It’s OK to cover it for brief periods when a modal sheet is displayed. Otherwise, hiding a tab bar can make people feel lost.)