THE OMNI SHOW

Ken Case: What Apple has done is they have augmented, the features of your own device you already have on that device can now be processed in new ways. Just like when they added Spotlight support to the system so that you could search your whole system for text. Now they've added richer understanding of what that text means so that you can start to ask questions about it. And I don't know what's actually possible yet because what's in the beta right now is limited compared to what they showed in the keynote. Whenever the timing is some of the examples that came to mind for how people might use this or look at my email for the day and make a list of all of the actionable items in that email now that you have that list. So I think that was maybe even very close to one of the examples they already gave, but then I can imagine taking the next step and saying, "Okay, well now send that list to OmniFocus." And now I have a list in OmniFocus of all of the action items from today's email and that was just done with a few words with Siri.

Andrew J. Mason: You're listening to the Omni Show where we connect with the amazing communities surrounding the Omni Group's award-winning products. My name's Andrew J. Mason, and today we have our post WW DC roadmap update with Ken Case. Well, welcome everybody to this episode of the Omni Show. My name's Andrew J. Mason, and today we have the CEO of the Omni Group in the house with us actually visually here to talk about the latest WW DC blog posts wrap up. Ken, always awesome to have you with us. Thanks for joining us. Thanks for hanging out.

Ken Case: Thank you for having me here and it's great to talk with people.

Andrew J. Mason: Ken, I think a great first place to start would be this, the launch of the Apple Vision Pro and Vision OS and the Spatial Computing. Talk to me about that overall landscape. How's it going?

Ken Case: It's hard for me to believe that it's only been less than five months now since the Apple Vision Pro launched. It's now a device that I use daily. I use in all sorts of locations. Last week I went to an automobile appointment and as I was sitting there waiting, it was just an hour long or whatever, but I got out my laptop and I put on my Apple Vision Pro so I could have my big workspace and sit there working and got some work done and then when the appointment ended, I went up to the counter and was talking to the guy and he was like, "So were you using that to work?" I'd never thought of working in VR before. People have thought about using it for gaming of course, because that's how it's been used in the past. But the exciting thing to me about the Apple Vision Pro platform and introducing spatial computing on Vision OS is that we have an environment where I can use that to make myself much more productive in this portable environment. I can take my laptop, I can take my headset and I can go anywhere and be very productive with a nice large screen, lots of space for my different Xcode windows or whatever I'm working on at the time. And so it's been really fun for me to use and I hear stories from our customers who are really loving it as well. It is still super early, it's only just starting to roll out in Europe this month and in Japan and other locations anywhere outside of the US it hasn't been available until this month and it's also any new platform that is this dramatically new, just takes a while to launch. People were super excited when the IBM PC launched and it started getting up to the point where it had a sale a minute. I think that was its peak rate that they were very, very excited about. So a sale a minute, well what does that mean? So that means 60 sales an hour of the working day is what they were talking about. So 60 sales an hour, 40 hours a week. If I do the math on that, that'll come out to 52 weeks a year. So that's 124,000 units, right? 125,000 units of Pros. I don't know what the sales numbers are for the Vision Pro, but I think we're pretty much on that path already. Now. There might've been a peak where a bunch of people ordered it early. We're clearly in the early adopter phase and I don't expect this to take off in year one or two. I really am looking at the five to 10 year kind of timeframe, but for me, I think it is doing very well as a platform and part of what it needs to do to succeed in that longer timeframe is to have the app ecosystem that really makes the platform an exciting place to be working. And so of course some of that is us building our apps and we're excited to be a part of doing that, but there are plenty of other apps out there as well from other places and it's exciting to see just how many new apps are appearing all the time there on Vision OS, as well as how many apps we're compatible in the first place coming from the iPad. That's great. A lot of new platforms have never had that kind of advantage where there was a bunch of software that already worked, even if it wasn't designed for the platform. So that's given it a huge leg up as well as just a starting point. It's fun to see and it's fun to think about where this is going and how it's going. And I can wait, I am waiting, but I'm looking forward to that future.

Andrew J. Mason: I feel like that's a really concise summation of the spatial computing landscape Vision Pro over the last five months since we've found out what's happened here. Talk to me more about our specific suite of software, our apps since we've last talked.

Ken Case: Well, since the last time we had one of these roadmap updates back in February, at that time the Apple Vision Pro had just shipped and we had just shipped OmniPlan, our project management app, as one of the first launch day app titles to be on the platform. And so we were very excited about that. We were very excited to get our own devices so we could see for ourselves now how it was to use this and to interact with it. And of course to started on OmniPlan was our first universal app. OmniFocus was our second universal app, and so to get started with bringing our second universal app over to the Apple Vision Pro as well, OmniFocus 4 having just become universal right before the holidays when we shipped version 4.0 and then 4.01 and 4.02, 4.03, 4.04 and 4.05 quickly, following up with, based on the feedback we were getting, here's some things that we can improve about the app. So that was where we were last time and we did then bring OmniFocus over to the Apple Vision Pro. That was a lot of fun. We learned some things along the way as we got these devices. I think Sal was the first person to really notice we have this great library of installable plugins and he wanted to test what happens now when you use your voice to control OmniFocus in this environment where you don't necessarily have a keyboard, often won't have a keyboard board? And so we started looking at how these hundred-plus plugins might be installed and part of the process, the way it worked before was you would go to a web browser, you'd see it on a page, you'd hit the download button, then on the Apple Vision Pro, you'd have to go find where that download landed and then drag it over onto the app. It was all possible, but if you're trying to install dozens of these plugins, that was a tedious process. So came up with the idea of is there a simpler thing we can do? We collaborated with Greg about what is that process that we could do, Greg being an engineer working on that code, and we came up with the look tap and approve process for installing those plugins now where in the Apple Vision Pro, you would just look at the plugin you wanted to install on the webpage and you'd tap on it and it would automatically open the right app with the right details, it would unpack it automatically and then just ask you to review and approve the thing. So that really streamlined that and we had that then ready to go when we launched OmniFocus as our second universal app for the Apple Vision Pro six weeks after the platform launched.

Andrew J. Mason: Yeah, let me say too, there's a little bit of, we always talk about skating to where the puck is headed and there's also this sense in which, "Oh man, the really the right solution we were looking for doesn't necessarily exist anywhere that we see." This is what I love about our company's culture where it's, "Well I guess maybe we'll just build this workflow ourselves from scratch." And I think that's one of the things that makes the Omni Group so respected in the community for such a long time is this willingness to say, "Well, we'll just forge this little slice of the unseen territory ourselves." I think that's really cool. So a lot of this wasn't even on the radar for the roadmap to begin with. So we have these things come out and what from there, where do we take it from there?

Ken Case: As we made that process better and easier, of course it was important to do, I guess Vision was alerted us to the problem, but we realized that optimization really improves all of the platforms at once. And so we brought that code back to the Mac to iPad, iPhone, and now you can use the exact same process on any of them where you go to that webpage, you might not be looking exactly, you do look of course with your eyes at the right spot, but you tap on or click on the link and it just opens directly in the app. The app does the in backing asks you to approve whether you actually want to install this thing so we don't have webpages spitting malware into people's Omni apps. And then once the approval is done, it's installed and that process became much easier and it's great to be able to share that code and bring it from the new platform where we start just being exposed to new ways of doing things over to all the rest of the platforms where that same concept just makes sense for everything.

Andrew J. Mason: It really does and it's so cool to see, as you were talking about Swift the last year, two or three ago, hey, we've got the code, let's bring it to everything and maybe some tweaks here and there, but it's just starting to really see the fruit of accelerating that process. I think that's really cool. Now this year's WW DC was a big one, Ken and I honestly feel slightly jealous because I hear you were actually able to be there in person with people. What was the overall feeling, the overall vibe, what was the overall sentiment from everybody this year?

Ken Case: Oh, you bet. It was a blast to be there. It was great to see people, some people I hadn't seen since before the pandemic, some people hadn't seen even in 10 years or more. It just had been a while since we crossed paths and WW DC has always been a good gathering place for the community to get together and talk about things and of course what Apple's news is for the week is one of the big things to talk about, but it's also just great to see everyone and get a chance to do things together and have some meals together and so on.

Andrew J. Mason: Absolutely. And that reconnection, that really is the community aspect of it. It's like, "Hey, you're a person in real life, so am I. It's so great to see you." But during the day one keynote, there was a moment of one more thing, it was anticipated and excited by folks. Do you mind sharing what it was and then also your thoughts surrounding seeing that happen?

Ken Case: Sure. So in the first half of the keynote, apple was very careful not to use the term AI anymore. They talked about machine learning. Apple has been doing work in this space for a long time now for decades really, but they were being really cautious about how they referred to it and trying to use the more accurate terminology like machine learning rather than just saying AI, which means a lot of things to a lot of different people and can sometimes just sound like magic, the latest magic fairy dust and often is being used or misused by companies just to try to make their stock go up. They think if they sprinkle little AI on it then people will say, "Oh, this is an AI company and this is going to be amazing and we better jump in on that." Whereas Apple is really looking at the long big picture and they've been working on these problems for decades, literally decades. When we came over from NeXT to the platform for Macs OS 10, they had things in there like doing summaries of text and speech recognition and text to speech. All of those things were considered AI at the time because that is a moving term that always means what's beyond the part that has already become routine for us. Those sorts of references were in the first half of the keynote and they talked about different things that you could do that might involve some of those technologies, but they didn't necessarily emphasize them a lot. And then we got to the second half of the keynote where they talked about Apple intelligence and that's their version of AI or their way to reference AI. Instead of talking about this very generic artificial intelligence acronym that the industry tends to jump on, they got very specific about what Apple intelligence means to them, what are the principles involved? And the most important principle from my point of view that came up during that was their emphasis on privacy. And of course we've always known this about Apple, right, that privacy is an important part of their design considerations as they build their products and build their services. But it was great to hear Apple emphasized that again and talk about how information that is being processed now with Apple intelligence is data that is already on your device when you're not collecting some new data from somewhere and you're not sending new data to some other party to go process and you don't have to worry about trusting that they are doing safe things with that data and not just using it to try to collect more information about you that they might use in ways that you don't want them to use.

Andrew J. Mason: I got to watch a little bit of this actually. No, I didn't get to watch this keynote but was able to see some of the examples they had that were presumably operating system-wide for I think the example is iOS where they asked a question and the question was able to draw from a person's specific context and infer meaning that was happening with just that one person's specific context, yet still keep privacy in mind because all of it was happening on device and they weren't keeping any of that personal data. That's really, really interesting to me and cool. What other things do you see that hey, this would enable maybe at an operating system level us to do or we will be able to do, I can say for sure at an operating system level before we start to maybe move into speculation for what that means for us? But maybe at the operating system level, what are some things that you see that you're like, "Hey, this is cool as an end result, we'll be able to do these things?"

Ken Case: Sure. Well, I think it's important to keep in mind that what this is about is not some external service that is operating on your data the way so many of these other, and the reason it's important to distinguish is because so many other approaches in the past have been that way where you would collect your data and you would send it to somebody else's computer and then that computer would do the work on it and then send back some results. And in the meantime that somebody else's computer has your data and who knows what they've done with it and how secure they've kept it and so on. What Apple has done is they have augmented the features of your own device. The data that you already have on that device can now be processed in new ways. Just like when they added spotlight support to the system so that you could search your whole system for text. Now they've added richer understanding of what that text means so that you can start to ask questions about it. And as I started thinking about this in the keynote, I don't know what's actually possible yet because what's in the beta right now is limited compared to what they showed in the keynote and I'm sure that more of this will be coming to future releases throughout the summer. And I think they talked about some of the features maybe being released by the end of the year, so I'm not sure they even will be launched just in the fall. But whenever the timing is, I think some of the examples that came to mind for how people might use this are look at my email for the day and make a list of all of the actionable items in that email and now that you have that list. So I think that was maybe even very close to one of the examples I already gave. But then I can imagine taking the next step and saying, "Okay, well now send that list to OmniFocus," and now I have a list in OmniFocus of all of the action items from today's email. And that was just done with a few words with Siri and that all seems pretty reasonable based on the current level of technology with large language models, being able to understand what part of the text is important, how to summarize it and so on. On the one hand it feels like magic and then on the other hand it feels very realistic for where the technology's at now. Or there might be another example, well why don't you find all of the people who are involved in sending those email messages to me today and then look them up in my contact database and figure out their relationships and then build me an org chart so I can see how all of those people fit together and who I'm talking to about what. And you would then leverage OmniGraffle to build that org chart for you. And so I can imagine that being possible. I don't know if it will be possible and part of the question for me is what do we need to do in OmniGraffle to make that possible? Assuming the language models that they're providing in the system are capable of understanding those questions and drawing the information from contacts and mail and so on, do we have to have a verb in OmniGraffle, the verb, their name for that is app intents? Do they have to have app intents defined for OmniGraffle that say build an org chart from this list of items with these connections or at what level do we define that so that Siri would be able to understand it and then do that task? Apple intelligent Siri, not the current Siri.

Andrew J. Mason: So it is in some ways very difficult to say, "Hey, this is exactly what we hope to accomplish in the coming months," because at one level we don't necessarily have it yet to mess with, but we know it's coming. And then on another level, what would you say to folks that are interested in being at that bleeding edge? Like, "Hey, I know that something's coming with the Omni Group. I love getting to use the software," and if there's any way that I can help inform and be a part of the process, do you have anything set up that can maybe help folks along in that slice of the journey with us?

Ken Case: One of the things we will do this summer is set up what I'm calling our summer exploration test flights where you'll be able to sign up for a test flight build and we'll have some of these app intents now defined in the apps and you can start playing around with them. The app intents fit to be clear that Apple is using for this Apple intelligence aren't new to this release, actually were introduced last year and we have some in OmniFocus already for things like changing the focus of the app that we just shipped earlier this month in OmniFocus 4.3. So based on the device focus, changing the app would then change what it's showing. And so we'll take some of our other shortcut actions that we've had in the app for years now, things like add an item to OmniFocus and so on, turn those into app intents as well so that then it's easier for Siri to invoke them and shortcuts can still work with them. Shortcuts, see any of these app intents and that's a way to talk to the app, but I imagine that we will end up creating a lot more app intents than we have now. I just don't know what they all look like. And I would of course love to hear from people about what app intents do you think are missing that you wish the app had so that you could use them in shortcuts so that you can use them as you build these workflows with Apple intelligence.

Andrew J. Mason: I know some of this feels like waiting for the stars to align in the right way before A, we can actually get started on some of the things that are coming down the pipeline, but also B, just to know what it is we should be working on. Is there anything that you are yourself, just on a personal level are particularly excited about the idea of happening if everything lines up the way that you hope it does? So there's a lot of contingencies here, there's a lot of speculation here. Let me just bookend this by saying none of this is a promised feature or idea or thought or a promise anything, but just really having a conversation about what it is that you're excited about if it were possible to do, is there anything that really fits that category for you in a specific space within our apps?

Ken Case: I don't know about everybody else, but my email inboxes are constantly overflowing. There's just so much noise coming in from so many places and part of this is that I've had the same email address now since what, 1992. It's always been publicly available and so I've constantly getting stuff in there and I have to filter and figure out what's actionable out of there and so on. If the computer that already has all this email on it can do a better job of helping me cut through all of that noise and figure out what's important to pay attention to today, then that seems like a really great time saver for me. On the other hand, I do worry, of course that's just an open inbox. Maybe people start phrasing their email messages to me in a way that says, "Oh, this is very actionable. Please make sure it's at the top of the list," and how is that going to attract people to adapt to these sorts of filters and try to work their way around them just like they work around the various language model problems now to get crazy results. But one of the things that Apple has been super careful about is that what they're doing is not ... I think they've steered away from some of the directions that the other parts of the industry are going that would allow you to get crazy language results. There's nothing in there that lets you ask it to write a bunch of instructions or a bunch of texts for you. So they dodge a whole class of problems where the computer's just making stuff up by saying instead, "Oh, I want you to summarize this or expand this," and it's your writing involved in the first place and you're the source, not the internet at large. Or similarly with their photo generation, which I might certainly want to use an OmniGraffle. I might be making a diagram and I might want to say, "Well, could you add a sketch of a server farm here in this part of the diagram," and instead of having to go find a stencil for that, it just goes ahead and generates one for me on the fly. That seems like a great thing for somebody who's not an artist to have at their disposal, but you can't say, "Well, could you make me a picture of Ken stealing something from the grocery store?" Because they don't make those sketches.

Andrew J. Mason: Why is that the first thing you think of?

Ken Case: Yeah, it doesn't do deep fake type images. It does stuff that is recognizably in one of these three styles that they allow and because of that it's really clear that what you got was not a photo, it was an artificial sketch or something and we're already used to sketches in our world and it's not a big deal to understand that a sketch is not necessarily a reflection of reality.

Andrew J. Mason: Talk to me about, there also comes with that, I've heard privacy concerns. People are worried about context specific things being shared in a larger space where their privacy isn't being respected. What's Apple's approach to that? What's their thoughts surrounding that? Can you give us a little bit of a breakdown as to what Apple had to say about AI in regards to privacy?

Ken Case: Yeah, absolutely. Well, I should first note that that is part of why we have not really been dabbling in the AI solutions that so many third parties have been providing over the last a few years. This stuff has been interesting, but a lot of the solutions that people are marketing at the moment, have been marketing, are around sending your data to their computer and having it do some work on it and you don't know how long they're retaining that data. They might have given some information supposedly in their end user license agreements around how long they keep the data, but even so that's not the same as guarantees that it's not possible for somebody there to have access to that data in the meantime. So for us, that was just a no go. We didn't want our apps sending our customers data to services that we can't audit. We don't know how safe they are for that data. They just seem like a big risk. The difference with Apple intelligence is that again, they are focused first of all on doing as much as possible on your local device and so there they don't have to send the data anywhere. It's data that's already on the device. It's just processing it in new ways and that's great. Occasionally you'll run into problems where it needs to bring more data to ... You don't want to wait forever for an answer and you don't want your phone to run out of battery while you're trying to compute the answer and so on. You want to send it off to a bigger CPU than you might have in your pocket or on your wrist if you were doing this on the Apple Watch, which to be clear, I don't think they announced that it's on the Apple Watch yet, but I have to assume that this is coming, right, that this is part of the plan. So Apple devised the solution that they call private cloud compute, and so they came up with a solution that basically lets you ... They've devised custom silicon. They've already done it on all of our local devices. We've had the Apple Neural Engine in our iPhone since 2017 and then in the Mac since the M1 was introduced in 2020. But these servers now in their farm have lots of neural engine capacity going on and they can work on these problems that would be much harder and slower to do on your local device. And so what Apple has done is they've worked very hard to make sure that you can trust that server just as much as you trust your local device not to be sending data somewhere it's not supposed to. So the data is encrypted when it goes to that server. It's never so you can't tap into it just from the communication channel. Once it gets in there, the system itself doesn't allow any sort of remote access from Apple's maintenance folks or anything else. The data systems are also encrypted and so on. So they're building this whole system as a system that you can feel confident is not doing anything shady with your data, but they didn't just stop there. They also made sure that it was auditable by outside experts, much like they make the images for the iPhone operating system auditable by outside experts and so they're providing them with an image of this computing environment that then those outside experts can poke and prod at and figure out, "Okay, is there any sort of vulnerability here that would let them get at the data?" It's funny, they used a terminology technique that has had its own baggage around the language, which is building things into a blockchain so that when they upload a new version of the software up to those servers, it has to be signed with a key that's on the blockchain so that anybody who is auditing is guaranteed that what got put up there on that server is exactly what they got to audit. It's not another slightly different copy that has some different vulnerabilities or capabilities in it. And so as a result, the software that the compute engine is running, the private cloud compute is running is something that is audit audible is something that you can trust Apple can't insert some backdoor into get your data and they can't be compelled or asked to insert a backdoor into by some third party. So it's a really big difference from how so many other companies are doing AI.

Andrew J. Mason: That's really relieving when you hear somebody say it's the difference between like, "Oh, we won't do anything with your data," versus, "We can't. We literally can't do anything even if we wanted to." It just is another layer of comfort for me as an end user knowing, okay, it's virtually impossible for somebody to get to that.

Ken Case: So the other scenario that can happen is that you might ask a question that Apple Intelligence just doesn't know how to answer. It's out of its space. You might be asking a question that they talked about basically handing things off to ChatGPT because ChatGPT knows a lot of information what has been trained on the internet. Now it will make things up. It's got a whole different set of parameters around how to use that information. And so they wanted to be really clear about when you're sending things to ChatGPT, what you're sending and that the answer that's coming back is from ChatGPT. It's not something that Apple Intelligence is telling you because what you're getting back from Apple Intelligence I think is you can feel a lot more comfortable relying on than you might rely on some other third party services. It's not just ChatGPT. I think it sounds like there will be hooks that can go to other services as well, but you don't want to be told to put glue on your pizza or whatever. So as you ask a question it can say, "Oh, I don't know how to answer that. But I think ChatGPT might, shall I send this question along." They just put a really clear dividing line there that says when your data is going out to one of these outside services that you might not trust in the same way that you trust Apple Intelligence and then that what came back again might not be as trustworthy.

Andrew J. Mason: Thank you again for spending this time with us, Ken. So grateful and just honored to have you with us. How can folks get in touch with us regarding some of the things that we were talking about today and find out more information?

Ken Case: Sure. Well of course you can send us email. All of our apps have contact Omni in the help menu or somewhere easy to find in the app. So that's one way to reach us and we try to answer every such email message booth in one business day. But you can also reach me personally at kc@omnigroup.com. That's my email address, the one that I talked about being published for decades now. So if that gets overloaded, which it can, and I apologize in advance, if you reach out to me that way and you don't hear back from me, it gets lost in the noise. You might also try reaching out to me on a social media channel like Mastodon. I'm on Mastodon.social as KCase. So KCase at Mastodon.social.

Andrew J. Mason: Ken, thank you so much. That's great.

Ken Case: Thank you.

Andrew J. Mason: Hey, and thank all of you for listening today too. You can find us on Mastodon @TheOmniShow@omnigroup.com. You can also find out everything that's happening with The Omni Group at omnigroup.com/blog.