Alyssa utilized OmniGraffle to create the brilliantly informative and aesthetically beautiful Path-to Newton project which visualizes the prevailing thought patterns and discoveries leading up to Sir Isaac Newton's discovery.
You can find Alyssa on the web at her Harvard page.
Some other people, places, and things mentioned:
Andrew J. Mason: You're listening to the Omni Show. Get to know the people in the stories behind the Omni Group's award-winning productivity apps for Mac and iOS. My name's Andrew J. Mason and today we're talking with Harvard professor, Alyssa Goodman about her use of OmniGraffle.
Andrew J. Mason: Welcome everybody back to the Omni Show. Today, we have Alyssa Goodman, who is a professor of applied astronomy at Harvard University. She's also the co-director for science at the Radcliffe Institute for Advanced Study, and she's a research associate for the Smithsonian Institute, and from what I hear, a very smart individual. Alyssa, thank you for joining us today.
Alyssa Goodman: Thank you very much. I'm excited to hear what we're going to talk about.
Andrew J. Mason: For our audience, one of the reasons we asked Alyssa to join us on the show is because she has developed this phenomenal, phenomenal data visualization, including Isaac Newton and how he came to his discovery, including many, many significant data points. And it could be so complex and it has so many data points, but it's so beautifully designed and aesthetically pleasing, that the right information just kind of pops out to you. So this massive project with many data points, all designed in OmniGraffle and so beautiful and we wanted to talk about that.
Andrew J. Mason: But before we head there, Alyssa, do you mind sharing with people how you kind of got to where you are teaching at Harvard and kind of in your current field of study?
Alyssa Goodman: Sure. I started out life in physics. All my degrees are in physics. I wound up in astrophysics through climate change. We can talk about that later if you want. But anyway, I became a professor in 1992 in astronomy at Harvard and yeah, I'm much older than I look. Anyway, so I came to Harvard in 1992, joined the faculty and I had a lot of interests in data visualization. And our dean in the 90s was Jeremy Knowles who found out about these interests and encouraged me to pursue them.
Alyssa Goodman: And ultimately, I started teaching data visualization at Harvard and made a lot of friends in that world. And we started a project called astronomical medicine, where we were using medical imaging software to look at high dimensional astronomy data. And that's turned into something called Glue, which is a whole other story, not what we're talking about today, but it means I have a sort of side career in data visualization, as well as astrophysics.
Alyssa Goodman: But what does that have to do with the Path to Newton or prediction, which is what I think we want to talk about? Well, the answer is I mentioned that I got to astrophysics through climate change and that's because I was an intern at Columbia, NASA Goddard at Columbia University in 1983, with none other than Jim Hanson. James Hanson is the person most famous for sounding the alarm about climate change simulations way back in the 80s, being something that we should pay attention to. And so I started being very interested in computer simulations and climate change and worried about climate change.
Alyssa Goodman: And again, what does that have to do with the Path to Newton or prediction? The answer is, in 2008, when I was done with a job leading a Scientific Computing Institute at Harvard, I took a sabbatical at WGBH, a public television station in Boston. And they asked while I was there, what I thought the most important thing that the public doesn't know or appreciate about science is, and would I like to work on that. And I said, "Oh, that's easy. The value of computer simulation." And they said, "Tell us more." And I said, "Well, the future of the world depends on the joint action of climate simulation, financial simulation, global health simulation, something that's very relevant right now. And people don't even really know what simulation is and they certainly don't teach it in school."
Alyssa Goodman: So my very smart colleagues there said, because I also used the word modeling and they said, "Do you mean 3D models? 3D printing?" I said, "No, no, no, that's not what I mean. I mean, forecasting, weather forecasting, but for climate, for the economy." "Oh," they said. And so over the course of that year, which was 2008 to 2009, might notice that the economy collapsed, there was a swine flu epidemic, oh and climate change kept going.
Alyssa Goodman: So we eventually designed a project that year, that morphed years later into what's now called the Prediction Project to explain the value of modern predictions, where you can associate uncertainty with them and numerical data driven prediction and where that came from, historically. And so we go all the way back to Mesopotamian sheep entrails and priests reading entrails on the ground.
Alyssa Goodman: And in between Mesopotamian sheep entrails and modern climate change simulation is a long time and we wanted to highlight kind of where you go from just kind of having observations and data about the universe and speculating about it, to having real predictive, mathematically instantiated theories, like the theory of gravity. And so Newton's theory of gravity in all of science, is probably the most robust and predictive theory that we have. So we decided to trace the history of that back to its origins and that's where the Path to Newton came from. And I think you have some specific questions about that.
Alyssa Goodman: But the Path to Newton is part of this giant prediction project, which arguably started a little over a decade ago and now consumes, I don't know about a third of my research life. And what we do, is we have a big online site where people can learn about prediction. We have formal online courses through edX and LabXchange, and we have two flavors of prediction classes at Harvard. One is a freshman seminar and one is a so called general education class, which is a broad class for undergraduates.
Andrew J. Mason: Okay, great. So we see this love of data visualization, all of these details, starting to work together. And can you talk to us a little bit about how does this lead you to the Path to Newton?
Alyssa Goodman: Okay. Sure. So basically, I have a bad memory and I'm not a historian. And luckily, my daughter is a historian, so she's taught me some things about how to think about history and how not to impose your own ideas and collect information, but really I have a terrible memory. And so we were trying to collect information in a narrative format about, if you started with Newton and you went backwards, what is it that Newton had to know in order to create his predictive theory of gravity? Now he did also invent calculus on his own in order to make that work, but that's a side point. Let's just say you get to the birth of Newton, what was known that was useful to Newton?
Alyssa Goodman: And there's a famous quote attributed to Isaac Newton about how he stood on the shoulders of giants. But it turns out that he actually made that statement if he made it at all, ironically about Robert Hook and was a dig. But let's pretend that it wasn't ironic. The idea was to figure out who those giants and what the giant ideas were and to see how far back we could go.
Alyssa Goodman: And ultimately, two diagrams were created. One is the Path to Newton and I'll explain how OmniGraffle played into that. And another one is something that we now call the Padua Rainbow, which I should also show to you and you don't need any fancy graphical software to make or see that. But it basically shows how you go from a world where you just kind of start noticing things around you, to where you record observations, to where you actually sort of make rules about the way the world works, to where you make real predictive theories and explanations. And so that's a sort of separate conversation.
Alyssa Goodman: But you asked what the elements are of the Path to Newton, and the answer is we were trying to find those kinds of events. Where did the observations come from? Where did people start writing them down? Where do they make rules about them? Were these ideas and rules and data, was it right? Was it wrong? Was it helpful? Was it unhelpful?
Alyssa Goodman: If you go back to ancient Mesopotamia, if I look here at the Path to Newton, the oldest date we have on here is 2000 BC. So, if you go 4,000 years of the history of the kind of physics that leads to Newton's theory of gravity, it's kind of hard to remember what happened, no matter how good a writer you are. And so we worked with some writers and it just wasn't going really well.
Alyssa Goodman: And so one day, my colleagues at HarvardX, which is the part of Harvard that makes content for edX, they basically dared me to show them what I meant, in terms of how this history of ideas worked. And so I can give you for your website, if you want, the whiteboard diagram that I drew almost from memory after doing a bunch of reading and research about this history. I drew a bunch of bubbles on a whiteboard and a bunch of arrows and a bunch of this person was important. And every once in a while, I'd have to look up a date on Wikipedia, because like I told you, my memory is really bad. When did Ptolemy actually live? How do those circles go again?
Alyssa Goodman: And so I made an exception rudimentary version of this, what you see now as the Path to Newton, and the huge swats of it were missing. For example, I didn't know much about India and Indian mathematics, I just knew it was important. And I only knew about sort of Avicenna center or Avicenna, depending on how you say that. I knew a little bit about the history of Islamic science as it related to medicine, but I didn't know much about physics. And so anyway, there were big blank pieces, okay? But all this stuff about Aristotle or whatever, eventually connected to all this stuff about Copernicus. So it was very Western centric, Western thought centric, which is terrible.
Alyssa Goodman: But anyway, so then we gave this diagram, which they were kind of surprised I could draw. So they saw that I wasn't sort of blowing smoke, to a writer, to a professional writer and he tried to write something which was just so over simplified in order to be possibly followed, that it wasn't bad, it just didn't capture all the information.
Alyssa Goodman: So, I went on sabbatical in 2017 and I was at a beautiful apartment in Vienna, Austria. And I'm sitting there with a bunch of history of science articles and with this article that the writer wrote and with OmniGraffle, knowing that I had OmniGraffle for other projects that I'd used it for in small ways. And I started just for myself, trying to build something in OmniGraffle so that I could understand A implied B and B implied C, but C doesn't really imply D and C kind of went to E, but then E came back to D. And it was so not linear that you really needed OmniGraffle or something that could connect a lot of dots.
Alyssa Goodman: But then what you also needed was the ability to kind of filter this information and to turn layers on and off, like, let me see all the ideas that were right, or let me see all of the ideas that were wrong. Or let me see what the Greeks did compared with what the Egyptians did. And so I used the ability to have layers and grouping and art boards and all that kind of stuff in OmniGraffle to experiment with filtering.
Alyssa Goodman: And it turns out that, that filtering, we eventually wanted to offer in the online tool that you see now, but we worked with a graphic designer to really fix up what I had done. So her name is Katie Peek. And so what I had done, some scientists might think look nice, but nowhere near as nice as what Katie Peek has created, the beautiful pink and green and brown and et cetera diagram that you see now.
Alyssa Goodman: So, but it turns out that all of that grouping and that filtering kind of capability of OmniGraffle was really useful to me in sorting out the information for myself. But then it turned out a really good graphic designer could use symbols about colors for which ideas were right and shapes for which were ideas and which were books later on, so that not as much filtering was actually necessary.
Alyssa Goodman: There were elements of the project that really are kind of a timeline. And so if you look at the top, there's stuff about the history of mathematics and the history of instruments and how do different instruments play into the story. And those are just kind of slightly straightforward timelines and we just use bars to show them.
Alyssa Goodman: But it's also interesting that in that same year, 2016, '17, I was on this full sabbatical and I was hanging out with a lot of humanities people at Radcliffe, and I was, and still am involved in something called the timeline consortium, where we're building a standard for time tag data, thankfully with support from the Sloan Foundation and there's a lot of big organizations that you've heard of, who were involved in using this timeline standard. And we were really talking about comparing linear series of events. You have the history of Mayan culture and the history of Greek culture, and they didn't happen at the same time, but you want to be able to have two timelines and slide them and be able to compare them and scale them.
Alyssa Goodman: And so we thought that was already pretty flexible trying to do that, but the historians said, "Well, wait a minute, what about more of a kind of mental map of a history of ideas? And then you need these connections between timelines that happened simultaneously in different parts of the world." And I was like, "Oh my God, we can't build that into the standard."
Alyssa Goodman: But what's really interesting is that the way that the Path to Newton came out, it's not a timeline. It's exactly what these historians were asking for. It's a kind of logical history of ideas that sort of has a timeline component, but doesn't. And so it's funny because I'm working now with people at Eon Timeline in Australia, as part of this timeline consortium, who are using the standard. And I'm thinking that version two of our prototype timeline software should integrate with OmniGraffle and actually let you have this kind of history of ideas, be a kind of very, very sophisticated timeline. So I'll have to talk to Kent about that.
Andrew J. Mason: Poor Kent. I feel like every single show I've been doing from here on out has somewhere embedded had a product feature or request put into it. But what's amazing about this document, is how you achieved this marriage between a beautiful display of sequential information. I mean, there's a wonderfully displayed timeline, but there's also this associative component to it, where you're jumping in a way that makes sense to the brain. There's so many data points there, but your mind just doesn't get overwhelmed by seeing it all. It makes sense when you look at it.
Alyssa Goodman: Anyway, what we decided to do was link from the diagram into places in the document, and then to link out from the document to deeper, richer information, rather than sending people from the Path to Newton out, mostly not directly to the web. The exception to that is, at the bottom, there are a bunch of books, historical books, and Harvard's library is trying to be very generous and they are in the process of scanning, they've gotten quite far, all of the books that Harvard owns that are on the Path to Newton, and making them publicly accessible in an incredible viewer that lets you zoom in and out and see the ink stains on the page. And also, they've done character recognition on all of them. So now there's this very rich resource for historians of science and for people who are just interested, where they can get to the original manuscripts, going back to the 1400s.
Andrew J. Mason: What advice might you have to somebody who's just getting started in this idea of data visualization? OmniGraffle is an amazing tool; it has so much capability. But if you don't know necessarily where even to begin with visualizing data or how to visualize data, this field is changing all the time. Like you mentioned, when you first got started there 1992, what's changed since then? There wasn't even an iPhone or commercial internet; it was just getting started. So it keeps moving all the time. Do you have any tips for people that are just kind of getting started?
Alyssa Goodman: We have a blog, actually we can plug our blog. We have a blog called 10QVis.org; that stands for 10 questions to ask when you want to create a visualization. And so we have a bunch of examples there, and we have a bunch of categories for people who are both new and not new, to think about it, to think about whether they have addressed those questions.
Alyssa Goodman: The one thing that's important about visualization, is that it's not an exact science. And a lot of people who come to me and want advice on visualization, want me to give them some kind of prescription. And it's kind of like the difference, this is an analogy I can give you that people might understand.
Alyssa Goodman: There are two kinds of people who use computers; two general types. The ones who want the instruction manual, and like, "Where do I click? And what do I do? And what is the order of things that I must do?" And then there's other people who just kind of mess around. And if you're going to do, I don't know, accounting, it's really good to be the first kind of person you don't want to be messing around. You want to follow some rigorous system, right? But if you're going to do science or if you're going to do data visualization, you really need a hybrid of both of those things. And in data visualization, it's even more the latter, the kind of mess arounds.
Alyssa Goodman: Some people comes to me like, "What color should I make this line?" And it's like, "Well, that depends on what you do with the rest of it." And so there is some sort of golden rules that you get from people like Edward Tufte, things like a data ink ratio, and you don't want to have a lot of ink compared to the amount of information; ink being pixels that aren't blank. The information you're showing, you don't want to have extra lines on a page because they make your brain vibrate and show you things that you don't want to be focusing on.
Alyssa Goodman: And so there are cute little kind of cognition tricks, and I don't mean to it in a demeaning way, I mean, in a easy to remember way. And for those, I would really recommend Edward Tufte's work. Our blog, the 10QVis blog. There's a little bit more directed toward people who are making visualizations that involve large amounts, or not even large amounts, but some amount of real data. So less information visualization and more quantitative data visualization. And in the visualization world, it's called scientific visualization.
Alyssa Goodman: But one thing that's interesting, is that people tend to separate words and pictures. And so I just use three terms that get used in the visualization community that I think should be banned as separate things, and that community itself is coming around to not doing this, but they used to treat information visualization separate from data visualization, separate from scientific visualization. Okay. And so the kind of thing that the Path to Newton represents would be information visualization, information graphics. An XY graph would be data visualization. A 3D plot of something, like a 3D view of a tumor or something in a medical scenario, that would be scientific visualization.
Alyssa Goodman: But the important part is that software like glue, like what we're trying to do and what you see and also the Path to Newton, is not separate those three parts; the sort of text data and spatial visual. We don't want to separate those parts of people's brains. We want to let them use them all together. And so that's why the Path to Newton has a textual narrative and a very interactive graphic, where you can kind of remember where you were in the chart, you can see the chart. But then you can also just directly go back and forth. So you can click on the text and go to the chart, click on the chart, go to the text.
Alyssa Goodman: And what glue does, just because you asked, is it does that, but also includes sophisticated kinds of data visualization. So graphs, and plots, and maps, and charts and things that people think about as let's just say, very sophisticated versions of the kinds of graphs you could make in Excel. Or if the listeners are familiar with Tableau, it's like Tableau, but it understands 3D volumetric data and arbitrary 2D maps and things like that.
Alyssa Goodman: So we could actually make a version of the Path of Newton in glue, but that would be serious overkill. We have not done that. But anyway, I guess the sort of shtick here is to not separate graphical views of things from narrative discussions or word-based discussions from data driven displays of information. And the only data aspect of the Path to Newton, is the fact that it's a timeline, or it's like a timeline; there is a sequence of real times.
Andrew J. Mason: Well, Alyssa, after knowing you for approximately 20 to 30 minutes, right now, as far as I can tell, you are the person that ranks at the top of the list of people I know, in terms of knowledge of data visualization. So I have a question about, what do you see at the forefront of this field? What's coming down the pipeline in the next 5 to 10 years. Do you have an inkling as to what that could be, what the field looks like?
Alyssa Goodman: Yes. And we're working on it right now, actually. And I think not enough people are working on it and so I do hope a lot of people hear this, okay? So in the gaming industry, there are amazing, very fast 3D graphics and there are controllers to let people move around in three dimensional environments. In the kind of visualization that's important to surgery and to understanding high dimensional data in other nonmedical sciences, for example, in my astronomy work, we studied the structure of the galaxy. And if people want an example of what this 3D visualization looks like, they can go Google the Radcliffe Wave, which is some work we published earlier this year that got a lot of publicity, so it's easy to find the interactive graphics that go with it.
Alyssa Goodman: But anyway, what we did there, is look at data in 3D and learned something new about the structure of the galaxy and we're doing that more and more. And clearly, I mean, you certainly can think about surgical applications for this kind of thing. And so what does this have to do with gaming? Well, you might've seen for example, that the HoloLens from Microsoft gets used to kind of train surgeons with virtual surgery or even gets to overlay information. So mix augmented reality, right? So mix the real world that you see with projected information in 3D and tell the surgeon where to cut and things like that.
Alyssa Goodman: And that's great and very high tech and has specialized applications. And we own a HoloLens, We have an Oculus, we have all kinds of things, but the device that makes me happiest these days, is called a Merge Cube and it's just a rubbery cube that has different patterns painted on the side. And what is this for? It's for, if you want to show, and I'll use the medical example, because it's easier for people to understand than the galaxy or the solar system. Well, actually no, let's talk about the solar system because it's less gross, okay?
Alyssa Goodman: So let's say, and I can actually show you this now, but it's a podcast, so I won't. But I have my phone and I have this cube in my hand. And on my phone, I have an app which will project virtually the solar system onto the cube that I'm holding in my hand. So I look at my phone or my iPad screen, and I make sure that the camera of the device sees the cube. And then I can project the solar system so that the solar's the cube in the view of my camera. So I see my whole office and this cube turns into a three dimensional moving model of the solar system. And then I tap a button on the phone to say, "Stamp that there, leave that there," and then I don't even need the cube anymore. And I can just move the phone around it as if I have this mixture, this so-called augmented reality of a world of data and a world of the actual world around me.
Alyssa Goodman: And so everybody knows how to move a phone around in three dimensions or move an iPad. Yes, you can do this with goggles as well, but you don't really need the goggles to do it. And so, one of the things we're doing is for example, we're adding to glue the ability to just tear off a window that has your beautiful 3D data in it and have it basically floating in the room and you can walk around it.
Alyssa Goodman: And then ultimately, we want to be able to make it so that people can make selections of regions within that data and can manipulate it in very sophisticated ways. And that's where we are looking at things like game controllers and gesture based controls. And so, a lot of people have seen Minority Report and so, some combination of voice control and gesture control.
Alyssa Goodman: But it turns out, we've been looking into this for a decade. And one thing we know, is that we need something that people are willing to do that doesn't involve a lot of special equipment or a lot of money or going to another room. And I think that game controllers might work for some people, but things like a decorative cube that costs $20 that you leave on your desk is probably more likely to work with somebody's phone and their iPad.
Alyssa Goodman: And so I think that's a very long winded answer to your question, but the short version is we have to make more use of the high dimensional nature of the rich data sets that people have these days and simple devices like people's iPhones and little anchoring cubes, like for example, this Merge Cube, can make it possible for them to use these devices that they have just kind of lying around, to have a really rich interaction with their data. So I think that taking some of that richness, that's been developed mostly for the gaming world into the world of science and medicine would be fantastic, and so we're working on it.
Andrew J. Mason: That is so cool. Talk to me, I know this is a little bit like my question previously about where to begin, but I hope you hear maybe the nuance in it. What do you do with somebody who says, "I just don't know where to go next with how I'm looking at this data." Any ideas on how to have maybe fresh eyes in being able to develop their data further?
Alyssa Goodman: Yeah. I have a colleague who actually was the original developer with another person, so the two people are Chris Beaumont and Tom Robitaille. And so Chris Beaumont was a graduate student here at Harvard and he was here when the phrase big data became very popular, and he used to like to use the word wide data instead of big data. And what he meant by that, and the answer to your question about what more can people do with their data, is consider the interaction of many different data sets.
Alyssa Goodman: So you don't necessarily need one monolithic, giant data set, but it might be more interesting ... Let's say you're doing some, oh, I don't know, epidemiological study, and you have numbers about case counts for, oh, I don't know, say COVID-19. And you have those numbers, but then you also have information about the demographics of particular neighborhoods. And you have information about genetic variability by age for some trait, but you have ages from the demography file and you have ages from the medical records. And what if you wanted to link together these three different data sets by age and see if there's any kind of weird correlations between genetics and location and COVID exposure or COVID susceptibility, or something like that.
Alyssa Goodman: Well, so normally you'd have to build some gigantic, so-called big data file. But not to plug glue again, but that is what it's for. So it glues together different data sets as well as different plots. And so Chris and Tom Robitaille were really the young scientists at the time who said, "This really is about the connection between different data sets and different resources and we should make that easier." If they find a way to connect different data sets and different kinds of information, kind of like what you see in the Path to Newton. In other words, you connecting their textual information, literary information, ideas, biographies of people. It's not quantitative data, but it's different kinds of information, and that's the value. It's not just a giant biography of everybody on that chart. It's a biography along with a history of ideas.
Alyssa Goodman: And so, in the COVID example, it's like, "Well, I don't just want death counts or case counts, I want to know about these people." And well, it might be hard to combine all that information, but if you use software and visualization creatively, you can make these combinations today. That would have been really, really hard to make, even 15 years ago.
Andrew J. Mason: So almost cross sectioning and slicing the data. I can almost see a 3D version to see a perspective you haven't seen before.
Alyssa Goodman: Exactly. And so when we say high dimensional, we don't mean ... And that's the thing about not using the term scientific visualization to mean literally visualization of three dimensional or four dimensional spatial data. You can make dimensions out of anything you want. And the software that does the best job of that is really Tableau. And some people have a hard time figuring out Tableau because of that model where everything is kind of a dimension or a measure. I'll leave that comment for those who know how to use Tableau. But there's a paper called Polaris, which was the visualization paper that was written before Tableau that really lays that out quite well for people who are very interested.
Alyssa Goodman: And so yes, that idea of cross sectioning information is very valid. And what I'm saying is that in addition to that, you want not to have to merge all the information into one giant data structure, which is what most people think you have to do, so they don't do it. And so the advice I could give, is learn how to use tools that let you integrate different data sets and have different views of them, because you can learn much more from wide data than you can from big data. So thank you, Chris Beaumont.
Andrew J. Mason: Alyssa, I can't tell you how thankful we are that you decided to join us on this show. Do you have any final thoughts or words of wisdom for people that are just kind of moving forward in the state of the visualization field?
Alyssa Goodman: I can tell you that the reason that the website for Path to Newton is not called Path to Newton, but it's called pathto.org, is that it turns out that this format is of interest to other people. And so we're currently working on the Path to Darwin, which you see a little bit about on the website.
Alyssa Goodman: The Prediction Project has a lot of video and there are links out to video from the text in the Path to Newton, and there will be in the Path to Darwin as well. So it basically takes all the kinds of modes of learning that people can have and mixes them together so that people can kind of choose their own adventure is the phrase people like, to go learn it however they like. And I think that's what really excites me about a kind of technology and learning, not porting PDFs and lectures to the web.
Alyssa Goodman: And that's great in terms of making material accessible to people who it was not accessible for before, but I think tools like what we're talking about here, make it a richer experience, where you can learn more in a more fun, shorter amount of time. And sometimes it's really important, like in case of epidemiology and sometimes it's just enriching, like the Path to Newton.
Andrew J. Mason: Alyssa, we are so honored that you joined us on the show. Thank you so much for your time here today. And I just foresee the audience gaining so much knowledge and insight from your time with us, so thank you for that. As always, thank all of you for listening today. This has been just such a cool show to be a part of, and we're just so grateful to have it keep going. If you want to keep up with us and what we're up to, check out the Omni Group at omnigroup.com/blog, or head to @theOmniShow on Twitter.