The Trillionaire Inventor (with Marta Karczewicz)

Invention is the wellspring of technology that makes innovations in 5G, AI, quantum computing, and video compression possible. Yes, video compression, the technology that enables us to take hours of 4K video on our smartphones. The technology that allows Netflix to stream 4k and eventually 8K movies and TV shows to your home. The technology that has democratized video conferencing on our devices and much more.

neXt Curve’s Leonard Lee had the opportunity to speak with master inventor, Dr. Marta Karczewicz, VP of technology and fellow at Qualcomm, about her career-long cotribution to a technology that has yielded billions if not a trillion dollars in economic value and continues to be a foundational technology enabling the advancement of digital media, gaming, XR, IoT, robotics, automotive, AI and more. Marta has over 800 patents to her name and has been recognized as one of three finalist for the 2019 lifetime achievement in invention award by the European Patent Office.

Marta & Leonard discuss the following topics in a chat with a master inventor:

➡️ Introducing the Patent Trillionaire, Dr. Marta Karczewicz (0:30)
➡️ The essentiality of video compression (2:50)
➡️ Why Marta is one of the leading inventors in the world (6:45)
➡️ Leonard upgrades Marta’s inventor status (7:49)
➡️ The foundational pillar of digital tech and applications (10:14)
➡️ Marta’s top video compression trends for the future (16:15)

Please subscribe to our podcast which will be featured on the neXt Curve YouTube Channel. Check out the audio version on BuzzSprout - https://nextcurvepodcast.buzzsprout.com or find us on your favorite Podcast platform.  

Also, subscribe to the neXt Curve research portal at www.next-curve.com for the tech and industry insights that matter.

Chapters

• 0:30 Introducing the Patent Trillionaire, Dr. Marta Karczewicz
• 2:26 The essentiality of video compression
• 6:37 Why Marta is one of the leading inventors in the world (
• 6:37 Leonard upgrades Marta’s inventor status
• 9:00 The foundational pillar of digital tech and applications
• 13:59 Marta’s top video compression trends for the future

Transcript

0:06

Next curve. Hey

Leonard Lee: 0:09

everyone. This is Leonard Lee, executive analyst at NextCurve. And I am here in beautiful San Diego. It's a beautiful day here. Right. And I am joined by Marta Kartevich. Did

Marta Karczewicz: 0:24

I get that right? Yes, you got it right. And she,

Leonard Lee: 0:27

she is the vice president of technology at Qualcomm and she is also. A fellow, and I just want everyone to know she is the most humble trillionaire that you'll ever meet. Very, very humble. And we're going to try to get her to talk about her technology, her background, and some of the revolutionary things that she's been involved with and has enabled through her career. Are you going to be able to handle this?

Marta Karczewicz: 0:59

Yeah.

Leonard Lee: 1:00

You know, cause last time we spoke, you were way, way too humble, and, but

Marta Karczewicz: 1:06

good.

Leonard Lee: 1:06

it is. And, I think this is going to be a great story for you to share with the next curve audience, because, we talk about innovation all the time, right? we overuse that term, and we also probably under appreciate invention. And, yes, we have F 35s flying over us, we're trying to do this, so you'll have to bear with us. But invention is also, I think, something that's probably under appreciated. Maybe not overused, but under appreciated. So, I really wanted to talk to you about that, especially in the context of the research and the work that you've been doing at Qualcomm as it relates to compression technology, right? And so maybe talk about your background a little bit, your role at Qualcomm.

Marta Karczewicz: 1:57

So, yeah, I have been working on, video compression or other forms of compression, for close to 30 years now.

2:07

Wow.

Marta Karczewicz: 2:08

started originally around, 95. I started in Nokia, been there for 10 years. Then switch, briefly to Microsoft and now, I think over 18 years, at Qualcomm. all this time, I have been working on mostly video compression. when I was starting, it was a very, very fresh area. We had just one, standardized video codec and one codec in usage. maybe I'll just tell briefly why do we even need video codecs. I think everybody knows what video is. We keep watching it. We have broadcast. streaming. zoom calls and so on. But what people sometimes do not realize that, after we capture it by camera, it has to be compressed. Otherwise, this whole ecosystem just wouldn't work.

3:04

Right,

Marta Karczewicz: 3:04

So just to give us some kind of, let's say, background, how big video without compression would be.

3:13

Yeah.

Marta Karczewicz: 3:13

let's think about what's becoming popular these days, which is, 8K video. Let's say 60 frames per second. Well, that's now I would say underestimate because people want to watch sport in like 120 frames per second, 10 bit. If you want to see the send the raw data, that would be around over 30 gigabits per second. I would say. Probably almost nobody has that level of Internet connection.

3:48

And let's remember

Marta Karczewicz: 3:50

that this numbers are being increased. So basically what the video codec is doing is grabbing this raw, raw data.

3:57

Yeah.

Marta Karczewicz: 3:57

And trying to, reduce the size without reducing visual quality.

4:03

And,

Marta Karczewicz: 4:03

this day's video codecs compress video. hundreds or even thousands of times. Before, you get it on your end.

4:13

Yeah.

Marta Karczewicz: 4:13

that's what I have been devoting my life to the Kodak from the times that I started, which I said the only Kodak, at, on the market was pec to, evolve quite a bit, just to give, some kind of, Comparison point at the codec that we were working now, can compress the data that MPEG 2 was able to compress because MPEG 2 is not able anymore to do everything that we are compressing these days. That's not going to happen. But if we take the content that MPEG 2 still could do and compare it with what we have now, we are using less than 15 percent of what MPEG needed. to get the same visual quality.

4:58

Oh, yeah.

Marta Karczewicz: 4:59

the top of everything, MPEG 2, couldn't do, what we keep seeing now everywhere, which is this high dynamic range and, couldn't exactly do, what we are using in teams, which is compressing graphics and text. I mean, it could do it, but the bitrate would. Probably explode. You might as well not compress it at all. I would like to think that I at least partially contributed to, this experiment.

Leonard Lee: 5:27

Too modest.

Marta Karczewicz: 5:29

Improvement. So I have been working personally on all the generation of the codecs after, MPEG 2. AVC, HEVC, VVC. And now, the new codec that we are starting, which, hopefully will be H. 267, I also have quite a few patents in this area, but last time that I checked, I think I had over, 800 granted patent in US and 250 in Europe.

Leonard Lee: 5:58

Wow, did you hear that? That's pretty amazing.

Marta Karczewicz: 6:01

And I've been actually probably mainly recognized for the patents. I have been one of the three nominees of Lifetime Achievement Award by European Patent Office in 2019. And I have some IP, awards. Also from Qualcomm, one of 2012 and so on and so forth. This is like the patents and I would say this is like the side product of my work.

6:31

Sure.

Marta Karczewicz: 6:31

so that's roughly what I do and who I am.

Leonard Lee: 6:35

Yeah so they call you the patent billionaire, right? And I actually consider you more of a trillionaire because I'll tell you why. I was one, I think I might've been one of the first. I consumers to buy one of those, DV camcorders. Yes. Right back in the day. And when I transferred the mini DV, one hour of mini DV footage into a digital format. Of course, this is 480p resolution. It took up 25 gigs of storage. And so, we look at the technology that you've helped to develop and continue to do research in has had on just everything that's digital, because keep in mind mobile, wireless traffic, 70 percent of his video, like you're saying media, think about outside of that internet traffic, probably the same order, of 70% is, if not more, is probably all video, right? And so imagine a world where we are still at four 80 p with the, video quality at 25 gigs. We would not have what we have today in terms of the. The quality of video and media experiences, right? Like you mentioned, broadcast.

Marta Karczewicz: 7:58

with the resolution. Why would anybody buy 80 inch TV if you Yeah. Exactly. So you see big blocks of identical color. I was like, what's the point?

Leonard Lee: 8:11

Right, right. Now, you know, going into the future as we look at higher resolutions, new, innovative Media formats, you know, I can only imagine you have vision on what that looks like, right? And, and, you know, again, like you mentioned the experience, these are the higher level experiences, higher quality experiences that we've been able to achieve, through a smaller data footprint. And that has enabled so many things, whether it's broadcast, whether it's, streaming, all these different modalities and ways of transmitting, distributing and storing even, right?

Marta Karczewicz: 8:52

would say, yeah, yeah., because I mentioned, I use the word what MPEG 2. still could do. And when you look how the video evolved, it's not that we can, compress what MPEG could compress, like almost 10 times better. But it's also that we enable a lot of things that MPEG do could not, do. Like I mentioned that. You go to any shop and TV likely is advertising high dynamic range with color gamut, and it's true. It, it made a big difference in the watching experience.

9:26

Yeah.

Marta Karczewicz: 9:26

Like the older video, like the maximum brightness that we see was like around 100. what we usually see as humans can go up to 10,000 current TVs are being advertised at two or even 3000 needs. So basically already the brightness that you see, which can be captured in display is much, much higher. contrast ratio, that's another thing that, before we may be wow. Were able to compress something that was. for like two to 10, uh, these days, that goes much higher. And what humans can do is like maybe two to power of 20 levels of Congress. And we have, I'm all at this place, but their camera, so we can show it. color reproduction also at the time was rather very far from what we can do now, because they are addressing this wider color gamut. So it's couple of times better now. Basically, the colors that we can reproduce and, the, the codex, especially starting from HVC were able to address it so we can compress this formats. we also can, compress things that became important while the usage of video was evolving, like, for example, during Corona virus almost everybody switched to. online experience and that included work and the zoom call and team calls, with sharing the slides, spreadsheet, or even, I'm still programming. So even showing like the source code via video, that became like became part of the normal. Experience. Yeah. And, past codex like did not really assume that you will compress anything but the natural contact. And the newer codecs like, HEVC Annex and VENVVC can do that. So basically, and they can do it twice better than the past codec, even if you look at all other tools. So we have the evolution and, what I see at least the evolution slowly going is, again, enabling, more formats because they become, more widely used.

11:44

Yeah.

Marta Karczewicz: 11:44

one of the examples is gaming. Let's start from entertainment actually still because what people do not often realize that all this multiple services that pop up with online gaming.

11:56

Yeah.

Marta Karczewicz: 11:57

using the codecs, actually video codecs, so basically GPU produces the content, a codecs grabs it, sends it to the end user, when you react to it, the reaction goes back, and then basically, so on and so forth, one of the Trends that we see now is the question. how do you use, the GPU information to do better and coding because, what we, for example, deal with in normal codex is, like if you think about video, it's really just, collection of still images. A lot of redundancy is that, From one image to the other. They are mostly similar. There is just an ocean of some objects, but it's not that easy to estimate. Well, when you have GPU, you can get information per pixel, however pixel move.

12:51

Yeah.

Marta Karczewicz: 12:51

So, okay. What new technologies can you do to do it? Another thing which is now very hot and also somewhat relies on video or maybe more than people even imagine is everybody talks about, AI, and video coding for machines,

13:08

which

Marta Karczewicz: 13:08

basically means that, we have all this cameras, we have cameras, not only, like security cameras, but there are, cameras in factories, warehouses. wherever something can happen and some intervention might be needed. So, okay, some of this content can be analyzed directly in camera, but usually it's actually being sent to the edge.

13:33

There

Marta Karczewicz: 13:33

it's being analyzed by AI on the server. And how it's being sent as video. So basically, now what we were looking more and more is that what we perceive as important as, humans when watching, it's not that important for AI very often. So basically, how do we change compression to make it, better for, AI?

Leonard Lee: 13:59

Beyond that, what's that one other thing that really has you excited, that you're really looking forward to, that you

Marta Karczewicz: 14:05

think

Leonard Lee: 14:05

compression technology is going to have a revolutionary impact on?

Marta Karczewicz: 14:10

I have one thing that is exciting to me from a technical point of view, which is usage of AI in codecs. Not only that it's being, you know, later used by AI, but how to introduce, neural networks into video codecs. That's already starting in the standards that we are working, that's already being tested. It's probably going to evolve at some point that the whole codec might be, neural network based. Yeah. But it's not going to happen overnight, to be honest, since, the main issue is the complexity. I mean, the codecs are designed that encoder can be very complex.

14:49

Right.

Marta Karczewicz: 14:50

But the decoder has to be very lightweight because it's running on, for example, mobile device and you don't want it to kill your battery in one hour or less. Neural networks are more symmetric in their design, so we'll have to figure out what to do with it, but they open new opportunities. I see mainly like two. I mentioned this video coding for machines that the quality measure will have to change,

15:19

Mm-hmm

Marta Karczewicz: 15:19

normal product, let's say have the very fi they have very fixed design. If you wanna change, the way that, you measure quality, you have to redo the whole encoder. Parts of the theod are very complex. With neural network, actually, you can just retrain the weights of the codec for a different quality measure. so that would make it, much more, user friendly when you change the scenario where codec

15:48

is used.

Marta Karczewicz: 15:49

We have been experimenting, literally, for years, how do we compress texture. Because, if you look, for example, around, you have tons of trees, right? the same if you look at, water, everything that has texture, the codecs didn't really progress that much. How to compress it, because it's random, it's very difficult to find what type of prediction you should use. It's almost like it's a noise. Neural networks, they wouldn't compress it exactly, but you basically can tell them, don't compress it exactly, but, mimic the statistic of this texture, mimic the statistic of this noise. And then when it does it, it does it, I would say perfectly. You can get, almost 10 times slower, betray when you get for the normal codex. And yes, if you, do the difference, the difference will be big between the original and the current image. But when you look at it, You can't tell. It looks fantastically natural.

Leonard Lee: 17:00

So is it like a smoothing effect?

Marta Karczewicz: 17:02

not smoothing. It just has a different error, measure. It's just like, these are statistical properties and these are the statistical properties. Try to match them. neural network will do it. So

17:15

yeah,

Marta Karczewicz: 17:16

because normal codecs, you are right. Short of like this graphic content, they rely on assumption that, the content is smooth with some edges in between.

17:26

Right.

Marta Karczewicz: 17:26

the motion is fairly, predictable. Well, again, like I'm looking at three in front of me. it's very textured and the motion is all over the place because the leaves are moving. Which other way? So that's one thing that I would say excites me. And then other thing which becomes, that might be a bit more longer term is the immersive video.

17:52

Okay.

Marta Karczewicz: 17:52

So we are used to, just watching 2D video. There was some excitement about 3D video, which kind of came back a bit. I would say, I don't want to call it a fake 3D video, but it is, because what we usually actually do, we just capture videos from two different viewpoints. Yeah. So it's a stereoscopic video and, it looks 3D, but, for example, like when we move our head, we see that objects move versus each other. when we watch this. 3D videos. Nothing is moving. So basically I was like, it looks 3D when I don't move. But when I move, it starts to look fake.

18:34

It's more like volume. It's missing the volumetric aspect. So basically

Marta Karczewicz: 18:39

we are working on multiple formats to be able to support. volumetric video

18:47

so

Marta Karczewicz: 18:48

that it will not only allow you like to some extent is just to allow more realistic 3D experience, but, there are also attempts to make it truly immersive, not maybe for everything, but for example, when you look at, sports, most of the events are captured by Cameras surrounding the stadium.

19:09

Yeah.

Marta Karczewicz: 19:10

And what we sometimes even watch on our TV. It's not even a real video. It's a video produced by, building and 3D model and capturing it like, let's say, view that you cannot see or camera doesn't capture, but you can recreate it by, capturing how this 3D model would look from this or that point of view. Yeah. So now the question is, if we can already do that, can we send this information to the, end user and then they can replace being, for example, receiver or be a quarterback or whichever they want to be. So basically, but that I see still as a longer term evolution. Because that requires also, I think evolution of the capture is maybe in some extent, to some extent there, but it Requires evolution of the display. To some extent you probably can already do it with XR. but that still will take time before it will be, that popular or that, broadly use that, production of such content will make, Sense from the cost point of view. So I would say it's like, Enhanced and, immersive. That's what really, really excites me at the moment.

Leonard Lee: 20:32

Well, you know what really excites me? That I learned so much just in the short conversation. And I love that you told me that I was wrong. See, I got schooled on my own podcast. It's like, but you know, I'm talking to a, trillion dollar inventor here. That is just, oh

Marta Karczewicz: 20:51

yeah. There's just so many

Leonard Lee: 20:53

patents. It's like, but no, you know, one of the key takeaways I've had, you know, I've already spoken to you about. Your background and many of the technologies that you're working on have worked on in the past. And I'm impressed by the impact. It's not like hypothetical impact. It's like real, real impact. That's had just an immense halo effect. Is also now that we've spoken again, really that foundational technology that enables, experiences to continue to evolve and advanced. And without that, we're stuck, right? And then the innovation across whether it's a streaming broadcast or capture and consumption, all that stuff just stand still. when people think of Qualcomm, they think of Snapdragon now. They used to think of Qualcomm from a, a wireless 5G perspective. But then there's this other thing that, Qualcomm is doing through the work that you're doing that, is really another one of those foundational pillars, so that's the other thing that I think, we're sharing with the NextCurve audience that they might not know. I mean, I don't think a lot of people know that you got this. Most of the

Marta Karczewicz: 22:02

codecs that we talked about are implemented on Snapdragon.

Leonard Lee: 22:06

Come on.

Marta Karczewicz: 22:07

Oh, okay. We have to educate

Leonard Lee: 22:09

The consumer doesn't know. A lot of people don't know about this stuff. You know, they just see a bunch of acronyms in the spec, you know, technical spec sheet. And then, but no, I, I really appreciate your taking the time out. of your busy day and, suffering the F 35s flying over him. By the way, speaking of compression, a turbofan engine uses compression, it compresses air. That's what makes, all these fighter planes go supersonic. Just like what she's doing, you know, she's doing compression that makes our videos go. Super what? Visual? Yes, but you

Marta Karczewicz: 22:45

can also think about it that way. That latest Top Gun movie wouldn't be as cool without the codex that we make.

Leonard Lee: 22:55

Oh my God. Wow. She is good. She is good. Well, Marta, thank you so much. I really appreciate the time and the education, quite honestly. And, I hope that we can continue to conversate every single time. I'm serious. Every single time I sit down and talk to you, light bulbs go off in my head because, I do broad TMT, like technology, media, and telecom research. And so as you were speaking, it was just I saw the future, I saw the future and it's absolutely wonderful. So once again, thank you. And, to, our audience, thank you for tuning in. We hope you enjoyed this, discussion and hopefully you learned as much as I did, about this wonderful individual who is the most humble, revolutionary, a disruptor. I could even call her a disruptor. She deserves that title disruptor. And of course, in all the positive and wonderful ways, follow us on next curve, WWW. Dot next dash curve dot com. And remember to follow, like, share, the content on our YouTube channel and we will see you next time. And once again, thank you so much.