MEP EP#65: Self driving cars with Josh Hartung from PolySync

Podcast Transcript

Host 1     00:10

Welcome to the macro fab engineering podcasts. And I am your guests, Josh Horton.

Host 1     00:16

And we are your hosts, Parker, Dolman.

Host 3     00:18

And Steven Craig. This is episode number 65. Our guest this week is Josh Hartung of poly sync. Josh is the CEO and co founder of poly sync, a startup that is changing the way that companies developed self driving cars, technology. Josh, is there anything that I missed in that description?

Host 1     00:40

Yeah, they're like 20.

Host 3     00:42

Probably a whole lot. Yeah, I think I think, where did we grab that description from off LinkedIn? Yeah, I'll link that. Yeah, it's so this is now becoming a trend, where we find some, like, really thin description of our guests. And we, we, we usually, like end up excluding a whole lot about them. So

Host 1     01:00

Yeah, cool. Just think is a company that makes stuff.

Host 3     01:03

Yeah, yeah. Did we miss a lot? Yeah. Well, okay, yeah. Well, let's jump into it. We want to tell us a bit about Polly sync and what you do there?

Host 1     01:14

Yeah, absolutely. So as it turns out, poly or autonomous driving cars is kind of a new industry, if you will. So all these all these technologies that have sort of come together to enable autonomous cars, or AI, you know, low cost, compute, you know, cars with actuators and things like that. They're all coming together, you know, at this incredibly rapid pace. And, and so as you might imagine, there's a lot of really interesting opportunities there for, for improvement. And the one that we're focused on is creating safe and deployable autonomous vehicle systems. So the important thing there is, is that, you know, early on in the engineering process, you're doing things like prototyping, and you're basically using whatever you can to make something, you know, work and, and, and, you know, get a sense of what that will take. Right. So so, you know, from a software sense, you're often using API's and libraries that you find available, either, you know, kind of in the open source community, or, you know, you whip together something yourself that works. Okay. But, you know, obviously, we as engineers know, like, obviously, you're going to replace those things as you get closer and closer to something that you, you know, people need to depend on. And, you know, people need to depend on autonomous cars in a way that, like, we don't depend on many other things. Right? Yeah, in a lot of cases, there won't even be steering wheels in these things. And so we're focused on solving the problem of how do you make not the capabilities of those cars safe? Not not? How do we like make sure we navigate intersections well, and don't hit hit pedestrians? But how do you know that your underlying software is going to perform as you expect? How do you know that the computing platform can tolerate different failures? How do you know that, that, that the underlying infrastructure of your system is actually safe? So that's the problem that we're solving with poly sync?

Host 3     03:11

Interesting. So how, what does the name mean? What's the meaning behind the name poly sync?

Host 1     03:18

Interestingly, the name came from kind of the early prototypes of of the system. At the time, I was the CTO of a distributor company in this space. And, and, you know, it was like four people at the time. And so what CTO meant was just me, flying out to all these different customers, you know, mostly in auto, but also in like military mining, agriculture, academia, and helping them with system integration with understanding the sensors that they were purchasing and architecting their system, and, you know, even going so far as to help them like design brackets and things like that. So sort of whatever. And one of the big one of the big gaps that that I started to see almost immediately was was that bringing many different sensors together into a place where you could actually use that data to make decisions and do do the things that a robot needs to do was actually a fairly tall barrier to entry. So an autonomous driving car, you've got so many different sensors, cameras, LiDAR, GPS, radar, ultrasonics, you've got an even then you've got down to like weight sensors in this in the in the seats and steering wheel sensors, capacitive touch sensors. And there's just a massive number of sensors, both high bandwidth and low bandwidth in these systems. And typically, they're all operating asynchronously, meaning they're not really they're not really receiving a trigger signal or something like that, where you can go, you know, I captured this sensors data at a specific moment in time. Instead, they're just sort of, they're all autonomous, if you will, they're all capturing data at whatever rate they sort of choose to.

Host 2     04:52

Yeah, it was it the like, average car has, like over 30 microcontrollers in it now.

Host 1     04:57

Yeah, exactly. And that's and that's how they deal with that. I know Cars, right? They've just got a bunch of different microcontrollers, a bunch of different like, a like an Arduino is essentially right, like all arrayed in a system trying to deal with the asynchronous nature of that sensing system. And and so it was so with policing without like, what would it be like to solve that problem in software instead on central computers, and that's the name. That's where the name polacek was born, which we're synchronizing into a central or a single time domain, a bunch of different asynchronous systems. So poly meaning many and sync, meaning sync? Yeah.

Host 3     05:36

Cool. So. So I guess you have some kind of hardware that everything talks to, and then the software is taken care of most of that, right?

Host 1     05:46

Actually, we don't really we deploy on almost anything, we're a software layer, pretty fundamentally. So in the automotive space, you got you do have like many different computers to run on. You've got different microcontrollers, you've got arm stuff, Intel, and video, all these different chips running simultaneously, how do those things talk to talk to each other? That's where we come in. We're the software sort of, we're a middleware layer that lets all these things communicate with each other.

Host 2     06:10

Yeah, actually, I was looking at the website earlier. And y'all Bill yourself as middleware. Right? That's called that's the policy core. Right? And so it's between applications, which I guess is like, what's what you're doing with the software? And then the vehicle? Oh, so you're like, you're in between that. So you don't modify what's originally on like, the you the vehicle? EMC?

Host 1     06:38

Yeah, you can like actually just download Polly sync and like, put it on your car or something, right? It's not, it's not really a consumer facing thing. The approach to using policing is really a sort of fundamental new architecture decision that you've got to make in building a car. So our customers are like tier ones, and OEMs one of the big problems they run into there is that their typical development paradigm is embedded development. So they're used to building you know, essentially fixed function devices. Right. So and then they'll either purchase those devices from their suppliers, you know, tier one, suppliers, these are like Bosch, Delphi, you know, denza, these types of come standard equipment, etc. Yeah, exactly. They'll purchase them, and they'll modify them. And essentially, they're building what amounts to like the the most complex, fixed function device, right, a car is a fixed function device. And the problem is that that really breaks down when you get to autonomous driving, right? This is a software defined system, it needs to be constantly updated to be changed, you know, Tesla's a great example of, of how important that is. Right? Always shipping software updates their cars. So the challenge, so So the challenge of, of embracing software is like a really, fundamentally different process for automotive companies to move toward, we're one of the parts of of making that transition,

Host 3     07:54

I would think that would actually be a difficult transition to make for most, most guys, mainly, because you Okay, so you have this sensor suite. And a hardware designer thinks of the sensors as just kind of an extension of their design, or even I guess you could say, like, the fundamental piece in their design, and that goes off to some microcontroller that they have control over. It's, I guess, destiny in a way that the, of what that sensor is doing, but by utilizing this, you're kind of taking that control, I guess there's a level of trust that they have to have in your software. Right.

Host 1     08:30

Absolutely. You know, and that's one of the biggest challenges of moving into Automotive is, you know, whereas in other industries, you know, everyone wants kind of the newest, hottest stuff that they can get newest, hottest software and, you know, newest version, all the time, automotives the the opposite for that exact reason that you just described, they want something that dependably works that they know that they can trust, so they can work on their job of sort of system integration or testing or something like that. So relinquishing that control is a real challenge. But once you do, the choice of functionality that you can deploy, is just so much more rich and so much more valuable to the end user. You know, it's it's the difference between, you know, the capability of, of a, you know, a Nokia flip phone, and an iPhone, you know, yes, you, you know, you as, as an app developer, move away from, you know, sort of the knowing everything about your system context, like an a Nokia app developer might have, you know, and instead you move away to a world of abstractions and API's, but what you get from that is, is is much more power to deploy on different devices, in different environments, you know, with different sort of different types of sensors. And that's the kind of paradigm that we're trying to bring to automotive is the software defined device.

Host 2     09:50

That's actually what it's gonna bring up is it sounds more like, you know, instead of having to write everything custom for, let's say, a Ford Focus, you can deploy Ford company ploy a car, basically, and then just change what it is as a car? And software? That makes sense.

Host 1     10:09

Yeah, absolutely. I mean, that's the thing that's really attractive to all these companies. Today, today, the product cycles for automotive, they're like five years, right from, you know, the napkin sketches to to a car that you can buy. I mean, that's just insane. What that means is that it, you know, is that the infotainment systems that you get the, you know, the different, you know, the design elements and things like that, those are like five years old, by the time you even get to, like, touch that screen. And so, you know, think about think about the design changes that have happened in like iOS or Android in five years. I mean, they're there, you look at those five year old versions of those of those UI and you go, wow, that looks old. And I think that we've all we've all had that experience is like the stuff that we get into cars, even new stuff today just looks and feels dated. And so the attractive thing about about working in a software defined context is is the ability to update that for your customers, and bring them a fresh experience and fresh functionality throughout the lifecycle of their product.

Host 3     11:13

Awesome, so. Okay, so on your website, you and your your products page. There's, there's two things that really show up. There's the poly sync core and the OSCC. Yeah. So you know, we've kind of scratched the surface on the polishing core. Can you tell us about the OSCC? Absolutely.

Host 1     11:35

I'm glad you brought that up. Because of course, that's the the thing that we've collaborated with macro fab to build. OSCC stands for Open Source car control, and it's a really fun project. The story of it is that when in our early days, you know, when we were a poor little startup, or I should say, less poor startup, or excuse me, more more for startup more poor. We needed a vehicle to test autonomous functionality on because until you can complete the loop of you know, sensing back to control of the vehicle, you're not really autonomous. But like, as anyone who knows cars have has, as found, there's no control interface for cars, cars don't have API's. No. And, and so that makes it really like vexing really difficult to to get control. And so how do I turn the steering wheel with a with a with a computer? How do I make the car brake? How do I you know, make it make it accelerate? Short of, you know, having a text readout that says like, you know, push the pedal down, dude. You know, attaching external actuators, you know, or buying, you know, research great equipment for like, $150,000 it just really isn't a good way to do it. Right. So, yeah, so how does, you know, first of all, how does a startup get into that world economically? That was our challenge. And then second of all, like, how do you scale a fleet on things that cost $150,000? Like, you know, just the cost become prohibitive? Yeah, there's the cost of the hardware. Right? So we were like, Okay, what what can we get that's just an inexpensive vehicle that's available worldwide that we think that we can hack and put an interface in and and Kia Soul I came out on top, it's just like really inexpensive car. They're super high quality. They're available in like, basically the same form worldwide. And, and also for a lot of companies. It's kind of a neutral brand. And that helps, you know, if they're going to do public demos and things like that, it's not a Ford. It's not a GM car. It's not Toyota, it's, you know, it's a Kia. That's the hamster car. That's like the comment we get most often.

Host 2     13:35

Oh, yeah, that's actually what they used to advertise that car, right?

Host 1     13:38

Is this hilarious? Hamsters? So yeah.

Host 2     13:41

So what what makes that car easy to hack then? So what what do you say is because like, if you say, oh, key is easy to hack, you know, a person who just says that, right? That's a bad thing.

Host 1     13:54

No, no, no, it's not easy. I mean, like anything, you know, it's you can compromise anything with physical access to it, right? So of course, you know, is

Host 2     14:03

That so y'all just tapping right into I guess the CAN bus?

Host 1     14:06

So no, like that's so there's there's two ways to do this out in the in the real world. So number one is to is to spoof a can system. And number two is to spoof an analog signals system. So I'll just give you some background on this. So so if you want to do it with a canned system, you're going to need to find a car that's enabled with a full control interface. And most typically, that's like a self parking interface. So you know, vehicles that can like pull up to a parking spot you like press a button? It can it can park itself. That's simply a car that you can, you can do full control with can't over can. Okay, that makes a lot of sense. Yeah. So what in what you do there is you basically you figure out the signals and you spoof it and you just tell it, it's in self parking mode all the time.

Host 2     14:51

Gotcha. Okay. Yeah, that makes a lot sense actually. Yeah,

Host 1     14:55

Right. So that's so it makes sense, but it's actually not particularly safe. because you're because you're like, you're basically violating the safety case of self parking, right? You're, you're telling the car, you're going five miles under five miles an hour, and you're in reverse all the time, you know, so on the highway, like, you don't know what that's doing to the system. So that's one way. And that's how that's how like the self driving like Lincoln MKZ that you see around or Ford Fusions, you know, there's various others you see, with a by wire control interface. That's how they do it.

Host 1     15:27

Yeah, so that sounds very much like a hack.

Host 1     15:29

It is a hack. And ours is a hack to I mean, there Look, there's no interface for cars, like no one designed a way to do this. Well, so you know, they're all shades of hacks, right? But you know, so the OCC approach is a little bit different, we go into the analog system. So the in particular, the Kia Soul has electronic power steering, meaning a brushless motor that drives the power steering instead of hydraulic system. It has throttle by wire, which most cars nowadays have now nowadays have which is, you know, there's no physical connection between the, the gas pedal and the and the throttle body that controls the engine. Yep. So those two, those two things are fairly easy to, to, to spoof. So on the steering side, which is the most critical one, the steering control works as a feedback loop on a torque sensor. So essentially, you put in some torque into the steering wheel, it senses that and the and the motor tries to assist and minimize that torque. Pretty simple. Right? And so so what we do with OCC is we go in, and we and we synthesize a torque signal, interrupt the ones from the actual sensors, we synthesize the torque signal, and we send that out. So we're essentially making the Motor Assist a phantom hand if you will.

Host 2     16:41

So this is the wheel. I guess, actually, the wheel does spin then. Oh, yeah,

Host 1     16:45

Absolutely. I mean, it's, yes. Cool. Yeah, yeah, it's really cool. So and then we and then we feed back on on steering angle. So essentially, where our stuff works is, is where the way RCC works is you send it a steering angle, and a target steering angle and a rate a rotation rate to achieve that, and the wheel just goes to it. And so

Host 2     17:07

How do you get the steering angle back then? That's closed loop. Yeah, so

Host 1     17:11

That comes there's two ways to do it with OCC it comes over the CAN bus because it's like a nice smooth condition signal. But you can actually get it from the the sensors as well. So the way that OCC works in Apple in that case is it's like basically an Arduino Uno board. And then we have a shield, which macro fab makes for us that has a digital analog converter on board and an interrupt relay and a can transceiver. And that lets us sort of make that its own little module. It's its own distributed ECU. So that's your steering ECU. And then basically, we do the same thing for for for throttle, right, the same thing is like there's two position sensors there, we just, we just get in there and synthesize those values as well.

Host 3     17:52

So So did you did you originally have to find these values empirically? or was there some kind of standard that you found for

Host 1     18:00

Us? Well, it's a it's a really interesting technique that not many people know, nowadays, which is known as an oscilloscope.

Host 3     18:10

So just watch what's going on?

Host 1     18:12

Did you know that many people are afraid of analog things now?

Host 3     18:17

Yeah, I'm sitting next to a guy who's afraid of

Host 1     18:20

Using oscilloscope.

Host 1     18:22

So yeah, so anyway, that's how we found those values. I mean, they're just, you know, they're just essentially, potentially amateurs. So you just you just measure the voltage change. Awesome.

Host 3     18:34

Yeah. You know, that's actually funny. I'm actually a bit surprised that even in today's day and age that the the, you know, position sensor of a steering wheel is analog. I, you know, I wouldn't be surprised if it was just, you know, if there was some kind of digital, whatever encoder sensor that would that would detect the well, that's Oh, and send that off?

Host 1     18:54

Yeah, that's a great point, actually. So in a lot of cars, there is a higher level protocol, like a can Protocol, or LAN, or there's, there's a bunch of different different approaches. That's part of the reason, you know, so if you had an Audi or something, or even a, you know, a nicer, you know, car, they would have that type of signal in there, because they tend to just engineer a lot more complexity into those more expensive cars. Sure. The Kissamos, for the sake of it is just a really simple car. Go ahead.

Host 2     19:22

Oh, no, I'm saying is they they put in added complexity just for the sake of it,

Host 1     19:28

Not just for the sake of it, right. The thing about the thing about API's Is that is that they are abstractions, right is is they are they simplify implementation and improve modularity, typically at the expense of of complexity, right, so you build an API, and it actually does make your, your system fundamentally more complex, but it doesn't make it more modular and more reusable, right. So for whatever their reasons, like that's a pretty broad systems engineering perspective on that, but that's one good reason to make a higher level protocol. There is For instance, you could use it in different places.

Host 2     20:02

Gotcha. Yeah. Like, what would be like a Merc? A West version one?

Host 1     20:09

Uh, I don't know what that means.

Host 2     20:11

Oh, Mercedes. Oh. Yeah. Trying to make joke fail, I guess.

Host 3     20:20

Sorry. I did. Yeah, it was over my head. I don't get this. Okay, well, okay. Next question.

Host 1     20:30

Let me let me finish on Oh, sec. Oh, yeah. The last one, which is, which is what I consider the real hack in this system is is a brake module. So the problem with with like, this, these old, you know, like basic ice, internal combustion cars is like they, they don't have any way to, to there's no electronic accurate actuation on the brakes. So there's a couple ways to do it. Number one, you can reverse. Like, you can hack the ABS module. That's not really safe. There. Those things are actually people to know this. They're only rated for like 30 to 40 hours of continuous duty. Really? Yeah, totally. So ABS modules, a lot of cost comes out of cars based on you know, uses assumptions. And, you know, think about how long I mean, your abs is running for a second or less every time it's running typically. So they don't need to run very long.

Host 3     21:22

Well, technically, you don't want your abs to run. Right.

Host 1     21:27

Well, yeah, but the when they do I mean, it's not for not very long, right?

Host 3     21:30

Yeah, right. Right, right.

Host 2     21:32

Oh, that man that makes Oh, like a sidetrack story, I guess is we're driving to old suburban in Colorado going down this some switchbacks. And I think we rode the abs for like, 3045 minutes straight.

Host 3     21:46

Oh, man. Oh, god. Yeah. Yeah. 5% of its entire life. Right.

Host 2     21:51

It's gone. Right there. Actually never thought about that is, is, you know, how long can the ABS pump basically work for?

Host 1     22:00

Yeah, I think you'd be surprised to find out how copped a cost optimize cars generally are. I mean, they're like, they're like an 9%, you know, net margin business or less like cars are and so every penny counts. I mean, you might you might be surprised what what these companies will do to save 25 cents. Oh, on the cost of a car.

Host 2     22:21

Oh, I know, I drive a mid 90s jeep?

Host 3     22:24

They said 25 cents everywhere. Right? The work? Yeah.

Host 1     22:29

So, yeah, so Okay, so the way I was going with that is is the brakes of the last thing. So what we do is we actually, we actually retrofit a brake module, a brake by wire module from a Toyota Prius, Oh, for 207 Priuses had like a really cool brake module, with like, a valve pack in it, a high pressure pump and a high pressure accumulate accumulator. And so what it means is that we can cut we kind of go in the middle between your foot, master cylinder and the ABS module. And we essentially like we it's essentially like a, an electrically driven foot, just creates hydraulic pressure, and is able to spill that out to the rest of the car. So that's so that's actually where most of the development effort went into is, is reverse engineering, this Toyota brake module. It's actually a denza module and then and then being able to control the valves and, you know, properly and, and, and things like that.

Host 2     23:24

So the Toyota just stopped using that module, or did they go back to the old school way? Or is there a reason why you have to use like that especial

Host 1     23:34

Yours? Yeah, it's a perfect, it's a peculiar unit, because it's what happened with Priuses. And like every car is that they just got more and more integrated, again, to save money. So in the oh, 4207 year Prius, there was a hydraulic power pack that did the brake by wire work. That's like, there's no smarts in it. It's just a pin out basically, to a bunch of solenoid valves and a pump, and a bunch of pressure sensors. So the pressure sensors are all analog again, and then the and then the solenoids are all like just coil packs, so so we just, were able to PWM them at the appropriate. I mean, it's kind of an inductive load. So you've got to have a special circuit for it. But other than that, I mean, we PWM those things, and we're able to control them. If we had gotten like, the next the next generation Prius, suddenly you've got, like, an ECU with an embedded microcontroller. It's expecting a whole it's got firmware. So it's expecting a whole bunch of canned signals. And suddenly, you've got a much harder and less deterministic system to work on.

Host 2     24:34

Yeah, you'd probably have to sit there and basically run a Prius and sniff that bus that's going into that module, trying to figure out what it needs. So you know, do its job.

Host 1     24:44

Yeah. Which you could totally do, and that might be a good way to do it, too. I mean, there's, there's more than one, you know, one way to do it here. That's the way that we chose is just what we felt was like the simplest way for people to understand if they were going to work on themselves.

Host 2     24:57

Well, especially if what that for a DIY kind of thing is simpler seven better. Yeah.

Host 1     25:03

Yeah. So on that on that track, I mean, we basically built this thing out of four different uses for Arduinos. Which we may go away from in the future, like with a little bit more custom system, but for the first iteration for separate Arduino modules,

Host 2     25:17

So now, so now your car's got, you know, 30 plus for microcontrollers. Yeah.

Host 3     25:24

So, so Okay, actually funny, in a way that kind of shoehorns, into the next question. Can you tell us about any of any issues you've had? In other words, have you had any failures? Have you had any microcontrollers just go wacko on you? Any crashed cars?

Host 1     25:45

So we're very careful with this system. You know, because it's experimental. There's, you know, first of all, we never, we never run it without anybody in the driver's seat. And then we almost never run it on like public roads or anything, we just run on close courses. When we were in engineering the system, it was just about jackstands. The whole time was like, we're not going to take a chance. You know, the thing is, we, I think one thing that strikes me generally about like, especially startups in autonomous driving is we like culturally, you know, people who've driven for a lot of years, you know, you just don't assign the same level of risk. Like you don't assign a real, like level of risk in your brain to what you're doing when you're driving. And when I

Host 2     26:27

Yeah, yeah, you're right is because you drive down the freeway and look to your right, and there's someone texting going 70 miles an hour,

Host 1     26:34

Right? I mean, it's like you're sitting there, it's comfort, comfortable, it's quiet air conditioned to cool your listen to your tunes. You know, it's like, it's about as stimulating as watching is watching, you know, a really boring TV show. And we and that's the way and that's the way we treat it. Yet. We're we're like carrying, I mean, massive amounts of kinetic energy with us that just can wreak havoc and destruction, you know, with the slightest of wrong moves. So, you know, so like, I mean, it's just like 3000 pound missiles driving down the road, you know, so we we don't like so we're really careful with that stuff. And we and we do a lot of work to make sure that like our safety drivers are, you know, aware and cognizant and just like we try and treat it with a lot of respect and we hold each other accountable for it. So

Host 3     27:18

Yeah, if you want the fear of God put into you drive a motorcycle on any of the freeways in Houston, right? And at 7070 miles an hour on a motorcycle is a different speed than 70 miles an hour in a car. It just is

Host 1     27:34

Yeah, you're so right. You want to feel small right a motorcycle, you know during rush hour.

Host 3     27:41

Oh, it's yeah, it's frightening but awesome. Yeah.

Host 1     27:46

I have kids now so I no longer have motor so I have a motorcycle I need to sell it you want to buy a motorcycle?

Host 1     27:53

Is actually doesn't work right now.

Host 3     27:55

But I need to bring it over the Parkers we got it. We got some carburetor cleaning to do here. Oh

Host 1     27:59

God. Yeah. The other reason I'm selling it is because I can no longer deal with having four carburetors.

Host 3     28:06

Oh, I've got two and that's, that's too too many.

Host 1     28:09

Because one doesn't work. Yeah, what

Host 3     28:10

One of them does not do anything right now?

Host 1     28:16

Yeah, so Okay, so mishaps so I mean, the biggest thing that's happened so far is we I mean, we basically follow the easy use all the time in early development we like if you send the raw there's all sorts of error checking that goes on and automotive systems and so that there's like the the torque sensors for instance on the steering are dual redundant. So they check they one goes you know, one goes up in voltage while the other one goes down. There's specific calibrations to both of them, you know, and so if you if you give it the wrong voltage, it'll fall out there requires like a dealer level service tool to reset so we end up having to buy one of those and even then, like we've had customers that have bought you know, sec cars from us and stuff, and they like get in there and start messing with the firmware and they like totally destroy like they like hard fault the system and they you know, they go

Host 2     29:02

How much is that? How much is a key a dealer tool cost?

Host 1     29:07

You can buy a Chinese knockoff version that that works even though it like, technically shouldn't because it's not for these year of kids. It works for like 200 bucks on ebay. Okay, that's not bad. Yeah, yeah. If you want the full system, it's like, I think three and a couple 1000 A year and maintenance. Yeah, cuz that's

Host 2     29:27

Because I've looked at for dealer tools for my Jeep, and it's like, the knockoff is like 250 300. And then there's a company that like ransoms I've done that before, right? But yeah, it's, it's just brutal on those on those specialty tools like that, cuz it's like, I can only use it on this model Jeep.

Host 1     29:47

Right, right. Well, and they're getting they're getting smarter about protecting them. Like before you could do these knockoffs. Like I had one of those knockoffs for my Volvo you know, and it works. It works fine and it was like fully featured, but then you you know, then You'd get up to the Modern cars. And it's like, they've got maintenance subscription contracts, and

Host 1     30:04

You better not connect that to the internet. Right? Exactly.

Host 1     30:07

I send you like a Android tablet and stuff. So yeah, they're getting better at it.

Host 2     30:12

Yeah, I think I remember my, my brother had like a BMW, and to reset the oil check light that would pop on, like every, you know, 3000 miles, you could trick the system by just grounding out a pin, right? Or you could pay a dealer like, you know, a couple 100 bucks to do that for you. Right.

Host 1     30:32

I think that was OBD. One.

Host 2     30:35

Maybe it was that weird, like, circular thing that that BMW had in their engine department?

Host 1     30:41

Yeah, I think that was actually that that was a little known fact, that was OBD. One, there's like a little LED that would flash. Yeah, yeah. Yeah, that was that was OBD. One. So OBD two is now the sort of protocol for, you know, if there is an open protocol for cars, it's, it's OBD. Two, and that's that little dongle that you plug into the connector under a dash. But before that, there was a funny little jumper that you would put in and a little light would flash and you'd be like, Okay, that was too long, and a short. My oxygen sensor, you know, so. That's great.

Host 2     31:16

So, um, so you talked about controlling just the physical aspects of the vehicle? What about like, maneuvering? How does the How does your system currently, like for the key, I guess? How did you get the outside data into controlling your vehicle then?

Host 1     31:32

Alright, so I'll caveat this by saying we don't do autonomous driving algorithms, if you will, right. So a lot of companies are out there building the brain of the autonomous car, there's, you know, the AI, and then you know, the object recognition, path planning these types of things, you know, even control systems. That's not really within, like what we do as a business. We do the nervous system, right? So underlying stuff with the network that lets information get where it's going and things like that. But having said that, of course, we we need to, you know, we needed to dog food, our own stuff. And so we've done quite a, you know, some work internally on, you know, basic autonomy demonstrators waypoint following. And you know, actually, I've just been, I spent the last few weeks teaching myself deep learning, which has been really interesting. Yeah, that's been the big thing recently. Yeah, it's a fascinating field. Would you like me?

Host 2     32:32

Yeah, actually, I Stevens, like, are you going to ask him? Why? Yeah. I'm like, Ah, I guess so. Yeah,

Host 3     32:39

I feel like learning in a deep sense right now. So can you please tell me what deep learning is all about?

Host 1     32:48

Well, so hopefully, hopefully, you know, I won't make audience members cringe here. Because, you know, I'm a few weeks into, to messing with the API's. And I've got a textbook that I'm reading and trying to teach myself this stuff. Effectively, deep, deep learning is kind of a, it's kind of a trendy term for, for sort of bigger neural networks. So neural networks have encountered sort of multiple rises and then declines in in, you know, fashionability over the years, recent improvements in the cost and performance of computing hardware, have really have really given birth to like this AI revolution that's going on right now. That's just colloquially colloquially termed, as you know, in a blanket as deep learning. And what that basically means is that, whereas we used to have networks, neural networks, where we had, you know, a few layers, one or two different layers, now, we'll have networks with many, many layers, you know, five 610, you know, 2030, you know, many, many layers. And the deeper you go, you know, the more the more the network is able to sort of store higher level concepts and create sort of what's what's known as invariant representations of the of the world or around it or, or within, or patterns within the data or

Host 2     34:08

The data sets. Yeah,

Host 1     34:09

Yeah. Well, I'm

Host 3     34:11

Really relieved because I thought for my entire life, I've been participating in shallow learning. And it's just, it hasn't done well. So I'm so glad I know about this now.

Host 1     34:21

Yeah. Pretty soon that, you know, the computers will do it all for us. Right. That's yeah. So yeah,

Host 2     34:27

Actually, Josh, what is that book that you're currently reading right now for that?

Host 1     34:32

Is a textbook called Deep Learning.

Host 3     34:35

Go figure.

Host 1     34:38

Let me see if I can find it. It's by a guy named Goodfellow and, uh, I'm looking up right now. Shoot it.

Host 3     34:47

He said it's a textbook, right? Yeah, it's textbook.

Host 1     34:50

It's not the place to start. Yeah, by a guy named Ian Goodfellow. Yoshua. Bengio. And Aaron kohrville.

Host 2     34:58

Yeah, actually, yeah, just if you Google deep learning Goodfellow. Yeah, it's like the first result.

Host 1     35:04

Yeah, this is like the this is like the the hot new, the hot new stuff in in, in deep learning, it's a great overview and it goes really deep. The way that I started out with this stuff is I just started to poke at the API's. So first I did, I did some experiments with, with TensorFlow. And now I'm on to kind of a higher level abstraction of that called Kerris, K era s, that was a really good way for me to get some quick introduction to the practical things and, and kind of look at what I didn't know. And that's given me a really good framework for reading this book. The dramatic thing, I mean, I'll tell you, like, the, you know, the seminal papers in deep learning were really were in, in neural networks were really written, you know, like in the 80s. So there's not, there's not a massive amount of like fundamental things going on. But what you're getting now is just these really impressive applications and tweaks to these networks, or to these to these approaches that are that are just fabulously capable. So you know, one of the one of the ones that's really in vogue right now are called a typical type of recurrent neural network called LS TM, long short term memories. And these things are able to, to, to store sequences of data now, and as well as spatial data. And they're incredibly good at like, there's some examples of where they'll actually, you know, in, you know, in some limited data set, they actually go as far as they're able to go from, you know, nothing to actually having learned English. And then being able to recreate correct grammatical sentences and phrases. I even saw one example where they trained it on the the like, like Linux code base. It was like a budget code base or something. And then they have, and then they had it, like regenerate, or they had it like, they call it hallucinate, they had it come up with a bunch of examples of C code after that. And it's generally it's what it's very bad at is tracking, like variabel allocation stuff, anything that would be long term, like, I need to remember that I allocated this variable to something, but semantically, it's pretty much correct. And stylistically, it just looks like a it reads like this gigantic C code base that it was trained on. And like, the crazy thing is, you imagine that this frickin network, it didn't know anything, it didn't know, C code, it didn't know English, it didn't know symbols, before it just started being trained on on this gigantic codebase it's just amazing.

Host 3     37:38

Wow, that's, that's really crazy.

Host 2     37:40

It's yeah. so on, so I guess, from the deep learning stuff, I guess so what, what's the, I guess? on the hardware side, it's kind of hard to grasp, cuz I don't do a lot of programming is like, what is the impact of that kind of, I guess, enough hardware and software behind it?

Host 1     38:04

The the impact to autonomous driving? Or, you know, you're just or what? Oh,

Host 1     38:09

Overall, it's just, it's,

Host 3     38:11

I would I would interject it right. Now, it seems like you could almost have this thing learn and characterize an entire system. And then learn from that, you know?

Host 1     38:24

Yeah, I mean, the so, so, right, your, your audience are engineers. So let me make this a really simple like, way to understand. So thank you autonomous driving neural networks, like be they deep or shallow or whatever, or just, or just a form of mathematical function, right? So, so, the, the fundamental thing is like, given x, what is Y? Right? So the most basic form of that is, like, we love linear regression, y equals mx plus b. Right? And so and so what a neural network is doing is whereas we as engineers have spent our lives trying to create models we look at, we look at systems and we create, and we create mathematical equations that that

Host 1     39:08

Best represent the system,

Host 1     39:11

Exactly. Neural Networks instead are able to learn that set of equations, now that they are okay, they don't generally give you the set of equations coming out of it, you don't suddenly, you know, you don't have like a you know, a set of you know, a set of independent equations that you know, that together when solved, right, what 22 Yeah, exactly, what you have is is a network where were those dependencies and and the model relations are all encapsulated within the relationships and the weights of a network. And so, all that neural network is is effectively a general purpose modeling tool. That learns instead of being sort of, you know, generated by by whatever we you know, we project onto the system as structure because How many times have you have you have you tried to like make a model for a system, you build some crappy prototype? And like, those things are completely different way off,

Host 3     40:08

Right? Every time,

Host 1     40:10

Every time. And so and so imagine instead of being able to go directly to the prototype and just characterize it, right, like, obviously, that's not practical in like most cases, but that's effectively what a neural network is doing.

Host 1     40:24

Okay, gotcha, gotcha. So, so

Host 3     40:25

A neural network isn't actually learning, it's still following a set of rules that you feed it?

Host 1     40:32

No, not at all. I mean, I think it is actually learning, you know, it, think of it in a linear regression case. So if I were to start, you know, I have a set of data and I want to, and I want to, and I want to fit it, right, I could, you know, sort of do the one shot, you know, statistical equation that we all learned in, in, you know, in school, or I could kind of start somewhere, right, within with, with hypothesis, and I could, and I could, I could run an optimization, right, I could step toward or away from the data and see and see if my fit was better or not. And eventually, I would arrive at some minimum in that process, right? Hopefully a global minimum, that that meant that like, my error was as low as it was gonna get, and I fit in, I fit the data that is effectively learning, right, like what else is learning than us than us trying to predict the outcome of something and being wrong? And and then trying again, and getting better at that?

Host 2     41:33

Huh? Yeah. So from what I gather, I guess, then it's It's Min Maxine the system?

Host 1     41:40

Yeah, I mean, it's, yeah, effectively, effectively, it's making a neural network learn is is an optimization problem. It's a it's a gigantic multivariate optimization problem. Yeah, that's why that's why like, GPUs are so are so good at it is like a, like, basically, it's just a massive matrix, like multiplication, and then a matrix add. Like, it's just, but we're just gargantuan matrices. Yeah. And

Host 3     42:09

Do it do it a couple trillion times.

Host 1     42:10

Yeah, exactly. Yep. Cool. So hopefully, I hopefully like your your listeners aren't like, Oh, God, this guy is a total moron. I think I've got it right. No, I think

Host 3     42:20

No, no, no, no. If anything, Parker and I are the morons here. Yeah, that's

Host 2     42:23

Me. It's like I that's actually why I asked for the book because I this is a subject I have no knowledge about. Yeah, I'm fine really fast. Because like, all I see is like the neural network, like, pictures that people post on Reddit, right? It's like, we're like post cat pictures and like, read a painting or something. Yeah. Yeah.

Host 1     42:44

I mean, the there was, it's, it's like anything, right? It's the result of a whole bunch of of fundamentally simple little operations that when added together and make something that's like super complex and crazy, but, you know, like anything, right? The basics are pretty simple, right? Mm hmm.

Host 3     43:00

Yeah. Well, this is the Mac fab engineering podcast, where we go from talking about automated cars to electronic learning, the philosophy behind what it means to learn.

Host 2     43:12

Yeah. Speaking of car, a self driving cars, so we go back onto that subject. We kind of deviated a bit. So how far do you think we're off from self driving cars? Like, I know, like, Tesla's got some automated stuff that goes on freeways, but like, to the fact where you said earlier where you can get a car that does not have a steering wheel,

Host 1     43:37

Right. Alright, so that's a nuanced and big question. So let's, let's qualify it a little bit. Um, there's two types of self driving, let's start there. One is one that we care about, and one is the one that we don't. The one that we don't really care about that doesn't really have much impact on humanity or anything, you know, like, our daily lives is the Tesla style, at least the way it is now. Right, which is a feature that makes your car

Host 1     44:08

More desire driving straight Ryan line, right?

Host 1     44:11

Like, you know, you're gonna see some accident reduction from that. But fundamentally, the like, most impact is that is that someone's gonna buy a certain car because they don't like driving on highways over another car. Right? Yeah. The, the autonomous driving that we care about, that's like, autonomous Ubers. It's a ton Exactly. Autonomous truck driving. It's, uh, you know, it's sidewalk delivery robots. It's, you know, whatever, like, the list goes on. You know, it's it's, uh,

Host 2     44:41

That's what I'm talking about is where you can wake up in the morning and then the car shows up in front of your house and you just hop in and that takes you to work right?

Host 3     44:50

Yeah, a cup of coffee and watch YouTube videos on the way to work. Yeah.

Host 1     44:53

Which you know, that brings up a whole nother thing like so now we now we have like, however many more hours in a car You know, where you're gonna spend more time on Facebook? Great.

Host 2     45:04

All this the whole thing is just getting to more social media. So this Facebook actually have a self driving Vehicle program.

Host 1     45:13

I don't know. I mean, maybe it'd be in

Host 1     45:14

Their best interest. Yeah, in the best

Host 1     45:16

Interest. Exactly.

Host 1     45:20

How long will it be, though, is your question I mean, I think I think it's gonna be just a few years near to start seeing these systems and in limited fleet trials in, in geographically, like in geo fenced areas. So in fact, Wei Mo, just like published the thing where they're looking for riders, and they're starting to do that, I think in Arizona. So I mean, it's not too long before, you know, for people in the, in the tight urban centers, you start to be able to get these things on, you know, the right trips, right. But the thing for robots is that the more that you can limit the operational scope, the better they are, the better it is. So so you're going to start out, you know, highway driving, you know, trucks and and inner city, you know, Ubers

Host 3     46:01

Yeah. And, and all of them are gonna be using Polly sync core, right?

Host 1     46:06

Some of them Yeah, absolutely.

Host 1     46:09

Be Arduino take the wheel, right?

Host 3     46:11

Oh, wow. That's, there's some trust there

Host 2     46:15

Will be the code word, right? Was it Arduino? Take the wheel. Oh, God.

Host 1     46:19

I will caveat that, you know, especially with the Arduino system, you know, you really, it's not engineered to be safe with without a driver, right? It's not like let this thing go without someone in the driver's seat. It is it is meant to be, you know, safe on closed course environments where you have a safety driver, there's a couple of ways a safety driver can disable that thing. But just, you know, for your for your listeners, you know, it's really important to qualify that we have an operational envelope that we, you know, are cooperating?

Host 2     46:48

Yeah, yeah. So, I actually, that's another thing I bring up is, because right now we have vehicles that have sensors, let them know when there's cars in the blind spots, sensors that will stop the car, if someone jumps out in front of the street, or, you know, the person is not paying attention, and there's a car stopping in front of them. This is more of a, like a opinion, I guess is Do you think that makes people worse drivers?

Host 1     47:18

I mean, that's kind of like asking, you know, does wearing a helmet make you like a more risky bike rider?

Host 2     47:24

Exactly. It's it's, it's same thing with a seat belt.

Host 1     47:28

Right? Yeah, exactly. I mean, I think that, you know, potentially, yes, but still I wear my helmet and my seat belt, you know, exactly. Yeah. It's up to everybody to sort of modulate their their own risk. And, you know, the safety things that we put, I mean, that's always the trade off for safety is that like, you know, if you add a safety device, then people will become more more reckless, more tolerant? Yeah. You know, potentially, the the numbers are telling us though, that we're seeing double digit declines in insurance calls it severity. And frequency, I think, is what they call it. So severity and frequency of, of accidents, in where there's like those type of protective sensors. So they're seeing like, more than 10% declines year on year of like, frontal collisions due to adaptive cruise control, or auto emergency braking. I mean, that's pretty significant.

Host 2     48:23

Yeah, it's actually like people say, you know, texting might be the worst thing ever. But the worst cause for like, I think it's one of the major cause for accidents is still not using your indicators on your vehicle. Still, number one,

Host 3     48:36

Nobody in Houston.

Host 1     48:38

I mean, I don't I can't quote this off the top my head, but I think like the largest, you know, cause of fatalities, just people not wearing their seatbelt. Oh, yeah. I think that like, you know, I, you know, I wished it before we did all this autonomous driving things, you know, we could we could solve the stupid problems. But apparently, apparently, you know, that's a that's, that's a harder one than making a computer drive a car. Is making people wear their seatbelts.

Host 1     49:03

Yeah. Where are the seatbelts? Yeah.

Host 3     49:05

So actually, you know, I haven't thought about this much. And we probably don't have the answer to but what happens to the auto insurance industry, once cars go to automated driving? Like, does that just kind of dissolve? In some way? I don't like I don't know, I would think it would have to still exist in some way.

Host 1     49:23

I think that's a great question. And I think that also the auto insurance companies are starting to realize that's a really scary and great question. Yeah,

Host 3     49:31

They might be going the way of blockbuster. Since Netflix is coming around.

Host 2     49:35

It depends on what you do for other vehicles too. Because like my jeep, there's no way you can slap a policy device in it because everything is connected to something else mechanically.

Host 3     49:47

Well, I mean, of course it would be it's not gonna be a steep cut off. Where cars go from, you know, not automated to fully automated. It's going to be some kind of curve that goes down but but still Like, it's gonna change the face of insurance. And,

Host 1     50:03

Well, there's a lot of there's a lot of perspectives on this, you know, on the one hand, you know, these devices do lead to lower lower risk while driving. And so you know, on that hand, your your your insurance rates should go down. But on the other hand, you know, they're going to lead to autonomous taxis, which are going to mean that less people drive cars, which means the costs are, you know, not spread out among so many drivers. And so and so then, you know, maybe you're, maybe the rates go

Host 2     50:28

Up. And that's the thing, too, is like, if the computers making the decision, and the computer gets in an accident, right? Is it the fault of the driver or drunk occupants at that point? Now, they're not even though it's the manufacturer, they

Host 3     50:43

Paid for the

Host 1     50:44

Well, that's a really, that's a really vexing one. I mean, Volvo came out last year saying we're gonna be, we're gonna be liable for any accidents in our autonomous driving cars. And I think that's the way it's gonna go. And I think that's, I think that's the right the right way, actually, you know, it's gonna mean that, that we put the appropriate level of rigor into these systems.

Host 3     51:06

Wow, that's, that's a lot of responsibility on the manufacturers side.

Host 1     51:12

You know, that's, that's absolutely true. And I think, but I think it's, it's, it's, it's appropriate, right. I mean, they're, they're building systems that that just that can make decisions that have life or death or death consequences for us, in the same way that if there was a malfunction, due to like, let's say, negligence in an airplane, you would expect the aircraft, you know, the aircraft manufacturer to be held liable for that?

Host 1     51:36

Sure. No, yeah, that makes sense. Yeah.

Host 3     51:40

Yeah. And we don't we don't have to play pay our own personal flight insurance to ride in an airplane. I mean, technically, you sort of do. Right. But, but it's not direct.

Host 1     51:50

Well, like any service, there's a lot of things that are rolled up in the cost of it, you know, right. Yeah. Right.

Host 3     51:55

Right. But but our auto insurance is not rolled up in the cost of driving in a car. That's

Host 1     52:00

But but it is rolled up in the cost of taking a you know, an Uber or Lyft. Right, I mean, the fee that you pay covers, you know, their their insurance, as well as the drivers insurance. So, you know, this, this stuff comes out. I mean, I don't think that, you know, in most cases, it's insurance that prevents the rollout of technologies. It can it can help to accelerate. And it certainly it certainly can have, you know, great financial impacts to those with vested interest. But, you know, I'm pretty confident that as well as the regulatory environment will kind of sort themselves out.

Host 1     52:35

Well, awesome. Yeah.

Host 2     52:36

This podcast went all over the map.

Host 3     52:39

Yeah, this this was everywhere, every direction. The automated car of this podcast is just driving in every direction. Well, do you have anything else that you'd like you'd like to add?

Host 1     52:53

No, I mean, I think we, you know, we covered a variety of topics here. Any one of them, we could go really deep on. So you know, it's been a fun time. I've enjoyed it.

Host 3     53:01

Oh, great. Well, we appreciate you coming on. Would you like to sign us

Host 2     53:04

Out? Oh, wait, hold on. We got to do the. So our listeners can check out Polly sync by going to the URL? Oh, good point. Yeah. Holly sync.io. Follow them on Twitter at Polly sync. And there have a GitHub which I think they have their open source car control module thing. And I think you just search policy technologies on GitHub, and you'll find it. We'll put all the links in the podcast description of course,

Host 3     53:31

Right. We need a codeword

Host 2     53:35

Or we did have one. Except I forgot it already.

Host 3     53:38

Let's let's come up

Host 1     53:40

With no it was a Arduino take the wheel. It Oh, that

Host 3     53:43

Was that a code with Arduino take the wheel is the code word for this. This podcast. Alright, just wanted to sign us up.

Host 1     53:52

Okay. But before we do shameless plug here, we're always looking for, for talented engineers, you know, interested in the space, or, or adjacent spaces, and especially right now around safety critical systems, you know, in some of the, you know, some of the more esoteric technologies out there, like, well, that's where I'll stop. So if you're interested, you know, please, please reach out to us. info@polysynth.io I read every one of those and we'd love to hear from you.

Host 2     54:21

Do they get bonus points if they drive a Kia Soul?

Host 1     54:24

Absolutely.

Host 2     54:27

And is it um, y'all are in Portland, right?

Host 1     54:29

Yeah, we're in Portland. Beautiful place to live. But, but you know, we work with people all over all over the world. So

Host 1     54:37

Awesome. Yeah.

Host 1     54:38

Great. All right. Are you ready? Yeah, go for

Host 1     54:41

It. That was the macro fab engineering podcast and I was your guest, Josh Horton.

Host 3     54:47

And we were your host, Stephen, Greg and Parker. Dohmen. Take it easy, guys. Later

Transcribed by https://otter.ai