AI models may grab the headlines, but enterprise success depends on everything surrounding them. In this episode of Ship Happens, Per Krogslund sits down with Colton Shaw, Principal Architect at Spectro Cloud, to explore why infrastructure, security, and governance—not just model selection—will determine whether AI succeeds in production.
Everyone is talking about AI models—but the hardest part of enterprise AI isn't choosing a model. It's building the infrastructure to run it securely, reliably, and at scale.
In this episode, Docker's Per Krogslund is joined by Colton Shaw, Principal Architect at Spectro Cloud, for a conversation about the often-overlooked foundation of enterprise AI. They discuss why incidents like Log4Shell transformed software supply chain security, how Software Bills of Materials (SBOMs) provide visibility into application dependencies, and why AI-generated code introduces new governance challenges.
Colton also shares lessons from deploying Kubernetes across highly regulated and air-gapped public sector environments, explains why edge AI is reshaping industries from defense to healthcare, and discusses how organizations can balance open-source innovation with enterprise-grade security, resiliency, and operational control.
Whether you're modernizing cloud infrastructure, evaluating AI platforms, or preparing your organization for production AI, this episode offers practical insights into building systems that are secure, flexible, and ready for the future.
(00:00) Welcome to Ship Happens
(00:38) Meet Colton and Spectro Cloud
(01:22) Why SBOMs Matter for Enterprise AI
(03:57) The Log4Shell Wake-Up Call
(07:19) Hardening Containers for AI Workloads
(09:33) Security Requirements in Regulated Industries
(14:16) What Spectro Cloud Does
(15:54) Kubernetes in Air-Gapped Defense Environments
(19:16) Why AI Infrastructure Is Different
(20:45) The Current State of AI Operations
(24:24) Budgeting for AI: Tokens vs. Talent
(26:28) AI Sovereignty and Avoiding Vendor Lock-In
(29:58) Building and Serving Your Own Models
(32:54) Choosing the Right AI Model
(36:52) Model Routing for Reliability and Resilience
(40:33) Designing Invisible AI Failover
(41:02) Managing Developer Complexity
(43:45) AI Development Environments at Scale
(45:04) Edge AI in the Real World
(52:28) Why Human-in-the-Loop Still Matters
(58:22) Governance, Audit Trails, and Responsible AI
(01:05:20) Open-Source vs. Proprietary Models
(01:11:46) Moving Beyond APIs to AI Strategy
(01:17:39) Closing Predictions
Colton Shaw is a Principal Architect at Spectro Cloud, where he helps public sector and enterprise organizations design, deploy, and secure Kubernetes platforms across cloud, edge, and air-gapped environments. His work focuses on modern infrastructure, software supply chain security, AI governance, and helping organizations operationalize AI in highly regulated industries, including defense and government. With deep expertise in Kubernetes, container security, and edge computing, Colton works with teams building resilient platforms that support mission-critical applications at scale.
🌐 Learn more about Spectro Cloud: https://www.spectrocloud.com
🐳 Learn more about Docker: https://www.docker.com
☸️ Kubernetes Documentation: https://kubernetes.io
🛡️ Learn more about Software Bills of Materials (SBOMs): https://www.cisa.gov/sbom
📄 Learn more about the Log4Shell vulnerability: https://www.cisa.gov/news-events/cybersecurity-advisories/aa21-356a
Per Krogslund: [00:00:00] So welcome to Ship Happens, a Docker podcast where I sit down with some really smart people to talk about AI and tech and everything in between, and also how it impacts people in the world. Today, I have the pleasure of welcoming Colton from Spectra Cloud, and, um, we're gonna talk about AI, edge, security, um, governance, uh, and everything in between.
Uh, I know you're gonna have some really interesting stories to tell, but first of all, give me a little bit about yourself.
Colton Shaw: Yeah, absolutely. Um, well, it's a pleasure to be here in, uh, sunny San Francisco. I'm from North Carolina. Well, cool San Francisco. Um, I'm coming from the 100 degrees out there. But, um, no, I've been in this, um, AI space for many years now.
Um, I work at Spectra Cloud as a principal architect, so, um, really what that means is I'm supporting public sector government [00:01:00] customers making these kind of decisions, whether it's infrastructure, whether it's AI, whether it's just understanding how to use it. It's really all of the in-between, and I've been working with these, most of these customers for many years now.
Per Krogslund: I'm happy you could be here, and I heard we both had time to meet up while we were both in San Francisco, so, um, this is not my living room. This is, this is a studio. But, uh, I think, yeah, this is nice with plants and everything. Yeah. So in extension with, with thinking about governance and thinking about these edge devices, um, and these new environments where you really have these systems running, um, how, like...
Security becomes increasingly important, especially for use cases like, like, uh, healthcare. Right. Um, so how do you as, as a platform really approach that?
Colton Shaw: Oh, uh, so as a platform, it's the ability to provide kind of the declarative. You know what's going in your infrastructure because you've designed it via, you know, Terraform stacks or cluster profiles, so you know what's being deployed [00:02:00] inside of that infrastructure.
Yeah. But that part, you've got to take a step deeper, too, for our organizations, because I'm sure your audience has heard of SBOMs and all kinds of, you know, acronyms around that. But really, in these organizations, these, these industries that I work in, it's really important to understand not just what I've deployed, but what's being pulled as a dependency from that, and then what's being pulled as a dependency from that, so then we can understand this full stack of security.
Because most of the large attacks, hacks, whatever, of the last five, six years have been because someone's been able to inject, um, either maliciously or just unaware, some kind of code that allows for a backdoor. And I know we're talking a lot about AI, and that's really where AI gets dangerous. Yeah. When you have a system, and no matter how much you attempt to govern the system, if the system has a way for a backdoor to access the internet or access data that it [00:03:00] shouldn't have access to because you've deployed some kind of software dependency that provides it that, then we're exposing data, then we're potentially making those decisions that you, you wouldn't want to happen in these, in these circumstances.
So super, super important. Yeah.
Per Krogslund: And I think, I'm not sure the entire audience knows what an SBOM is. Uh, so it's software bill of materials. It's very... It's a security concept, really. But I think it's interesting in the sense of when people deploy a Docker container somewhere, um, they are not always sure about what's actually inside that container, and that's what really a software bill of materials does, is that it gives you, like, a very clear list of packages and versions and where they come from and so on, so that if there is a security issue with any of these components- Your security team will very quickly be able to identify, okay, what are the sy- what are the systems in our landscape that is impacted by that security issue?
Colton Shaw: Right. And I mean, you can look at, like, as an [00:04:00] example for why SBOMs are needed, the Log4j stuff that happened a few, what? 2020, 2021. Um, I was at a... My company was trying to figure out where that would exist. Like, do we deploy it? Engineers, customers are asking, do you use that? Is it- Yeah ... and so having to then chase down where it is, it, it becomes a task, especially for customers that are deploying in these regulated industries, is that a vulnerability I need to worry about?
Yeah. Because SBOMs have been around for a little bit, but they're not... They haven't been mandated for a very long time. I know the EU is one of the first, um, regions to mandate it. I know, um, here in the US it's now mandated across most systems to have that data. But before that, you would hope that you don't have it, you'd spend time trying to find it.
Um, and so now you're able to, with an SBOM, you're able to identify, kind of think of it like a file tree for anyone that's looking at it, right? You have the root that you pull in, right? I'm gonna, um, NPM pull or, or Docker pull an image. Yeah. And then that's now my root, and then I'm [00:05:00] building those layers down underneath it, and that's all part of the SBOM.
Per Krogslund: Yeah, it's all from the operating system to the application layer, to the dependencies, to everything. Exactly. Yeah.
Colton Shaw: I've,
Per Krogslund: I was, uh, I think it was the Log4j breach that really triggered this whole conversation about SBOM, because Log4j is everywhere and I, I was also, I was, um, I was head of product security at a company called Zalando at the point.
They had like 10,000 microservices in production, and just, like, chasing down, like- Yeah ... where is Log4j? And the answer was, like, everywhere. Um, but really, like, we had no means of being able to say, "Oh, which of our critical services has Log4j so we can prioritize?" We couldn't do that at all. We had no idea.
Right. Um, so it took a long time, um, to get to the bottom of it, and the end of it was just like, yeah, we just need to roll out everywhere, like, um. So everything was recycled, and it was an enormous time-consuming event to do that. Um, and there was a lot of, like, there was a lot of services that hadn't been touched in years.
Like, they were running smoothly and fine in, like, a closed environment, but we, we don't know if these are impacted, so we will, we will just update them.
Colton Shaw: Well, and that's, that's actually a good [00:06:00] point because that's why it becomes really important for health care and, and defense and all these other industries, because I'm creating- An appliance of some kind, right?
Hardware, an MRI machine, a drone, a tank, whatever it is, doesn't matter, but you're not gonna be software updating that all the time. You're not gonna be managing that, so you really wanna be sure what's in that infrastructure. And then when you need to update something, you need to have the ability to swap out certain layers and, and manage that because I can't take the whole stack offline for some reason- Yeah
to manage these, these security updates. So it's important to know what's in there, how it is, and that I trust it when it's shipped.
Per Krogslund: Yeah. Yeah. And it's also, this is not like updating your Windows computer every Tuesday. Gotcha. This is like you might update a part, uh, of that stack that will not bring it back to life again.
It might nev- it n- it might never start up again. Like you might- Right ... get a incumb- um, incompatible version of something that just breaks how it boots, and it will never start again.
Colton Shaw: Right, and even on top of that, when I deploy into these systems, I've [00:07:00] certified the stack that exists today- Yeah ... in the way that it exists today.
So for me to update a component of it, it might not come back, but at the same time, it might change the scope of that component. And now I have to recertify the entire device, which takes months to years sometimes.
Per Krogslund: Yeah. So I think another thing about SBOMs and how they kind of fit into this whole security story is, of course, I think seeing SBOMs and really seeing like an inventory of like, oh, this is, this is what is in my container, also really triggered people to say, "I don't actually need like 80% of the stuff that's in this container."
Like this is- Right ... this is like a multipurpose kind of system we set up. Like, but we actually don't need all the packages in there for our specific application, right? Um, so that's, I think that's where people have started thinking about distroless and like hardened containers and, uh, and really bringing down the number of, of thing- moving parts, really.
The fewer parts you have, the fewer parts could break.
Colton Shaw: Oh, absolutely. I mean, you have NPM packages for is it even- Yeah. ... is it on, right? [00:08:00] Like, we could build that into our, our stack, but absolutely, the fewer that I have to worry about or harden that layer on what can happen, the fewer gaps that I have as I move into this, um, AI world.
Per Krogslund: And I think, uh, it's also because, yeah, we don't really know what's gonna run in these systems tomorrow, what AI's gonna do with them, and what kind of exploits they can find. So just reducing the sheer number of, of, like, the size of the attack service becomes, like, critical.
Colton Shaw: Absolutely.
Per Krogslund: Um, and I think since when we, the, the team at Docker, they also work on hardened containers.
It's interesting to see what, like, the difference does. Like, some of these are, like, reduced in size by 90%. It's, like, a massive amount of stuff that gets removed compared to, like, developer just takes, like, node latest and then to, like, a, like, a hardened node version. It's like, it's, it's, yeah, 95% sometimes in time.
So it's, like, size reduction and package reduction as well. And it's just stuff you don't have to deal with breaking anymore.
Colton Shaw: Yeah. And, and Docker Harden is a, is an interesting topic too, [00:09:00] because you allow that ability to then take away other attack vectors that aren't just dependencies- Yeah ... but maybe I'm removing root access to it, or maybe I'm generating those SBOMs because I don't wanna have to do that for everything already.
So all being provided for, for me. So that takes some of that thought and... off of me as an organization or as a developer. So I'm making that, my life easier there by knowing that that layer is not something I have to worry about as much, as manage, I have to think about, but it's not something that I'm in control of necessarily.
Per Krogslund: Is this when you deal with your customers, is this something that they see as a mandate now? This is how we do things. This is now a, a critical part about thinking about security, this f- kind of like hardening things and so on. Whereas, where for a lot of developers, actually it seems like a little bit of a niche topic out in the real world, but I guess for your customers it's not.
Colton Shaw: Oh, no, it, it is. It is the number-one ask for most conversations. If we go into a customer, they like our product. They say, "Okay, well, show [00:10:00] me your SBOM. Show me your CVE. Show me how you do real-time scanning. How does all that happen? Because I need to know that when I choose you or use your platform, that it's secure."
Yeah. And then that goes up into my customer's mandates, right? They're mandated to know what gets shipped where and how, so this allows them to make that decision effectively. Yeah.
Per Krogslund: So do you think this is gonna spread from, you can say, these very regulated industries into, you can say, normal developer scenarios because of AI?
Colton Shaw: Yeah, I mean, you look at AI as a good example. I wanna use AI to manage my infrastructure, or maybe another example, I get an alert from my infrastructure that's a system is down, and it makes sense- Yeah ... for AI to go analyze what's happening in that system. Well, I can't just throw AI at it and give it, you know, the keys to the kingdom because who knows what's gonna happen from there.
There could be vulnerabilities I don't know about baked into it [00:11:00] by accident, on purpose, doesn't matter. But when I am able to provide a hardened container, that sandbox for it to pull from, I'm able to say, "Well, I trust my AI that is non-deterministic to go and make its decisions and pull its understanding from this container that I trust and control, and I've limited the blast radius as small as possible."
So now I can let AI do what, what it's good at, and AI's very brute force, right? It's gonna throw 100 things at it to analyze, and it's gonna do that when it's analyzing what, why your service is down. It's gonna continue trying to iterate on it, but it's not able to move outside of that box. And I think we're gonna see that, especially for developers and stuff as we move, as we shift AI kinda downstream to be more commoditized, to have access to it as a whole.
You're gonna want it to also have access to other services. You're gonna want MCP server layer and all of that on top of your AI. [00:12:00] And if you provide it the ability to have images that are mostly controlled and ac- you understand the access that they have, now you've, you're able to give it, give your whatever AI systems more available, availability for pulling that data to then surface you information.
Yeah.
Per Krogslund: So in a sense of the higher degree of security, you also have the higher degree of autonomy or, or possibilities with this.
Colton Shaw: Exactly.
Per Krogslund: Um, and then I think the other part is when developers use AI to generate code, they might generate some insecure code, they might install, like, an insecure library and so on.
So you would catch that in that process by, by really sticking to SBOMs. And then you have... Then it goes into, like, the normal security mechanisms that your, your security tooling will find that, oh, Claude installed this library that's actually, it's jQuery 1.2. It's like it's old enough- Right ... to drive a car.
So, um, those, those tools will also fit into that, like, SDLC that we really, I guess, trying to optimize for AI now. But we, we, right now we're really talking about [00:13:00] optimizing for speed. Um, and then- Exactly ... these different mechanisms that, you know, you could say this, like, traditional security work of SBOMs and governance and all these things might, might come in, become really, really handy in actually producing things at speed, but also at, like, high security.
Colton Shaw: Right. When I know what's going into my platform and I allow my developers to utilize Claude or whatever AI system I have underneath it, but then I have that layer of security that's generated in a PR via an SBOM, and then we can at least make a justification for why the version changed, why this dependency was added.
So then I am more aware for an organization how that's being used and what's being used inside of either what I'm delivering internally or externally.
Per Krogslund: So developers might not have any idea what code is being written, but they will know the, they'll know the SBOM.
Colton Shaw: Right.
Per Krogslund: And that will follow the system into production, and you will hopefully have things that will capture like anything that's like very broken at that process, right?
Colton Shaw: Right. It's the same as like having [00:14:00] tests on a- Yeah ... on a, on a platform, right? You run tests to catch regressions and know if you broke something. It's the same as an SBOM is the same concept, right? I know that I've added a new dependency that either changes something that I gotta worry about or ex- expands the scope.
Per Krogslund: I just wanna go in and talk a little bit about Spectro Cloud, because it's not a household name. Uh, but you've been around for, like, close to a decade. Um, you're now one of the leading clouds, you can say, for AI infrastructure. But maybe tell the audience a little bit more, what do you do, in your own words?
Colton Shaw: Yeah, absolutely. I think Spectro Cloud is really attempting to simplify what can be quite complex. So running Kubernetes clusters isn't easy. If you're running one cluster, maybe you can manage that. Um, but you have these large organizations running hundreds of clusters, thousands of clusters, edge devices on top of that.
So that gets really difficult to do. So bringing that down to a level that anyone can do that, [00:15:00] standardizing that across the board so it's predictable and repeatable, is really the core of what Spectro Cloud does. Um, there's a lot of facets that you can tack on top of it, right? When we deploy stuff, we deploy on top of immutable operating systems with declarative stacks, if you want all the buzzwords.
Um, but really what that boils down to is simplifying what's often hard is that infrastructure.
Per Krogslund: So it's really the, the making, the scale- It's the scaling part you wanna make easier for customers. Like, these people who have, like, thousands of clusters, and that brings a certain amount of challenges in there for them.
Colton Shaw: Absolutely. Scaling, um, and that, that just ability to redo it in a predictable fashion. Because if we're, if we're running a dev environment, you want that dev environment to be as close as you can to production. And if I make a change, tweak it, put that change in code, deploy it somewhere else- Yeah ... um, and, and just keep, keep on the same.
Yeah.
Per Krogslund: And I guess you also, like, again, looking for on the, the, l- looking at the people you work with, like, it seems [00:16:00] you also decided not to make it any easier for yourself by working with, like, regulated industries like health care- Oh ... and defense and all these things, like air gap environments. Um, how is it working in that space?
Because, like, most people who use Kubernetes today, it's like, "Yeah, we'll get it as a commodity on AWS and just go with it." But that's not the choice that, you know, very regulated industries and defense has, really.
Colton Shaw: Oh, absolutely. I think it's a huge misconception, like you said, that you can just deploy in AWS, you have the resources, and it doesn't matter, and I have internet connectivity, right?
Um, but oftentimes, I work in public sector, so US DoD. Um, and when you go into these environments, it's air... So you can't bring anything in. Um, so I've, I've been into some environments to where you have to bring in every package on a CD that has to be validated before you enter that environment. There's no access to internet.
You have a little, uh, red ethernet cable if you need something. So no access. You gotta bring in documentation. So that's really where having a simplified platform helps [00:17:00] in those environments, because they're not just calling out to ChatGPT and Claude and asking the question. These guys have to know what they're doing.
Yeah. Um, so it's, it's very complex in those environments. And then there's a lot of requirements. Yeah. Um, because you have to know what you're deploying and why. So- You end up with these organ- these people that are often used to commercial deployments saying, "Why don't we just go faster?"
Per Krogslund: Yeah.
Colton Shaw: Well, you don't realize that when you're in regulated industries, energy, health care, defense, all of that, faster often means risking lives in some direction.
Yeah. And so there's faster with an asterisk that we need to validate what's happening. Yeah.
Per Krogslund: Because you have... there's so much at stake. If things go wrong- Right ... you really need to be sure, make sure you know what you're doing.
Colton Shaw: Right. Absolutely. And so everything has to be trusted.
Per Krogslund: Yeah. This is interesting.
So when you work in these environments with these developers who work in these kinds of environments, like if you said they don't have access to the coding agents and all these things that everyone is [00:18:00] so excited about, or at least like everyone has adopted in, like, record time. They don't use that at all, or...?
Colton Shaw: Not, not, not at all. No. Um, they have limited fashions of it. Okay. So again, d- working with US defense, so they build their own kind of, um, LLM, so they can operate with- Okay ... which aren't quite as smart as the, you know, large Claudes of the world. Um, so- No, no. They'll be lagging
Per Krogslund: behind, right?
Colton Shaw: Exactly. And then those don't have access to certain environments.
So you do go places to where you just have to use what's available via the internal documentation or your brain- Yeah ... and that's, that's it. Um, and then sometimes you can abstract. So they often have, um, what they call high side, low side. So we'll build... And that's, you can often think of that sometimes too as like dev and prod in, like, more of the commercial world.
But you build on the low side to where you have access and you've abstracted some of these things away. So maybe what I'm designing today isn't, um, you know, a missile system. Maybe it's abstracted to where it's just a thing that does something on my developer's [00:19:00] computer. So then I can use the large Claudes of the world, and then I can ship it over to the high side, make some changes, and then now it is a missile system, or now it's an MRI machine because we've moved it over to that secure environment to where you can't access those tools.
Okay.
Per Krogslund: Gotcha. So, um, you describe yourself as an AI infrastructure company, a platform company. What is the difference here between... We've, we've had infrastructure companies like, especially cloud native for the last- Mm-hmm ... ten, 15 years. It's been, that's been the default for the world. But now we're all of a sudden talking about AI infrastructure.
What is, what is the, what's the significant difference here in running AI infrastructure versus what we've all been used to for the last 10 years?
Colton Shaw: Oh, there's, there's so many different layers on top of that, right? You look at mar- today's infrastructure, ignoring AI for a second- Yeah ... you have different kinds of servers.
You have, you know, you gotta worry about RAM and CPU and how large those can be. However- Those, that, that's, that's one layer. But then you throw AI on top of it, and now we're talking how many [00:20:00] GPUs can I stick in there? They're energy, um, inefficient, right? So we're pulling all that energy, and that's just the hardware layer of that.
Then we have to think of what can I deploy on top of those? Are they gonna be as outdated quickly? So that, that often happens now, especially with how quick AI is moving. My hardware might be outdated in, in a year or two.
Per Krogslund: So this is a legit concern for you as, like, an infrastructure provider that-
Colton Shaw: Yeah, I mean-
Per Krogslund: buying something that goes old too fast, basically.
Colton Shaw: Oh, yeah, all the time. Yeah. And that, that's part of what we're attempting to do, and a, a lot of these large companies are as well, is if you can abstract away the hardware to an extent, you can extend the life cycle a bit. Yeah. You can make it so I can manage it easier, I can use it longer, and I can shift its focus as we go.
Per Krogslund: Gotcha. Okay. So again, so you have AI on top of the, like, Kubernetes infrastructure that you're providing. So what's your experience been with customers? I guess customers come out and say, "We wanna, we wanna have AI infrastructure because we're gonna..." You do AI for whatever reason they wanna do it- Mm-hmm
right? But like, what's, [00:21:00] what's been surprising to you when you talk to customers about, like, where are the challenges here? Like, why is this hard? Like...
Colton Shaw: Yeah. I, I think- They don't quite know what they don't know, right? You could say, "Today I want AI infrastructure," and that's, that's cool. You have money you're gonna throw at it, that's cool.
H- um, but there's a lot of layers that go on top of that. It's how are you gonna pass the GPUs to your applications? How are you gonna manage the models? What models? Who's vetting these models? How are they getting there? And then how are you sharing access to those, right? So there's... That's why we, I think we see the default right now to just go with the large models of the world- Yeah
that we can, that are public because it's easy. I don't have to manage that. But when I really need to manage it, there's so many layers- Mm ... to think about. And then that's where, like abstracting that away to an extent and saying, "Well, you have your hardware, now we're gonna standardize that, and now we're gonna make that layer easier."
'Cause anyone listening, if they've tried to [00:22:00] hand roll, like local, just local AI hardware, right? I do it in my home lab all the time. Yeah. It's a lot of work, and if something goes wrong, you and Claude are spending hours trying to figure out what went wrong- Yeah ... because you did something, you know, odd.
Per Krogslund: Yeah. And that's a home lab with a single user.
Colton Shaw: Oh, uh, yeah. Yeah. That's a home lab with a single user. This is not
Per Krogslund: 1,000 clusters. No. It's not like capacity planning anything. Exactly. Just that one. Yeah. Exactly. Do you think that's because AI model serving or inference and so is just so immature that we haven't gotten to a stable point yet?
Like the... I think we all kind of take Kubernetes for granted today, but it's, it's, it's been a massively maturing part of the ICT ecosystem. It's been like a standard across everything. So now we're just used to things just work. Right. But now we're kind of starting over with like entirely new technology, and we just haven't reached a point where this works in a stable way.
Colton Shaw: Yes. I, I would, I would agree. I mean, even when you layer Kubernetes on top of that, um, most of the drivers and everything are still beta or alpha- Yeah ... for that. Um, depends on what stack [00:23:00] you pick, right? If you want Intel, um, AMD, like how that's gonna pass through, they're still designing those... We're building the plane as we fly it- Yeah
as a kind of example. So what I have today, there might be d- Big production bugs that I gotta work through because the software's just being tested out, or a new GPU comes out, or new hardware comes out that's never been tested on that. You've gotta sit on your hands while your users or your use case wants a, an AI workload that uses that.
Yeah. So you've gotta figure out how to bridge that gap.
Per Krogslund: Yeah. But isn't that also kinda exciting? Like, you know, we-
Colton Shaw: Oh, yeah It's very exciting to be in that world because you never know what's gonna change. I, I... And I'm sure everyone has experienced this, but three years ago we just had a single ChatGPT, and that's what you used.
Yeah. And then now it, it's everywhere, so to the point where we're abstracting basic boring tasks over to AI, which, which is what I'm most excited for, for organizations especially. I think it's [00:24:00] really cool to embed AI into all these apps, but when you start to abstract it away and when an organization can deliver that in a seamless way, so maybe I need meeting notes, maybe I need topics, like, in an easy way that takes effort off of me as a human- Yeah
to put that onto that which is really, really good at that, and I'm not trying to deliver these really complex workloads- Yeah ... then you really can see the value, and that's super exciting as well.
Per Krogslund: Yeah. So also, um, for the, you know, VPs of engineering and CTOs out there, for the last 10 years, they've kinda known what they had to deal with.
It's been more standardized things, things that are commodity. We start offloading the infrastructure teams into just, like, managed AWS or, or Azure services or whatever, right? So it was more of a thing of, like, yeah, we might keep some stuff ourselves, but the rest is commodity. We don't wanna deal with that.
Like, that's for the non-regulated industries really, right? But now they're like, um... Now you see CTOs and VP of engineering be- being back at conferences, back at, like, investigating what does this mean for me and my budget and my technological [00:25:00] strategy going forward, like what's on the tech radar for next year.
Mm-hmm. Like, like, these things have been standing still for the last, like, five years basically. There's not been a, like, a massive change for how companies run their infrastructure basically. That's my outside perspective, perspective at least, like.
Colton Shaw: Yeah.
Per Krogslund: Yeah. I, I- But now it's changing. It's changing rapidly.
Colton Shaw: Oh, yeah. I, I, I think you're right. To maybe hit your point a little bit too, like, you look at the SREs of the world- Yeah ... the people that are responsible for the infrastructure, they have standardized response plans for everything, and it's, it's become so predictable that it's become automated, and that's, that's fantastic.
Yeah. But now when you overlay this whole layer of AI attack vectors as one, it's much less predictable for how your system could be compromised. But for two, just delivering models to your users and how you wanna do that, it's also a whole different world. And, um, you're seeing these VPs, CTOs of the world that had to make the decision, do I- hire more people or do I pay more [00:26:00] for tokens?
Yeah. Right? We have to make that trade-off. And now I think we're seeing the shift back because one day the shoe's gonna drop and tokens are gonna go up 'cause they're subsidized right now. So now I need to think of hosting my own software. Where am I gonna do that? Because I've already made the decision to invest in it.
I can't rehire a ton of talent and ge- expect them to be where I was, so now I've gotta pivot to local or managed AI infrastructure.
Per Krogslund: And y- I guess you're in this business as, as well. You see this as a, as a provider, I guess, for also the kind of customers you have are also looking for, like, alternatives to depending on a single vendor.
That is, that's always a dangerous spot to be. And I think- Mm-hmm ... I think a lot of... Especially, again, uh, coming from the European side, uh, we saw a lot of customers, a lot of developers being very surprised about Fable, the Claude Fable model being withdrawn from, like- Oh,
Colton Shaw: yeah ...
Per Krogslund: non-American usage, basically.
It's, it's out again now, but it was, like, a bit of a shock to the ecosystem, and I think it made everyone realize, oh, we built up a [00:27:00] strong dependency on a single vendor, and now we don't know w- if this is gonna go away again or anything else. But I think everyone realized the danger of where they were in.
Um, so we see a lot of, a lot more people now thinking about, oh, should I use GLM or should I use, like, one of the other, like, models that is... It's not the same, but it can, it can get the job done, right?
Colton Shaw: Right. Well, it's so funny you say that. So that was a super big, in my realm of the world- Yeah ... Fable, because of what Fable's able to do to the NSA internal systems.
Yeah. Um, I had a coworker that kinda gave me an analogy that I'd never thought about. So if you think, you know, I know guns are a hot topic, but assault rifles specifically are highly controlled across the world- Yeah ... right? Because of what damage they can do. I think we're moving to that world for AI-
Per Krogslund: Yeah
Colton Shaw: because the models that we can create now just can be so unpredictable in the damage they can do- Yeah ... that now they have, not have to be controlled, but you wouldn't want someone to [00:28:00] use them to attack your company. Yeah. So we have to think about that factor, and we can't secure and update all of these systems because a lot of our large infrastructure is, it's not legacy, but it's been around for a while, so it's gonna have more vulnerabilities associated with it.
So now I have to look at, how do I secure all of that, or how do I remove the attack factor?
Per Krogslund: Yeah. Yeah, that's a different part I'm thinking. So there's, like, two big players, right? There's OpenAI and there's Anthropic. Mm-hmm. And you could say you, from the US side, you can regulate that by just saying you cannot export this to the rest of the world.
But, like, it- that's- it's not happening in a vacuum, right? You also have several models coming out of China, like open source models. You also have, like, s- open source models in, in Europe as well. So you have all these, like, smaller vendors. They don't have the same resources, but again, it is likely just a matter of time that they will hit- Mm-hmm
like similar capabilities, right? So, like, an export ban is probably not the, like, solution here.
Colton Shaw: Yeah. I, I would agree, and I don't know that there is a perfect solution for [00:29:00] it. Um, so I'll, I'll be very interested to see where the governments and the markets of the world end up taking it. But, but you're right, because, like, even locally, I can run some really, really powerful models, especially if I buy some more GPUs.
Yeah. I can almost recreate Claude. Not quite, but almost.
Per Krogslund: It's close. It's close. And yeah, it'll just get closer and closer. Yeah. Yeah. I heard, uh, Michael Poland, he said, um, maybe we should think about AI as kind of nuclear. Like, we were able to contain nuclear, uh, weapons by agreeing as a world that this is, like, a really bad idea.
We need restrictions. We need, like- Right ... we need agreements about, like, what we can do here as nations. Um, and he's just, just maybe we should think about this. Maybe not now, but down the line we should think about this. Like, if this keeps getting more and more powerful, we need to think about this as the world really.
And he was also- Right ... he was also, he was also thinking about in the, in the context of, like, how you can say AI content and fake videos and so on is impacting people. He's more, he's more looking at it from that perspective rather than, like, cybersecurity. I, I,
Colton Shaw: yeah, I would agree with that.
Per Krogslund: Yeah.
Colton Shaw: That as a whole, yeah.
Per Krogslund: Yeah. So, um, so that's a [00:30:00] little bit of a, a tangent into, like, you can say world impact of AI. But what if, if we go a little bit more local and you think about, like, your customers and how you see developers using this. Uh, on the topic of sovereignty and the topic of, like, vendor lock-in and choice, uh, like, as a, as a platform provider and also with these different sectors that you serve, like, w- where do you see how...
What possibilities do they have? What options do they have? Like, we are, we are thinking about this in a world where we have two big vendors that is, like, serving most of the world right now. But- What are, what are you seeing in the custom space and in the platform space about how people can work with this?
Colton Shaw: Yeah, I, I think that actually goes back to what I, what I said earlier with the CTOs are kind of making that pivot from, you know, how do I pay these large vendors, how do I budget it, to how do I build it myself? And I think that's a really fun pivot, in my opinion, because that allows for a lot of this customization of models, but it also removes that lock-in.
So it, it means that I'm not just beholden to my cloud bill when it comes in every month and I have to pay it, or I don't have [00:31:00] a product anymore, right? So it removes that so I can move local, but then I also get control over what I'm able to do in that environment, and I can build a lot of really fun ideas outside of that.
So I can start to create my own models. I can pull open source models, and then I can serve those up to my users in different ways and give them access to more trained content, which is really awesome. But then I can start to give it access to more of my stuff, too, which means that my models can be smarter, right?
We can train these on top of my code, my internal knowledge base, and I can trust it, especially as, like, a proprietary company. I can bring that internal. And then there's products like Spectral. We have our Launchpad for AI as an example that allows you to deliver models as a service, right? So you're able to control that infrastructure inside of it and then deliver that to your users in a predictable way.
Per Krogslund: Yeah. So when you think about that, like, what is the technical leverage here that organizations have? Like, well, how would that work? Like, you can say you [00:32:00] make models available, but is there, like, different means they can build into their infrastructure to deal with this? Like, how, how would they approach this, like, in a more, like, detail level?
Colton Shaw: It, it kinda comes down to their capabilities, and that's part of the learning gap that ends up having to happen or abstracting that away to an extent. You can train models. You can pull open source models. But the journey starts, in my opinion, we're pulling models that exist today. We're testing those models, and then we can deploy them on top of the infrastructure I have.
Yeah. And then we can use a tool like, you know, Launchpad for AI, and we can push those on there, and I, I'm taking away a large burden of management from that- Yeah ... putting it on top of the tool that already does it quite well. And then as my journey expands, then we can start to have some engineers building our own models or training them via RAG pipelines and all of that, and then design those.
Per Krogslund: Yeah. And I guess, yeah, so that's the first step. So I, I talked to, uh, Pia Nilsson from Spotify a couple months ago, and she said, like, interesting, "We are using [00:33:00] all the models. It's not because we want to, or, or, like, all the agents, but, but we wanna measure it. Like, we wanna measure what is the difference between the different vendors we have available today."
Mm-hmm. "So we at some point can kind of decide, yeah, this is better for that thing, so we can route that kind of work in that direction. And here it makes... For these kinds of tasks, it makes sense to go in that direction. Uh- Mm-hmm ... but we don't know right now because s- it's such a black box. Like, we don't- Right
know if it's OpenAI or Claude or GLM or whatever, like, uh, Op- like Deep Seek. What are they actually good at? We need to kind of test it and try it out and see where does these things perform." Um, and I think that's interesting in the sense that at some point There will be a competitive advantage for the people who went from passive consumer to, like, really poking the system, understanding the system, understanding the capabilities of saying, "We invested in actually knowing what we're dealing with here, so we can make a, like a rational choice- Right
instead of just rooting for AI vendors as like a sports team," which is kind of what s- I feel that- Yeah ... this cause on LinkedIn is, right?
Colton Shaw: Absolutely. I, I think that's a really good point [00:34:00] because there are mo- each model has a different skill set associated with it, and I think as engineers we often might default to our favorite one, and that's just what we use for simple tasks because that's what's, what's there today.
Um, but, so as an example for w- what I do at my house, which I think, yeah, I'm kind of vetting out some of these things that I'm working with, um, a lot of our defense partners for. But, like, at my house I'm building parsers for pictures for receipts as like a kind of boring... Right, I wanna figure out what's on the receipt and, and pull it up into stuff.
Yeah. And you could throw Opus at it if you want to. You could throw Sonnet at it, but they kinda suck at it, but they'll do it.
Per Krogslund: Yeah.
Colton Shaw: But through different testing, right, Haiku is the best at it. Yeah. It doesn't do any better. But then there's other models, right? Some of the smaller Google models, the... I forget the name, but they're like mini models-
Per Krogslund: Yeah
Colton Shaw: are even better at it, and they're the cheapest to run possible.
Per Krogslund: So- And they're actually good at this task, like-
Colton Shaw: They are fantastic at this task, as opposed to throwing an Opus that will spend t- [00:35:00] 20 seconds thinking, and then it'll give you a response, but you know it's not the best response, versus some of these trained at it.
So I think organizations internally, I, I would agree, they're doing... They have to do a lot of that testing- Yeah ... to see what works. And even at Spectra Cloud, we have our favorites that, that we pick as individuals because those fit the tasks that we do. So whether I wanna use ChatGPT because I'm on a, um, you know, maybe a customer success team and it's really good at meeting summarization and just general research, or I'm more tech, I need to use Claude, or I run my own with some of the Llama models and stuff- Yeah
you get to pick that.
Per Krogslund: I think this, this is also fascinating, is the sense of, like, uh, we've, we've had this motion of, like, infrastructure is not a competitive advantage anymore. We're just gonna offload it to AWS. It's commodity. This... We add value somewhere else in the chain. But all of a sudden it's actually like, oh, we actually need to go back and look at infrastructure as a strategic asset that we really need to [00:36:00] understand to build up these capabilities, and it does make a difference if you know one model over the other and you're constantly evaluating and so on.
It's not j- just like picking Kubernetes anymore. It's, it's much more different. Right. It's like totally opposite. Actually, the, the, you get the value from this by specialization instead of commoditizing.
Colton Shaw: And I, I think you save a lot of money doing that too, because you realize that running a 300 billion parameter model at home is expe- or not at home, but in your organization- Yeah
and serving it up to dozens of people is really expensive. But if I know that this task requires a 8 billion model or 24, right? I can make that decision, and then I can load multiple models onto shared GPU. Yeah. And then I can run dozens of them at one time versus one for a few people.
Per Krogslund: Yeah.
Colton Shaw: So you end up expanding what you're able to do by moving those tasks off and what they have access to.
Per Krogslund: Yeah. So yeah, so, so comparing this, like, overprovisioning in cloud to, to AI, um, like developers don't have to make those choices anymore for [00:37:00] cloud. Like, there's- Yeah ... typically at least. Like, there's typically a platform team, there's typically processes in place that kinda helps developers make those provisioning choices today.
That's, that's the maturity of the field. It's quite natural. But we're not there yet with, with, with AI, right? Right. So it's more like AI sh- like, developers shouldn't make those choices, like, probably in the future.
Colton Shaw: Right. They, they definitely shouldn't have to, because as a developer, I think if you have to make those choices, I think it extends the scope of your role outside of producing quality code and, and managing that to, what am I using right now?
How is it impacting my line of work? And, you know, is it being good, or should I try something else? Yeah. So what, what we're seeing, and I think the kind of market will need a lot of, is like model routing, as an example. So I make a request up to a shared service. There's a whole big backend behind that shared service, but on top of that backend, it can dictate based on the request which models it goes to.
Yeah. And then what it's able to do, because each model is [00:38:00] gonna be different token consumption based on that. So maybe I'm, I'm asking a simple task, right? What does this function do at a high level? I don't need to send it to the largest, baddest model available because that's a waste of resources, waste of what I have available, versus I can send that and that context over to a much smaller model that can deliver me, um, an adequate or if not better response.
But as a developer- I don't need to know that. I, I would argue maybe I, if I'm interested in it, sure, but like in my day-to-day job, it doesn't matter. It's res- returning a result that I'm able to then move on to the next task in my job.
Per Krogslund: Yeah. Yeah. That's also interesting if you kind of go down the line of how routing might help in the future, is then you can, you can start dividing your traffic into saying, "Okay, let's try, let's try to do like a, like a green-blue deployment," basically.
Mm. Saying, "Okay, let's try to route like 10% of these tasks into that model instead and see if there's like, like significant degrading in service." Right. Uh, "Oh, this actually works perfectly fine. Let's just move everything here because we get more money, like more, like, work done for the [00:39:00] same kind of money."
Colton Shaw: Or my developers are complaining.
Per Krogslund: Oh, yeah, yeah. Yeah.
Colton Shaw: Yeah. They don't, they don't like the responses. Okay, that's a bad model.
Per Krogslund: Yeah, exactly. So, so you, you will see this over time, I guess, the, the field with mature organization will start making strategic choices of like choice of, of model and so on. And again, going back to the Fable story being cut off from Europe, that's probably also what we're gonna see, like this need for sovereignty, that, that organizations really will probably have like, um, a portfolio of models.
They'll have backup solutions, and they might lose access to some, and maybe a new release will degrade some other parts, but it- they really need to stay in control, right? Right. They can't just be single vendor. They need to figure out like not just depending on one model for everything, just with cost thing, but also, yeah, uh, performance and then in the end, being in control of what they can do as an organization.
Colton Shaw: Well, it goes down to an age-old topic of operational resiliency. Yeah. Right? If I'm running a SaaS platform, I need to have my disaster recovery figured out. I need to have my failover figured out. There's all these very [00:40:00] well-known topics that I need to do as checkboxes. As models or, or AI as a whole today, we don't do that because, you know, I, I use Claude, I pay Claude, and Claude's available.
Yeah. When Claude's down, what do developers do now? We kind of sit on our hands, and, and you don't have access to much, and you wait till it comes back up, watch some YouTube or something. But, so the ability to have that failover, the DR happening all for it- Yeah ... I think that's what model routing will end up going to because we've abstracted that layer away to the point to where, uh, the operation still continues.
Obviously, something can happen, take it all down, but that's, that's different story. But the operation continues. I'm able to go to a different service. But as a, as a technical person, no matter what your role, you don't need to care, especially if you think like Slack as an example today. You're on Slack.
You don't know if they've failed over to, you know, AWS East- Yeah ... or West. You just know Slack had a quick blip. "Oh, great. I can now message my people again." Yeah. That's all you care about. Yeah.
Per Krogslund: [00:41:00] They figured it out on the behind the scenes.
Colton Shaw: Exactly.
Per Krogslund: So, um, so again, these, these things about, like, what developers do, what do they really need to know today?
So we've worked on shifting ops on developers, like with DevOps, then we shifted security into it with, with, uh, SecOps and DevSecOps and all these things, like shifting everything left. That's been the mantra of the last 10 years. Yeah. And now we have... So today, developers understand cloud, they understand security, operations, uh, some even, like, certain SRE topics as well, like just maintaining and operating their services.
Uh, and now we have... Well, and then they, of course, know Kubernetes and, and, and observability and- Yeah ... and now they also have to understand AI. Um, and maybe it, this thing will probably mature so they need to understand less, or at least make less choices, but they do need to understand the capabilities of the technology.
Are we putting, like, too many e- eggs in one basket of having developers understand all these topics? Or are we more seeing, like, a shift in capabilities that, like, just, like, [00:42:00] I think we can all agree it's probably not as important anymore to write, write code. But it seems like there's still, like, enormous, like- amount of topics that developers really need to understand is only growing.
Colton Shaw: Yeah. I, yes, developer, especially nowadays, um, and I know you'll hear a lot of devs in the market that like, "Oh, I, I wanna write code. I, I, I do this for my job." Um, but when they get into an organization, you're throwing... They have to think about a lot of stuff now, which is error-prone to an extent, and I think that's, that's where SRE teams and that's where dev containers and that's where all this stuff came from, to try to remove that mental load so they can focus on what they know.
Because you can't know everything, you can't manage everything. I mean, even in my role, right, I'm more sales technical, but there's a whole world out there that I can't know it all. Yeah. Or, you know, I'd stay up all night watching videos and learning and studying, but to be effective in my job, I have to know kind of my scope and- Yeah
[00:43:00] how to deliver that and, you know, some stuff around it. So I think as we go for AI and, and developers and, and all of that, I think it's cool and, and they, they wanna know it, which I think is... 'Cause it's the biggest thing to happen in our industry for a very, very long time. Um, but the intricacies behind it, I don't think most average developers need to worry about that side of it, because they as a developer need to worry on how is the code as a whole orchestrated and, and architected well.
Per Krogslund: Yeah.
Colton Shaw: And that's, that's a big topic by itself.
Per Krogslund: Yeah. So this is more like a bump in the road, and we'll get back to that focus at some point once we kind of mature these different systems and processes.
Colton Shaw: I would hope so. Yeah. Yes. I, I think we're seeing the market shift in that way. Um, especially as we move to managed AI infrastructure stacks, we're seeing that movement back to an organization is gonna provide you with the tools to do your job effectively- Yeah
and you're gonna be able to do that well with those. Um, and [00:44:00] we're, we're seeing that shift back, but right now, you know, in six months something could be new. Yeah. So I, I'm excited for the world when devs can have like a, their own little box at home that can do it, because the closer you get the AI to you, oftentimes the better it can be.
Um, so I, I, I... So I used to work at- Cisco as an example, and they would ship to you a little VPN, a little router on top of it, 'cause they know to get access to you, you gotta bring it close to home- Yeah ... so then I can have less latency inside of that entire organization, right, to access all the backend services.
Yeah. I have a feeling one day we'll move towards more of that as well. So I've managed stacks of, um, large organization boxes, but then developers, much like laptops, might be shipped, um, dev boxes as- alongside it, and that's your local. And then we shift everything up, but those are managed devices with models on it- Yeah
that are all controlled that, you know, internet goes out, your job can still move forward.
Per Krogslund: Yeah. So you kinda get this hybrid model of you have something on the edge, [00:45:00] and you might fall back to, like, a bigger cloud resource if you need it.
Colton Shaw: Exactly.
Per Krogslund: Yeah. Yeah. Um, that's actually also, when we say just to, on being on the edge, 'cause this is also one of the things Spectral Cloud does, uh, with, uh, edge infrastructure, edge AI.
Um, and I think that's probably, that's, it's a topic we don't really talk a lot about in the industry. Um, we are talking about these are big centralized services, but there's a lot of things happening in edge use cases- Mm-hmm ... uh, like bringing AI into, like, very close to where the data and the f- like, actual, like, physical thing is happening.
Um, so I know you as a company have, like, actually quite a lot of, like, experience with it. And I've, uh, we talked about it before. If I feel like this is really interesting, I know you can't really mention names because of the, the kind of clients you have, but maybe you have a couple, like, examples of how, like, AI is actually deployed in, like, some really interesting use cases already.
Colton Shaw: Yeah. Yeah. So there's a few... So again, kind of pulling from a lot of my public sector stuff I do, um, we're seeing for drones as a really cool hot topic [00:46:00] is we're pushing AI down onto small devices. Yeah. Or maybe they don't have AI, but inside of that ecosystem of a drone or a restaurant- Yeah ... right, you're gonna have a lot of data that exists there.
Yeah. And because that data exists there, and because bandwidth isn't Always available everywhere. You can't take all that data, ship it up to your large strategic cloud, and allow that to analyze and then push down decisions to a restaurant or a drone or, or anything else. Yeah. We're moving that AI to these smaller form factor edge where the data's available, and it can make those decisions, whether it's what terrain looks like or whether it's optimizing restaurant flows.
Yeah. Um, so we're seeing that movement down to what we'd call edge nodes, but I think when we say the term edge nodes, people think, like, I'm throwing a Raspberry Pi in in the middle of, right, the Middle East, right? Right. Like, that's not really an edge node nowadays. Edge nodes are everything. You think, um, restaurants-
Per Krogslund: Yeah
Colton Shaw: have edge nodes. I used to work in, in retail [00:47:00] for cellular sales, right? And we had POC devices. Those are edge nodes. Where I'm putting my credit cards on, those are edge devices that have processes running on them. Maybe it's containers inside that store. There's a little tiny stack of infrastructure equipment, and nowadays you're seeing a lot of AI deployed inside those stores to allow that all to function- Yeah
when, you know, connectivity's strapped or just moving it down to where the data's available because latency costs. Yeah. Right? If I have to call back to big cloud every time we do anything- Yeah ... it's, it's a lot of delay that, in those industries, doesn't matter if you're in defense or restaurant, the delay costs- Yeah
money or lives.
Per Krogslund: Yeah. Yeah. I guess a, a simple example is, like, you don't want delay or potential, like, issue with connectivity when a customer's about to put their credit card on a thing and buy something for you. You want that to be fluid and fast, so yeah. Exactly. Yeah. Um, I guess it's also, like, a good use case for edge nodes here in the con- when, when the data is so sensitive, like health data for instance.[00:48:00]
Um, like this is not something you can... You can't send health data into the cloud to, like, a random, like, AI vendor.
Colton Shaw: Right. You want that all to run locally. You want it all to be trusted- Yeah ... locally. Um, and so that's where you would, you would manage that- Yeah ... inside of your small stack- Yeah ... for health data.
Per Krogslund: Yeah. And so this is already a space you're, you're in, and this is already something that's, like, deployed in production, which I find really fascinating because, like, if we ag- if we're gonna go to my favorite topic of the LinkedIn discourse because I think it's so absurd. Like, there is, there is these opinions out there of, like, either it's like AI is a bubble, it's gonna dest- destroy us all.
We all gonna, like, f- or it's just, like, fake. It's not useful or valuable. Uh, and then there's, like, these, like, AI hype people. I don't know what we should call them. Like, AI, AI maxers occ- is the new term Yeah. Yeah, yeah. Um, where they're like, "Yep, software is gonna be free. It's like everything's just gonna be automated.
Uh, everyone will have their own version of software." And so, so there's, like, there's, like, extreme opinions out there. Yeah. So that's why it's, like, so it's so interesting to see actually that you have [00:49:00] organizations, health care, restaurants, like, really concrete businesses who actually have AI running already in production, and it's actually, it's giving them value.
They're solving things that they couldn't before.
Colton Shaw: And if it's not working, they're learning. Yeah. So then they can iterate on it again. Yeah. And then they can deliver it in a repeatable way to learn off of that. Yeah. But, but yeah, I think to your point, though, the world is not gonna be one of those extremes.
Software is not gonna be free- ... or AI is not gonna take over the world, or maybe we don't know that yet. But, but I think that that middle ground is what's the most important for organizations to think about because I want to bring the cost of my software down. I want to allow AI to operate inside of my infrastructure.
How do I do that? What am I thinking about? How am I deploying those components? Um, and then how am I making my whole organization more effective? Um, so a- as, as an example, I, I was reading recently someone said that if you pay a developer $100,000 a year, right, and then you can make them [00:50:00] 20% more effective, so that would cost you $20,000 to buy another developer.
You could probably do that for less each year through AI and make them more effect- you do it all day. Yeah. Because you're, you're making your operations more effective because you're, again, back to the very beginning, you're offloading a lot of those simple tasks in that middle ground, right? We're not bringing our software free.
At the beginning, we might be making it more expensive because we're learning, but then once we've learned those lessons, we can start to optimize on top of that and really start to deliver specialized models- Yeah ... add on that routing, um, and you really have full control there. Yeah.
Per Krogslund: So going, going into this, like, they're learning.
They're building things. They're building real things. Uh, what has been the biggest surprise in that process with, like, working with customers and see how they do this? Like, when, when you can say you're actually trying to build a concrete thing, it might not work as you're investing. This every- something, especially with tech, that's so new.
Mm-hmm. But what's been your, like, biggest surprise in this, working with organizations doing these kinds of things?
Colton Shaw: I think, uh- They're more ahead [00:51:00] than sometimes you think, um, which has always been very interesting to see stepping into an organization who already has a well-thought-out idea for how they wanna build it.
Um, they... And, and there's, there's large connected. I was, I was at a conference this week, and, um, you don't think how AI is used in defense that much, right? 'Cause you're like, "Oh, it's not that trusted." But, um, we were working with an organization, and they were describing to us, um, kind of this hub-and-spoke AI model that they deploy out in edge sites all around the world today, and they've been testing it in production for a while now.
It's essentially AI running on a Jeep with edge sensors that pull in all this data, and it makes decisions on the Jeep, and then the Jeep has access to, you know, Starlink if you want the internet or an upstream to then push that data up. So that aggregation story of the data, what's happening with it, how it's working is, is well on its way to being figured out, and they're learning, and that's-- I [00:52:00] think that's the most important part because it's not gonna probably work exactly how you want today, but we're iterating on it, we're building, and we're identifying those challenges.
Um, and I, I think that's, that's been the most surprising. As someone who... I, I'm, I'm-- I love AI and everything it does. I'm very passionate about using it. I think everyone should learn it. Um, to see a lot of these org- organizations that you would naturally think that they haven't done much for it yet are sometimes ahead of you, and you have to catch up
Per Krogslund: So you said with the Jeep is interesting, right?
Because it's, it's, it's taking compute and AI into, like, hostile spaces, but also spaces that are really close to people, and this is really, in the end, what, what we are here to do every day. It's, it's not to... Like, running code in the data center is one thing, but it's really, like, bringing this technology close to people and solving, like, very human problems a lot of the times.
Um, so again, like, yeah, it g- it's, it's challenging to have, um, a system running on a Jeep, I guess, uh - Right ... or running in a hospital- Right ... because it's a, it's a, it's a very different environment than, like, a pristine data [00:53:00] center in the middle of a desert.
Colton Shaw: Yes. A- a- and that's what I think people don't realize is how complex...
So we take AWS for granted, like, we can deploy in there, how complex or how difficult these edge devices, essentially compute these edge environments, right? So we've seen deployments for running AI alongside, um, hospital hardware and stuff deployed in that. And so you don't own the hospital's hardware, and you often can't make changes to that hardware because it's gone through a lengthy validation process.
Yeah. But you wanna give doctors, as an example, the ability to surface additional data in a secure way so they can make decisions for their patients, so they can make recommendations. Yeah. So how do you do that? How do you manage that? How do you deploy that at scale? All the, all the networking challenges that go alongside that.
Um, people will often take those for granted or forget about that, but that's where we've seen the most success, is [00:54:00] because we're not touching that layer that is almost untouchable in health care or in military because that works, but we're giving those doctors that data. And that goes back into human in the loop as well because when we're run- even this Jeep as an example, it's, it's not driving itself, right?
Someone's driving it. Yeah. But there's sensor data. Or you look at planes. I mean, taking AI away for a second, there's sensor data in planes. It can probably autopilot. I'm not a pilot, but the pilots are making decisions for what the plane is doing based on the data that's available to them today, and you're seeing that in health care, throwing AI on top, in defense, throwing AI on top.
It's how can a human make a decision with the information that they have- Yeah ... today.
Per Krogslund: And so, so this is like an additional layer. It's like adding value or adding like additional context to their work. It's not just about... We are not gonna AI-fy doctors. That's not the thing. Like-
Colton Shaw: Right ...
Per Krogslund: the, the uniquely human thing is still to be able to decipher all these things and make a decision in the end, right?
So you're not making decision systems as such, but more like [00:55:00] context providing or data providing or of like doing some of the work so humans can make better choices.
Colton Shaw: Right. Surfacing and aggregating the information. Um, as an interesting example, so I've been working in the Air Force for many, many years now, um, and there's a term called an Air Operations Center.
It's, it's an AOC, and that's basically where they would centralize their opera- so think of it like a company's headquarters. Yeah. They're gonna centralize there, and one of the big problems that they had was there's so much data available, and humans manage that data. So how can a human, unless they're up 24/7, right?
You're running shifts with these humans, but how can they make decisions for people running real missions? How can they consume that data, internalize, make a decision, and then how do you remove the junk data? Yeah. Because oftentimes, in any industry, you're gonna have tons of data that, um, is just not relevant.
As an example, AI is taking a picture of terrain. That's a tree. That's a tree. That's a tree. I [00:56:00] know what a tree looks like. I don't care- Mm-hmm ... what a tree looks like, or that's a healthy, um, you know, lung- Yeah ... or something like that, right? Like, I wanna surface what's valuable to make that decision. Yep. So, so the, this AOC, Air Operations Center, um, they created the same concept.
So they brought data into a large library of data, so called a UDL in this context, and they're able to then have AI aggregate over that information, pull out what people are doing at certain moments. So am I flying over this country? Am I doing this? And then I can surface it to the person that makes that decision and says, "Well, this human, or this group of humans are flying over this country, and there's this weather system moving in.
What do I do? Are they at risk? Are they not at risk?" So that human can make a decision for the lives and safety of everyone downstream for that, because you don't want that abstracted away to AI because models are unpredictable right now. They're, they're not deterministic. Um, I had, uh, [00:57:00] a partner I was working with, um, a few years ago said that they can make AI deterministic.
I'm like, "I, I don't think that's how it works." Mm-hmm. Uh, maybe one day, but I, I think just the math behind it does not work in that way. So a human needs to make those core decisions. Yeah. And then you can abstract the boring away, the data aggregation away, and all of that.
Per Krogslund: All right. Um, and I guess that's one, one of the things we can kind of compare...
It's cool to s- hear about these, these use cases being done, because I can see the parallel between me using Granola to understand a meeting with a lot of people talking at the same time, and I just get the most important points out of it, right? Yeah. That's, that's like a very local, one-person use case.
But I can see how this is, this is... Behind the scenes, in a lot of contexts, this is expanding. We're using the same capabilities as understanding a meeting and people yapping on top of each other, but... And but here you have, like, AI being really good at understanding, like, medical data or landscape photos or whatever it is, uh, to really digest it down to something that a human can then make a decision on top of.
Colton Shaw: Exactly. Yeah. Simplify the boring for that.
Per Krogslund: Yeah, [00:58:00] yeah. And so being an s- edge, like, AI provider, you have all these things you have to deal with. Again, like, making a system work on a Jeep is one thing, and making sy- the system work in, like, a old, like, f- I don't know, I wouldn't call it prehistoric, like, IT environment as a hospital can be, I guess.
Yep. Uh, or air-gapped environments where you don't even have access to the internet. So you have a ton of stuff to deal with to make that stable and predictable and I guess even, like, approved. Like, um, so that brings me kind of to, to a topic that not a lot of people like to talk about, but the whole, like, governance.
Like, when we deal with these cases of, yeah, doctors making medical decisions or even, like, military making military decisions and then, um, how do we govern all this? Like, what are the mechanisms that we can really do? Like, I've... As a simple person who's using AI to code with, I can do governance of, like, locking down what files it can see or locking down what kind of websites it can go to and so on.
Like, that's, that's a very simple governance model, uh, like sandboxing things down. But I guess, like, when you talk about governance in those [00:59:00] kinds of scenarios, it gets much more complicated.
Colton Shaw: It does get much more complicated for that. I think governance comes in many flavors for based on what you end up doing.
Um, and that's where dividing the- AI layer away from the actual deployment of software layer, so... Or from the hardware layer. So AI is not running on our MRI machines. AI is running alongside somewhere else, and then h- using that data. And so governance, it's really gonna be based on what data has access to, how am I controlling the data it has access to, or the decisions that it can make autonomously.
But then I'm bringing a human in that loop every time something important needs to happen, so then we can make sure it's moving in that right direction, um, alongside i- its path. So whether that's making a medical decision or, or a decision to move a vehicle or anything like that, we can bring that in.
And then for these infrastructure, these, these secure environments, it comes down to [01:00:00] testing and at least getting some form of predictability around the model. So that goes back to your AB testing as an example. We can't do that in production for a hospital, right?
Per Krogslund: No.
Colton Shaw: But- I hope not ... it also goes down to the audit trail.
So we can, we build an audit trail behind it that someone can... A doctor can go back and say, "Well, how did we get to this conclusion? I, as a medical professional, disagree with the system that came out of it, so how can I understand the decisions that it made to get to this outcome?" Yeah. So then either I see something that I didn't know before, or we can adjust the model and provide that feedback up, um, and then we can make those adjustments as, as needed- Yeah
to manage those models.
Per Krogslund: Yeah. I think that, that's an interesting point of this, of, of the, that you as a person with, who has, you can say, the education, the knowledge, you might disagree with the conclusions of an AI- Mm-hmm ... system at some point. Uh, and there it's... I think that's, that's essential, right? That we don't get to this point where, uh...
I don't know if you've ever seen the, the, the joke about, like, computer says no. I don't know if you've seen,[01:01:00]
there's, like a, a English sketch show, but there's, like- Okay ... a, a, it's, like, just a person trying to interact with a government official and is like- ... "Can I, can I get this thing? How do I get the help?" And she's just looking at her computer, and it's like, "No, computer says no. Sorry." It's like- Yeah ... if you ever tried to deal with an airline to get, like, a rebooking, that's the experience, right?
Yes. It's like, "Nope, system says no, you can't do that."
Colton Shaw: Yeah.
Per Krogslund: And so, and again, those are kinda, you can say, they're annoying, but they're kinda harmless in the sense that, yeah, you can't get your ticket as you wanted to, or, like, you can't get the form, or you have to do additional things. But, like, in a, in a medical situation or, like, a military situation, you probably don't wanna get to a point, "Oh, computer says no, you can't.
We are not gonna operate you because computer says no." Uh, like, this ability of humans stepping in and still being in control, uh, so the system is a servant, not a master, uh, is, like, that seems to be more and more important, right? Because we are putting autonomy into more and more important processes.
Colton Shaw: No, I, I would totally agree.
It, it really comes down to the ability to also know what's bad output. Um, and so that's where I think we lose [01:02:00] some- We lose some ability with that when we start to favor spending on AI over humans to an extent because you lose the person that can make that decision or has that context and has that understanding.
Some layers, sure, if you wanna analyze large data at scale, okay, maybe that's more of an AI-shaped task. Yeah. But what that data means and then how do I dig down to know is it seeded with bad data? Do I have outliers that are feeding it and AI is making a bad decision? I mean, you, you have your friends i- in your life that, um, will ask a question to AI and that's the answer.
Yeah. Right? And I always have to tell people, teach them how to, to query it- Yeah ... basically, because it's, it's not the question you ask necessarily, is how you ask the question. What... Are you feeding it a bias? Are you feeding it to then deliver you a, a result? And that's not a good question, right? And so how do we do that and then build that into how we serve our models across all these [01:03:00] platforms- Yeah
and what they do with that data.
Per Krogslund: Gotcha. Um, I had a, I had a conversation with a, with an engineer from, um, like an AI drone company. Um, they build military drones. Um, they're equipping them with AI, and they have a... They, they have this very interesting discussion. They're looking for a way of like, "Okay, how do we do governance on these things?"
Colton Shaw: Mm-hmm.
Per Krogslund: Uh, because they're gonna be autonomous, and, like, they're unarmed right now. They're just used for reconnaissance and reconnaissance sense and stuff like that. But at some point, someone's gonna put a missile on these things. And then it's like, what do we do when the AI drone does a war crime by accident?
Not because it's evil. It's not like being... It's not like Skynet thinking, but just, like, it inter- interprets data wrong- Right ... and just decides to, like, fire at something that it's not supposed to fire on. That's like a very, like
Colton Shaw: Uh, that's important. Yeah. I mean, you don't want that... So that's a, that's a fun conversation because I think that goes back actually to what you said for sandboxing for containers- Yeah
right? And I've seen similar examples for drones, and I've been involved in a lot of that [01:04:00] discussion- Yeah ... is you end up having to split the decisions out, right? Yeah. So on a drone, as an example, I might have a container or a sandbox that allows it to perform certain functions, and if it wants to do another function like fire a missile, like do something that could be destructive, it has the ability to go upstream- Yeah
to the next layer, and that next layer, most likely not on the drone, but that next layer is responsible for that decision, whether it's a human or a system that then identifies a decision and takes it up another layer. I mean, you have connected planes now, right? So you have clusters of fighter jets that have...
The fighter jet manages these clusters of devices around it, or pulls from all these sensors and pulls from all this data. It's the same thing that, that jet's making a decision based on the data that it has and then feeding it to the human. So in, in the drone examples, often they're deployed or used by a group of humans.
Mm-hmm. So there's a group of humans, many layers removed potentially- Yeah ... but we're able [01:05:00] to surface that data up, make a decision, and then send that decision back downstream. Yeah. But you wanna get as close as you can to that drone for the decision-making because that loop is what's really important.
Yeah. Whether that's in health care, whether that's in warfare, that loop is really important and how quick that can happen in accurate, predictable way.
Per Krogslund: That is interesting. So I wanna shift a little bit, um, back and make some of the topics here. So we see, we see this technology being applied in so many cases, and it's really, it's a capability that's really changing how we think about solving these problems.
And so, we have, of course, we have the frontier models now who can do a lot of these things. We've also talked about there's a lot of these models that can solve some of these problems even better and, and more cheaply, right? So we get up in- into a mix, and that's, that becomes really interesting for you 'cause a lot of these models are open source and just freely available.
So all of a sudden, we actually have, like, really powerful technology that's just, like, freely available. If, of, of course, we need a GPU, and, like, you need a machine and so on. But it's... If you think back, like, 20 years in this industry, [01:06:00] like, back then, to get a database, you had to get a contract with Oracle or Microsoft.
Like, just like- Right ... getting basic technology is like... But that's, like, gone. It's just like you can find, like, really advanced technology on the internet today. Uh, that's just like it's a, it's a model that we wouldn't have thought would exist, like, 10 years ago. It's like, it's like nearly... It, like, compares with, like, alien technology if, if you think about it in that way.
Oh, yeah. But you can just download it from GitHub. Like, it's, it's available on Hugging Face. Oh. Yep. And then you can start having these, like... You can start applying this interesting technology to, like, very complex problems without asking anyone for permission.
Colton Shaw: Uh, right. And there's pros and cons to that, but I think it's, it's a good move for the industry, but it's also a scary move because you don't know bad actors, um, hacking organizations with these models.
But at the same time, that level of a- of availability gives the organizations some more choice that they have. And you look at the rise of open source as a whole. As an org, I can fund open source, I can sponsor them, I can support them, then build stuff, and then [01:07:00] I can pick between all the options that I have, and so now I can interswap those layers.
We see the same for models in organizations now. Because they're surfacing up, I can test them, I can pick between them, I can pick what works for me. I can pick this one for monitoring faces. Um, there's, there's a bunch of models for that and this for that. And so I get to pick that. But on the flip side- You don't know if you have someone...
Well, you, you definitely have people using models to attack organizations now too. Yes. Um, but we've also, because it's available, we've commoditized it to an extent. Now, at least for the most part, organizations have access to what is out there too, to make decisions on how they protect themselves from that.
So i- it's an interesting world I think we're going into.
Per Krogslund: Yeah. Yeah. I think it's, it's, it's interesting when you see the news and you see these people being very critical about, say, oh, yeah, we can see all the bad things right now.
Colton Shaw: Mm-hmm.
Per Krogslund: Because we can see you know, Nigerian scam emails are getting much better.
Like, it's not a broken English anymore. More believable, yes. Much more believable, right? And you can [01:08:00] see also phishing emails much more believable, and it can be automated at scale, right? You don't need to... Like, attackers don't need to sit down and understand the, the context or the personality of the individual victim that they're trying to phish.
They can just automate it by looking at their LinkedIn, right? So you could just like... They know what they've been dealing with, so it's like, "Hey, blah, blah, blah, this person from this department needs you," and they, they get all the context automatically, right? Right. So they don't need to, like, they don't need to make, like, a artistically made email anymore.
They can just automate that. Uh, but of course, we see all these things up front because, like, scams and so on are not a regulated industry. They don't need to think about governance and compliance. They don't care. Right. Right? But it's interesting that we can... We have some people who have, um, who have, like, a very basic IT setup but can really start applying this, like, technology to, to do something like this.
It's, it's of course scary, but sort of interesting in the sense that we really lowered the bar of, like, access to this thing. So we will start very likely see much more, like, positive use of it down the line once we kinda hit the mainstream of where people are maybe not so financially motivated of just, [01:09:00] like, banging out a billion emails- Right
to kinda scam people, but more we will get, we will get students or teenagers trying to bang something interesting really using a model because they just have access to it.
Colton Shaw: Right. I, I think- That's just where the market i- the whole world is gonna end up going is, is making these more accessible. But I think that's also a good thing because you have the downsides of it, like you said.
But when we make that more accessible, then we can... We don't control, right, Big Brother control of what you can do for AI. Um, I think I hear a term like the AI underclass, right? Like, you don't want that level of control because then what ends up happening is I can do stuff better, faster, and more efficient than you can because I control the access to this data.
Per Krogslund: Yeah.
Colton Shaw: But when we push that down to allow really creative things to happen, we then allow more people access to that, more... less control, more, more... Maybe less control is a bad term, [01:10:00] but, um, kinda more ideas that come out of that that don't live in a small vacuum. So then you'll see this kinda, in my opinion, surfacing of, um, really cool ideas from people that are able to harness their local AI and, and make that decision and control build themselves.
Per Krogslund: So you could say the positive version of this is that there's no, there's no barrier of entry to the, like, core technology. Anyone can use it, so this is not a competitive advantage anyone to, like, have that. But it is certainly a, a competitive advantage of money because, like, GPUs are expensive. Right.
Colton Shaw: It's gonna co- It's just like Bitcoin mining and stuff like that. Yeah. If you wanna do it, sure, you can have a GPU and you can run a few. Yeah. You might make a few bucks. Yeah. But if you really wanna do it at scale, someone needs to fund it, and you get into a whole different story with that. But yes, it, it's not as accessible because it does cost money to, to do all those.
I think we'll see it get smaller one day and, and where you have CPUs that can run it. You have unified CPUs and GPUs that can run stuff better, so it's getting smaller, um, which makes it more accessible, so [01:11:00] you hit both sides of that coin.
Per Krogslund: Yeah. So if you like, the, the competitive advantage of having access to a large amount of GPUs in the future might be less important because we will get models that's good enough with lesser resourcing.
Colton Shaw: Right.
Per Krogslund: Yeah.
Colton Shaw: And then it goes down to where you put them and how you're doing it. Yeah. So that complexity of scale for deployment for my organization, right? Maybe I don't wanna have a centralized large data center with models running in it, but now I can stick a GPU or I can stick a small AI inferencing device at every location, in front of every computer, at every developer station, and we've kinda moved that around.
But then I have to think about how I manage all of that, how I support it, um, how I update that, how I make sure the models are doing what I think they're doing.
Per Krogslund: So it brings me to kind of, I don't know if this is a question or more, uh, uh, an opinion decis- disguised as a question, but more, we have all these people today who say, "Yeah, we are like AI first and blah, blah, blah, because we are like, we're using Claude to [01:12:00] generate code."
That's, that's not really it. That's just, you're just a passive consumer of a piece of technology, right? Right. Uh, you're just, you're just getting stuff from the internet, like you're getting generated text, fine. But more this thing you're talking about, this thing of like being able to make model choices based on your use case and your financial situation, like your, your budget for a given service, your return on investment of this thing, making those choices.
That's when we go into, as you can say, engineering, like thinking strategically about this technology, not just if I'm choosing sports team one or two of AI providers, but really thinking about, oh, we, we could go with a small like model for this specific thing and offload that, but over here we need that thing, and this thing is running on the edge and so on.
So that's really thinking strategically about your, your AI landscape all of a sudden.
Colton Shaw: Exactly, and I fully agree that you see orgs that wave the flag- Yeah. ... that I'm, I'm AI, but it's just... I mean, even all the AI companies that really just offload to an API token with Claude- Yeah ... or ChatGPT. Yeah. But, um, [01:13:00] the orgs that are doing, in my opinion, really cool stuff are the ones that are thinking about, how can I make knowledge more available inside of my organization?
So am I running a RAG pipeline against my internal data, my conversations in Slack, my code base? Am I providing like a vector-optimized ability to query this so now I've made everything more efficient? And that all goes to the AI story 'cause I can't generate that without embedding models. I can't do that without having my AI journey figured out and designed in a way that I can scale and support.
Again, failover, you don't want your organization to stop when you have your, you know, one set of models down. But those are the organizations that I think are really gonna see this growth in this next phase of technology because they're, they're harnessing what's available and delivering creative solutions- Yeah
to that, whether it's monitoring their infrastructure, whether it's providing, you know, knowledge base internally, whether [01:14:00] it's people asking, "Has anyone asked this question before?" Do, do, do, here's a Slack message. Because it has the availability for that data. And now human in the loop, but we're able to make the better decisions with- The less free time that we often have, so now we can move faster as an organization.
Per Krogslund: Yeah. So, looking at this, like, looking at the delta between the, the, the passive consumers or, like, the, the Claude users and then to, to organization and customers you've dealt with that, that's already at that point, like- Mm-hmm ... where do you see the... Like, where is the skill gap? What is, like, organizations want to move from passive Claude consumers to understanding these things on a technical and strategic level so they can make better decisions.
And, like, how do they need to move forward? What is the... Like, what's the steps here? What do they need to really, like, learn?
Colton Shaw: Yeah. It's, it's a huge technological skill gap. I think it's very, very complex to just say, "I wanna do that on my own." Yeah. "I wanna spin [01:15:00] up my own vLLM clusters, and I wanna manage my data.
I wanna do that myself." Yeah. It's a huge level of learning. You're hiring teams to do that for you. Um, so I think these organizations partnering with, uh, a vendor that is able to deliver that, right? So they're able to take that layer away, and essentially, what do you have access to? What are, what is the budget that you can do that within?
How do you wanna do that, and what are the outcomes that you want? And then we can build that in-between layer, because that's the hard part that changes every six months, that you as an organization don't wanna think about. If you go back to the AWS analogy, you're not having to think about what's happening with how your RDS database is being managed for your data- Yeah
because you've abstracted that to AWS, and you just know, "I want Postgres, you know, 14.7 or 16, and how do I go between the two?" Because that's what's important. So I think the same for AI, right? I don't wanna have to think about how my shared models exist and all of this. I just wanna [01:16:00] know that it's handled, right?
I wanna know that I can deliver the outcomes for it, and I can start to think creatively without having to get really, really deep in the weeds. So then I can solve problems for my organization without worrying about setting that up. And that, that's where when we say, like, Spectra Cloud model as a service, that's really what that comes from, is the ability to say, "I don't, I don't worry about this layer because I've provided the hardware.
I, you know, know I have the expectation of what's there. I know what I can do with it, and now I can add new models. I can swap it out, and it happens dynamically in a way that I can predict. And then I can start to move up the stack." Yeah. "And I can start to worry about what models are better, or how do I deploy them?
Or how do I build these pipelines- Yeah ... and what do I deliver from them?"
Per Krogslund: Yeah. And then you can learn. You can measure, you can learn, you can iterate.
Colton Shaw: Exactly.
Per Krogslund: And then you, then you move forward, and, like, there's not a lot of progress of, of just consuming an API point for tokens. Like, you learn [01:17:00] nothing. Like, you do.
You get, you get value out of it. It's not like it's valueless. It's not what I mean, but more like if you, if that's your only consumption, you're not really measuring anything. You don't have any control of how to improve the service because you're just sing- you're choosing a single vendor here. Exactly.
And they decide what to serve you with, basically. And they, they decide if you, they wanna throttle you, as they do. It's like- Yep ... uh, right? And at some point they're also gonna decide to raise prices, right? So it's, there's, there's like, there's a limited amount of things you can really learn, even if you measure all the activity you do by interacting with a cl- with Claude, for instance.
But it, it won't bring you ahead. Like, you won't learn a lot of things that will make you progress into the next point.
Colton Shaw: Exactly.
Per Krogslund: So I think we're nearly out of time, and I wanna wrap this up. And I think, um, I think what you just mentioned about the skill gap is a good learning for anyone who's watching this.
Uh, but also if we look at this historically, we say, like 20 years ago, it was a... It was everyone believed they should own the hardware, and then that became less important. We offloaded that to the cloud. Now we're kinda like going a little bit back [01:18:00] again. Yeah. Infrastructure is not a commodity. It's like more like a strategic choice that you need to understand.
But then, like, what's gonna happen from here? I know it's h- really hard to make any kind of, like, uh, predictions. But f- if we think about, like, if we're sitting here again five years from now, like, what has then changed? What do you believe that people have completed? Like, uh, we've talked about the thing about, like, software will be free and so blah, blah, blah.
That's probably not gonna be the case. But, like, what, what do you think fundamentally changes in the n- next five years?
Colton Shaw: I mean, I think it, it goes on most of the topics that we've hit on so far. I think really we give the ability for AI to be almost a commodity at the point to where I don't worry with the little intricacies of who I'm calling or...
But it, it exists and I can use it effectively. But then I can also remove kind of that boring complexity. I think that's really where we see value as humans, that we want, we want more free time. We want the ability to make decisions with [01:19:00] more information without having to spend hours researching, you know, what does this one thing mean, right?
So to, to aggregate that data up to us. So I think allowing that to be everywhere, to an extent, in a way that allows us to have ourselves in the loop for the decisions for our lives, for our companies, but we're able to do it in a way that makes us more effective or gives us the ability to have more time for the things that are important.
Which that could be learning if you're an organization. That could be free time if you run an AI at home for stuff. Um, so you're just removing that layer of, of not work, but time, right? Giving back time wherever it is. Which often is one of the hardest things in an organization to have, is the time to deliver whatever you need.
Yeah.
Per Krogslund: So I think last question. We are... We actually... This is not my living room. We're, we're in, we're in San Francisco in a studio , but walking on the street here, every bus, every train, every, every [01:20:00] billboard is like something something AI, something something agents, something something AI, AI, AI. Do you think in five years we'll stop seeing AI on billboards, not because AI goes away, but just because it is like, "Look at this fancy service that can do this for you"?
But it, it likely contains AI, massive amounts of AI, but it's just that's not the selling point. The selling point is what it does to you- Exactly ... not, not, not the components in it. The outcomes.
Colton Shaw: Yeah, the outcomes. Yeah. The outcomes are, are what's most important. I think, yeah, AI is sexy so you gotta throw it on everything- Yeah.
um, because that's how you get big investments. Yeah. Uh, um, but I, I would agree, because I think that's what's really most important, is not what you use AI for, but it's the outcomes that are driven from that. So sure, you have AI inside your meetings that provides you a summary of it, but I don't care that the AI provides the summary.
I care that I have the summary and that I have action items, so now I don't have to take notes on everything- Yeah ... because I have that in there. Uh, um, so I think we're gonna start to [01:21:00] see hopefully that removed and, and go down, commoditized then to the point where it's more efficient because of the software that I've overlaid on top of the AI making the smaller decisions for us.
Per Krogslund: So it's all about the outcomes. It's not about the technology. Yes, it's
Colton Shaw: all about the outcomes.
Per Krogslund: Yeah. And I think that's an amazing note to stop on. And, um, thank you for joining and thank you for the conversations.
Colton Shaw: Absolutely. And,
Per Krogslund: uh-
Colton Shaw: Thank you for having me. It's a pleasure ...
Per Krogslund: it's been wonderful to hear about AI being used for real things in real environments for real people.
Um, that's always refreshing. So again, thank you for sharing all these stories and your perspective on all the things. And, um, this was a Docker podcast called Ship Happens, where we talk to, like, interesting people, like Colton, here, about AI. And, uh, we will continue to do so, so see you in a future episode.
Thank you for listening in.