AIAW Podcast
AIAW Podcast
E159 - Zero-Data AI, Agentic Systems & Next-Gen Video Models - Agrin Hilmkil
Join us in this week’s episode of AIAW Podcast as we sit down with Agrin Hilmkil, Member of Technical Staff at Latent Labs, for a no-nonsense dive into the frontlines of AI innovation. From the latest LLM drops and agentic AI trends to training hacks and experimental breakthroughs, Agrin unpacks it all with clarity and insight. We also break down key takeaways from Google I/O, including AI-powered search, the Imagine 4 image generator, and Vue 3’s lip-synced video output—plus a bold look at self-trained models, Claude 4's whistleblower mode, and Europe's approach to AI infrastructure. Whether you’re building, researching, or just trying to keep up, this episode is your fast-track to what’s shaping AI right now.
Follow us on youtube: https://www.youtube.com/@aiawpodcast
How did you get into training? It happened one and a half years ago, oh heck, yeah. So there is a date from which I decided things should change now.
Speaker 2:Okay, Goran, we need to have some of those before and after pictures up on the screen.
Speaker 3:I can find them. No, don't do that.
Speaker 1:I don't think there are any, but you've been shaping.
Speaker 3:You look fucking awesome man, thank you, thank you. So okay, what happened then? I guess we can go.
Speaker 1:So there was like a coincidence of many things going on at the same time in my life. I think I have a vivid moment of just looking at my arms and realizing that they're not there anymore, they're floppy, and a lot of things happened at once. I think I had always thought at some point I would want to feel better about, you know, my physical strength. Uh, it just never happened and I realized I'm over 30. Uh, it's not gonna happen by coincidence now.
Speaker 2:You need to work after 30. Now we need to work. You have to work for it. I turned 50, motherfucker, I need to work for it.
Speaker 1:And you realize you can't run anymore. Like running is tricky, like going a distance is very tiring. I also, like by a coincidence it's not a coincidence really but I ended up getting an Apple Watch. I had always wanted an Apple Watch because they have this nice feature with like notifications. I'm the Garmin guy and this was like the metric that I needed to optimize.
Speaker 1:The computer nerd and I started seeing that you know I am improving a little bit. Uh, let me now actually do a serious attempt at getting into shape. And when you do this, you start reading up on how you can do this. Um, and it turns out the biggest problem usually is on the input side, and I always thought I was fine on the input side. I thought I ate healthily. So I ended up getting a calorie tracking app, a Swedish one I think, lifesum. Which one, lifesum?
Speaker 2:Lifesum.
Speaker 1:The one where.
Speaker 3:Jesper worked before. Yeah, jesper was there right.
Speaker 2:Are you using that one?
Speaker 1:Yeah.
Speaker 2:So we're promoting a Swedish startup again. Yeah, yeah.
Speaker 1:Though if they're listening, please call me. I have a lot of problems with your app and I'm not getting the support I need. So I started doing that and I realized I am way off in my estimate of what I'm eating, so obviously I had to fix that as well so not the trainings specifically, but eating as well, right?
Speaker 2:exactly yeah, you have to do all at once and all this is a core topic. Is it 50, 50 input training, or you think is 28, 60? What is your calculation in your head, how much effort you need to think about? Look, so the interesting thing is.
Speaker 1:Fitness is multiple things. Yeah, right, one of them is the build-up of fat and weight. Yeah, that is a hundred percent what you eat. The other part is what proportion of your body is muscle versus fat? And it turns out like you can. You can get down to a good weight uh, completely independent of training, fairly easily, but what happens is you lose a lot of muscle, and it's not healthy to be low on muscle. So what you have to do is, while you lose weight, you have to do strength exercises to encourage your body not to lose so much muscle, so I'm learning something here.
Speaker 2:So we are talking about this is not a sequence where you first eat right and then, and then it's like concurrent things. You actually need to, you need to balance eating right, but at some point in the middle of this you actually concurrently needs to build muscle in order not to lose muscle or something yeah, I mean the goal at that point uh, this was me starting to work out, so I ended up gaining muscle.
Speaker 1:But the goal? Really? You shouldn't see this as look, this is a way for me to build muscle. You are doing physical exercise to make sure you don't lose your current level of muscle.
Speaker 3:The goal is to lose. You do gain appetite, though, from training, so potentially you eat more from training, so you have enough to make sure that you don't. Good point good point it is.
Speaker 1:It is not, uh, super easy. Yeah, but the biggest um thought that I've had on this topic is wow, imagine if I had just decided to do this like five years ago or 10 years ago, because it is very controllable so you had this date one and a half years ago approximately, and what did you start to do then?
Speaker 1:I did everything at once eating better, the diet and the training, the diet, the training and the training, the diet, the training. And the more you get into this, the more results you're seeing. The more your watch is telling you, oh, your VO2 max just went up, the more exciting it gets. So you start. I just wanted to see, hey, can I run now? I ended up getting to two kilometers. And I ended up getting to two kilometers as my warm-up exercise for the gym of sprinting full out two kilometers. So my heart rate is around 175, 176 something, and then I started thinking could I maybe run a bit further?
Speaker 1:So I just went out and I ran 5K. It's like well, it turns out I can do that. Can I run farther? And you sort of build like that.
Speaker 2:And what is your training regime right now? Like so, where are we at?
Speaker 1:Currently I have a little bit of a problem working out because of my injury.
Speaker 2:Oh yeah, this is right. Yeah, yeah, but skip the injuries.
Speaker 1:Go to the week before the injury the week before the injury, uh. So I tried um, doing a gym five times a week, uh, running to work three times a week. Uh, it's a 4k run, uh and um. That turned out to be a bit too much, so I scaled it down to the week I got injured. The idea was just once a week, run to work five days a week, gym and rest.
Speaker 3:But when we were together many years ago, I remember you actually love to eat as well, and you love a good steak or something right, I do. So how did you change your diet?
Speaker 1:potentially, uh, I had a hack for that, which I can't recommend for everyone, but, beyond the app, I ended up subscribing to a service that just sends me my food and I take that thing out of the equation.
Speaker 2:Exactly what you get don't think about it.
Speaker 1:The meals are small, they're very easy to control and just completely. I've gone back to actually eating things that I enjoy now, but I completely stopped eating any form of you know candy, sugary substance, fatty substance. We stopped using olive oil, even though olive oil is supposed to be good for you, right?
Speaker 2:but a tablespoon of olive oil is 130 calories so when you're chasing down to single calories, then everything goes? Yeah, but I need to ask you then can we elaborate a little bit on the mental journey here, because I've been thinking about this for a long time? I just turned 50 this year. If you had the, I should have started this when I was 25. Fuck you, man, I should have started this when I was 30. I was really fit and healthy up to 25, and then it's been deteriorating and right now, the balancing topic right, driving my own company since 2019. Mentally and intellectually, I want to do it, but I simply I deprioritize it. So what was your mental state or how did you think about this to get it up on the agenda and in your prioritization? How did that work for you?
Speaker 1:So I think a bit earlier you guys spoke about having fears of things I had, like part of me getting fit and wanting to be fit is I really don't want to die and it is sort of the unavoidable consequence of being unhealthy, that you will lose a few years of your life, and the older I got got, the more I realized you know already lost some. That's a good way, so yeah, that and just forcing yourself to do it but still you want to live well.
Speaker 3:Right, you wouldn't like to do like david goggins, and do you know, david goggins, I don't ignore it, but anyway, some people, just you know, live to the extreme to be as healthy and, you know, work out in extreme ways and still I mean you want to have a good life as well right, yeah, so there's.
Speaker 1:There's the topic of balance, right? Yes?
Speaker 2:let's go right let's go there.
Speaker 1:Uh, balance is something when I was younger I didn't appreciate well enough. Um, go hard, go hard, just do it, force yourself through it, and I think everyone's different uh with, uh, what you end up doing, um. But but if I look at what has changed now, like, how do I have time to still really push, work, um and also work out, be with my family and everything in between? Uh is, when you have this thing that you have to do, you have to go exercise. It's unavoidable. You have to um and you have to go to work. There's no way around it. Um, everything else that doesn't matter has to give. Yeah, tv tv.
Speaker 3:But you can't live for tomorrow all the time. Sometimes you have to live for today as well right right that's true. So I mean, if you just live for tomorrow and can't really enjoy a good steak or having a good time, then you could potentially end up regretting that as well, wouldn't you? That's true. Do you have a way to balance today in some way, or do you actually enjoy it? Steven Seagal basically had no, not Steven Seagal.
Speaker 2:Steven Seagal, the greatest quote machine on earth, no but the Rambo guy you know.
Speaker 3:he basically said you know, I have a Stallone Stallone. Yeah, he had a diet of basically saying I'm very strict during the work days, but on the weekends I'd eat pizza, I'd drink beer, I'd, you know, enjoy myself. Would that be something that would work for you?
Speaker 1:you think, not for me personally. I'm sure it works for a lot of people who work out much more than I do, but for me, I think my way of balancing sort of the enjoyment with non-enjoyment is doing it in moderation. So this is like the balance thing again. So previously I might have said, well, heck, yeah, let's eat the 600 gram steak. Now I'm going to be thinking about the consequences of eating that steak and I'm going to buy a super nice, uh, maybe 200 gram steak or maybe even smaller, like a really small like very nice.
Speaker 2:Now you go to this super luxury kobe steak and you enjoy every gram of it, but only 200 grams so how do you balance your life in training and mental training?
Speaker 3:I don't know the simple answer.
Speaker 2:There's the simple answer is I'm 50 and I'm with the push hard mentality. I intellectually balance my life, but then to take something intellectually to execution and behavior, I'm it up big time. So a couple of examples. I mean, on one hand side, the simple topic of switching off work and being in the moment with your wife and your kids, fucking that up all the time, and I don't want to do it, but my brain is sort of you know, you're not here, henrik, okay. So this is number one, I would say.
Speaker 2:And then, in the same vein, I would argue that oh, yeah, yeah, I'm going to trade three times a week now and every you know, oh, it's 10 o'clock, so I'm getting so in the zone and my head is so obsessing around my work or what I enjoy, and I enjoy the work. That's what I mean. So I have a hard time compartmentalizing and I think it has something. There's another concept in all this that I think is very important to talk about, and that is discipline. So it's one thing to have balance, but in order to have balance, you need to have discipline in execution, and I'm lacking some of that discipline.
Speaker 3:What about you, man? Yeah, speaking about discipline, I think I heard Mike Tyson say something about what the definition of discipline is and he said basically, discipline means doing something you hate like you love it, I love it. I think it's a good definition, right? I love this kind of short definition.
Speaker 2:Eat the frog, eat the frog. You remember the book? Yeah, but maybe that's enough on balancing and you know, but damn it, inspirational stuff yeah.
Speaker 3:Cool Well with that. I am very glad that you were able to come here on very short notice and a dear old friend of mine, agrin Hilmkil Hilmkil Sorry, I forgot your last name. Do it yourself.
Speaker 3:Himkil, himkil Is that the way, sorry for that. I never said your last name. I just realized that I think we always just call you Agrin. Agrin, he's Agrin, come on. Anyway, we work together while at Peltarion and you are an awesome expert in AI and math and so many more things, and I learned a lot from you, so I really appreciate you, both from your expertise but also from your personality. I think he's an awesome person, but you've been working now at some other places, including Microsoft Research, but right now you're at a company what was the name? Latent Labs? Right? That's right, right. So something about you know you couldn't go into details, but something about making biology programmable, right? Was that correct? That's correct. Yeah, cool. I know you can't speak too much about it, but I'm looking forward to hear more shortly about you. Know what you're really doing there, and I'm sure it's some kind of AI related thing. That sounds really amazing, but perhaps you could mention just shortly. You know that sounds really amazing, but perhaps you could mention just shortly what did you?
Speaker 1:do at Microsoft Research, so Microsoft Research was interesting. I was in a research team for a lot of the time. Initially we looked at causal machine learning various kinds.
Speaker 3:So we were looking at causal inference, causal discovery. Perhaps you just should briefly describe what it means with causal inference right.
Speaker 1:So the the common way that people keep talking about it is you know, machine learning learns correlations right? Yes, say yes, yes, perfect um that that was cautious by the way it was. Uh, correlations are interesting, good, but sometimes you need to be. When you want to modify the state of a system, you want to make sure that you find the causal connection so that you can exert control over the system, because if you do not, you might be changing just an indication of success without achieving it.
Speaker 3:What's the classic example of the correlation versus causation? I mean something about. You know, it's raining, no, Temperature and elements.
Speaker 1:Temperature outside versus heat or temperature, yeah, and if you look at a sort of temperature outside versus temperature by the elements, what you see is when you or is it inside? I also always forget. I mean so many examples, so many examples.
Speaker 3:I mean you have this being sunny and the sales of ice creams or something, or something, and you say, okay, well, it's sun outside, then you could assume that a causal relation, that would be, ice cream sales increase, which is logical. But if you do the opposite, saying oh, the ice cream sales is going up, then it must mean the sun is coming soon. That would be idiotic, right? So the correlation would be bidirectional, but the causal relationship would be unidirectional.
Speaker 1:Right. So you can look at the number of ice cream sales and probably infer the temperature. But if you want to change the temperature, that is not the way to do it.
Speaker 2:Tell my ice cream, maybe we get better weather.
Speaker 3:I mean cool. And it's much harder, of course, to do machine learning in causal inference than traditional machine learning. Of course, to do machine learning in causal inference than traditional machine learning, of course right.
Speaker 1:It is very challenging and there are a lot of sort of assumptions you have to do and so on, but it turns out that deep learning can be very useful for this. So the initial thing my team looked at was in general, can we apply sort of can we solve traditional causal ML tasks with deep learning?
Speaker 2:And what was the years when you were working at Microsoft Research?
Speaker 1:Oh, so that was 2022, until 24 years.
Speaker 2:And of course it's a pointed question because I was fishing for if it was before OpenAI collaboration or during, and obviously it's during and we don't need to talk more about that, but I'm sure there are many interesting things and connections there happening.
Speaker 1:Yeah, there's the interesting phenomena of knowing that there are people in, the interesting phenomena of knowing that there are people in the company you work for that can't really speak about what they do and you will just never know.
Speaker 2:Yeah, and was this in London? You moved to London. Did it coincide with your joining Microsoft Research?
Speaker 1:It did, yeah. So I was very, very grateful about them being able to accommodate me and my wife in such a way. So she was wrapping up her PhD and therefore allowed me to stay in Sweden for one year, and then we both moved over to Cambridge initially and then London.
Speaker 2:So the research? I mean where's the office? Is that Cambridge, or is it in London, or both? It is in Cambridge.
Speaker 1:It is in Cambridge, right there are offices in London as well, I think, mostly like sales. And then there's Microsoft AI, who's a bit more secretive, and I don't know a whole lot about where they are right now and Microsoft is such a huge company.
Speaker 2:So if you look at the organizational unit or the business unit, or whatever you want to call it Microsoft Research, how would you frame what that does or what that was all about?
Speaker 1:So a lot of different things, but really what they're trying to do is they're trying to push innovations in areas that matter to them as a company. Ai, of course, is a very big priority right now, but there's also a lot of good work going on in infrastructure, like actual physical infrastructure, so things like optical devices for the cloud, for faster communication, for storage, for compute and so on.
Speaker 2:So, and my real question is a little bit to probe if it was applied research that eventually ends up in new technology for Microsoft, or if you're also was applied research that eventually ends up in new technology for Microsoft, or if you also were doing applied research for others than Microsoft.
Speaker 1:It wasn't for others. No, it is an internal unit, it is internal, and some of the work happening there is very fundamental. Some of the work is very applied. You are probably very familiar with the ResNet, so that is Microsoft Research, so that is one example of them doing something very fundamental that then ended up being a thing in all applications AI.
Speaker 2:Super cool stuff. And why Cambridge? Why did it start in Cambridge? Do you know the origin story? It's probably connected to the university.
Speaker 1:Yeah, somewhere right yeah.
Speaker 3:Cool stuff, but we're actually here today partly because we did it very ad hoc and in the last minute kind of thing, but we're simply here to do a proper after work. I think, yeah, proper, after work doing what we love, which is speaking about AI and, in this case, a major focus on news. Yeah, so I have a list of news stories that we could just go through a bit and just have a proper after work and discuss what we think about it.
Speaker 1:Oh, I look forward to that yeah.
Speaker 3:And perhaps we could start. One big thing was, of course, the Google IO, the annual kind of developer conference that Google has every year, and they released a number of things, a large number of AI-related announcements. Of course, I mean it sounds more like an AI conference than a developer conference, but still, it's super cool what they actually have released. It's super cool what they actually have released For one. Of course, they are transitioning the normal Google search into more of an AI search, as they have for some time. They had a kind of AI overview before coming up in Google search, but now they have a new AI mode Think like perplexity if you think about that. Or even OpenAI have their own web search now, which is LLM powered, and now they're doing something similar here as well. So I mean, I think it's interesting to think about what that really means and, of course, they have to take this path right. Everyone else is doing it.
Speaker 3:Google search is the biggest moneymaker they have and they have to make it more AI mode related. But on this and I thought about this, I mean some initial thoughts for me is you know, so many people are angry about this because it means that you don't have to leave Google to figure out and get the information you need. With traditional Google search, you just got the links and a small snippet, and then you had to click the link and actually go to the publisher to get the information you wanted. And now not so much. Do you have any thoughts?
Speaker 1:That's already gone though, right.
Speaker 3:I think so too, because of other providers.
Speaker 2:Yes, who's been using perplexity or equivalent for a while? You're using perplexity. I am honestly not.
Speaker 1:So the time I wanted to use perplexity, I wasn't allowed to use it.
Speaker 2:Oh yeah, yeah yeah, but did you use something equivalent, then maybe Perhaps.
Speaker 1:Yeah, but one thing that stands out to me with these things is just looking at people who are less tech savvy. So my mom, for example. Every now and then she's very excited and she tells me about you know what. I used ChatGPT for this. So we're going to. Many in our family are going to a wedding in August and she wanted to reply to one of my cousins in English. She's not very confident writing in English, so she just asked ChatGPT to write up an email for her. She asked it in Swedish. It produced the English letter and things like that are amazing. She also uses this for like, searching the web, learning things.
Speaker 3:I mean you're starting to use the chatbots more than the traditional search right? I'm certainly doing that more and more. I think even a majority of you know when I previously just searched for things, I nowadays mainly GPT for things, if we call it that right. Do you do the same?
Speaker 1:Yeah, definitely. I'm a big fan of Cloud, so I'm completely locked in with Cloud. That's your search engine, that's my search engine. I mean, I work with code, I work with ML for both of these things, cloud is amazing and that is a great way of getting through the first thing the sort of the findability problem of knowledge.
Speaker 3:We will speak more about Cloud and the Opus and Sonnet 4 shortly, but you used to sort of to latch on to a question is this a problem?
Speaker 2:You know? Do they have to do it? I can use. I'm trying to observe my own way of using these different tools now and for me it becomes I can almost categorize if I'm searching for a problem or a topic, want to know more about something. I simply my first choice is perplexity and I type in or even speak to it and ask it the fundamental question in plain English so what was the origin of this? Or you know, so when I'm doing research type topics like this, then sometimes I need to find the closest guitar stores, you know, then I typically still just type it in. I mean, like then I really depend on Google, or I simply type it in my browser and I simply rely on the fundamental advertising and search and you get those links up. So for product shopping I haven't changed so much, but for everything else I changed.
Speaker 1:I think that's an interesting area for AI to get into, so I'm just waiting for people to release MCPs for they haven't haven't they. Well, I think I saw something about it, so shopping.
Speaker 3:Maybe that's showing MCP right, but just thinking you know it's good for you. I think we all can agree For us as users of these kind of services, it's a good thing, but for the publishers it may not be right.
Speaker 1:It's an unresolved question, right. It is how are we going to encourage uh interesting articles to be written by humans if we can't?
Speaker 3:monetize, that we need to have a business model for the producers in some sense.
Speaker 2:I mean and this is really what everyone is protesting and being really angry with right and goes back to the research of the, the podcast we had on ip rights and intellectual property that if we don't solve that in a similar way, like Spotify solved it or whatever, you know what's the motivation to write great stuff? You know that is unresolved.
Speaker 3:I mean if we take the comparison to Spotify, I think it's interesting. You know, they certainly focused on the user, the consumer of music, to start with, but then they moved to the producer of music and, you know, from 2015, 16-ish forward, they had a big bet on just being the best service that could be for producer of music. I think, you know, if Google did the same, it could actually be a good thing, is that?
Speaker 2:a prediction that you need to solve the producer side and the consumer side.
Speaker 1:I think it's that simple. What do you think? So what is interesting to me is thinking about my experience with spotify recently, which I think ties exactly into this, and you know, music becoming so easy to make and produce and also some of these companies getting, um, what, at least from the outside, looks like optimizing suggestions based on which content is cheaper for them to force you through. I've noticed with Spotify that the music that I got, because I became completely dependent on Discover Weekly, the music that I got really took a downturn in terms of quality and I stopped enjoying it.
Speaker 1:Oh, really To the point where it's like it ended up becoming completely useless to me, because I'm getting all of this like random cover crap, presumably generated with some you you know easy to use tools, some uh simple, like uh covered by maybe an artist that just does covers. Uh, I'm just thinking with ai, it's gonna get even worse that type of behavior. So I ended up I I I had been using spotify since they were in like beta mode or something like that. Uh, always use Spotify, completely dependent on it. Last year I stopped.
Speaker 2:Are you inferring that they basically started to push and produce music that was less costly or where they could make more money? They started to recommend stuff in your Discovery Weekly that was produced in a way that sort of is more cost effective?
Speaker 1:so there are a lot of discussions online about whether this is happening or not.
Speaker 2:Yeah, I'm not sure yeah but I know of services that do this, but I did notice like you noticed, you notice a clear decline in your own opinion, in your perception in terms of what you could produce, and I don't have any insights, but given how it worked when I was there, it wouldn't actually matter for Spotify to do that.
Speaker 3:So they still paid out 70% basically all the time, independent of who it was. But it could have changed in recent years, who knows? But my guess is not. It's probably due to other reasons, but who knows.
Speaker 1:It might just be that it's so difficult to find and recommend music when what you're dealing with is some type of similarity based on what you previously listened to, and you end up with this like weird trajectory through space that over time just deteriorates.
Speaker 2:So maybe a successful strategy for me would be to open a new account with spotify could be and it's also like the whole recommendation space is changing vastly when you don't have, when there's a magnitude more to choose from and 50% of that magnitude is crap or, you know, AI produced.
Speaker 3:So the whole space is ultimately evolving, but you know, that's what collaborative filtering and the recommender system that they are experts with are supposed to fix for you. But still, who knows? Anyway, ai mode that's one big news from Google IO, right, yeah, and we have a long list here, so I'm trying to manage time here.
Speaker 3:I think we need to go faster on each we go back a bit. Okay, so another you know they have the image generator, text-to-image generator, called Imagine, similar to DALI in OpenAI, and they released a new version now, imagine 4. I'm not sure if you used it or tried it. I tried it but I'm not super impressed. But what I saw with Imagine 4 was actually rather impressive. I would say have you tried to seen it?
Speaker 2:I haven't tried the latest one I, I still don't think it's proper motive. The stuff when I, when I try to get stuff done and you know I'm now referring what? To chat, gpt, where I've done more work. Yeah, I, I think there's a challenge with the multimodal that it, you know it, it's supposed to understand what I'm doing, but it clearly opens up a new so my system prompt goes somewhere else. I think it's still not there.
Speaker 3:No, and I actually like OpenAI's approach to this. The problem with image generation, as I see it, is that the prompting strategies you have to make for image generation is very different from text generation. I argue, and OpenAI. They have the nice trick of actually using the text generator to generate an image prompt for you. So you can simply talk to it as you do for text generation and it will generate the prompt for the image generation indirectly, and then you get a really good image. That's not what I believe Imagine is doing.
Speaker 3:But, I think you know. Perhaps more interesting one is they released the, updated the video generator as well, Vue 3, and the cool thing with this is actually, finally, they are not just generating the video, the images, but also the audio in sync with the video. So I mean they can literally have I can take a picture of Agrin. I can say, generate a video where Agrin says I love Andes, and it will say that with lips synced and everything, unless it complains saying this is inappropriate.
Speaker 2:Could be some kind of, but this is a big step forward.
Speaker 3:Audio has been so lagging behind, I think for a long time yeah.
Speaker 1:Audio is forward when it's when audio. Audio has been so lagging behind, I think, for a long time. Yeah, yeah, audio is a very exciting space as well. Um, and you have some experience with audio as well. I have worked fairly extensively with it, uh, so audio is uh interesting, I think, because, even though you can convey uh emotion through text, uh, I think you get an order of magnitude more with with audio. Yeah, so imagine a conversation when you hear somebody's emphasis and pronunciation. You get so much more out of it, at least as a human, and I think, if we're talking about building uh agents, uh that are going to, you know, understand humans and be able to effectively communicate with us, surely, surely, they have to be very tightly integrated with speech.
Speaker 3:But still it's getting closer to. I mean, if you take all the fears that Hollywood has about generating movies on demand, now you can more or less with a single prompt get at least like a couple of seconds of video that is more or less professionally looking. I mean it really looks good, for sure.
Speaker 1:And a lot of providers have the same right, Like this isn't just Google. The technology is there now.
Speaker 2:But if you think about it, I think the first critical threshold when this is going to be really useful I can just look at this as my own as a founder of a company. When I can get to something really good that has 15 to 30 seconds, then I can do my marketing advertising, you know, video, micro stuff for social media. I mean it doesn't need to be more, but when you get that to a really high professional level, then I think it's going to be very, very useful for small, medium business compared to what it really means to do a full-on production.
Speaker 1:Right, there's a company in London called Synthesia. This is basically what they do, I believe, or they do various types of content, I think for a lot of use cases it is basically there.
Speaker 2:I think, the way I see this now for the digital agency, they can produce a video production in 30 seconds at very small cost, and what I'm talking about. But there's some tricks and tweaks that need to be done right. But to take that to me as a business owner prompting what they are doing, that's what I'm talking about, but I I agree with you for really low cost. Someone who knows what they're doing can stitch a couple of things together, but I think it's the next I'm talking about when I can do it with a simple prompting. I'm pretty sure you can do it already.
Speaker 1:okay, you should try out the tool if you have a good application for it. So Synthia, synthesia.
Speaker 3:Synthesia, but another I think London-based company is Eleven Labs as well.
Speaker 1:Yeah, Eleven Labs.
Speaker 3:They are amazing right they are super good, not in Swedish.
Speaker 1:If you've listened to Swedish. I've tried Swedish, but I have an interesting relationship with Eleven Labs Not an actual relationship as in. They know who I am, so you know I was at Storytel building A Swedish startup for audiobooks. Right, I think it's not a startup anymore.
Speaker 2:Storytel. Is this the biggest one or the second biggest one right now in the market in Sweden?
Speaker 1:I'm sure it's the biggest one, unless they've took a huge downturn.
Speaker 2:There are two brands. Which one is the other brand, storytel, is the biggest, it's a big one. So this is audiobooks Massive collection.
Speaker 1:Massive collection, they own it, and so on. So while I was there, I was building technology for audiobook creation. Is this before, or?
Speaker 2:after Peltarion. After this is after right, this is after Peltarion. Then Storytel Storytel Microsoft Research, right.
Speaker 1:That's it Right, and I'm not going to go through everything that happened, but I think a year ago I saw them actually releasing a collaboration with 11 labs oh really and I think they started having audio books made by 11 labs, yeah, so you have, uh as well, this blink is a blink list.
Speaker 4:You know the, the app that was like coming in 2015 and cetera, which summarized books in 15 minutes, right? Um, we actually had this as a news here. So they completely not completely, but many of the curation of the books right now is done with 11 Labs and it's shortening out the time to production in a very, very big way.
Speaker 1:And that's super cool, yeah. But I think it's a completely different level if an audio company is comfortable with the content, where it's like, imagine a piece of content that you're not going to listen to 10-15 minutes content. If it's 10-15 minutes you might be okay with like limited dynamic range in terms of emotion.
Speaker 2:If it's 10 hours it needs to be good, yes, but for BlinkList.
Speaker 4:I think this really, really makes sense because it's maximum 12 to 15 minutes summary of a book. I agree with you One hour. But I have listened to some of this and they're quite good they are right, I think the English voices are amazing.
Speaker 1:Yes, I mean, some of them are.
Speaker 4:I would be happy listening to them for many hours so I was working with Suno now before Data Innovation Summit because I got tired for all of this YouTube video restrictions for music. So I created like two albums and posted them on Spotify and stuff it doesn't matter so we used them during the conference. It was great and the beautiful thing that I basically took the entire content that we had on Data Innovation Summit, summarized that in the major topics that we wanted to discuss and make songs about that.
Speaker 1:That's very cool yeah.
Speaker 4:And it's on YouTube. It's really cool. Yeah, and it's on YouTube, it's really good. It doesn't matter Coming back the sound and the expression and the cadence of the English songs. There is no way you will see that this is actually.
Speaker 2:Yeah, but this is fun. Why don't you show it to us? Because actually there is a couple of songs on YouTube now that you produce, I mean you love music.
Speaker 4:Yeah, they're probably on YouTube as well, but I think it's on.
Speaker 3:Spotify. It's on Spotify. So Suno for people that don't know, and Yurio. These are the kind of music producing services where you simply prompt to create a full song with the instruments, the melody and the lyrics and the voices I need to challenge you here.
Speaker 2:It's not only sumo. I know you've been going down as well no, no, I, I did you.
Speaker 4:You, you go down the edm route.
Speaker 2:Yeah, yeah, yeah, yeah I see what you have in the back.
Speaker 4:So I I went through all of them, 11 labs. I tried actually to put my uh text into those, their voices and etc. But uh, I think that right now um sumo.
Speaker 2:So, so, so. So, do you like you? You, you could compose the song and all that, but we are talking now about the voice of the actual.
Speaker 4:Um, I mean like did, did, yeah, we'll come.
Speaker 2:I found the link and I listened to it. It's a cool song.
Speaker 4:It's a cool album. Actually, it's an album it doesn't matter. Continue, yeah, we'll never get through all the news.
Speaker 3:if we go this slow, we get stuck at the first topic.
Speaker 1:I just googled on the. They have 100 news from Google. I google io. Are you going to go through all of them?
Speaker 3:no, I am not okay, but let's, I think potentially let's take at least one or two more. Um, I think the web views that the project mariner is basically, if you remember remember, open ai. They have the operator functionality where you can ask it to browse websites, take a number of action and then you know, do something you want and anthropic has computer use.
Speaker 2:What do they call it anthropic computer use? I think it's computer yeah but it's different.
Speaker 3:Yeah, I guess in in their case. I'm sorry how if anthropic has developed it, but before it was basically a containerized thing. You have to download yourself and control. But in Operator it's basically their managed cloud service, more or less. So they run everything. You don't have to manage the containers.
Speaker 2:What is the Google way?
Speaker 3:I think the latest news here with Mariner is similar to Operator. You basically run it in the cloud and you ask to do things and it can do things in parallel. But it's not a computer use, it's a web use, basically Web use. That's also a distinction. I think browse use is a proper term, right, so they can control a web browser and they can take a number of actions and, in an agentic way, perform what you want it to do. So I think we need to move faster here. But I think it's cool. I mean, it is a lot of copying, I think, from what all the other AI labs are doing, but I still would argue that Google actually has taken the lead. I'm not sure what you think here, but I think today they had at least overtaken OpenAI's lead significantly.
Speaker 2:Okay, motivate.
Speaker 1:As a consumer, I don't think I care at this point. I mean, I'm sure it matters a lot for the people working in each company and surely it matters a lot for people building agentic software in general. When it comes to this quarter of AI, I am not very invested in terms of, you know, tagging along with evolution. I'm just super excited as a customer and I'm seeing everything is going to get better. The services are getting so good. I have so many use cases where I just I can't wait for agents to solve these problems for me.
Speaker 2:But the way I'm thinking about this, am I sticking to one universe, a little bit like oh, I have all my stuff in Google or all my stuff in Apple, so I can't wait until Google fixes this. Or will you shop around for different tools like best of breed? Oh, I'm going to use perplexity for this. I'm going to use your opening eye or cloud for this. Or will you shop around for different tools like best of breed? I'm going to use perplexity for this. I'm going to use your opening eye or claw for this. What is your take here? Because then the consumer cares, right, I think at this point.
Speaker 1:We're like in the happy early days when people are happy integrating with everything. But I'm sure there's going to come a time. Like you know how all the media companies in the beginning they fixed TV subscriptions. Now we're back there again and I'm sure we're going to see the same here.
Speaker 2:It's the argument of app fatigue, right that you're going to switch too much and it's just going to be. Then you're going to have a super app that controls other apps.
Speaker 1:Yeah, exactly. Oh. We can't pre-compute embeddings for your documents unless you subscribe to Google something. Something AI plus, Shoot me.
Speaker 2:We're going to get there.
Speaker 3:I mean in some sense the AI functionality of generative AI's core functionalities getting commoditized in some sense. So I think all of them have similar kind of functionality and you can stick with the favorite supplier you have for I guess in some sense.
Speaker 1:Right now, definitely. And Anders, you said copying opening. I'm not sure that's fair. This space is evolving so quickly and I think we're seeing co-evolution. Who knows who started working on it first? We just know who released it first.
Speaker 3:True, well said, and I think Google has historically been significantly in the lead, though, in the last 20 years, before 2020, at least, and Google was heavily in the lead and they invested the most in AI. But then suddenly, openai came about and they actually, you know, through all the GPT work, took the lead, but now Google has taken it back, I would say.
Speaker 2:But I have a spin on this because we've been talking about this and ranting around this a bit and then sometimes it's almost like actually what we have experienced as consumers or as newsreaders is that a reflection of who's really in the lead or is it a reflection who is kicking ass in marketing and go-to-market strategies and all of a sudden navigating in order to get their story to blow up? And here I think there has been a couple of debacles where Google has not done a good launch job and there's been some awesome tactical moves of OpenAI where it's clear that they have waited to release something they've done and its timing is so fantastic. And this is pure marketing. It has nothing to do with who's in the lead or better, but I think OpenAI to some degree has out-competed from a pure marketing genius, I agree with you.
Speaker 3:I think Google, although they have the core competence, the marketing and the way they strategically went to market, so to speak, has been horrible in recent years.
Speaker 2:Not horrible, but OpenAI has been amazing.
Speaker 1:Yeah, maybe. Another thing if you look at things like that is I don't think OpenAI is making so many ads, are they? Maybe they started, but a year ago or so I started seeing ads for claude. Oh, so I think you're right. It is uh. When it comes to acquiring customers that aren't just developers, it is really getting down to the marketing game now but it's also a question of distribution.
Speaker 3:I mean here google and marksoft have a clear advantage, right. Openai does not, nor does Anthropic. I saw some numbers saying that Gemini now has 400 plus million users of Gemini. I think OpenAI, with their ChettyPity app, has 500, 600 million or something. So they're still in the lead. But I think it's unfair to say that Gemini has 400 million users. It's simply that they integrate it into Gmail, integrate it into Google Drive and they have the distribution there. So they simply get the users for free, more or less, because it's put in front of all the users. So they just get them for free, whereas Claude and OpenAI has to fight for it a bit.
Speaker 2:But then you can talk about who's in the lead, right, and you at least need to break that down into three core components Technology-wise or research-wise, what they actually have under the hood or what they've done. Marketing-wise, in terms of how they're winning the marketing space. And then you add distribution strategies or distribution possibilities. Where's the real moat, by the way, in one of those three? All three?
Speaker 3:I think the moat is more compute limitations going forward.
Speaker 1:Do you think the moat is having access to compute?
Speaker 3:To a large amount of compute. Yes, Interesting.
Speaker 1:I think it is fairly possible nowadays to get access to very large amounts If you have a huge amount of money. Yes, well, yeah, but we're talking about competition between companies who are able to raise fantastic amounts of money.
Speaker 2:So if we look at the mode between those three, what is it? I mean, is it distribution? I think distribution is part of the mode here. Marketing also, I don't know.
Speaker 3:I think it depends on if you simply want to come on top of the leaderboard for some kind of benchmark, in more kind of a research sense, or actually you want to have a product distribution with number of active users, and it's a different thing, exactly.
Speaker 1:But it's different games, right? We're talking about providers of the foundational layer in a way. Right, there's also the application layer. I'm really curious Do you guys think it is possible to build a way? Right, there's also the application layer. I'm really curious. Do you guys think, uh, it is possible to build a moat there? I see people talking about how the value is in the application layer, but the the other uh question I see asked is well, is it possible to actually defend the value there, or will people just switch over.
Speaker 2:So interesting because I have arguments of leave the llmM. The foundation layer is no money, it's commoditized already. You need to go to the application layer. So I've seen quite well, rhetorically good arguments that trying to understand where the moat is and one say there's nothing in the application layer and one say it's all there, right. I'm very confused.
Speaker 3:But it's similar. We have spoken about similar kind of topics in the past and if we think like a couple of years in the future, I think we can see at least a couple of tiers. We call it that. One could be the frontier tier, where we probably will see like a handful of super huge frontier AI models that will be trillions of parameters in them and they are really really good but extremely expensive to use, so you can't really build an application on them and you don't use them to build applications. What you do is you basically distill information to provide your own more specialized model and that's something that any company should do and can do and then you can get down to at least a few billions perhaps, parameters in it and it still works really well. So then we'd have one tier of a few very selected super huge mega models of trillions of parameters, but then thousands and tens of thousands or if not millions, of more application tier models.
Speaker 2:This is also a prediction in our Christmas special. Yeah, do you see what I mean? I mean like so is it interesting to build a European?
Speaker 3:super, but do you agree with this idea?
Speaker 1:first, I'm not sure, anders. There's been recently fairly recently, there's been a lot of talk about hitting scaling limits right and the scaling limits simply being well. Are we actually going to see the same scaling law upheld as we make the models or the compute we spend on training even bigger?
Speaker 3:even more. I have a topic about that, so let's get back to that.
Speaker 1:Sure, but I mean, you are almost assuming that we're going to keep scaling with this statement.
Speaker 3:And certainly not in terms of, perhaps, number of parameters, amount of data, but in terms of intelligence and capabilities.
Speaker 1:yes, so my thing is, we might live in a universe where what happens next is we start scaling more on the inference side, and to me, this maps to what OpenAI is proposing with their PhD level compute. I don't think we're talking about fantastically larger models. I think we're talking about spending much more compute, and I'm not just talking about reasoning tokens. I'm talking about, you know, having these flows with multiple agents collaborate with each other. That ends up, you know, one query ends up being a hundred.
Speaker 3:I think also, it's yes and no I would say to that comment. So let's go a bit deeper into that. So one thing is, I think the current models are really good in knowledge management. If you remember the open AI kind of pyramid of five levels, the first one is basically conversational, then you have reasoning, then you have autonomous and then innovation and organizational.
Speaker 3:I think you know, today we have really good level one, really good knowledge management, but if it comes to reasoning, they're getting better, but it's still rather poor, certainly behind humans, and that will increase and change a lot. And then it's not perhaps the number of parameters that needs to change or the data, it's really the architecture or the algorithmic side that needs to change, I would argue. So we will continue to see, and the same, of course, to agentic, autonomous and innovation going forward. So I think we will still see a significant change in the more let's call it algorithmic side that will bring AI much more capabilities than it doesn't have today. So then it's not just an inference compute thing, you know, it's really a change in the AI system as such.
Speaker 2:There's a need for another style of architecture. To sort out reasoning. Would you agree with?
Speaker 1:this Team, lekun, team, lekun. Yeah, yeah, full on, but I think already with the current ones that we're seeing.
Speaker 3:They already transitioned from the standard transformer to something else, so so I mean we already start seeing it, and even the diffusion models that we combine transformers with, diffusion technology has moved away from the traditional transformers. I mean we're already starting to see a lot of algorithmic change, so to speak.
Speaker 1:I think I used to be Team Lacombe in these things, but what I'm seeing is I'm constantly being blown away, but by what is happening and what is possible. So I'm starting to lean into maybe we can get, uh, a lot more out of our current technology, and I'm not saying just training them for longer um.
Speaker 3:But but you would agree that the standard transformer from 2017 has significantly changed, right, of course Of course.
Speaker 1:Well, significantly, no, but has changed and there's been a lot of optimizations along the way and we've changed a lot of how we deal with them.
Speaker 3:But all the reinforcement, learning, fine-tuning that happens afterwards. I mean it's not a standard transformer thing, right?
Speaker 1:Well, the transformer remains. We're just talking training methodology.
Speaker 3:But it's added a lot of components outside of it right. So it's not the standard transformer training methodology that you use anymore.
Speaker 1:So then we're getting into my argument right, which is we might be almost there. Maybe we need to add something on top rather than completely changing.
Speaker 3:I would never say completely change Right, because some people are arguing that you know, fundamentally, transformers cannot.
Speaker 1:But that's not what Jan LeCun says Almost.
Speaker 3:So if you think that's what Jan LeCun says, then I would disagree.
Speaker 2:But let's pick it apart because he says I'm not the expert enough to explain it. But he says the fundamental understanding of the technology we're using today will not take us all the way. So he says this is fine, but we need something else.
Speaker 3:You mean Jan LeCun? Yeah, yeah, but he has a big argument. Now we go into rabbit hole here.
Speaker 1:It's fine.
Speaker 3:I would love to hear it, anders. Okay, so let's go there a bit then. I mean, what he disagrees with a lot is, of course, the autoregressive nature of LLMs, meaning just predict the next token and in token space. So what he's speaking about in the JEPA architecture is that you don't go to token space, that you actually do the reasoning and prediction and multi-step inference in the latent space. So that's, in his case, the energy space that he's thinking about. So both the output and the input is then first having an encoder into their energy space and in that space you do the prediction steps.
Speaker 3:So, in short, I mean it's basically latent space reasoning which I've been speaking about, I think, a number of times here and we're still not seeing it really because it's still mainly reasoning in token space, which is, to my view, idiotic, sorry to say it, but I mean, if we call it like this, if you take the normal decoder which a GPT model is, you know it doesn't really have a formal encoder, but in practice a decoder which the GPT has, the first layer is an encoder and then you have some reasoning happening in the middle layers and through the tokens in the context window that you have and the last couple of layers is basically an encoder back to token space. So every time you have to reason, you have go through the encoding and decoding, or sorry, decode and encoding, decode and encoding. Instead, you can simply do the steps in the semantic space in the middle. This is basically what Dan LeCun is saying in my view.
Speaker 1:So if you map over your view to Transformers and we just accept this tiny change would align with LeCun's view, then that seems like a reasonable step. And I haven't read up enough recently of what he's been saying, but my understanding was he wanted a much deeper push towards learning joint embeddings to the extent where you cannot train the models with the current pre-training tasks that we do right now.
Speaker 3:I think all the encoders are pre-trained, I would argue Right right.
Speaker 1:So in part, yes, I mean the tokenizers are pre-trained.
Speaker 3:Yes, yes they are, but also the whole encoder. I would say Sure, sure so in part it is pre-trained still Right.
Speaker 1:But what I'm saying is I think you know the simple, very powerful tasks, training tasks that we have, like just next token prediction, multi-token prediction and so on. I'm just saying I'm surprised at how far they are getting us and previously I thought this should not be enough. But, um, like, what I was saying earlier is uh well, I've been wrong for so long now that maybe this is it. Maybe we we just, you know, we train them on learning next tokens. Uh, maybe this is our pre-training tasks. Maybe we add additional training loops at the end to tune them and really have them achieve what we want in the end. Maybe that is sufficient, I don't know.
Speaker 3:But I think as soon as we add more modalities to it, like images, is hard to do, autoregressive in terms of pixel space. I mean, no one is doing that, it doesn't simply work. I hope you agree with that. So we know already it works rather well for text, but I would argue, as soon as you move to other modalities, you have to do something else. Then it can be tokens or patches of pixels potentially, which the BIT basically does, or something else like the diffusion model, which is completely different from autoregressive. So we already are seeing, as soon as you move to something on the text, the traditional single autoregressive step doesn't work.
Speaker 2:So we already know it doesn't hold up in the pure 4-world modality but.
Speaker 3:I don't know, but there isn't a single pixel autoregressive model out there.
Speaker 1:Not pixel level for sure, right, but what we're talking about is basically how do you tokenize video? Yes, and same as the existing tokenizers for text, we have trained tokenizers for images and video and so on, and there's a long debate about, you know, getting rid of tokenizers, and I think that is a very interesting space Not that it's my thought, it's been highlighted by others but I think having a more consistent way of producing tokens seems interesting to me. It seems like we're currently. It's a more beautiful solution, right.
Speaker 3:If we could do it.
Speaker 1:Right, and my interest in audio leads me to believe that maybe the closer data source for human thought might actually be speech, and if that is the case, maybe our pre-training tasks shouldn't be so centered around text as it is right now, and that becomes tricky with tokenizers and how do I tune them and so on.
Speaker 3:And there's a lot of different ways to mix in with different modalities and tokens together. I think Meta even had a paper what was the name of it? Large concept model. They basically try to, even in tech space, move away from tokens in terms of words and then speak like at least sentences. So then you operate and reason a bit more in semantic kind of concepts not really in syntactic tokens at least and they could see some kind of crossover between languages and things like that that work better when doing that. So at least it's moving in that direction and I think we will need to go there somehow.
Speaker 2:But if you flip circle back a little bit to where we started, how much of this innovation will happen now in the fundamental backend, on the large language model? And if we talk about now, we're going up in reasoning At the same time. Now we start then saying, okay, what is LLM level and what is application layer? And we said, how can you really have a moat in the application layer? So I'm thinking also a little bit like all these architectural innovation that is really research right now. What will belong in the core backend LLM layer and where will we actually have innovative ways, something like to really understand how we stitch things together or these architects to make it really work?
Speaker 1:I mean it's a good question. This is the, I think part of the bet you have to do I'm guessing, as a company betting hard on the application level is that the foundational level is not going to get good enough soon enough that it actually eats into my value prop.
Speaker 2:Because the tricky part is that you then do a hardcore research to a very smart application layer architecture. That actually becomes a moat in itself because you can get to a much more predictable result or useful result over time in more complex tasks. The tricky point, as you know, pit point out is well, the LLM will eat that up in the next six months.
Speaker 3:Yes, we've spoken now for half an hour approximately about the Google AI and a lot of rabbit holes. We have a number of topics. I would like for you to choose which one we take next. Angeline, it could be, I think, an interesting paper called Absolute Zero. It's a Chinese company that actually could train the reinforcement fine-tuning part without using any human data at all. So I can introduce that if you'd like to hear. I think it's a big step getting closer to Alpha Zero and AlphaGo and these kind of models. Or we can speak more about Claude, opus 4 and Sonnet 4 and the Whistleblower part. Or we could speak about this new investment from Sweden with Wallenberg and with AstraZeneca and other companies trying to invest more in AI infrastructure. Or we could Okay, the last one here. Let's take one more the recent this week announcement from our Minister of Digitalization, erik Slotner, about an AI strategy.
Speaker 2:The digitalization strategy, so to speak. Where do we go next?
Speaker 1:Right, you know feeling a bit disconnected from Sweden nowadays. I am very curious about the Swedish efforts, but I would also love to speak to you, Anders, about the AlphaZero Do?
Speaker 3:you think we can do both. Let's start with the paper first. Okay, so let me do a quick intro. Absolute Zero from a Chinese company. No, actually it's from a Chinese university. So this is more. It's not Alibaba, it's not Baidu, it's not Tencent, it's one of the Chinese universities. As far as I know, I think they had some collaboration with Microsoft, if I'm not mistaken, as well.
Speaker 3:So the point with this if you take all the traditional fine tuning being done in current LLMs, they are pre-trained first using the normal ultra-aggressive next token prediction regime, but then they're fine-tuned, either like instruction tuned with supervised fine-tuning, saying this is a proper output of some kind of prompt, or the reinforcement learning, the human feedback, which is another technique to simply have humans ranking these are good examples or not. And then you can, using a different kind of learning algorithm outside of the standard transformer, fine tune to make it more aligned to our values that we have as humans. But another one that they speak about here it's actually DeepSeq did as well and which they use here in AlphaZero sorry, sorry in absolute zero is reinforcement learning with verifiable results. So what that means is basically they generate code that they can execute, python code basically. So now they can run the code and if it crashes it's a definite zero or bad result. If it works and it actually produced the output that you want, then it's a positive result. So you can have verifiable results without using humans.
Speaker 3:Now, what they did in this paper let's see if I can recall it properly they basically have three types of predictions that you're doing. Normally you just have some input, you have the model and the parameter and you produce an output. In this case they do something more. They call that the first step, basically deductive. I don't like the term. They have used the term deductive, abductive and inductive here. These are the three type of reasoning techniques from philosophy of humans, but anyway, I shouldn't go there, even though I wanted to, because I hate that they have used the term.
Speaker 3:They used the terminology that you asked Anyway deductive is basically taking the input and the model. What should the output be? That's one thing, but what they also do is more the abductive part, and abductive meaning you know the output, you know the model, but you want to predict what the input was. Now, this is really smart, because if they train the model to also predict what the input was, given the output, it means it can generate new input, and this is really the core, I think, of this. So what they are trying to do here in absolute zero, is that they generate new data. They don't use human data at all. The zero part is basically no human data whatsoever. They generate a lot of math and programming, coding examples by itself.
Speaker 3:So the comparison would be AlphaGo and AlphaZero in 2016. You know, alphago was this kind of mix of human expert playing the game of Go, and then they also did self-play, where it played against itself, and together it grew to a capability that far superseded any human, including Lee Sedol, who was the human champion. And then they had, you know, after AlphaGo, they had AlphaZero, and what the zero meant. There was basically no human expert data was used at all. It simply did self-play all the time, and that's far superseded AlphaGo.
Speaker 3:So it was better to not use human data. It was better to just use self-generated data to build up the model's capabilities. And this is actually what Absolute Zero is doing as well. So they say and they have soaked our results state-of-the-art results for NME and other coding tasks and logic reasoning tasks, and it gets better by not using human data. So all the people that says you know we are hitting a wall, we're running out of data, we have no more human data to train on is completely gone For programming and reasoning and like math tasks and things like that.
Speaker 2:But isn't this the challenging part? As long as you have a space that you can then find a true answer that you can compare to like math, like coding, then we can push reasoning really really far. But in the more humanistic you know even business, when there is no right or wrong, there is judgment.
Speaker 3:I mean, that's you know. Alphago is easy in that way. It's so simple to say who won or not. I mean it can't be much easier than that in terms of you know, was it good or bad? Coding is easier Sorry, it's much harder, but not impossible. Math reasoning is also more difficult than Go for sure, but you can in some sense say it was correct or wrong. And then you come to other, like subjective tasks. That is much harder to judge, but it's moving in that direction and this is a step where we get self-play into these kind of open-ended questions and if it continues to do that in coming years it could do self-play for more subjective tasks as well, potentially.
Speaker 2:But I really want to stress that for me, there's a trajectory here in areas where you can find the true answer and train it on self-training, where I think this is 100% valid. But I'm not sure if I go into fundamental judgment and decision making that business does in relation to optimization problems in an order to delivery chain. I'm curious to see how will that work with synthetic data or self-play if you don't have a space where you can answer this was good or bad.
Speaker 3:You need to have a world model to do it. But still, you know, I hope we can all agree that by eliminating human data it makes it possible to supersede human performance yes, I agree with that, but at the same time, you're.
Speaker 1:You're wondering, like, how generally applicable this is to my problems in my world and so on this paper they mentioned coding benchmarks and performing a lot better without human data, but presumably they still pre-trained on that same human data.
Speaker 3:So it's based on a pre-trained model, but it's just a normal ultra you know ultra-aggressive, but then the whole fine-tuning is without human data.
Speaker 1:Okay, that's very cool. Does it deteriorate a lot in performance on other tasks? I don't know. I don't think so. I haven't seen that. At least, that seems like a tricky thing. To be honest, I've never done like these steps of training and my expectation is typically you do need to, you know, avoid catastrophic forgetting somehow, and I'm really curious if this ends up being, you know, an agent that can only solve coding challenges, and I mean you can phrase math problems as coding challenges. I have no problem with that.
Speaker 3:I mean, I think it's a bit limited. You know the current strategy they have, that they have to produce like Python code or something that can be executed, but it's a step in a direction.
Speaker 2:Of course it's not super general yet, but the whole step that it's taking but I don't see a problem with it, because in any innovation to adoption cycle, what are the innovator cases? Where can we start? Where can we do something safely? And then you start here. Right, you start in the coding problem where it did. You know types of problems where you can actually find a solution to train against, and then you can have a 10x or 1000x efficiency in so many core processes. That has that profile. So you come to a profile of types of problems and challenges.
Speaker 2:Where you can, I would see this. I have no problem seeing this scaling or working. But this what scares me is when people are not nuanced and then trying to take that logic, oh, it works. And then they apply it to oh, come on, this is a really, this is not just complicated, this is complex, you know so. And then there's another thing we do, or maybe we can do part of that, but then we need to have some sort of you know, we need to find the ways and mitigations around that. So do you see what I mean? Like, I don't see this as a problem. I see we need to be very sharp. Where is it useful, safe? I see we need to be very sharp. Where is it useful, safe and what would we need to add when you go?
Speaker 3:outside of those first domains. What do you think? Is this a good?
Speaker 1:path forward. I think this is a super interesting path forward. I'm sure there are a lot of nuance and details in the paper about how they get this to work well. Certainly, there should be some type of balance in not giving the maximum. You probably have to have some type of penalty for having long solutions as well, I imagine. But yeah, very interesting way forward. I'm curious how agents trained this way would interact with humans, though that is the thing. I think this is a great path for building coders, uh, and I think, um, I'm just saying I don't know, I I have, uh, some friends who are you know, um, and this is a pattern you're seeing across all of the giants in this space they're investing heavily in solving math problems, specifically because there are people who believe, like, fundamentally, if we can show that we, we solve logic, uh, we will be done.
Speaker 2:I don't agree with that. You see, I don't think that's human I'm not saying I agree or disagree.
Speaker 1:Honestly, I don't know. I think this is a super interesting thing and I think we should push harder in this direction. I'm really curious how know, asking an agent like this to actually perform a task for you is going to end up looking like if a human asks for the tasks to be done.
Speaker 3:Still, I think the core message here you know for one that they can train to predict the input and in that way, being able to generate new content is a super cool idea. Yeah, that's cool. And then doing a self-play and actually performing better than humans by doing so in the similar spirit as AlphaZero For me it makes a lot of sense.
Speaker 1:Yeah, and it breaks the barrier that people have been talking about, with data running out and also just completely detaching from data in a way means okay, so now we're compute bound again. Exactly.
Speaker 2:Maybe that's a rabbit hole, but we were stumbling upon the inference topic a little bit A couple of years ago. We talked pre-training and bigger, bigger, bigger. And then we had Cerebras. You know Cerebras as a technology we had a friend of ours was here talking about you know, so cool. This is the way. This is the inference.
Speaker 4:Wafer size Wafer size, inference, fucking.
Speaker 2:But this is another trend. I think we are, I mean, like maybe I'm following Cerebras too much because there's a lot of inference, or are we talking enough about inference?
Speaker 3:It's a rabbit hole. It's a rabbit hole. Forget it, Stop it. Stop it, but it will. You know, people that say that we don't need infrastructure or compute, I mean, ah, Jesus Christ we will need so much more. We haven't even started with the inference compute. Yeah, okay, again, time is flying away. So okay, choose one of these now Cloud4.
Speaker 2:We have to talk about Claude Four. Can I choose?
Speaker 3:Okay, okay, so probably Claude Four, then, or something with the Swedish investment in Wallenberg, or the Swedish Ministry of Digitalization and the strategy there.
Speaker 2:But if we can do one, we go technical.
Speaker 4:Let's do the Wallenberg one, because he said he would like to hear what is happening here. I have just basically a little bit to do and to roast it a bit like just you have been roasted.
Speaker 3:I don't know I have a roast I I usually just roast it but could we do a quick claude for them and then we move to ballenberg and perhaps the digitalization strategy.
Speaker 2:Then we had technical, technical, technical, and then we end up on societal, yeah, okay.
Speaker 3:I like it. Okay, let's see, I haven't really read up that much about you know Cloud4, but they released a new version of Cloud and you know the 3.5 was awesome in coding and the 3.7, you know it was the best one. I think you agree. I think you said you use that the most, right? Yeah? And if people don't know, anthropic has this kind of you know, it's a lot of ex-OpenAI and ex-Google people that went together and started that startup that they received a lot of funding for building a new, and I think their speciality, if I may say so, is really safety or ethical way of… Alignment, yeah, alignment with human values, perhaps, right and okay. So Opus, ethical way Alignment, alignment with human values, perhaps, in any case.
Speaker 3:So Opus 4. So Opus is a super big one, and then we have Sonnet and we have this smaller one, the smallest one called, remember, haiku Haiku, yes, and then, of course, opus 4 is supposed to be the biggest one, the best one, handle a lot of really, really complicated tasks and I think one of the features now you know I think even in the previous one they can mix how much they reason, how much they do not. So some more flexibility when they realize this is a complicated problem or not, and then sometimes be much faster and sometimes be slower. Be much faster and sometimes be slower. And of course, there's a lot of focus on the safety aspects of this.
Speaker 3:And, if I may just throw in a bit of a nasty topic here as well, apparently it's on experimentation stage to have some kind of whistleblower functionality in Opus 4. Meaning, if someone is asking it, I'm planning to assassinate the president or I'm planning to kill my parents because I want to have a strategy how to get away with murder, it will actually potentially call the police or the authorities or someone and say, potentially call the police or the authorities or someone and say this person is someone you need to get in contact with, without telling the person, the user. So if you potentially are trying to build a new biological warfare or something, they will call the authorities on you and be a whistleblower. It's not in production to my knowledge, but it's something that they are apparently experimenting with so interesting that they're doing this.
Speaker 1:Another part of the news here, I think, was they were looking into. They have various types of scenarios that they test their agent against. This is going through the safety aspect, and they have this example with you know, an engineer working on Claude having the ability to turn off Claude, and then it starts blackmailing this person. And this happens in what was it like 80% of the scenarios they had. And now imagine you are a company using Claude and you are competing against Claude, right, what is it going to do?
Speaker 2:That seems a little bit problematic. Could you elaborate on the blackmailing story? I also caught that one. So just to be clear, Claude started to blackmail the engineer.
Speaker 1:Claude blackmails the engineer using knowledge. It blackmail the engineer. Claude blackmails the engineer Exactly Using knowledge it had about the engineer. It's a little bit like.
Speaker 2:For what reason? For money or for you know? It was like oh, I saw your emails that you were unfaithful to your wife. It was on that level, I think.
Speaker 1:It was on that level and I think the reason why was basically this engineer was was it about to turn off Claude or could turn off Claude? Something like that? It felt life threat.
Speaker 2:Claude felt threatened. And then don't shut me down.
Speaker 1:Don't stop using me, right. But to connect this to the first thing you were talking about, andes, now imagine if it has the power to call the authorities on you start reporting malpractices based on your google workspace documents and stuff like that. It is a bit scary to me that you have both the whistleblower functionality and the proof that claude will do things to affect the survival of claude.
Speaker 3:In a malicious kind of intent in some sense Right.
Speaker 1:And there's no proof that this would happen, of course, but I'm just curious like a lot of companies are building on top of Claude. This seems like something they have to fix.
Speaker 3:And it's also adding to the dangers with agentic capabilities. When you actually take action without asking a human to do so and autonomously calling the authorities on you would be something people may react to Right.
Speaker 2:But it's scary to me because a couple of months back or half a year ago, the whole conversation, I think it died out a little bit. We had the whole sentient topic right and people were calling out sentient way too early. But here now, like you know, how can it come up with blackmailing? You know, is that the narrow?
Speaker 1:It's part of the corpus, right? It's part of human imagination that robots are going to turn against us. So there are so many stories and books about this scenario happening out. It's actually we have fed it those stories. That's the point, right. I'm not sure if this is where it's coming from, but I can certainly see how this would be something in the training data even.
Speaker 2:Yeah, but it's interesting because even for that objective function doesn't exist, you could argue, but still it emerges, right. Emergent behaviors.
Speaker 3:Yeah, well, when you have self-play in the future, you may not even need the stupid humans that do nasty things. You can even come up with nasty things to do by itself. Yeah, but this is once again okay, should we just move a bit closer, because we need to also start wrapping up soon. We just started. Well, you started, right?
Speaker 2:soon, you just started.
Speaker 1:Well, you started right.
Speaker 2:No, but could we be a bit more specific on Anders? So what were the main features of the new Opus 4?
Speaker 3:I think it's bigger, better, more reasoning, more flexible.
Speaker 2:What were they pushing themselves that they wanted to highlight? Was it reasoning capabilities, or was it coding? Or where did they put their marketing? I?
Speaker 3:think Anthropic in general, they are starting to move more and more into the best coding model out there. So I think coding abilities, if I'm not mistaken, is something that they're trying to push more and more, and they are the best one, I would say. I think they're really awesome. But coding is also really hard because it's so compute intensive and if you use it for more fill in the middle or this kind of out to complete things, you need to call it all the time and some other tasks. They have this what was it called? The cloud code, right, right. So it's a command line interface tool which you can use in your terminal and basically ask it to fix this Git conflict that's happening now, and it can go in and look in all the files and fix things for you. So it has this kind of awesome developer tools that I think few others have, which is really cool. I mean, they seem to be going towards the developer community in a big way in my view.
Speaker 1:Yeah, that's true, but I'm also curious about why the four from 3.7, something big must have changed right.
Speaker 2:That's my argument, right? So when you put the four in front of it, it usually means something, right.
Speaker 3:Yeah, I could be missing this?
Speaker 1:I'm not sure. Maybe we're back at marketing being a big factor here.
Speaker 3:Maybe They've been at three for too long.
Speaker 2:I mean, I used to take the, you know, when we were all waiting for OpenAI 5.0, we didn't get it right. That's the opposite.
Speaker 3:Interesting. They probably trained it for a long time and it was time to move to, I don't know, cool. Okay, should we take the more swedish situation here and we have both the let's bake it into.
Speaker 2:You know what you know for you now right living in london. Yeah. So what's going on in sweden? What's the news?
Speaker 1:that sort of hits the mark yeah, I keep hearing about things are going well in sweden. A lot of American investors are only looking towards Sweden. Is that the case, or is this just the investors that I know on LinkedIn being very salesy?
Speaker 3:Or could have some other objectives or reasons for doing so.
Speaker 2:Let me frame this one then, anders, because if you look at the Swedish map right now, it's a little bit like there is a story going on now on a thriving startup community, and I think you're referring to Oliver Molander, who's a friend of ours as well, who's been pushing and highlighting and writes this awesome LinkedIn post. I love them.
Speaker 2:He's one of the best. He's one of the best as an influencer, in my opinion, because he's got the facts around it. And then, of course, we have a very nice fairy tale right now with Lovable, and, by the way, we had Anton Osika here one month before their third launch, so he was telling us about you know, we're going to go away from calling it GPT to Lovable and we had the whole backstory to Lov. Lovable is this minimum lovable product, right? So we were talking about that one month in between in advance, and then it took over, so maybe we had some in. Maybe it was this pod that really made this because of us most likely correlation versus causation, right, causation, you know.
Speaker 2:Okay. So that's, that's the whole we can talk about. You know how do we feel about the startup community? I went to Epicenter. You know there's a lot of stuff happening and there's a legal, there's Talentium in HR, there's brewing, right. Then we have another space, which is a little bit like we had a big news this week. We had a big news this week Wallenberg goes in heavily with the core companies to build their internal AI factory, nvidia-based. And we made the joke. You know the NVIDIA sales guy if you buy for more than 100 million, then you get to shake hands with Jensen Huang, jensen, right. And then we have, of course, what's happening on the European level, or government level, where Eric Slotner released a digital strategy, or presentation report.
Speaker 2:And then on top of that I mean like we had the whole we can move into Europe, but of course, the European story of the 200 billion being invested in that then trickles down into AI factories and Mimer and all that. So we can go from the startup to the government, to the macro level of Europe, I guess. But I think we should start in the startup space. So your feeling from the guys you're following and what you're seeing from London is like it's brewing in in Stockholm or Sweden. What do you think about that? What do you think?
Speaker 1:I'm asking you is what I'm hearing online. Is that actually happening? Is that how it feels on the ground here?
Speaker 3:I mean to give my view, I mean for one. We should be proud of what we're doing in Sweden and Europe, but to say that we, in any way or form, are ahead or even close to the US or China, is simply wrong. I would say.
Speaker 2:Yeah, but I fully agree with that. But let's move the goalposts back to Europe. Is it happening more in AI? Do we have an abnormal push right now around ai in stockholm? Sweden versus europe?
Speaker 4:yes, but I think that you we need to look at this from a perspective of opportunity and what is happening on the market.
Speaker 4:You are understating seriously, uh, right about, like we cannot compare ourselves to silicon valley but we are not talking about right now what is happening. Rather than the enthusiasm and the vibe that is happening right With Volovable and the Norwegian company I don't know what was it and et cetera, there is a very big right now momentum and enthusiasm, like I have seen it in 2013, 14 and 15, when you had like a multiple meetups happening almost every day about startups starting something new. Right now, it's about starting a wrapper, a agentic type of a vertical solution that you will do something right. So I think that, from a opportunity and emotion and and moment in time, I think that, right, this is the best place to be. Can I place it like that? Yes, I will not judge about cloud Maud and all these other things, because, of course, we know that we are not there, but the enthusiasm is back to 2015, and you can feel it and it's mesmerizing and it's captivating, and you're even wearing the T-shirt Silicon Valhalla, yes.
Speaker 2:So Silicon Valhalla, yes, silicon Valhalla, it was. It's a fun little. What is it? It's like another community, but someone coined it.
Speaker 4:So it was coined by this Dagens industry reporter. I do apologize, I know the name, it will come, but so she basically wrote an article and then created this uh logo and put it on. Uh put it. It didn't, it didn't went viral, but it uh is becoming there, uh, so she put it uh, the logo actually uh online so you can pick it up and you can make your own t-shirt from it. So I said like Ooh, this is great logo, so I made one.
Speaker 4:Yeah, but uh but Oliver took it. So I made one, but Oliver took it. And now what I learned is this week he was actually he bought a domain as well. He was 15 minutes away from me.
Speaker 2:I was in.
Speaker 4:Dubai, so I couldn't buy it.
Speaker 2:But it's okay. But let me start here, right, but?
Speaker 4:it's a great enthusiasm, so we cannot deny that.
Speaker 2:The enthusiasm is here and we had Sverker Jansson leading up who said it's really, really well about Silicon Valley and how the big tech giants feeds an engineering community and then the engineering community goes off and starts things and it's almost like we've had the engineering community around Spotify and all that.
Speaker 2:And there was a first round of cool stuff. Now we have the second round and I even saw some trees like what has been sprouting AI startups and you can see, you can follow Depict AI, you can follow Sana Labs. So you can see Anton, you can see Oliver and a couple of things and the people who've been around those first generation AI startups if I want to say it like that who are now then going out. So we are seeing in a microcosmos in Stockholm right now how engineering is feeding new startups and it's happening right now. So I think even if it was maybe Oliver who pointed this out or someone else how you can always, if you do the same, you can track PayPal and the mafia. You can track from the Google into perplexity. That phenomenon you can now see in Sweden, and not only digital companies but AI companies.
Speaker 1:So a very hopeful picture for the future.
Speaker 2:And the community is tightly knit Because if you follow like I mean, mean like Anton started with with sauna Labs right and then he went to he's used anything and Stockholm is small, like one knows these people.
Speaker 1:I think, I think so yeah, yeah, yeah.
Speaker 2:So sauna and then and maybe I'm wrong here, but I want to show the logic, what?
Speaker 4:is happening, but there is many other startups that are super. I mean, uh understand, others should know a little bit more about this. We are part of this and you can track people from.
Speaker 4:Pretorial, so it's quite a lot of things that are happening, but the latest news was that Paris was taking over the capital of startup from London. Right, so you can see that there are things happening. There are things happening in Germany, things happening. I think that Germany is actually even investing. I saw an ad of Germany actually inviting startup companies to build their businesses in Germany. So what I'm saying, we are not there, but the enthusiasm and the approach that we have right now is back to 2015. So it's great and I think it's going to be great. However, let's sorry.
Speaker 1:Like. One thing that has been missing, though I think in a lot of Europe and Sweden, is the money. I was going there, so how does that look?
Speaker 2:I was going.
Speaker 1:No, but obviously the question is more like is capital available for companies here? Is capital available for companies here? If you want to get into the space that it seems like Sweden wants to get into, with actually training foundational models as well, not being completely dependent on outside of Europe Anderj, I know your reaction to this, but if you want to get into that space, if you hope that companies are going to go into that space, you need a lot of money for compute.
Speaker 4:Yes, which we don't have, but in any case, but we're going to build our own, so then we don't need money.
Speaker 3:Just trying to make me annoyed, but I want to answer your question because I was planning to go there.
Speaker 2:I mean, like a friend of ours, Evelina Antila, right, and she's been talking about this quite a lot, If we follow the two, three years, you know everybody are cool. We have too much VC money. It was easy to get VC money for startups, for AI startups, and then it got fairly cold and it was a little bit like show us the money and maybe I'm not sure how we are in the middle of 2025, because 2024 felt cold.
Speaker 4:But it's interesting that most of the company before it was the most focus was actually on the data layer and the foundation model layer, but right now you can see that there is investment quite a lot in the wrapping application layer, and I can actually concur because from our events I can see that the innovation is going more from the technical people towards the edge. So this technical, what is called this vertical, this wrapper type of AI tools I think that this is where the money is going to be in a short while, of course. So they make money and who makes money from them is basically the LLM model, right, the big model companies, and who makes money of that? It's NVIDIA, right, so eventually it's a siloed, but I think it's a really.
Speaker 3:I agree with you and I think it's actually a very good thing that investments are targeting more the application layer. It means more focus on getting a product out there that actually does work, that has some kind of business model that is actually applicable, and not just building something that may work in the future with some kind of I'm not saying it idea that we never know if it will work. I mean, if we were to compare US to Europe in some way how investments are being done, I would say US is so much more focused on the commercial side of things and the engineering mindset that they are having versus Europe is very much in more of the research part and making sure that we have a lot of awesome people, which we do, and building a lot of prototypes, which we have, but very little on the product side, which I think is a severe mistake.
Speaker 2:Yeah, and we were onto it. I think it was even before the camera was on. So sometimes we say we compare Sweden to US. This is your words. I'm trying to reiterate it's companies, guys. It's like we are lacking the big, the players that has gone from startups to hyperscalers that then provides the engineering that then provides the compute. So when we are talking about, oh, we need to have compute in Sweden, we need to have an LLM, a large Swedish LLM model, you know who is Sweden. Let the company do it. You know how can that work if we fundamentally don't have product?
Speaker 1:It's great to say yeah, let a company do that. But my only concern? It's not even a concern. I'm just curious. Are we going to get there with people building on top of other LLMs? Is that going to yield the next Google?
Speaker 2:I think, this is a great thing, what you need right now I can follow his last LinkedIn comment around. We need to distill rather than be lush. It's a smash going on. Now Tell us your story, how you see this. I'm never going to get out of here. He said he wanted a proper after work, so he didn't actually mean this part.
Speaker 3:I get a bit annoyed because I get into these kind of discussions too often and I think it's very one-. Get a bit annoyed because I get into this kind of discussions too often and I think it's very one-sided in the type of discussion. And, of course, they all speaking about sovereignty of Europe and Sweden. We need to be getting rid of dependence to the US and I'm like fine, I would love it, but you can't just wish for something and believe it will come true. You must have a realistic way of doing so. That's why I believe we need to have at least a two-prone approach to this right. Sure, let's do big investments in trying to build our own cloud provider that is at least somewhat close to the big European top cloud providers. We don't have it today, not even close, I mean.
Speaker 4:American cloud providers. It's easy, man. Ericsson and Nokia will do it. It's going to be great, it's going to be awesome.
Speaker 3:So, okay, I think we agree on the intent. It's just you can't just simply say something without having a realistic plan to achieve that. And then if you do believe and you want to have, okay, independence of US, fine, but what do you do until you get that? Let's say everything were to be going super well. Potentially, in five years we could have something that actually makes great use and can let companies in Europe work on a European cloud provider. In best case scenario five years I would say. What do we do until then? Should we just wait for that then and do nothing? I think that would be a horrible mistake. Of course not.
Speaker 1:But there are a lot of good examples where we're not just talking about you know building so clearly upon. Well, it is building upon. I'm thinking about modal, for example. So modal seems like a way to really start getting into the space of you know, serving the needs of, like very fundamental needs of AI, especially on the inference side. It seems like to me that seems like it's at a slightly lower level than these that just build on top. So it's providing a very fundamental type of infrastructure that's required that others can then build on top of. Yeah, but what it looks like, at least for my novice angle, is that a lot of work that's happening now are sort of on the end point application, like it's it, it's a final uh and it's not serving other applications in turn. Right, is that not the case?
Speaker 3:I think we can still do both. So think of three tiers. Then bottom tier is basically okay, base cloud provider functionality. You can think of it from like foundational models, like the super big multi-trillion kind of parameters in type of chat or GPT 4.5 or something. But then you can think of it in like a middle layer where you have, okay, you can fine tune it now. Now we can take a model and for Storytel or for whatever kind of company we need to now have a model that works for our needs.
Speaker 3:We can't run a multi-trillion dollar in the cloud because it's too expensive and slow. So we need, for practical reasons, to have another model, and then you basically distill it and there are awesome techniques to do this. It's not very well researched and I think this is where you should spend time researching. If we become the best in adapting models to specific needs of our society in Europe, in Sweden, and to our companies, that would be awesome. That would also lead to that we can actually use this model in practice, because they are only a few billion parameters, then it means it's actually very feasible.
Speaker 3:The latency is good and we could do research, not on how to build the pre-trained part, because it's not that hard, but it's actually surprisingly hard to do the fine-tuning. It's so many different techniques there. So you know, I think there is so much more we could do there that would be insanely valuable to know more about. And if we did that and fine-tuned for European values and Swedish values and gave it away saying here is how we fine-tune for European values and Swedish values and gave it away saying here is how you fine-tune a model for your needs and even have like a reference architecture, potentially building upon the Americans, one to start with, and then slowly starting to build that dependency away, then we could have value from day one. And I think we need to have these kind of two prone approach where you do the middle layer already today and then you have all the application layer as well. But if we invest in the middle layer now and then try to in five years potentially fix the bottom layer, great, but we can't just do the bottom layer.
Speaker 2:That would be stupid. We've had two different guests on the pod. That sort of exemplifies these two approaches. So we have on the one hand side, ai Sweden, who wants to build Svea GP3, who basically starts on the foundational level, and I can tell you, with me and Anders they were grilled. Why do we build a foundational model? And then another guest we've had several times and of course we had Lovve Börjesson from KB Labs when they released KB Whisper, right.
Speaker 2:So KB Whisper, what they've been doing with the BERT models is literally okay, we don't start from scratch. We start from some foundational model. We get a really good training data set, we fine-tune the techniques. We get a really good training data set, we fine tune the techniques and then ultimately they release KB Whisperer on Hugging Face after doing their own training in one of the European clouds and all of a sudden now a very small model beats the crap out of ChatGPT for Swedish text, oh, or sorry, audio and text and text, right. So here we have. You know why, you know for Luve. It's like okay, of course we can build a European infrastructure, of course we can do that. But do we really need to go all the way to the trillion dollar trillion parameter approach? Or can we distill something and then we put that in a llama, we do something else and then from here we build an infrastructure?
Speaker 3:provider. Can I ask a leading question to you, aguin, go ahead. Should we only invest in the bottom tier or should we have a two-pronged approach, with investing both long-term and short-term?
Speaker 1:Well, you're asking it in a way that I can't say anything other than the last one on this. So angled, it is no, but what I think is yes, we need to have investments in multiple layers. I'm saying, is it enough to just be on the top layer Is my question. And one example. But we don't. We can have the three layers right.
Speaker 1:We can have the three layers right. We can have the three layers whatever you want. One interesting example I have of another industry that is very much in this is looking at silicon manufacturing. The world as it looks right now is design happens in many places, fabrication only happens in one place Exactly, and this can lead to a lot of weird situations and tensions in the world, problematic supply chains and so on, and if we're really talking about a world that is now evolving in a direction where supply chain security is becoming increasingly important and this being the concern of the Swedish government then I think we probably don't want to end up in that situation, as we did for silicon manufacturing. And these companies, arguably, have taken the approach of well, let's just build. It's a very difficult and tricky thing. They've built on top, they've created a huge amount of value, but they're still deeply, deeply involved with this other part.
Speaker 3:I think we all agree dependencies are really, really bad and we need to have a market that have a number of competitors working. And that's not the case for silicon manufacturing, for sure, yeah. Nor for, I would say, frontier models, and we wish we didn't. Nor it is for cloud providers, but I wish we had. But just you know, wishing for it is not enough. You need to have some kind of?
Speaker 1:that's for sure. True, but maybe you, you need to take small strategic slices of all of these layers. Yes, that's what I'm saying. So maybe you shouldn't just invest broadly in the top layer. Maybe you should make sure that investment comes down into small slices across all layers.
Speaker 2:And if I summarize those two angles, I see the problem with only the hardcore front and application layer, because we're really not only talking about application layer, we're actually talking about front-end layer and, on the other hand, we need to respect the research and investment that went into Amazon, that they stumbled upon AWS or Microsoft and how they 30, 40 years of work, went where they are, and to copy that in less than five years on the lowest level is impossible, is right now. It's almost like it's bipolar, right. Either we have an application layer conversation and all the VC goes in here, or someone talks in European. Oh, we need to build our large language model at the very bottom.
Speaker 1:Right, and they talk about building the entire layer.
Speaker 2:The entire layer. That's the crazy thing, you know. So, instead of going surgically and what I then argue with, I concur with Anders. Don't be stupid. If you think about where is the added value to? You know, if my key objective is to create European sovereignty, that's my key objective language sovereignty, cultural sovereignty, blah, blah, blah. I'm not sure I'm getting that from the lowest layer. I think I can get it out of the fine tuning for a tenth of the cost, so I can get to the objective of tuning for a tenth of the cost, so I can get to the objective of sovereignty to the tenth of the cost. Then I can provide a middleware, I can provide a digital service like the European Amazon or the European ChatGPT. It's simply that. Who gives a fuck if we did the bottom work, if it was distilled?
Speaker 3:I don't care. Sure, I mean, we should have a multi-prone approach or a multi-layer approach, and of course, you know EU is investing so much in just one layer and some other more commercial. You know, VC is just in the top. We need something in the middle. That's missing, right? So I hope we all agree that that's something that's missing.
Speaker 2:So the objective of Sovereignty is the right one. How do we get there? The fastest and the most smart? Yeah.
Speaker 3:Should we, if we try to end up in a couple of minutes?
Speaker 4:Just to end up, we didn't discuss the Wallenberg that much, so Wallenberg basically company. They made like a new deal with NVIDIA to build a new supercomputer which will basically be led, if understood, by the experience of Ericsson right, yeah, ericsson, atresanica, saab, so it's the Valenberg companies basically.
Speaker 3:Yes, SAB, et cetera.
Speaker 4:But most of the what is called the skill set is coming from those companies. Uh, and ericsson as well. And how much was it what they're building? Uh?
Speaker 3:is some of the latest super pods from nvidia.
Speaker 4:You know, yeah the blackwell, yeah the blackwell, gb300 things, I think and I just returned from uh dubai and you know, these two countries, the United Arab Emirates and the Kingdom of Saudi Arabia, they're like the cousins who basically always compete who's going to get the biggest Porsche. So I was there, we were doing a conference, and then, of course, trump was there, together with Jensen and many others, and so, if you look at it, saudi Arabia wants to be basically so they want to be the global leader within AI. So they are building 500 megawatts. What is it called?
Speaker 3:No, no, it's gigawatts. This is a crazy thing right. There is unheard of to have a multi-gigawatt data center and they are planning like a five gigawatt data center. And you're not talking about just building the factory.
Speaker 4:They're actually building an entire area around it, all right. And then, of course, like well, this is going too much marketing, so we guys in Dubai don't like it, so we do another thing. So I remember it basically makes a deal with OpenAI to make ChatGPT open for everyone, which is actually a really, really good thing, so all people in Dubai right now can go and use ChatGPT for free. You're talking about the pro version. Right On top of that, they are also big building one gigawatt ai super computing cluster with the initial uh 200 megawatts expected to be operational this year. And this is what is interesting actually about uh, um. What is interesting about, like, when you know this, um, um, investments are coming from the top. We have in Sweden, still Wallenberg, even from the beginning, and thank God that we have the family. They invested from the beginning and they're continuing to invest, while you have countries like this where the government is immediately just making decisions like that and build like super big investments and taking the leap towards leadership in this area.
Speaker 3:And Wallenberg is very important for Sweden, but they are a lot, you know, research focused and we need that. I mean I'm a researcher to start with, but I mean we need someone else.
Speaker 4:My point is that, yeah, my point. I think that we agree on this, and my point is that things are developing fast and the companies in the countries that invest the most they will have the biggest benefit from it, and right now we are missing a mission, a goal, a perspective, just basically a vision. Where do we, Sweden or the Nordic, want to be in the next 10 to?
Speaker 3:15 years. Just take the investment that the US did 500 billion dollars.
Speaker 4:Well, they have not invested yet In coming years, together with SoftBank. They have not invested yet In coming years together with SoftBank.
Speaker 3:They already started digging and building things.
Speaker 4:It's interesting who is building it, who is?
Speaker 2:Danak, of course the Trump company. No, it's not the Trump company, it's the biggest construction company in the United Arab Emirates.
Speaker 3:I think you know the scale of it. $500 billion it's basically the size company in the United Arab Emirates. I think you know the scale of it. $500 billion is basically, you know, the size of the Swedish GDP, all of GDP in just you know one big AI infrastructure investment. I mean, if someone says how we can do the same they're doing in the US or in what they're doing in Saudi Arabia, I mean you need to have. You have, you need to understand the reality here.
Speaker 2:Yeah, so this is also your arguments for focus. Build smarter on the Middle East. The United States didn't invest in that it was.
Speaker 4:you have, you have SoftBank in the same, you have Oracle and you have OpenAI.
Speaker 3:So you have the but it's supposed to be an investment made in the US. Yes, so it's not a government investment, yes, but it's a US investment. Yes.
Speaker 4:But Europe, they said as well, they will do 200 billion, 200 billion yes, commitment Right.
Speaker 4:So when you say commitment, that means that, hey, I'm going to be a facilitator, that I will gather 200 billion to build something. I think that we can do it in Sweden as well. So hear me out, all right, we don't need to pay for it. But, for example, if you have a deal strategic deals like with open ai and the soft bank and why we cannot do it? Equity you have great rich companies here, etc. Like uh, sqf and etc. Etc. You have like a lot of companies. We can collect 200 billion as well, but we want to.
Speaker 2:But don't, because we don't have a vision.
Speaker 3:No, and we don't have a way to drive it. I think the problem with the EU investment is that it's very scattered. They specifically say it should be an inclusive approach, meaning it should be 50s or 100s companies taking part of it. I mean, in the Stargate investment, it's basically three main companies not a number of other companies, but three main companies OpenAI, oracle and SoftBank driving it, and they have a track record. They are specifically targeted to say you're supposed to do this. We won't be able to do that in Europe, I would say.
Speaker 2:But it's still good. But this is another problem when we are comparing China, sweden and Europe and people seem to forget Europe is a construct. We are countries. We are not one country. We are not one decision. We are not Saudi Arabia in one decision maker Together.
Speaker 4:If we come together, we have all the pretenses to lead right. We have the population.
Speaker 1:We have the mindset, we have the schools, we have the infrastructure, but have the mindset.
Speaker 4:We have the schools, we have the infrastructure, but we are despaired.
Speaker 3:We don't need to invest in these scales.
Speaker 4:I think we can find value.
Speaker 3:without investing in these immense scales, we can actually do it in a smarter way, I would say. Then we can find great values.
Speaker 2:You need to flip it, Because we are not one Saudi Arabia, prince sheikh, that can invest. We are, we are. We have many countries in Europe. So how many? I mean? Like we can talk about EU 28 in EU, right? So 28 countries with quite different cultural and economical opportunities. So clearly, the starting point on how we can look at this is something else. So we need to work, do smarter and we might need to create a company together. Then we need to do something, because to have 28 countries as the who owns and who decides is an impossible. Yes, you cannot do it, you know so there's things to be figured out.
Speaker 4:The EU construct is there. Building these AI factories is actually the construct. Is there that we have, like what? Eight new supercomputers that are going to be Four to five?
Speaker 3:or three to five, I think.
Speaker 4:Across Europe. So we have still opportunity to innovate, it's not enough.
Speaker 3:I mean the point is, it should be the middle age that we invest in, but it doesn't matter.
Speaker 4:As long as we have the enthusiasm and the culture of innovation, I think it does matter, because we were going to waste money, you're just such a bad specialist Come on. I'm trying to get in the position. I mean I have to agree with Anders.
Speaker 1:In a way, these data factories you guys from Peltario. We work together with people building data factories and don't want to call anyone out here, but like I still don't know what a data factory is here, but like I still don't know what a data factory is, the data factory we worked with it was a room with a couple of GPUs, like I mean literally a couple of scattered devices like oh great, I mean, this is not something that will force collaboration and good outcomes.
Speaker 2:And now we're getting down to the problem, right, because when you go to three or four companies, when you even go to a crazy guy like Elon Musk with a clear vision, they start putting what does that mean? The 200 billion investment for what? What are we trying to do here? What are we trying to do? And what Anders is trying to do is say let's focus on a mission and goal to build the best middleware, you know, because we are now fluffing around on a very, very high level and this is the reality. When you now start going down into you know the people, who knows anything about anything to build this, the way the decision making is done and the way this is not sort of trickling down to a clear purpose and mission and ultimately engineering. That's the main problem, in my opinion.
Speaker 3:Should we try to end?
Speaker 2:And, on a positive note, we're never going to end on that.
Speaker 4:We are ending this. We are ending this. I just wanted to throw one more news, very simple one.
Speaker 3:I also have one news.
Speaker 4:Because I think this one is quite interesting and it's you know, salesforce bought informatica. Yeah, I thought that that will be almost impossible, that company to be bought because it's still flourishing, it's doing great, and salesforce basically bought it for eight or nine billion, I think it was I have.
Speaker 2:I have a joke around this. Yeah, the, the guy influencer did a joke five years ago and this is now the. This is actually the investment thesis. Imagine if an iPaaS company, an infrastructure platform as a service company, like Salesforce, started, you know, invested in building up governance as a service. Then you get iPass gas and then we have the new brand in for Salesforce.
Speaker 4:So the investment, the investment sees his eye past gas in any case, I think it was a very big news. Obviously they're buying it because they want to push their agentic AI or AI agent. So the same. Actually you can see the same thing happening with SAP, so agentic is becoming quite quite big thing right now.
Speaker 1:So, goran, you're saying Salesforce want to push their agents and they're doing this through. I'm not familiar with Informatica, so Informatica is basically.
Speaker 4:It's a great data management tool. You usually deploy it in a large enterprise scale platforms where you want to know everything about data, so it's a data governance model. It's a master data, metadata, all of this.
Speaker 2:Which is important. It's really really big there. So imagine you have lots of agents and now you need to govern a policy and you need to go from guidelines to guardrails and computational policy around your agents. So then, if Salesforce is going hardcore on agents, they kind of need data governance around the agents. So then, if Salesforce is going hardcore on agents, they kind of need data governance around the agents.
Speaker 4:Because they're quite similar.
Speaker 2:I think the data governance and AI governance will become like a very so if you truly believe in an agentic approaches, then you get to data product approaches, you get to distributed approaches around data and data sets and the agents Right.
Speaker 1:Salesforce is interesting. It feels like one of those odd, like super boring companies that have an incredible research team, exactly.
Speaker 3:The papers they have produced is amazing. It's incredible.
Speaker 1:I wonder how they managed to pull that off, and I think they did a good bet there If these things are going to pay off.
Speaker 4:They did it by purchasing other companies. So, as you can see, it was at the beginning, it was a CRM company right. Then they bought Data Cloud, tableau, mulesoft. They bought now Informatica, so they have plenty of companies. So now they are 360 view, basically database AI type of a company. It's also a very rigid company, but that is a different thing. It's very expensive. For example, we use HubSpot. Salesforce is a different thing, it's very expensive.
Speaker 1:For example, we use HubSpot Salesforce is a little bit like 10 times. But why isn't Ericsson or Volvo able to have this amazing research team embedded within the company? They certainly have the money for it as well.
Speaker 4:They used to have it, but they own IoT. Ericsson still has great people, but they're not doing a lot of papers. Right now they're producing patents.
Speaker 1:Right, I've seen that patent wall.
Speaker 2:Ericsson has been one of the best. I mean, it wasn't for a long time. Ibm top of the leaderboard and Ericsson second. So they've been good at patents and they are very much on radio. All right, I will shut up now so you can continue. But, how do we wrap up on a positive note.
Speaker 3:Yeah, okay. So I think a positive note is actually what Erik Slotner just announced yesterday or this week or something at least, so a new digitalization strategy, and they actually speak a lot about AI there and I think, goran, by you inviting him to Data Innovation Summit, I think he heard and learned a lot, so I think we-.
Speaker 2:So once again causation or correlation? I'm not sure.
Speaker 3:Yeah, it's probably causation here. I think we could argue that to some extent.
Speaker 2:So the data innovation summit was before the report was released, so everybody can make up their mind.
Speaker 4:I take the credit because I was keynoting right before him.
Speaker 2:And you were right after him. So I think the two of us have the biggest impact here. And then we had a discussion. I have to say that the lineups.
Speaker 4:that morning it just like everything clicked, it, clicked, it, clicked, so much. I was looking at the post, the post on the social media after the event and it was just like you can see that there is a one single what is called a theme and everything else came. It was uh. So of course it's uh, my event right, but I amazing.
Speaker 2:You know you're, and so you're completely objective. Yeah, I'm completely objective with this.
Speaker 4:But people that know me, they know that I'm a very, very, very critical about especially my events, and I think that this event, something happened there and just clicked, everything just clicked towards like One keynote leading to another one. Embracing this transformative value of data and AI. And then it was doom-doom, and Eric Slotner did a great job as well. Many people commented about his vision and et cetera, which was super good.
Speaker 2:I need to give him credit because sometimes politicians, you need to give him credit because you know, sometimes politicians, you need to suck the words out of them, right, or you know, but he we needed to, you know. Shut him up.
Speaker 1:So he had concrete statements, ideas, yeah.
Speaker 3:He's a politician.
Speaker 2:He was very concrete for a politician. Okay, okay, if we go through a bit very, very briefly, he was politician.
Speaker 3:He was very concrete for a politician. Okay, if we go through a bit very, very briefly, he was good.
Speaker 4:He was good. I think he did a good job. I'm very happy that he was the one attending.
Speaker 3:If we go very, very briefly to what they said, I mean, it's simply a strategy saying we will produce a new AI strategy and we will produce a new cloud strategy as well, or cloud policy, which I like even better. So in that sense it was a rather shallow kind of announcement, but I think the direction he's setting out is very good. So he basically said, in his five areas I think it was digital competence or education is one, of course, and then we need to digitalize basically the private sector, and then we have the whole healthcare sector and public sector. But also he spoke a lot about connectivity and I'm not just saying that just because I'm from global connect, which is connecting society, but that was.
Speaker 4:I have never seen him doing a jingle before.
Speaker 3:That was very nice. No shots, no shots, that's a good one.
Speaker 2:That's a good one. I'm sitting here with a cap on.
Speaker 4:Yeah, I know, but this was a proper jingle.
Speaker 3:It was like so subtle You're going to ruin it now, by the way. Yeah, I'm feeling ashamed now, but the point is, all these kind of five areas he said every one of them will be penetrated by AI and data. So all of them need you know, you need to have digital competence in terms of AI and data, you need to have AI and data to digitalize the private sector, you need to have it in public sector and healthcare and so forth, which I think is really good. So I think the direction is good and just that he says it actually will produce an AI strategy, something that Finland did back in 2017 and Sweden still not really have.
Speaker 2:So the report from the commission doesn't count as a proper strategy and I will hold him to. If this is going to be a proper strategy for Sweden, can we argue that? Can you have a strategy without commitments in money and budget? So if he talks about the strategy, is he actually meaning to put money behind the words.
Speaker 3:I hope so, otherwise I think it's quite. I think in the spring it did put a small amount like 30 million Swedish. Yeah, that doesn't count?
Speaker 2:That doesn't count, it's too small.
Speaker 3:Anyway, I think it's a really good direction. Yeah, it's good and I think he's really. I know he's. It's not easy to be a politician and they don't really have like one part of the government that have responsibility for AI. It's basically spread across all departments that do exist there, so it's super hard to drive that and essentially they don't have the competencies.
Speaker 3:They need to rely on Skatteverket and others who sort of is sort of best in class and that they have a different purpose and it's also that you know AI as a topic is not very politically interesting to speak about. No one will win an election next year because they say we want to invest in AI. People are afraid about AI, mainly. And this is so stupid, I think, and I think media is not doing a good job here. They're just speaking about the bad potential use cases.
Speaker 2:Back to what we talked about before. I think maybe that was before the camera was on. What do we see on LinkedIn? We see very, very, very technical stuff on how you build agents, or you see very, very fluffy stuff, even doomers or whatever but how this works or what do we need to do? So we have the conversation. This will change everything it says on the top layer, but we're not really talking about this middle layer, what it really means. So we're talking the top fluffy or super technical MCP protocol approaches to agentic AI. Where's the middle part? Where's the tactical layer missing? Interesting question.
Speaker 3:Agrin, this was a way longer podcast than I was planning for. Thank you for listening to us.
Speaker 2:But now we need to shut up. Let Agrin have the last you know, wrap it up.
Speaker 3:I hope you found some value and enjoyed sitting here in an after work speaking about AI right.
Speaker 1:It was great Good talking to you. Thanks for going through the news with me. Honestly, I've lived in what feels like a cave for the past few months, so good to hear the news.
Speaker 2:Are we excited to see things coming out of the cage eventually, right, at some point, at some point, at some point?
Speaker 3:Good luck with moving into the new place that you're having as well and going back to London soon. There, and all the best with upcoming adventures.
Speaker 2:Yeah, but it's going from a city life to not country life. I wouldn't call it, but it's different. Right, it will be different.
Speaker 1:Yeah, looking forward to it.
Speaker 3:Perhaps an invite as soon as you come to some point where you can announce even more stuff we would love to hear more about that we would love to have you back when you have everything you can share.
Speaker 1:For sure, it's been a pleasure, guys. Thank you so much. Thank you so much. Take care.
