Michael hosts the Talk Python podcast, co-hosts Python Bytes, and runs courses at Talk Python Training. We discuss his recent shift to mainly self-hosted open-source options for everything from marketing emails to analytics and more.
Will Vincent 0:00
This episode is sponsored by Mailtrap, an email delivery platform the developers love try for free@mailtrap.io
Hi. Welcome to another episode of Django chat, a podcast on the Django web framework. I'm Will Vincent, joined by Carlton Gibson, hello, Carlton, hello, will, and we're very pleased to welcome back Michael Kennedy from talk Python. Welcome Michael,
Michael Kennedy 0:23
hello, hello. It's amazing to be here on your show. Thank you guys. Good to see you. Thank you. You
Will Vincent 0:28
are one of the original inspirations for this podcast. I was, I think, a guest on yours, and we're just talking off air that was over five years ago, so it's, uh, time flies.
Carlton Gibson 0:38
I can't believe it's been five years.
Will Vincent 0:41
I know, I know. But if I said talk Python, you you have this whole constellation of things now as well, right? You have Python bytes, you have your training courses, I feel like I'm missing something. Is that the full side of things? That's
Michael Kennedy 0:53
pretty much it? Yeah, tiny bit of conference speaking and stuff, a little bit of YouTube, but primarily talk Python, Python bytes and the courses around it.
Carlton Gibson 1:02
So you mentioned conference week. I saw you flashing by on the news this week or last week about some keynote somewhere exotic. Look. You been up? Yes,
Michael Kennedy 1:11
yes. I had a wonderful time. I got to go to give one of the keynotes at Pycon Philippines this year, which was very excellent. I think I'm over jet lag, and then we had daylight savings. And so, you know, that's, that's,
Carlton Gibson 1:28
does that mean the kids get up earlier or later? I can't remember.
Michael Kennedy 1:31
They are forced to get up earlier than they want, right? Okay,
Will Vincent 1:36
we lose, we lose an hour, yeah? But
Michael Kennedy 1:39
it's glorious. It's glorious. I live in the Pacific Northwest, in the US, and it is like we have two seasons. We have summer and rain, and also that comes with darkness. It'll be dark at like 430 in the winter and rainy, yeah, and the sun was out till past seven last night. I feel like summer's already here, even though it's not. So no regrets. Yeah, it
Carlton Gibson 1:58
makes a big difference. So
Michael Kennedy 1:59
I'm on Team daylight savings. Let's go. So
Carlton Gibson 2:02
how did you go in the Philippines? Tell us about your keynote. What was the what did you talk about down there?
Michael Kennedy 2:07
The keynote was super fun. I was called the state of Python in 2024
Carlton Gibson 2:13
Oh, that's good, because I wanted to ask you about that. I was like, What are your
Michael Kennedy 2:18
hot takes? Michael gone takes. Well, it's, you know, I think it's, it's interesting to consider that in the Python space, it's so distributed the directions and it's, it's like Brownian motion, almost like stuff is just moving around, and like whatever channels it takes and where traction gets for various projects. That's where the future goes. Right. Even amongst the core Devs, there's a bunch of them working on things. So if I went to a keynote, or you went to a keynote at Apple, Tim Cook, Tim Apple would come out, and he would tell you, this is what we're doing with our new stuff, and this is what the team has been doing. You go to Microsoft build or Google IO, there are people come out and say, here's the direction we've decided for you and us as a community. Hope you like it. There's none of that in Python, right? But partly because, who's going to speak for the 10 different web frameworks and the 1000 different data science things and all that kind of stuff, right? So I tried to put that together. I tried to say, like, I Where's Python been last year, where what's important, and where is it going in the future? So, fun stuff. Okay,
Carlton Gibson 3:34
so my, I mean, one question, one not question, one problem I have, you know, I'm quite involved, but I'm very involved in Django, and I don't know anything that's going on over there, and all of a sudden I'm like, Oh, wow, there's this brand new thing that, how do you keep up? How does, how does one keep us kind of overview of what's going on in the different areas?
Michael Kennedy 3:50
I have a bit of a cheat card in that regard. Yo. You got Python bytes, right? I got talk Python. And I got Python bytes with talk Python, I can, I can dive into deep topics. Like, one of the big things is, you know, the whole astral UV stuff, yes, that's going out, right? And so I'm talking to Charlie marsh in three hours or something about UV, right? So I get to ask all those questions and kind of figure that stuff out. But then Python bites, like, 678, topics that you've found every week, and you can not just get to you have to find another six for next week. And so it just keeps you kind of on top of always searching. But
Carlton Gibson 4:32
for the, for the, for the rest of us, we should just listen to your podcasts, right? That's yeah,
Michael Kennedy 4:37
take it on Coast.
Will Vincent 4:42
So do you we talked about this last time. I mean, when you're so involved in talking about and producing things around Python, do you still get to code a little bit, right? Because you had recently, we're going to talk about infrastructure and deployment, but like in terms of actual Python, Python, do you still get to keep your hands dirty? I. Yeah,
Michael Kennedy 5:00
yeah. It seems like I would just be busy talking, right? But there's a ton of infrastructure and stuff going on. There's kind of a whole DevOps side. But, you know, the the talk Python training stuff is, it's basically its own SAS, right, evolving over time in lots of ways. I just noticed I was doing some stuff in GitHub. I'm I'm trying something different. I'm always busy, you know, I talked about all these different projects, and then I have people working on the courses, and I've got, you wouldn't believe how much email, and it's always just pulling me away. And so I've, like, big ideas for stuff that I would like to add, and it's always like, well, maybe next week. So I was working on something I'm calling platform week where turn off the email, turn off, have no meetings. Who knows, maybe even I'll go stay by the beach or something. I'm just kind of more alone. I haven't decided. But just sit down, come up with a plan, like, if I had 40 hours to just write code and sort of tackle the bigger problems that I've been thinking that I should, what would I do? So I was working on that, and I realized, just for the courses I'm on, issue 1690 or something like that, it's mostly me, but also a little bit, a little bit of stuff is in there, around, say, like our mobile apps, where I worked with another developer, and things like that. But that's a lot of code,
Carlton Gibson 6:20
yeah. And what's, what is it now that it keeps you excited, like, when you you're coding away and you're actually just focusing on the code side, what the exciting bits that you you like? Oh, yeah, I like doing that, or I like this,
Michael Kennedy 6:30
yeah, there's just always stuff to learn. There's always stuff to get better at. You think, Okay, I've really got this, this web stuff, nailed. And then htmx comes out, and you're like, Oh, I got this changes everything, right? I know you both are big fans of that. And, yeah, you know, like, Okay, I got hmx nailed. I got, I got the web stuff nailed. Oh, maybe we could deploy it in a whole way, better way. This whole thing I've been doing, you know, is I could completely change that as well as it's never ending. I think the fun thing is, is it's, every time you kind of go through that process, it's better. Like Python is better. The web frameworks are better, the infrastructure and tools, like databases and web servers all the way to the host. Like the whole spectrum is like, Oh, this is even cooler than last time I dove into this aspect of it. And so it's, it's worth, yeah, no, sometimes, yeah.
Carlton Gibson 7:24
The the excitement is, like, last few years, it's been like, Oh, wow. We've got, you know, new layout things on the front end. We've got, you know, CSS finally catching up with where we wanted it to be. And, you know, and then on the back end, all the async thing, stuff has come through. And it's, it's always fresh.
Michael Kennedy 7:41
Yeah, absolutely. I just, just yesterday, I talked about mountaineer, the web framework. And few weeks before, that was fast UI, not fast API, but fast UI from Samuel Colvin and team. And those are, those are both super interesting projects. There was, like, Python deeply integrated with a React interesting ways. I don't do a ton of React, but you know, like, when you write a function in your web app in Python, in mountaineer, it automatically creates, like, an API connection for you, and it generates a TypeScript type safe, like front end thing to call that, and keeps them in sync. There's just a lot of a lot of neat stuff out there, right? Well,
Will Vincent 8:20
it's good to have that attitude, right? Because I think you can also, after a decade plus getting this, you can fall into the trap of thinking, this is just reinventing what I already know, which is a little true, but it's also, you know, generally better, which is why people are taking the time to reinvent the same things. Yeah, but that might be more where, if you you know, if you're three years getting really good at something, and then, you know, a manager says, We're switching and get up to speed. And you're like, I have a feeling that's more where that, that feeling comes from. Or you're, you know, the market changes, and it's, yeah, it can take a little time to feel like you're catching up, even though you've been deeply in one of those areas.
Michael Kennedy 8:59
It can, and if you're, if it's not your choice to go down that new path, that could be frustrating, right? You're like, ah, we were really good with just server rendered code. What are we doing? Like, why do we got to change all this? And honestly, I still think that that's, that's true. I think it's it's pretty amazing, which I guess is why HTML is also really interesting. It's not as bad in Python, though, right? I mean, we could be in JavaScript, not knocking the language necessarily, but it just changes so fast the lifetime, you know, and and backwards compatibility seems to be something that people are like, Ah, maybe if it's convenient, you know, I just saw that they're working on Alpha tailwind form like, all that does is fill me with dread. I'm just like, Okay, what is not going to work anymore? But I don't quite realize. I'm quite excited combination of 10 classes are going to break. Yeah,
Carlton Gibson 9:51
I'm quite excited about the tailwind fourth, because it looks like it's bringing it a step closer to CSS. It looks like, you know, using native layers, right? Than the tailwind layers and things like that. So it might be, you know, that might be nice to because it's not the case that you have to stick with your tailwind forever. You don't have to do everything in JavaScript, right? But you can work it back into your style sheet. And if they go closer that way, I think it might take away a few of the kind of religious criticisms of
Michael Kennedy 10:20
it. Excellent. Well, it looks exciting, so we'll see. Will Are you a tailwind fan? Do you believe in tailwind?
Will Vincent 10:29
I have a redesign of my main site that is not publicly live yet, that's in tailwind, but I have to say, I'm not a tailwind expert. So I worked with designers. Imagine that, who did an amazing job?
Michael Kennedy 10:42
I think we talked about that on mastodon.
Will Vincent 10:47
Yeah, maybe yes, that's right. Yes, you were, you were looking for, yeah, I used Yup, cup. We're great. Who do a lot of work in the Django space. So I highly recommend designers. They're a lot cheaper than programmers, and it's so, yeah, so I've, I mean, I definitely wouldn't say I'm super comfortable with tailwind, but I'm sort of really happy about that. Yeah, I have enough to deal with with your tailwind. Telling curious, yeah, yeah. Just like, I like, I'm, I'm pretty familiar with Bootstrap. I've used a lot of Bootstrap, and there was a discussion of, like, should we use Bootstrap? Should we use tailwind? And it's like, well, I know that whatever it is, we'll probably have to change it in five years, and tailwind seems pretty established and a little easier for me to maintain once someone else does it, but I don't have to worry about architecting it, I mean, because so that was the decision. And, you know, it's a it's a little annoying to have to have a separate server running to, you know, have NPM going, but it's not the end of the world. Like, yeah,
Michael Kennedy 11:58
yeah, that's true. It's nothing. But you do have to figure out, okay, well, there's these little scripts, like watchers and stuff I've got to write,
Will Vincent 12:04
yeah, and then deploying it. Like, is, you know, it was a day to, like, fully figure out, like, okay, like, even just on Heroku, which is still, like, what I use, like, it's like, okay, wait, where's the thing for this? I mean, that's, you know, we both work on courses. Like, I'm like, How does anyone figure this out? Like, why? Why does this take so long? Why isn't there just, like, an answer, like, I will pay money for an answer, like, someone just, like, saved me two hours for 10 bucks or 20 bucks. Yeah, but I guess it's the change is too much, and maybe it's not worth anyone's time. But So anyways, so, yeah, I'm trying to fight the don't change things. Attitude on all this,
Carlton Gibson 12:41
oh, just, just on that. I saw a thread this I saw a thread this week, and it was from a front Ender. And they were like, Look, all I hear is, front ends too complex. Front ends gone off the complexity cliff. Front ends crazy with its complexity. But what they're like, their argument was, what we what were we supposed to do? Just sit there and not look for the better solutions, not try the better things, not progress. And at each step, their argument was that, you know, it made sense that we brought in Webpack, it made sense that we switched to this. It made sense and, you know, they were just searching for a better solution. And it's all it's all right for us, Python desk, or you JavaScript as you crazy. Was that kind of, I think, Well,
Will Vincent 13:20
I think Well, I think it's two things, right? It's the people pushing the envelope. Are large companies with big staff who have maybe needs that the rest of us don't have. So does your average website need all these things? Probably not. And at the same time, you know, selfishly, we all want to feel like we understand everything going on in our web app, but obviously, as it matures, that becomes harder and harder to say with a straight face.
Michael Kennedy 13:45
Yeah, it's probably true. Also the big teams, it's probably more true the big teams that they don't not, the people working on don't necessarily understand everything, all the moving parts, right? Because you got teams right, specialized team members that deal with, like, hey, how do we keep it in production? Right?
Will Vincent 13:59
Right? Well, maybe that's a segue to infrastructure, because that's one of our big topics, and you have a number of notes to dive in, I guess, have you, you've changed your infrastructure or you're thinking about it. Where does all this research come? Oh
Michael Kennedy 14:14
no, this is all this is done. This is kind of the infrastructure equivalent of what I refer to as the platform week. Like, I'm just gonna take some time, focus on this problem, put everything down, suffer the backed up inbox later, and try to dig my way out and get a bunch of changes. I think this was really the impetus for this. Yeah,
Will Vincent 14:38
I was gonna ask, how, why do you why dive into this, right? Like, what? Where is that point where you say, you know, because you're, you have traffic, but you're not Google level, right? Like, how do you decide it's worth now, it's worth customizing things beyond just, you know, paying a Platform as a Service bill? Yeah,
Michael Kennedy 14:55
absolutely. So just to, I don't know. That I would say it's big and it's not totally small. I'll just give people a sense we do through all the talk Python things, we do around 9 million Python backed requests a month, and we do about 10 to 12 terabytes of traffic a month. So it's a decent amount, but it's not it's certainly not Google, like that's a minute on Google or especially YouTube or something, right? But it's a non trivial amount of sort of traffic, right? So for a while, I had, so let's take a step back. So for a while, at least in the early days, I kind of had this, what I think is not really the way anymore that a lot of people think about. I had this belief that the cloud was small commodity things, right? The cloud is the cloud is best at giving you a bunch of small little things. They're super cheap. I can get a $10 thing here, and at 12, you know, $12 thing there, if I want to optimize the CPU to be an Intel, whatever, and so on. And I can piece my stuff together with that, right? So I have a server for this. Server for that. I got some back end servers. I got some failover so I got, I ended up with like, eight servers, sort of in that constellation of things. And I thought, Okay, this is great. So I've got the main stuff. I've got failover servers. If I need to do maintenance and say, like the kernel is patched and there's a security patch on the main one, I've got to reboot that server. There's not too many interesting tricks I can do besides just making it reboot. And so what do you do? So do you use some kind of load balancer to switch it over to the cheaper backup one, or, you know, something like that, right? That was kind of how I was doing things and and it real. I realized, like, I've got, yeah, go ahead, go Karl.
Carlton Gibson 16:48
I just wanted to bump him, because, like you said, 9 million requests a month, so I just quickly did the maths. There's about 6000 requests a minute or something. It's quite a lot, right? So you can't just reboot your server because you've lost 6000 requests?
Michael Kennedy 17:02
Yeah, it's a lot. It's all of your
Carlton Gibson 17:05
just running your blog, and it's like, oh, one person maybe,
Michael Kennedy 17:11
yeah, carry on. Sorry. So
Carlton Gibson 17:13
you so how do you handle the fall over and how do you handle the low band? So you were saying, but yeah,
Michael Kennedy 17:19
well, I was just sort of cycling between these servers and using, you know, sort of IP magic and stuff like that to, you know, sort of devopsy stuff. But what I realized was one, I have all these servers to maintain, and I've got to patch them all, and it's just a bit of a hassle. And then, I mean, you could make a Platform as a Service argument, that's that's certainly a thing, and you could talk about Heroku, but there's advantages not being a Heroku as well. And then what I realized is, well, if you got to take the database server down as one of those things, they're all going down anyway. You know, there's, there's, there's probably 10 seconds of downtime per month. That's unavoidable, unless I get into a more complicated database cluster with replication, and now it's like, okay, I'm not Google, I'm not YouTube. It doesn't justify that level of uptime. I'm willing to take 10 seconds to downtime a month, very rarely, like more if I screw up something. But you know which does happen, but not that often. But anyway, I'm like, there's kind of this single point of failure anyway, and I have all these servers, and then that was one half, and the other half was and I think this is where a lot of interesting stuff is going to happen. In the conversation here is there's actually a ton of open source similar things to what you might use, right? So let me just give you one example, MailChimp. MailChimp used to be cool little startup. You could get your mail over there, and you could sign up people, and you could make a community, and it was amazing. And then it grew, and it grew, and then it was bought by a financial company that just wanted to turn the screws on folks. From what I can tell, I'm sure there's people who don't just want to turn the screws. So no offense, but it's, it's got a slightly different business model, and it doesn't have that like, I'm supporting the small the small person anymore, right? And my bill is just going up, like $100 every two months. Like, hey, we've guess what your bill is now $100 more, sometimes because there's more subscribers, sometimes just because, well, we've added a new feature you're not using, so it's $100 more. Again, you know, it was up to, like, I don't remember, something like six, $600 a month or something. I was like, All right, this is getting out of control, like, and so I realized there's, like, a couple of really cool options for just self hosting, something an open source version of this, all right, well, it got these eight servers. Where do I put this self hosted thing? Well, I'm sure not going to put it like right next to anything else that has access to my database that I. Because I'm not in control of that code. How much do I really trust it? You know, I trust it, but there's trust it, and there's put it on inside the room with you, you know, like that, that animal looks friendly. Oh, it turns out it's really aggressive at night. I can't believe it's in my house. Like, what have I done, right? Like, that kind of thing. So I'm like, All right, well, I need to set up something to kind of manage that, right? Like, all right. Well, maybe I need to set up another server, or maybe I should just, this is ridiculous, and I should start using something like Docker or containers or something so, like, that's one half of the the story. And the other one is I was interviewing or reading different people talking about, like, this cloud, commodity, a bunch of small things. Maybe that's not the right way. Like, for example, I got a chance to have Mark rosadovic, the CTO of Azure on talk Python. We sat down and we were talking, and it's like, oh yeah, we're moving towards more consult more big servers, rather than a bunch of small commodity servers. And they have some that are up to ridiculous levels, like single servers you can go get that are, you know, 1700 core machines with many terabytes of RAM. And they every, every time they put out a new one, they have a different adjective for it. So it started out with Godzilla, then Super Godzilla, and now the one that they have that's really big is, like, mega, Super Godzilla beast or something like that, right? But they're, they're running out of ways to describe, like, how ridiculous it is. So, so that in the David Heinemeier Hansen stuff, like leaving the cloud. I don't know people have been watching that, but, like, similar thing, we're gonna get, like, a couple really big servers, and we're just gonna use modern software to sort of DevOps our way through this better than and so I kind of went down that path because I'm, I'm like, All right, well, I want to take advantage of not just Python, but all these cool open source things for both running your code, for for extending services, for getting away from unfair Sass to having not just unfair Sass, but you know your your GDPR statement has, here Is this other service worshiping in your data to at a very high price we pay for the privilege of doing so. And then here is their GDPR statement and their cookie statement, and they're this and they're that, and it just it makes operating an online property a little bit of a pain, a little bit fraught with that kind of stuff, not to mention blocking, right? Like, if it's some kind of API endpoint that uses their domain, dot JavaScript thing, and maybe that's blocked as tracking, right? So there's just all these challenges of integrating with other people's stuff. So there's a lot of advantage to running things yourself, but then you don't want to let the wild animal like into your house.
What do you do? Yeah, okay, good.
Carlton Gibson 23:05
So I want to. So one thing I want to pick up on before we perhaps switch over to the container side of it, it's just on hosting your own email list. Is the big Gary beast that they always wave at you, is deliverability, is getting your email delivered. And if you self host, how do you handle those deliverability issues? How do you Yeah, that's
Michael Kennedy 23:22
certainly a concern, right? So there I people should disentangle two things. There's running your own mail server, and then there is running the database that holds the list of people on your mailing list. So for example, one of the things that I chose, there's a ton of them you could choose, but I chose something called list monk, which is free, self hosted, Docker. First type of I don't even think this one has a hey, you can pay us money. List monk, dot app, yeah. So when you go there, it says, Well, how do you use the thing. It right there. It says, Okay, what you do is you literally use this, say, create a Docker compose YAML file that has a list, Monk, latest, and it also has, you set up Postgres. I think it's Postgres. There's basically two, two. Servers that you got to run there. Yeah, the uses Postgres thing and then just the web app, which I think that's written in Go. Yeah, I'm pretty sure it's written in Go. But it doesn't matter, because you don't have to think about it says here, here, use. This is how you plug it into your infrastructure, right as part of your system. So back to your question, Carlton, that plugs into what people are probably already using to deliver mail, so it has like a drop down for are you using Amazon? Ses, are using SendGrid. Are using Mail gun, which you've already got your. DKIM and PK, whatever stuff set up, and you because, because you're already sending a ton of mail out of that, right?
Will Vincent 25:05
Yeah, right. Okay, right. Because that's the thing, is that when you like running a platform as a service, you have your transactional emails already, you know, someone signs up, reset password, that kind of thing through, let's say, SendGrid, and then Sep, you can separately have your database that holds your newsletter, which you also send through the same what's, what is it? Ses? What's, what are they even called there? There's some acronym for
Carlton Gibson 25:28
SES is the Amazon one, right?
Michael Kennedy 25:30
Simple, yeah, simple email service. But I there's something, yeah, I'm sure there's an alternative, an opposite of transactional email service. But, yeah, yeah. So
Will Vincent 25:43
they send it this, they send the emails, yeah, the email
Michael Kennedy 25:46
sender, yeah. So Carlton, I'm not hosting my own mail thing that seems fraught, and also it's not, yeah, exactly, and my reputation is 100% at SendGrid, so long as I'm well behaved, maybe it's 99.9 or something now, but
Will Vincent 26:04
I gotta put on my old man hat, like 2010 when I was at a company called Quizlet, we had millions of users, and we had to do all the, I guess, primarily transactional stuff that we sent a couple, a couple marketing in quotes emails. And it was just, like, just stored in like, a PHP app, and, you know, it's basically impossible, then it's worse now, to just send a million emails, or 2 million emails, unless you have this built up reputation, right? Because Gmail, Yahoo, like, it's, it's undoable. And I think mail gun actually had just come out of Y Combinator at the time. But, like, yeah, so, like, you can send them, but they're no one's gonna get them because
Michael Kennedy 26:44
they're not going anywhere. No, they're not good. It's, it's
Will Vincent 26:48
a nice little gated garden there, but, but, yeah, that's for anyone who's thinking, I'll just self host at all. Like, it's not going to go through, because there are these guardians around reputation. And if you get a certain number of spam, it's actually, this is what I tell, like, my non technical friends. I'm like, if you someone's sending you email, just flag it as spam, and they will stop. Like they will hard stop, because unsubscribe is just kind of like a option. It's like, please unsubscribe me. There's no force there, but you hit spam their, you know, their credit score gets dinged essentially, and they will automatically pull it. Yeah, yeah.
Michael Kennedy 27:24
Actually, set up a thing that will listen. It'll set up a web hook that'll listen to unsubscribe events across different things. And so sorry. And Well, that's obvious, but spam requests and bounce requests and those types of things that are not typically we're sending back responses, and through SendGrid, they'll tell you, hey, somebody, somebody marks you a spam or something like that, at least on the major platforms, Google and Microsoft and stuff. And so you can then unsubscribe them, so you don't keep sending it and keep adding to your bad score. And so that's, right, that's some of the interesting things that you can do. It took a little bit of writing code to make it happen, but
Will Vincent 28:06
seems like that should be built into one of these services, built into
Michael Kennedy 28:10
this list monk thing, but it somehow doesn't come back from SendGrid correctly, I don't know. And I'm like, It's it matters enough that I'm gonna just take an hour write a little API that just listens to this email doesn't like you for some reason, unsubscribing now,
Will Vincent 28:25
and as this primarily, this is for, like, marketing emails, as opposed to transactional. Yeah, exactly, yeah. So the transactional you already had through, I forget which one you said that was all, yeah, saying, Great, this, right, right? Because the marketing ones, that's where MailChimp and Convert Kit and all them come in. And yeah, they were actually there's a Django hosted one button down, yeah, but
Michael Kennedy 28:47
to pay for it, but certainly down as well. Yeah, that looks cool. We're gonna
Will Vincent 28:52
have Justin Duke on a little bit later to talk about awesome. But yeah, that seems to be the progression, right? It's like, Hey, we're indie, and then all of a sudden, Intuit owns you. Yeah, exactly they. And it's like, free for the first couple of 1000. And then it's like, Wait, how much is my bill? Yeah,
Michael Kennedy 29:07
yeah.
Carlton Gibson 29:11
It's always struck me that email providers are quite expensive. You know, you look at them, and it's always like, okay, it's only 30 bucks, but it scales up. And it's like, Oh, wow. Actually, it gets it gets expensive. The transactional
Will Vincent 29:22
ones aren't that? No, no, no, the pen, the marketing ones, yeah, the marketing ones, like, marketing ones. It's like 100x Yeah, like, markup. And it's like, you're still just sending an email, like, and my issue is like, I want, like, a base version, like, I don't want to do a drip thing. Like, I just want to send email, and I'll have, like, personally, I'll happily pay someone not to just do that, and, you know, so we're going to talk to Justin from button down these other solutions, you know, because I'm not at a scale, but there seems to be no in between, you know, broadly speaking, right? It's really, I guess, designed for larger companies and the marketing departments, once they get hooked in, like, they're not. An API endpoint. They're not looking at this. They're just like, just throw money at it and move on.
Michael Kennedy 30:06
Yeah, absolutely. And there's, uh, I don't know, I guess, I guess that's just the space, but there's that whole area is just, I kind of feel like they're just overcharging. Now, you look at places like HubSpot and MailChimp, and they're like, You know what, it seems totally reasonable that it's $1,000 a month to have a few landing pages. You know what? I'm not really sure that that's true. Oh, by the way, you can pay that in 12 months, you know, a yearly chunks as well. So it's like, wow, okay, right? Well, there's no loan. I need a loan to get, like, marketing software, right?
Will Vincent 30:39
I mean, there's a company here in Boston that just IPO, that's Django based, or was originally Klaviyo, with a K that I believe also does, like, like, if you look at a lot of like, e commerce companies, they handle, you know, marketing emails, essentially. And I know there's, is it still around? There's a company called drip that was doing, oh, yeah, drip, is that, that kind of thing, yeah. But like, clavio is like that, but like, ten billion or whatever it is, so like, huge companies, but, yeah, I guess there just isn't a market for, like, solo or smaller places, right? They don't really care, right? And it's like, well, you can go open source if you can. Otherwise, we'll just, we'll take your mind. Yeah. Anyways, button down. Shout out. We'll talk to Justin about it. I think he's trying to tackle that challenge, but the space in general doesn't, doesn't focus on that. This episode is sponsored by mail trap, an email delivery platform. The developers love, an email sending solution with industry best analytics, SMTP and email API, STKs for major programming languages and 24/7 human support try for free at mail trap.io.
Carlton Gibson 31:46
So Michael, you decided you were going to stick everything all on one but one big box. What kind of box did you get? And where did you get it? And is it a physical thing you have to drive to the CoLo and change the hard disk at two o'clock in the morning? Or is it, I
Michael Kennedy 31:57
think, you know, I mentioned the DHS for leaving the cloud, and those guys went and spent half million dollars on servers. Like, that's not, you're also not. You're also not. 37 signals, probably for the general people listening, who would consider but it's totally reasonable to say I'm going to go to somewhere like DigitalOcean or Linode or somewhere. One of the things that has struck me as like interesting, but you often get sort of too deep into the machine, and then you're stuck is just go to AWS and look at the console of services that are there. You know? It's like, you gotta zoom in just to see what they are anymore. There's so many sort of thing, right? What is going on here? And, you know, it's, it's totally reasonable that all right, this, this database thing is, like, starts at $20 a month, okay, that's fine. And we got this function list, and then all we're gonna need caching for this thing, then over there, and then all of a sudden, your code doesn't run. In general places. It only runs in this one particular cloud, and the aggregate of this because you now started doing too many, SES, sorry, s3, transactions, reads and writes, so now that's expensive, but you got nowhere to go or whatever, right. Bandwidth is high then, then you're kind of stuck. And it's, I think it can be frustrating, and also can kind of take you away from just the simple Python solutions. Like, then, how do you even test that kind of stuff locally, end to end? You know, it's like, well, I don't have all these. I know there's containers you can set up, but like, at that point, you're running a different set of complicated infrastructure just to write code. So answer your question. Carl, then just I got like, a $60 server over a DigitalOcean. Just put everything on that, right? Just really interesting. But you
Carlton Gibson 33:48
get a lot of, you get a lot of stuff for a $60 server, right?
Michael Kennedy 33:51
You do, yeah, it's eight gigs, four CPUs. What's really interesting is the bandwidth. So if you did 10 let me just do the quick math. If you did 10 terabytes of traffic, the CPU, the EC two, is actually not as expensive as it used to be, but if you did 10 terabytes of traffic on Amazon, it's $921 a month, right? It's a nine cents a gigabyte, right? Which doesn't sound that bad, but then you multiply it by themselves, and it goes up Digital Ocean. You get most of that for free. It's like, bandwidth is like 50 bucks a month. I was like, Oh, that's interesting. You can that's quite a big difference, though. That's it's not a non trivial difference. And you know, it scales linearly with how much traffic. So as you get bigger, it's just it's it's different. Why is?
Will Vincent 34:49
Why is that? Is that Digital Ocean trying to compete with AWS in one of the few ways they can? Or what?
Michael Kennedy 34:56
Every machine that you get, every basic. Here of a machine that you get at DigitalOcean. I think Lenore is this way too. But since the my acquisition, I don't really know anymore, you get one terabyte of included bandwidth with each little machine you get. And if you get big, one big machine, I think I get six or eight terabytes of credit for outbound traffic, whereas on Amazon, it's like, well, it's just nine cents a terabyte or a gigabyte, it's not so bad, right? But then it adds up.
Carlton Gibson 35:29
So I think there is a quite generous starting thing for I mean, most, most small sites wouldn't pay anything for bandwidth, yeah. And
Michael Kennedy 35:35
even if you were a small site like it also it just doesn't matter, right? But if you start shipping bigger things like MP threes and videos, yeah, videos, video,
Carlton Gibson 35:45
as soon as you're posting videos, you're like, Ow, yeah,
Michael Kennedy 35:49
yeah. Well,
Will Vincent 35:51
and you, yeah, you so you don't use, like, I don't know if it's still the case. For a long time, Vimeo had like, a unlimited for like, $99 a month, or something. Is that? Yes, a sigh there. Talk to me about hosting videos.
Michael Kennedy 36:06
See videos. So yeah, I had some stuff going on with Vimeo for a while, paid for their business tier where you could white label, rehost it. And it was, it was actually a really good deal for a while, and it solved a problem, because video delivery, CDNs, all that stuff is super, super hassle, right? You know, re encoding stuff so it streams the right? You know, on Android, it wants a different size and format than it does on iOS than it does on Yeah, it's a nightmare. So they handled it. I was happy to pay them, and then they changed their policy like, Oh, if you have too much bandwidth, then you're going to start paying there was a famous example. I can't remember the exact details because it's been so long, but there was, this is somebody who was not going through the business plan. They just put, like, as if you were on YouTube, they put up a video and it went viral as a creator type, not somebody trying to create a platform or anything, and they got a letter from Vimeo saying you're gonna need to pay us, like, $1,000 a month for the bandwidth that you're viral video is costing us. And I what I just put like, you're a video. We posted a popular video. It wasn't popular at first. It became popular. And yeah, they, they, they had some changes in policy that just really kind of put the quash on that. I don't remember the exact details, but around that time. It feels like it was five or six years ago that it was, it just things changed there, and it didn't make any sense, you know, kind of like MailChimp, like it was cool for a while that you could do stuff there, but it things change. Well,
Will Vincent 37:54
what is, I think it's, this is also Boston based. There was another, like, more expensive, better Vimeo. What is that one called? There was, there's like, one other one, right? Something like that. Yes, the Wix is the building. It was like, something, yeah, something like that. I can't remember, but his pricing was even from the start, was like, kind of outrageous, I think, yeah, it's probably back five, six years ago, I was looking at this. It was still, it was like, I was like, who can afford this? You know it's like big companies, I guess big companies with marketing departments that just want to pay money and have it solved. But yeah, I guess it had all this analytics and everything. And, yeah, yeah. Anyways,
Michael Kennedy 38:31
awesome for the right group, but not for the wrong group, I guess.
Will Vincent 38:36
So, what are you doing today? Yeah, you're no, you're right. It is Wix. You were correct, yeah, okay, they're still around. I guess I was thinking about Weebly, all these weird something
Michael Kennedy 38:46
that's like Wix, though, for building, building web apps. It's like a Squarespace competitor. I don't, yeah, yeah, remember exactly what that is. So basically, I real. I through this whole process, I realized, oh my gosh. There are an amazing set of open source projects that sometimes you can pay small amounts of money to, or, if it makes sense, you could also self host it. Right? This list, mug example I gave you, you have to self host it. But another one that's really, really interesting is the web analytics, you know, Google Analytics alternatives sort of thing. Probably the best known one would be plausible@plausible.io you guys know that one? Yeah, yeah. So those, those folks are based, I think, in the EU, and I'm much more concerned about GDPR. And I think what you guys know Carlton, maybe you know you live there in your neighborhood, there was some ruling, either France or Germany that basically said, Google Analytics violates the GDPR to some effects. And then another one, which of those pairs that said Google. Fonts violates the GDPR, right, which is even a little more subtle,
Carlton Gibson 40:07
but you guys, so the trick, I mean, I know more about the Google Fonts once it's something I've looked into and considered, and I always, if I want to host a web font, I will always download the web font and host it locally, because, actually, they
Will Vincent 40:18
don't let you do that anymore. Carlton, yeah, they do. I
Michael Kennedy 40:20
went, No, no, no. I
Carlton Gibson 40:21
was there yesterday downloading Google. Google
Will Vincent 40:24
does, yeah, but, but if you like, because I went through this with the design process, even all these fonts, I was like, we're talking to designers. I was like, I'll pay a couple $100 to download a font, and it's like, Nope, you can't anything that's not Google Fonts. Basically there. It's a SaaS as a service. It's yeah, and it's so expensive, right? And I'm like, you know, like, there's, I forget the font that, you know, open AI's font that they're using is, you know, I calculate the bill based on, like, obviously they're getting a discount, but it would be like, 10s and 10s and 10s of millions of dollars a month just for the font. And, yeah, anyway, so just side rant, like, yeah, just, let me just download the thing. And, yeah, I guess Google lets you download them. But all these other fonts, I was like, oh, I want a Bougie font. I love fonts. I was like, Nope, that's like, basically not possible for for the majority of Yeah,
Carlton Gibson 41:13
okay, anyway, so I deal with them because I don't want to for my you know, even forget the EU regulation. I don't want to put my use people come to my website in good faith. I don't want to then have tracking stuff, embeddings in there. I'm 100% you know, big, big, mega core can form their shadow profile. I'm just like, No, I'll download it and self host it, and that's fine. So the thing with the analytics is like, which are the which are the ethical ones, which are the ones which don't do exactly that?
Michael Kennedy 41:47
I'm gonna leave some out for sure, but so plausible.io. Is really interesting. They've got a just analytics as a service, which I think probably makes sense for most people, right? But it's buried deep down in there. I mean, it's not buried, but it's not front and center. Click here to self host it, but you can self host plausible as well. The one that I'm using is something called umami, which is more, yeah, I've heard of that umami.is it has a pricing version where you can just go get it your own for $9 a month, or you just go, it's got a GitHub 20,000 stars, and fired up your your Docker compose file and and off it goes. And what's really especially interesting about analytics is the so I actually took Google Analytics off, like, four or five to four years ago, and just turned it off because I'm like, it is not just a privacy violation to basically have it, but it is also kind of useless, because so many apps, web browsers, pie holes, everything, just block Google Analytics outright, and so you're getting a really partial view of what your web traffic is doing. But with anything self hosted, you can put it on your domain and it just looks like part of your web app. So for example, for umami, I self host it in a container on the same big server. With all the other things, I've got like 15 things running over there, in there, it's on the same domain, the same IP address. So when browser look at it, go, Yeah, that's fine. That belongs to part of this web app, right? Yeah, it's just a and you get near 100% unless you turn off JavaScript, you're getting almost 100% accurate view into what's happening in your world. That's yeah, which is obviously most pretty that's a pretty unique thing. Most things are not like that, but web analytics are so subject to this. Privacy blockers, Tracker blockers, ad blockers, which I not ranting, I guess at all, I think that they're serving a very important role. But, you know, as a web site builder, you end up on the receiving end of, you know, other people misbehaving badly, so you're untrusted as well, kind of like the same email story, right? You know, yeah, people have abused the system. We don't trust anyone anymore, unless you're Gmail.
Carlton Gibson 44:14
Back in the back in the day, you didn't have a web blocker, but then you had pop up. So we have to have a pop up blocker. And, I mean, it's, it's just warfare. I'm not turning it off.
Will Vincent 44:24
I'm surprised there isn't like, a documentary on like, you know, the dominant web blockers, right? Because the ownership structure is deliberately unclear. And it seems, seems like they're owned by, like, Google and Apple, like, but there just isn't like, for real. Yeah, you know, not to get to Joe Rogan on this, but, yeah, there, that's right, yeah, deep
Michael Kennedy 44:47
analytics Don't, don't
Will Vincent 44:49
let I think it's, I'm pretty sure that I remember taking like, 10 minutes once to look at it, and it's like, well, we don't know who owns it, but there's all these carve outs for these companies. And, yeah, yeah. Like, who would want to. Who would want to own, who would want to own it, and has a lot of money, like, right?
Michael Kennedy 45:03
Like, we're gonna block everything except for our ads, you know, something like that, right? That's right, yep, yeah. So I actually run next DNS for my DNS, and that that stops everything that's beautiful, even my television has ad blocking built in. My all your iPhone apps or Android apps built in ad blocking because that network endpoint is unreachable.
Will Vincent 45:29
So if you so, like Amazon Prime Now for their videos that you just get a blank thing when it tries to put ads on there.
Michael Kennedy 45:36
I, you know, I haven't watched in Amazon videos since they put ads on there. So, yeah, I think generally, the stuff that's in the stream is kind of coming through the same source, but most of it, you know, even if you go to, if you go to some mobile app and you hit a thing, it opens up an embedded browser, like, the ads are gone. Every now and then, my daughter asked, like, Dad, could you put the ads back on on the network for a little bit? I'm like, Why do you want that? She's like, well, if I watch this ad, then I'll get a credit in my game, and I can't watch that. We're winning.
Will Vincent 46:10
But yeah, no, I was thinking this because my kids like for the longest time, you know, they've never seen ads on a screen, and now one of these things are something they want to watch. And they figured out that my kids like Legos, and so now it's like, hey, I want this Lego Okay, what does it cost? What? Oh, my God, you know, like, so it's, but it was, you know, in a way, it was this was, is this golden time of kids not seeing ads like, you know, because growing up like the radio that, like you just bombarded with it, right? Whereas most kids, you know, well, I guess you know, if you pay for YouTube premium or get a family plan, but anyways, yeah, so when someone looked at that and was like, you would pay more to not see those ads
Michael Kennedy 46:54
exactly, probably exactly. So, yeah, I think anyway. So back to the analytics. I think you know, there's all these different ways in which that that information either is somewhat ill gotten by sending it through a big tech or just just corrupted because it's so partial from, you know, the different levels of Ad blocking. So I think that that kind of self hosting stuff is there. I want to give, I'll give you guys a whole long, interesting list of things here in a second, but I want to kind of just step back for a second and talk about, like this eight little servers versus one big server. And if you think about you could say, All right, well, the big server is roughly every app has about the same RAM as it had before, right in a at least in a sense, right, if they're all still using a decent amount, but if you have a bunch of small servers, and there's some spike in, say, one of your APIs, or there's a traffic spike in one, like talk, Python has, like, a big spike. For some reason, Python bites had one because some credit was trying to DDoS it the other day. But it was fine. I didn't mind anyway, if for whatever reason, good or bad, it has a spike and it's on this sort of commodity, bunch of small pieces world, it quickly runs into the limit, like, well, that server has one CPU, or it has two CPUs, or whatever it gets when you chose it, and that's it. But if you put them all together in some kind of like Docker cluster type of thing, those spikes almost never overlap, right? The reason that the courses would spike the first is the reason that talk Python versus Python bias versus the APIs that run the mobile apps, whatever, there's no, almost no, correlation between the spikes of one and the other and so it can. Each one of those has almost all the compute available to scale out and do its thing. So it's really interesting to think, you know, like, I've got this little umami Docker thing running and this list monk thing running, and all the other apps and web apps running, and they might have these different spikes, but they all have, like, a higher headroom to do whatever they need to do for less, like I'm paying 40% of what I was for the eight servers, the little baby eight servers to get one bigger one, and you get more compute out of it's really interesting.
Carlton Gibson 49:18
Yeah, no, I like that. I like that. It's
Michael Kennedy 49:22
lovely. Yeah, it's fun to think about. But so, all right, so I mentioned a couple of these things, and neither of them just just before you got
Carlton Gibson 49:30
gone, but just before you do that, let me just jump in. So you're running. So you said 15 services on this on this box. So I presume that isn't just some sort of long running screen session with, you know, shell, it's 15 tabs open and
Michael Kennedy 49:45
Docker containers, I believe maybe 17, with the different databases stuff. But, yeah,
Carlton Gibson 49:50
if you got some nice UI that, you know, helps you manage those, or, Oh,
Michael Kennedy 49:55
not really, but I, I it didn't come with one, because you just got Docker. And then compose and different things. But I used rich, you know, the Python library again, and textual, and did basically an in memory join on the report that a bunch of Docker and Linux things return, and it'll create a table. And then with rich, something that's really awesome is one you get Keller, as I'm sure you know, but what people maybe don't know is Rich has a context manager for live redrawing of content. So you say, with live, here's the table, and then you just start changing the table, and it'll change the UI without even, without even flickering or anything, just over SSH. So that's basically that. And Are you all familiar with BTOP? No, oh, B top magic, so BTOP, it's github.com/aristocrat, dot com, slash, aristocratos. Slash, beat off. Yeah.
Will Vincent 51:04
A monitor of resources, yeah. But
Michael Kennedy 51:07
what's amazing, they don't have, if you scroll down to the graphics section, like, maybe half the way down that page there, it's, you know, this thing I described with rich. This is not based on rich, but it's like kind of that to the logical extreme. It looks like a Atari or Nintendo 64 game, almost, and just live updating of all these different panels. Here's all your CPU across individual and total aggregate CPU. Here's your network traffic, your processes. This is what your disk and memory usage looks like, all updating live with graphs and stuff. So like, that's kind of the thing I've been using. Just like, look at the server. What's interesting is, when you're running Docker containers, all the processes in Docker all show up as separate processes in in the process monitoring tools, right? So you can actually see this part of my Docker container, like this worker process for this service is using this much RAM and this much CPU, and here's the graph over time and stuff like that.
Will Vincent 52:13
It would be super cool if just thinking about like your custom stack here, it's almost like if some naive person came in and you just had these drop downs like, Oh, I'm using MailChimp. I'm using these different things. And then it was like, here's, here's a recommended other, you know, I don't know, you wouldn't monetize it in any way. So what's the point? But yeah, because it, as much as things are custom, there is some degree of generality around all this, which, I mean, you know, Carlton, I almost hesitate to ask, you know, you have spent quite a bit of time on a Django specific thing button that was going to do some of this for Django. Yeah, I guess it's just hard to draw that line and get enough people in where it's like, yeah, I'll pay. You know, it's not as much as all these services, but it's enough to someone put a wrap around top of it? Yeah, no,
Carlton Gibson 53:02
I think there's a market there.
Michael Kennedy 53:05
You know? Yeah, no, it's
Carlton Gibson 53:07
going really well. So I'm so I stopped following and I mean, immediately, in order to pack the bill, I pay the bills, had to pick up a new thing. So I've been working on a project called paths, which is an app in the environmental and social governance space, but I'm deploying that with button on exactly the tooling that I want to use, and it's all going great. And it's, it's super and I think there is a market there. It's just a question of productizing. So you build an app, and then you you want to productize it. And it's, it's sort of three times the three times the time to productize something as it is, to just build the app that works for you. And it's that bit that is tricky, because I'm already building one application for as a full time job. How am I going to blend build another? Well, I slowly tick away at it and I'll roll it out over time now. But for me, it works. It works well, it's lovely. I think there's a market there, but it needs somebody to sit down and, you know, have the time, and where do you get that time, without the funding to back it and just raise money? No, but as soon as you raise money, then there isn't a market there anymore. That's, that's the rub. That's so what have we talked about on this podcast? We've talked about mail. Chump used to be cool, and then they sold out, and they then they became really expensive. We talked about, I don't know, a Vimeo that did used to be cool. It was the best offer for hosting videos on the internet. And then obviously they need to make more money to satisfy their investors. And all of a sudden it's, like, priced out, and it's corporate only, and it's the same in this deployment world. Is like, Heroku. What happened to her Roka? Well, it got bought out by, you know, the big company there, and then they, like, we can't do this free plan, because it just doesn't. And that sounds right,
Michael Kennedy 54:45
yeah, yeah. I don't know how much their policy changed, but like, they they kind of got down the same path. Yeah. You can't
Carlton Gibson 54:52
have it both ways. Either it's a small bootstrap thing, which you need to kind of grow organically, or you take outside money and then. You can't, you just can't serve the hobbyist developer or the small, independent developer you're looking at corporate. And there's no way around that. That'll
Will Vincent 55:10
be our first question to Justin.
Carlton Gibson 55:15
He's doing it that way, right? He's a small thing, but it takes
Will Vincent 55:18
a lot of money, a lot of intent, and a lot of and at some point, right? Like, maybe you don't want to work on the same thing for forever, right? And so, I guess, you know, and tech is so weird because it's infinitely scalable, whereas, like, a more real world example, like, Was it like the, like, oatmeal, like to get from this, like, Bob's something or other. It's this old guy on it and, like, Red Mill. Just, yeah, Red Mill, right? Like, you know, he could have sold to mega Corp. He sold it to the employees. So now it's employee owned. Like, where I'm from, in Vermont, there's a number of companies that have done that, but you know, that means that Bob, or whomever, he could have made a lot more money, right? But, you know, yeah, and I guess it'd be nice if there was some tech equivalent. I guess maybe there are some a little bit, but more around, like agencies, I know, like torch boxes, employee owned. I guess it's really hard around products to do that in the tech space. Yeah,
Michael Kennedy 56:20
and the stuff that has running costs, right, infrastructure, bandwidth, right, compute. If
Will Vincent 56:27
you raise any outside money, then it's gonna
Carlton Gibson 56:32
Yeah, no, but yeah. Thing about outside as soon as you take the outside money, you need the bigger revenues, and you
Michael Kennedy 56:38
need sticking on your 10x and you better find a way to get it. But, yeah, I mean, not
Will Vincent 56:43
to make this all about 37 signals, but they famously took money from Jeff Bezos, like, way back in the day, because I think they were facing some of these pressures, and it allowed them to, you know, stick it out right? Like, maybe there's, you find a benevolent billionaire Who'll buy 10% for some amount of money
Michael Kennedy 57:02
camp app I want to do exist. I'll just make sure it doesn't go away. Yeah, yeah. Their whole leaving the cloud series that David Hyneman and Henson wrote is super even if you disagree with them, it's super worth your time of looking through it. It's quite interesting. So you guys are, I'm sure, familiar with awesome list, right? So one of the things I want to just give a shout out to is, kind of, like this whole thing we've been discussing kind of that. So if people check out products.containerized.com, that is, like an awesome list for, you know, pick your self hosted thing. So you want to do blogging. Here's, here's the choices you want to forum, here's, these are all open source, self hostable, right? Discourse, or, do you want vanilla? You want, you know, node BB, or there's a marketplace. They're all alternatives to, you know, Shopify, I guess, would be the marketplace equivalent, right? And they've like, this is the one that's voted most by by the users. You want chat like Slack, well, rocket dot. Chat is like the winning one, apparently here, but there's, you know, is that 12 other options, transactional email? Is
Will Vincent 58:14
it win? Winning by GitHub, stars, or they have some other what it's like?
Michael Kennedy 58:20
Reckon it gives this little like, thumbs up, and I think it's whoever runs this is, like, this is the one we've sort of given our stamp of approval to more out of them. But, yeah, that, yeah, no, it's very cool. That's interesting, right? But then you end up like, well, if I want to use any of these, like, well, what infrastructure do I run this on? You know what? I mean, you've got invoices back? Yeah, you're back to that, right? But I don't know.
Carlton Gibson 58:46
I mean, every one of these things, there's a massive learning curve to picking them up and taking them on, right? It's not, it's it's open source, hostable, but it's not free.
Michael Kennedy 58:53
No, it's not totally free. And I totally agree with that. But what I would tell you is, you will find a pretty common theme of here's a Docker compose file. Do you have a place that can run Docker Compose? If you can do that, then you can probably self host this pretty well, right? You got to have some kind of backup, but almost always says Postgres. So you just run a Postgres backup on the server you set up and figure out how to restore that later, whatever that means for them. You know, it's, it's not as dawn. Once you start down this path, you're like, Oh, I've already got 10 of these things running here. What's What's two more,
Carlton Gibson 59:29
what's 11? Yeah,
Michael Kennedy 59:32
because it's Docker, like, you know, you don't have to trust it as much. You still got to trust it with your data. But it's not right in there with all of your other stuff on the inside of the firewall and those kinds of things. And so it's, it's not as risky, but it's pretty interesting to go down this. You know one, one thing, I guess, also two things here. One is, have you heard of alestio? E, L, E. S, T, i, o, elestio. So these, this is a, it's not that cheap, but it's, it's kind of empowering. It's just this company that you say, what is your provider? It says, hosted on Digital Ocean, AWS, filter, hertzner, Linode, scaleway, etc, etc, even on prem. And then you say, Well, which one, which open source project. Do I want? So do you want clickhouse? You want OpenSearch? You know, whatever Redis and they'll just set they'll manage the backups, keep it running and keep it patched for you, almost like as a service on your own infrastructure, for like, a bit of a cloud premium. But you just go in there and you type in, you know, one of these crazy things. There's a really good chance that they have it right there. I think it's Yeah. Well, pretty fascinating.
Carlton Gibson 1:00:50
Love a link, yeah. And it's yeah, all of these things that cloud premium, it's like, well, what are you buying? What you're trading off? Because in my day, there was when I was a youngster, it was literally a sort of old guy who, like, had a beard, no, but it was, it was like, yeah. Like, it was a proper, big beard and recumbent
Will Vincent 1:01:10
bike, you know, he, he had a
Carlton Gibson 1:01:13
full time job, and he was like, you know, whatever time of day, at a page, like, it was never, like, seem, seemingly, never sleeps. But didn't do much. Didn't speak much, but literally, huge job was to keep the the actual physical server going. And so when the cow came along and they were like, well, you could get rid of that guy, and that guy didn't get got rid of because, like, yeah, you know he was, he was just part of the furniture. You
Michael Kennedy 1:01:35
wouldn't get rid of him. But that cost just disappeared.
Carlton Gibson 1:01:39
And so, okay, the cloud server was more expensive than running it your own, but it was a lot cheaper than hiring system.
Michael Kennedy 1:01:45
Absolutely, absolutely. And
Carlton Gibson 1:01:48
so you think, oh, you know this service here that's going to be an extra few pence per hour. But hang on, what are you trading off? Are you trading off? It took you two days to work out how to deploy this versus clicking.
Michael Kennedy 1:01:57
Or do you need to hire somebody to manage all these things? Or can you just, like, pay an extra $100 a month? Yeah, it's, it's really interesting. One Django, one that's really worth maybe you give already had these, these folks on but have you looked at post hog,
Carlton Gibson 1:02:13
post Hall? No, I've
Will Vincent 1:02:15
heard, I've heard that. No, we haven't. We haven't talked about,
Michael Kennedy 1:02:20
I think I'm pretty sure it's self hosting. Let me just double check
Will Vincent 1:02:24
single platform to analyze test observed, yeah, itself,
Michael Kennedy 1:02:27
self host. Yeah. Post hog, what is the domain ending? Post hoc.com. Provides open source product analytics, session recording, feature flagging, AB testing, all of that as a service or self hostable, and it's in Django,
Carlton Gibson 1:02:44
right? Well, we're getting them on there, right? I had, now, you bring up. I had seen it before. I didn't know I was built in Django. So that's, yeah.
Will Vincent 1:02:53
I mean, isn't that kind of cool? Like, even, even the two, even the three of us, right? Like, who do podcasts on this? And it's like, oh, there's this thing with, like, 17,000 stars that we didn't really know about, that lot of other people knew about, somehow. And how
Michael Kennedy 1:03:08
do we not Yes, almost 17,000 GitHub stars and written. And it's a really cool looking app, single platform to analyze, test, observe and deploy new features for your web apps and stuff. So yeah, that looks, that looks quite slick. Maybe. Let me give you all one more category of interesting, yeah, before we close it out, or we can keep going as much as you want, but before we miss it, and I know will that you're a fan of the Heroku style of things, right?
Will Vincent 1:03:41
Yes,
Michael Kennedy 1:03:42
I am. And then you end up, go ahead,
Will Vincent 1:03:46
yeah, I just say, I put a preface with at my scale, it's I'm more than fine. And,
Michael Kennedy 1:03:52
you know, I know there was a lot of drama around Heroku. I don't want to, I don't really want to give any hate. Like, they shouldn't be required to run the Internet for free. They're just run by one person.
Will Vincent 1:04:04
No one else does either, yeah, exactly, no one else really does either, exactly, you
Michael Kennedy 1:04:09
know, like Digital Ocean doesn't have free they have a $4 server, which I don't know to me, feels like free, but no, it's not free, right? It still covers it for, like, the super, super indie little sites and just little small things and so on. But as part of this whole deal, I came across Doku, D, O, K, K, U, are you familiar Doku? Doku? Yeah. I
Will Vincent 1:04:32
mean, this was
Carlton Gibson 1:04:33
one familiar with it ran something like, If you used some kind of UI, a bit like this, but you don't do it with the terminal. But, yeah,
Michael Kennedy 1:04:41
so dock is pretty interesting. It's kind of, it's got this, you know, create your own. It's, it builds itself as open source pass alternative to Heroku. And you just, kind of, somewhere, you create an app, and then you create a proc file, and you get pushed to some branch, and that makes it go. I talked about that on Python by. Some people are, oh, that's interesting. And somebody wrote in, I can't remember the name of apologize. Said, That's cool, but have you seen coolify? C, O, O, L, I, F, A, y. This is like one of the same things, but little bit little bit more full featured and a little bit better UI and a little bit more supported self hosting, alternative or Heroku, Netlify, vercel and servers. A
Will Vincent 1:05:26
lot of features there. Yeah, free,
Michael Kennedy 1:05:29
free, SSL, automatic database backups, web hooks, automation, notifications, monitoring, all the all of that in just a Docker compose up,
Will Vincent 1:05:41
basically, let me ask, let me ask you as as an educator, like, what do you tell people who just want to, like, learn web development, right? I mean, because, yeah, like, this is kind of the issue is, like, at the same time that most software engineering jobs are around infrastructure, actually, probably, you know, if around the web, but getting started, like, I don't know, what do you say? Someone comes to you is like, I don't know what Docker is. I just have this. I'm so excited. I just learned Django, and I want to host something. And it could be, yeah,
Michael Kennedy 1:06:13
let's put the data science folks off to one side, because that's a whole different level. Okay, yeah, right, because they actually have a web app, but they need to somehow get it on the web right, turn this
Will Vincent 1:06:23
spreadsheet into a web app with crud and everything
Michael Kennedy 1:06:26
exactly which is more of a but. But for the folks like I have a web app, it does just what I want on my machine. How do I make it on everyone's machines? Right? So I think there's two paths. As an educator, you send people down. One is just don't think about it. This is what a connection string looks like to a managed database server. You get pushed to Heroku or something, some Platform as a Service equivalent. Job done. That's kind of the I want to make it happen, and I don't really want to think about, worry about, or even hardly know about, necessarily, like, what do I need? I just need it to just go. And if I can get it here, these people will take care of it, right? That's the minimum path, I think, and that's probably the right path for a decent number of people, just like I wanted to get it to go the other side. This isn't what I've ended up doing more, and I don't know if people are like, Michael, why are you doing this? To me, is show them how to set it up on Linux, if you because, if you know how to set it up on Linux, you can run it on VMs. You can run it on what do you got to do? To do a Docker container? You put run or copy in front of all the Linux commands. You know what I mean? Instead of saying CP, the Docker file, thing is copy. Instead of, you know, apt install, you say, Run, apt install, right? Capital r, u n is the front of that, that thing to make it a Docker command. And so if you understand Linux, you understand Docker. And that opens up all the containers as a service. It opens up Kubernetes, if somebody else is running the cluster like it opens up that whole world, I think. But there's a higher bar to say, like, Okay, well, I'm a Windows person, and I generally don't use the terminal, and you're making me work in Linux with no UI. What? What has become of me? Right? There's that challenge. So I think there's this bimodal you want to send somebody, people to one end or the other. I
Carlton Gibson 1:08:27
think there's a trade off as well with the passes, that it's kind of like a dead end, and that you I've described it in the past, the cul de sac, you go in, and it's great to begin with, and it's lovely and pretty, and there's flowers around, and I suppose the end, but you get to the end and you're like, oh, actually, I want to do something further, and you have to kind of go to kind of go back all the way out, out where you were to start again to get onto a more complex whereas if you go straight to a VM, it's slightly harder to get there to begin with, but you can grow an awful long way before you need to learn anything else at all. And I 100% agree with that, yeah. The the sort of example that comes to mind is with channels, which is the, you know, async add on a WebSockets ad on for Django, we the number one problem we have is people having difficulty connecting to Redis because they've got a pass and they've they've got a Redis that they've spun up in the cloud somewhere, and it's not simple to Connect, because you're doing it over the network, and it's got to be HTTPS, and it's got to be this or the other. Whereas nobody in the history of redis had a problem connecting to Redis on local host, right? It just never happened,
Michael Kennedy 1:09:32
yeah? Like, if
Carlton Gibson 1:09:33
you've got a VM, you can spin up Redis sitting next to it, and you're just, you're off, and you start. You never hit that set of problems. And until you know much bigger scale, and you're like, oh, I need a much bigger Redis. Well then, okay, you can put Redis on a separate box, but by then you've, you've kind of got the chops, whereas, like, people really struggling early on because they've gone the pass route, and then they needed something which wasn't delivered by the pass in its sort of cul de sac of first offer. Yeah.
Michael Kennedy 1:10:00
Yeah, yeah, yeah.
Will Vincent 1:10:01
I think part of it too is it's enough to create a website and market it and do all those things like, in of itself, and then to add in all this other bit. It's kind of like it's rare to have the success and the knowledge and the time to do what you've done. Michael, yeah, whereas most people, it's like, on your own, put up a portfolio stuff, and then go join a team, and then, you know, have the time to focus on area. It's pretty it's very rare to do all the things you're doing at the scale that you're doing. And I know you have some help, but like, still, it's usually most people are overwhelmed with No,
Michael Kennedy 1:10:39
I totally agree with you. And look, when I started my stuff, I ran it on Python anywhere, and I'm said, Here's my code. Run it. I don't want to think about it
Will Vincent 1:10:49
also bought by someone else, yes, Anaconda, yeah.
Michael Kennedy 1:10:55
Then I slowly, slowly over time, kind of thinking like Carlton said, like, Ah, it's just, I need a little bit something different, right? You know, I think it was something like, we're gonna have our maintenance period. Will it be down for four hours? I'm, like, four hours. I'm not really psyched on that. Maybe I should learn a little bit enough to control my, my, my maintenance window to be shorter than, you know, four hours. That's, that's a long time, and stuff like that, right?
Will Vincent 1:11:22
And I know we're coming up on time, but how, how did you dip your toe in, right? Because it does feel a little bit like a boom, like, like, were you able to do that gradually, or was it No, I need to,
Michael Kennedy 1:11:33
no, super gradually. It was. How do I get, like,
Will Vincent 1:11:36
moving services, yeah. And then
Michael Kennedy 1:11:40
just, well, at the beginning, I just used SQLite, because, how much, how, how, how, read heavy is a podcast, right? It's not that read heavy. I mean, sorry, it's not a write heavy. It's read heavy, right? So that's actually pretty much fine for something that you updated a couple times a week,
Carlton Gibson 1:12:02
you know, it's only one person updating it, right? It's one person. So
Michael Kennedy 1:12:06
you doing that? Yeah, exactly. And so I started out with that. And, like, I, this is easy. I can, I can productionize this because I just let the file be there, basically, you know, run it on on Python, anywhere. And, like, all right? Well, maybe we need to move a drone server, okay? Well, maybe we need a better database for analytics, like recording stuff that's happening and stuff like, you know, and it it grows, but it grows, you know, it's, you look, you look back, and I go, look at all this stuff that's like, it's crazy. How, why did you choose this? Well, it started like little, little baby steps. And, you know, after you work with a while, it's like, well, that's not, that's not for me. That's not that complicated, but I know that it is for others, right?
Carlton Gibson 1:12:49
No, the best advice I ever had was from, you know, senior, when I was a junior, was like, Don't only add the bits that you need as you need them. So, you know, don't worry about varnish caching and stuff, you'll get to that if you need it, like,
Michael Kennedy 1:13:06
care about that, yeah,
Carlton Gibson 1:13:07
then yeah, then you can dedicate the time, yeah, you should be
Michael Kennedy 1:13:11
so lucky as to have a problem where you need that honestly, yeah, exactly,
exactly, right?
Will Vincent 1:13:16
Or, as I sometimes I like, tell people, like, when it's a problem, it will be overwhelmingly apparent. Like, you don't need to, you know,
Michael Kennedy 1:13:24
how will I know when a mind database needs to scale? Like, well, yeah, when it becomes a premature optimization, like,
Will Vincent 1:13:31
there's a quote on that, like, yeah,
Michael Kennedy 1:13:32
and that's true in code, and that's true. And like, how you try to run your code?
Will Vincent 1:13:36
Well, we could keep going on and on, but we're a little bit over time, we'll have links to all these things. I should we'll put a link to you. You have you mentioned HTML. You have a relatively new Django HTML, folks should check out that seems. I hope that's doing well for you. Yeah,
Michael Kennedy 1:13:54
yeah. It's doing well. Worked with Christopher Trudeau and that, and it's going well,
Will Vincent 1:13:58
yes, he's kind of, he's like, your Django person, right? He does Yeah, a lot of stuff, yeah, not to limit him to that, but I, when I see talk Python stuff that does Django, I'm like, oh, yeah, exactly, yeah. So thank you for coming on our podcast. Of course, you have your podcast Python bytes talk Python. We'll have links to everything training around Django stuff, if they need it. And this was really fun. This deployment is such was just an infinite topic. And I think for you to come in as pretty much a solo person at scale, that's a really interesting perspective, as opposed to someone at a large company who came in late, you know, Kate didn't, didn't start from the beginning, and it's just like, well, here's my little piece of it. But everyone wants to know, how did you how did you get there?
Michael Kennedy 1:14:44
It's, yeah, it's really interesting to think of. I mean, my takeaway is, it's so easy and so affordable. I guess easy is not the right words. It's so accessible, right? You know, for. Yeah, $100 a month or something, you end up with infrastructure that looks like world class infrastructure, right? It's ridiculous. So it's all within reach, and hopefully people got some cool resources as well. Thanks.
Will Vincent 1:15:13
Thanks, boy. So we are at Django chat.com and we'll see everyone next time bye, bye, bye bye. This episode is sponsored by mail trap, an email delivery platform that developers love try for free@mailtrap.io you.