Craig Kerstiens works at Crunchy Data, a provider of managed Postgres services. He previously worked at Heroku, Citus, and Azure Postgres among other companies. We talk about hosting providers today, the challenges of managed databases, Heroku’s early years, and more.
This podcast does not have any ads or sponsors. To support the show, please consider purchasing a book, signing up for Button, or reading the Django News newsletter.
Carlton Gibson 0:05
Hi, welcome to another episode of Django Chat podcast on the Django web framework. I'm Carlton Gibson joined by Will Vincent
Will Vincent 0:12
Hello, Carlton.
Carlton Gibson 0:14
Hello. And today we've got with us Craig Kerstiens. Thank you for joining us so much.
Craig Kerstiens 0:19
Yeah. Thanks for having me. No.
Carlton Gibson 0:22
I'm really excited to have you on. So massive hero of mine. For people who don't know, you're, I'm gonna say you're the you're the database guy, right? Yeah, I think I definitely been pegged as that, like, basically, it's been a lot of time and dev tools and around Postgres. I keep joking that like, the next time I do something, it's gonna be not Postgres and back to dev tools. But no, it's No, it'd be Postgres again, oh, no, it's
Craig Kerstiens 0:47
third time, and I'm done with it. But now I've been around, you know, Postgres and in that community a while, but really, you know, a mix of dev tools and kind of work with a bunch of dev tools, companies know those circles, well, spent a long time at Heroku. Like came on Heroku not to do anything Postgres at all, I came on there to actually build and run their Python team, so and ended up in doing Postgres there again, so, so.
Carlton Gibson 1:13
Okay, so that's a nice little segue. Well, how does one start, I'm in a Python lead. And now all of a sudden, I'm, you know, get further and further into the Python. Well, just one, one index at a time.
Craig Kerstiens 1:24
Yeah. Um, so I guess I can, you know, back way up, right. So I've been at like fortune 100 companies had joined a small database startup way back in the last recession, not the current one. But um, and they were they extended Postgres to become a streaming database, and basically do MapReduce, MapReduce on data as it came in. So yeah, back in the day, at a time, like complex event processing, CEP was a common thing. So streaming databases were a thing. There was like Aster data and tiza Greenplum. And I joined Trevizo. And it was interesting. We had I think MySpace was a customer. Like, if you remember back when MySpace was like, hey, what do?
Carlton Gibson 2:12
We do? Yeah, that's, that's where I learned the web and MySpace.
Craig Kerstiens 2:15
And like, we were handling all their ad traffic, right? Like, how do you like process all this in real time and manage it and report on it. And so I joined there, they were 15 people, that grew to be about 30 people dwindled back down to 15. And it was kind of ready to move on and did some consulting bootstrap my own thing. Actually, in the wedding space, it was one of those like, scratch your own itch is right was engaged, getting married at the time, and was like, built this product, didn't know anything about marketing, and like got written up in like the Wall Street Journal, like top wedding apps. Like, it's like, oh, wow, cool. And then I got married. And I'm like, I hate this. Like, I was literally going to meet up talking about the seasons stress cover. And I had known some of the Heroku folks who had been there did some consulting with them. And, but it's like, it's all Ruby, right? And I'm like, I don't want to go work at a Ruby company, like Heroku was the cool thing at the time. But myself being more in like the Django world, I'm like, yeah, it was like, this is cool thing I had heard about, right, but I'd never experienced it. And so I show up there. And I'm doing a little bit consulting, some business analytic stuff and other things for them. And they're like, so let me show you this thing. We got this, like, build pack thing. And, you know, when I was bootstrapping my own thing, we spent two or three weeks like getting our deploy process good. And like, cool. You tar it up, you ship it over here, like everyone did this, right. And like to have stable builds and releases, and I forget, we were running on a y node VPS or something, you know, like everyone did back then. And I was just like, wow, there was three weeks in my life. I wish I could get back because I could have just git push, and it was up and running. And Python was slated to be like, the, I think, the next the second or third language, and it got pushed back because of Java and others. So it like ended up being one of the last major ones really released at the time. But I was like, Okay, who's releasing this who's driving this? And they're like, Oh, we don't know. Like, we just have this thing. Like, we built it. And here it is. And, um, so I ended up joining to like, I was the first kind of build pack maintainer there. Then I hired Kenneth because I'm like, I'm not actually a good Python developer. Like I know that ecosystem and world but like, there's so much better out there. So I, I joined Heroku to kind of focus on that, not Postgres at
Carlton Gibson 4:42
all. Okay. Okay, fantastic.
Will Vincent 4:44
Can I just go back and ask, so how did you get into programming initially do sound Did you have a like undergrad degree? Like, why? Why? Why did Python hit for you? Because this is before Python was kind of the hotness.
Craig Kerstiens 4:56
Um, so I started coding it like Ah 10 Like, to me, like codes always been an interesting means to an end. Like, I like solving problems, but like, to me that intersection of like code and technology and business. So like my, my degree in college was actually over in the business school but a minor in CS. So like, I, I generally say, and like, you know, the best of the best, like developers, I'm like, I can't keep up but like, put me in your average eight to five engineering job, right? Like, I think the people listening to like, the podcast, right, deeply care. But there's a whole world of developers that like, the show up, write some code and go home and don't think about it right, and don't care if they get massively better. And I guess I'm kind of envious of that at the time.
Will Vincent 5:43
I mean, I was just gonna say that doesn't sound I mean, maybe not the not caring about getting better. But the rest of it sounds good.
Craig Kerstiens 5:48
Yeah. I mean, like, I'm kind of envious, like, Hey, you can turn it off, right for me. You know, I'm out here in the Bay Area. And it's like, everything is code or tech constantly. So I appreciate the like, you know, the family friends that are doctors and nurses and not in tech, right. So I don't I don't subscribe to the idea that you have to truly, to be good developer and contribute. You don't have to like live and breathe it and say this is you know, the end all be all. But yeah, to me, always that intersection of tech and business was interesting. We, I think it was a colleague of mine, Daniel Farina that like, he was like, Yeah, this Django thing. It seems like it has good docs. It's like, well, opinionated, it's well structured. I mean, this was back in like, was it like, oh, nine, two, maybe that we got started on it. Like early days. It was kind of data rails. And it's like, the docs on Django seemed better. So I think it had the time, the batteries included tagline, which I loved. Like, it's like, great. I need off. I need an admin panel. I need like, it was all I mean, all of the things that are still true today. Were incredibly true way, way, way back 15 years ago.
Carlton Gibson 6:58
Yeah, right. I mean, that talk about how you might you might as well ask for a pony because it's got, you know, it's got all these things that like, now, you know, when people say, well, where's the pony come from? And you tell that little story, they sort of glaze over because they're like, but every web frameworks got that. But you got to understand that no, really, every web framework didn't have that.
Craig Kerstiens 7:17
Yeah, I mean, it was, you know, you look to the Ruby community, the Rails community, and there's like, still five different off, and finally it off gem one out, but for the longest time, it was everyone was changing every, you know, few months.
Carlton Gibson 7:29
Yeah. And Ruby is still part of the mean, you know, Rails is still very much a batteries included full stack framework, right. So expect?
Craig Kerstiens 7:36
Yeah, I mean, to me, I think Django goes even further, like, Rails is close to it, but like, misses some of those pieces. And I have to have to be careful, because, you know, I have many, many friends in the rails Ruby community who to
Will Vincent 7:46
whom we had DHH on here. So we, you know, we're, we're all compatriots trying to solve the same problems.
Carlton Gibson 7:52
I mean, like, okay, so what we've got, we've got you talking about Rails, I mean, one thing that people often say about Rails is it's more opinionated.
Craig Kerstiens 8:00
I think it's more opinionated. I think there's less batteries included, I think, right? Like, there's not an admin dashboard, right? There's not user management, there's not like, there's there's a few things that now it's standard, oh, yeah, you pick this up off the shelf. But I mean, even 10 years ago, that wasn't I decided, you know, like, how do you do user management rails,
Will Vincent 8:23
I'll just say one of the things that rails had as an advantage was that Heroku, you could pretty much just start it and push it to Heroku. And it worked. And that I think, the rails book, you know, the first chapter, I remember when I was learning Django, back 10 years ago, was just like, you know, doo doo doo, boop, you're live. And there's nothing like that on Django. So like, I have a couple of books on Django. And my goal was actually like, I want to get you as quickly there as we could. But it's still four or five, six steps, just because so that I think comes to I like that distinction you made of batteries versus a pin of you know, guardrails. But definitely feels like a beginner still thinks Rails is like, gets you what you need. It's like, kinda, but then once you actually know what you're doing, you're a little bit more on your own. But, but that's what we'll talk later about this. But there's the whole news post Heroku, hosting platforms and variant, like, I just wrote the docs, the Django Doc's for fly.io, which is one of them, because they didn't have any. But it's interesting how there's not quite like a Rails bias. And there's also you don't use Git at all, like, in fact, if you don't have a Docker ignore file, your Git will be pushed up there. So like, it's a whole kind of different. It's similar but quite different in subtle ways, if you're used to the Heroku way.
Craig Kerstiens 9:36
Yeah, I think we're gonna have a few interesting years. You know, I don't know if we want to get into it now or later on, but like a few interesting years where, I don't know what the I mean, and Heroku is still there. Right? The free tier is gone. But I think it's, you know, it's an interesting time and to me, it's kind of the little bit the writing on the wall, right? Like, I saw someone that was, you know, X Heroku used to work at Heroku. And was, you know, probably it was the right thing to do to get rid of the free tier. And then came back a few weeks later and was like, Hey, I just ran like a Rails, girls bootcamp and like, I didn't think about the implication of this, like, like, developers learned on Heroku. Like, if you were at a Django bootcamp, pi ladies, like, I mean, they every highlight these bootcamp ran on Heroku that I am aware of aware of, like, I helped with a bunch, like we sponsored stuff. And it was like, here, it's just easy, right? Like, getting a dev stack is still a huge pain. And you didn't have to have a full dev stack to be able to push up the Roku and see something at the end of the
Will Vincent 10:39
day. Yeah, every, like, you know, my books, everyone's everyone's stuff is broken now. Or not broken, but you have to pay. And it's less than having to pay. I mean, I think like we've Carlton, Carlton, I've had some discussions, like there's a valid thing to say, you shouldn't have unlimited free, because there's a real cost and there's fraud. But you know, not like the those one of them told us they fail want to be the second place, right? They want the marketing benefit, but not the hassle of being the free place. But it is a Yeah, it's harder, it's harder to learn. I mean, I'm trying to tell people like I'm switching to fly just to my books, because I need to pick one, like you. Hopefully, you paid for this book and didn't steal it. So like paying a couple cents to host a site like, you know, that should feel normal, right? Like it. There are real costs involved. But it is just a drag on the whole cycle. And to your point, it's going to be several years before, there's a clear winner, right? Like a lot of these are Series B and C, they've raised money. Probably one or two, you'll get acquired, one will win and the rest will go away. So in the interim, it's frustrating for people teaching beginners. Yes, Carlton?
Carlton Gibson 11:44
Well, I, I mean, unless you want to say something to that point, Craig, but I wanted to ask you more about the DX side of it, the developer experience side of it. So Heroku clearly got it. Right. And from your perspective, which is an amazing set of having been there and worked in it and created it. What do you think? Sort of like the key things that people should look out for or that are missing from the new ones? Or, you know, what do you think the sort of the Holy Grail, it's like the,
Craig Kerstiens 12:15
yeah, two pieces in there, right. Like at Heroku. We, we agonized over dx, like, we spent so much time discussing and debating, you know, like, I mean, at the time requirements, dot txt was not a standard. Right? Like, you go, I mean, I remember speaking at a PI code camp in Miami, asking how many people actually had a requirements dot txt and an audience of Python. And there were like, three hands. It was like, it was not a thing. And it was like, but then you go and talk to people, and how did you manage like a reproducible build, and it was all over the map. And now it's like, you know, whether it's, you know, Pip M, or like, what I mean, I know there's plenty of debates on but it's like, okay, requirements, dot txt generally works as well. And like, well enough, a standard now, right without it being is that I'm not aware of it as a standard, but like, it may be right, right, codified somewhere. But we agonized over that and kind of we looked across language communities and said, What do we need to push as an agenda to like, not for us, but to like better improve? How can Python go and learn from Ruby? And how can Ruby learn from Java? And how can database URL was a canonical thing? And like I had, I forget how many debates I had, whether it's with Jacob or Andrew or who they were like, No, you just do this? And I'm like, No, you shouldn't be putting config in code. And they're like, Yeah, but it's like, and, man, it was some hard fought battle. And, like persistent connections in Django, I remember at a Django con Europe, standing up giving a talk, and Andrew to argue tooth and nail with me that like, it's not a problem. And I showed him the chart of like latency to do TLS. I'm like, Okay, do you connect your database with TLS? He's like, Yeah, you should, it's secure. And then I showed him the TLS negotiation, and it's like, oh, here's 100 milliseconds. And like, that's just to grab a connection. He's like, like, and I asked how many people so this is Andrew, he knows how to run production, Postgres, right. And I'm like, Alright, how many people are using PG bouncer? Because that's what you should do he tell you to do that. And there were like, three hands in the audience out of like, 400 people. And it was like, I love publicly trolling him, right? Like he's sitting there on the front row. And I'm just like, I'm giving him this like subtle stares. I'm asking the question in the audience. Because he's like, No, it's not a problem. This is what you do. And everyone says to do this, and we agree that is the right thing to do and no one does it. So I'm being highly opinionated and pushing for the thing that was right, right, like taking the long database URL wasn't a standard for three years, right. And we just gradually pushed and knowledge and had the conversation and built the community and the connection, right. Like, I have a lot of very good friends having never contributed a line to core of Django at all, that like when they're in when Jacobs in town, like he's coming over for dinner to catch up. And I mean, that invite goes broadly out to like, anyone I know, in the dev world, like, I'm very happy to, like have, like, it's a small world and community that we're a part of, but I'm really pushing for that, like, this is the right way to do things. And, and just seeing it through not giving up on it. And that's a hard thing to measure. Like, yeah, I you see it when you know it? And if the answer is go to docs, or oh, here, like, here's this error message, or here's how to tweak, like, it should just work. Like it should just fade away in the magic.
Carlton Gibson 16:07
Okay, good. That's really nice, because I swapped one thing I sort of see with the a lot of the new batches that are trying to cover every possible use case and every every possible way of doing it. And because of that it's quite complex. There's too many moving parts. Sometimes it's it's,
Craig Kerstiens 16:23
it's hard. And I I'm excited by fly in what they do, I think a lot of what they're doing with the networking stack is really interesting. Like there's a whole bunch out there, and some are more exciting to me than others. And they're, they're exciting because they have an opinion and a vision. I don't think any of them have nailed the same simple developer experience of Heroku yet.
Will Vincent 16:47
Well, they don't know that maybe this leads into crunchy data. I don't think any of them truly have managed databases yet. Which, you know, is like they had they'll give it they'll give you Postgres, but it's just a running in a VM. It's not, not like Heroku has it and not like dedicated services like crunchy data. And my sense, my just from the outside senses, it's taking them a lot longer to do that, than they thought it might.
Craig Kerstiens 17:11
It's, it's, it's hard, and I wouldn't wish it on and it sucks. Someone to run my databases for me. I mean, at the time, it Heroku we we joke that we had all these Rails developers asking for database and we thought, how hard could this be? Turns out it was a whole lot more work than we ever expected. Like Wally, which did all of the like, underlying disaster recovery, continuous protection stuff. Like, we were good and lucky. Like we wrote that in a weekend, thinking, hey, we needed something like and shipped it. Luckily, about a month before the great EBS apocalypse. I think that was April 2011. I don't know if people remember when like Netflix read it, like the entire internet was down for a week. It was terrible. And like, a lot of times, like when Heroku would go down in the early days, like we would go down. And we check and see like is rented up and Netflix up. And if they're down to like, we're screwed, like we need to wait for like,
Carlton Gibson 18:10
yeah, it's the whole availability zone. It's gone. It's not just
Will Vincent 18:14
like Northern Virginia is offline and half the web is gone.
Craig Kerstiens 18:17
Yeah, it was. That happened. Like we I mean, AWS was offline for a week in April of 2011. Pretty much. And we had shipped Wally, just like, luckily, a month before. And so we were able to recover databases with no data loss, whereas RDS just told customers like Sorry, we lost the volume, like back then. Yeah. And so I mean, I'd like to say that was amazing foresight. And like we knew what we were doing. It's like a little bit of Lucky timing. And, I mean, now we very much know what we're doing, like a very experienced team that, you know, at the tail end of our time, at my time, there it was, after five years, we were running, managing, like 1.5 million Postgres databases. We joke that a one and a million problem was once once a week across eight engineers, so very small, proficient engineering team. But I know Kurt over at Fli really well, in his viewpoint is like, it's at this point, it doesn't make sense for a pass to also run a database as a service. Like they're specialized enough. Like it's a distraction, I think, I think we may be past the point where, like, if you want the best service or Postgres, you come to someone that does Postgres. You want the best service for Redis you're gonna someone that does Redis you want the best paths. You go to the past for the past and it's I know Kurt's gonna you know, ping me after this and be like, alright, Craig. So that means you're gonna run the databases for fly right? And we we go back and forth every few months and we have customers that use fly and use crunchy and like, I've run a pass. I've worked at Heroku and ran a lot of Heroku Postgres but a lot of other areas of post have Heroku is, well, it's a lot of work to do either one of them. And right now we're really focused on Postgres really, really, really well.
Carlton Gibson 20:09
Okay, so given that we've talked about crunchy, let's dive in there. So this is a managed database service a bit like rds. But, you know more as a boutique most more bespoke more. So I'm a big fan of rds. I always recommend people get a managed Postgres. Why should I use crunchy data? what's the, what's the?
Craig Kerstiens 20:31
Yeah, so I mean, I'll give a few reasons. Right? Like, one, we contribute about a third of the upstream code to open source Postgres itself, right. So, you know, if you care about supporting open source, right, like, we contribute to a lot of it, right? If you're like, how do I contribute to Postgres? Well, guess what we, we employ people that contribute to upstream Postgres, about a third of the code. So there's, uh, you know, there's sometimes not a lot of love loss for Amazon right out there making a ton of money, how do I, I contribute in a meaningful way to open source supporting us is one. We, we are just Postgres like, at whether you gotta, you know, Microsoft or Google or RDS like, they, they have people that run databases, and they have, you know, app dev teams and all sorts like, we're just Postgres. And we also understand it from an app dev perspective. Like, I've been trying to make a positive impact on Postgres. And for app developers, like at Heroku, we contributed a lot to get JSON be committed to, to Postgres itself. We, situs when I was there, we donated a portion of our equity to the postgres community and foundations. So like when situs was acquired, it meant money for the postgres community. So like, we're deep in there, and because of that, you're gonna get a great Postgres experience, within RDS like you open a ticket. And you, you may or may not get to the product team, like,
Carlton Gibson 22:04
No, you know, you won't. I'll say, even if you won't say that, I'll say that you won't.
Craig Kerstiens 22:10
And for us, right, like, if there's a bug in Postgres, like, I can reach over to a colleague and have them upstream, a patch to it, right. So like, there's no issue too deep there. I was told by our marketing team, I can't say like, our support isn't crap. But like, our support isn't crap. Like we give really good support. Like I tell people, like open up a ticket, ask a question, like, and I think we have a unique like, it's not on the product side, necessarily. I think it sort of isn't sort of isn't in that, like, I get app developers, like, and app developers are terrible with your database. Like it's true. Like, how many don't run with PG bouncer, and you should, and like, even if you have persistent connections in Django, you're like, No, I sent my connection pool and like, that actually wastes memory and Postgres, like, you've reduced latency of establishing a connection. But every idle connection to Postgres is about 10. Meg's of wasted memory. It's like, oh, so I should run with PG about Yes. But like, one connection pool is not the same as another connection pool? Yeah. Like, you're like, you're 20. I want I'm like both, you want both of them.
Will Vincent 23:24
But you're talking about performance things. I mean, I remember, I was at a startup called Quizlet, which has decent traffic, and it was MySQL. And, you know, at one point we did, we didn't have like, a lot of permissions for who could go in to do something. And the founder went into change, record and dropped the entire database, which was an oops, we got it back. But then after that, we had like, Percona, on the highest level of support to, you know, to your point of like people saying and not doing I mean, if you're running a website or running a startup, like you're definitely not focused on the database on the database, to the degree you should be just because you have a million other things to do.
Craig Kerstiens 23:59
Yeah, no, I mean, I laugh a little like situation, but like, I've seen it before. I've had customers Friday at 4pm be like, Oh, we ran our test suite against production. What's the first thing in the test suite like truncate database, like, oh, and I'm like, Alright, take a deep breath. Tell me exactly when you ran the command, right? Do you have and we can like restore your database to exactly no database data loss, like it's going to take a little while, but like, we can watch the right ahead, log replay, and see when it like the exact time that it happened and give you the exact moment before you dropped all your data. And so like we have your back there. I mean, we've got some stuff that detects like n plus one queries in your database coming soon. Like, no one of you in the Django ORM have ever written an N plus one query at all, I'm sure right.
Carlton Gibson 24:49
Yeah. Use adult group stuff.
Craig Kerstiens 24:55
And it's, it's not that like developers are bad but you got other things to worry about. You can't be an expert on everything. And like, I found the Django community loves Postgres, but also you, you're missing half the features and half the perks of it, because you don't have time to dig into it. Well,
Carlton Gibson 25:13
this is, well, this one. So one of the, one of the things that crunchy data's like on my radar for is that you've got support for PL Python. So the, you know, embedded Python in that was RDS like you go down there long list of which supported PL is and pythons not there. And it's like, but, you know, I I'm not gonna go and learn. This one did to write that stuff. So there are more features that as well, like, you can get a richer Postgres experience.
Craig Kerstiens 25:39
Yeah, I think it sums it up for like, richer Postgres, like, because we are everything Postgres because we believe so strongly in it. Like, I want to give you everything that Postgres can do, right, like, I, you know, when we ship PL Python, I'm like, I wonder what F and we ship it also with Syfy NumPy, and pandas in there. And I was just like, you know, what, I wonder if I could write a recommendation engine inside my database. I started like tweeting this, like when we shipped it, and I was like, I should have tested that I could do it first. So why could this how wrong could this go? Like 20 lines of PL Python and two hours later, and like, here's a recommendation engine inside my database. And there's all sorts of reasons you shouldn't put that in production. But maybe you should. Yep. Oh, well, this
Carlton Gibson 26:27
is one question is how much? How much of my application logic should I move to the database, because I remember years ago, and it must have been years ago, because I was using MySQL, and I got to learn about triggers. And I was like, this is going to be quite good, because, you know, round trip to the database, that's costly. Let's see if I, and it was fine. To sort of start with, but then it started getting out of hand, and I had no real management of it. So the things that I missed, and that I still don't know if they really exist are source code controls, version management, and like testing, decent testing. And so like I, you know, one question I have for you is, okay, let's say I use PL Python. How much of my logic should I embed in there? And how can I have good software engineering practices, so that I can roll back that I can test I can show and other frameworks to help me do this. Because if I was doing this, in Python, I just use Django. If I was doing it,
Craig Kerstiens 27:17
there's a little bit more than there used to be, but it's still not perfect, right? There's still not the easy release process version management as much as you'd expect. Testing, there's a little bit more with things like PG tap, or like, you think about the the other kind of supporting pieces, right that I can go and pick a fork of my database. So I can grab a snapshot of production on the staging. And I can run this as like, we have more tooling around the database of like, I mean, we have customers all the time, that take a copy of production, moving to staging like via automation on a Friday night, slim it down. So they don't, you know, developer, other developers aren't working with a one terabyte database. But it's like, here's the latest 10 gigs of production data. So we have realistic test data that we get every Monday morning, right? Every every developers get a real copy. And, you know, I think they built it in less than a day with all our API's and automation and like the, hey, we give you these constructs to build and work how you want, right? And we see organizations work very differently. So we haven't said, here's the codified one way to do this. Now we have thoughts of how it evolves. But it's still I think, early for that type of CI CD for your database, right? So it's, it's not perfect. And that's why I don't think you should always go like gung ho, like, let Yeah, swing all the way back and do all of the logic in the database. I think if you're doing data intensive operations, that's a good line where you can do it easier in the database. And my, you know, Postgres, I think can do a lot. And so it's start there, instead of adding another tool or system, full text search, start there before you add elastic, because how are you getting the data into elastic? Now you've got two systems to maintain geospatial stuff, post GIS is great. Like, you start to talk about a recommendation engine. And it's like, it's 20 lines of PL Python, and it's going to be slow and not great, but you refresh it every hour or something like that, right? Now, if you were to do this, otherwise, you're going to be like, Okay, I'm gonna go and install the Vizio and do Change Data Capture from my database, feed this into Kafka, I'm going to run some model and spark and now I'm going to cache the results in Redis. And now, and I like that's not an exaggeration from 90% of companies. And now I've got five systems to maintain versus like 20 lines of PL Python inside my like, and your point of like, well, how do I version it? How do I test it? Well, manually, like it's 20 lines of PL Python as opposed to five systems, right?
Carlton Gibson 29:47
Yeah. Okay, that's super as well because that's, that's that ties it I saw you tweet something about how, like Postgres and neither rails or Django, you know, but then being like the money stack, like if you just want to get something Tom,
Craig Kerstiens 30:00
it just works. Like it's it's, it's, it's it's profanity, okay, on the podcast, you're not good. You get the idea, get whatever done right. Like, that's, that's what it is oh, it's like, I just want to end. One fun thing a total kind of non sequitur, but like, one fun thing for me of like running a past or databases of service is seeing everything like people build like, I talked to one person who's he he surveys snakes for Canada like that is his, like half, like engineer and half like out in the woods surveying snake and populations of snakes. And he uses like, post GIS, because it's like the best geospatial database. And he's like, Yeah, I'm not a real developer engineer. I'm like, you totally are like, the stuff you're doing here is, he's like, Yeah, but not a real eight to five software. And like, because he's his backgrounds and snakes, but he's using like post GIS. And so getting to see all these fascinating things like, coming full circle, right? To me, like tech is a means to an end. It's a fascinating one. And so by being at the end of the dev tools, Database as a Service intersection, I can see like, you know, we have one person that our team jokes is like, the cow social network that like tracks, cows, like it's an app that like, tracks, their eating habits and other things. And it's like, who, how is this a business and a start up in like, what, and we know this, like, walk down the street and like, walk into a retail store, I'm sure there's a problem. You can see tech solving. So it's, it's fascinating to be at that intersection, and like Django, Postgres, Django, or rails, to me, they're very similar. Just just work. Like they're not new and shiny. They just work. And now I can go focus on solving a problem.
Carlton Gibson 31:52
Yeah, and you haven't got 15 bit moving parts, which then you need three engineers just to keep the moving parts in the nevermind about right, the application logic.
Craig Kerstiens 32:01
Yeah, yeah, absolutely. And so that's why I mean, I totally appreciate the question, right. I'm like, Okay, well, how do you test this? How do you version this? But sometimes you may not, if it doesn't change once a year, and you, you know, how did we how do we test things before? Well, we would like extra code review and eyes on it. And like, you know, manual testing. It's like, Oh, my goodness, crazy that we could do those things again.
Carlton Gibson 32:22
I've got such an evil devil on my shoulder going oh, yes, yes. It's like, it's like when I got the remote development exam extension in VS code, and all sudden, like, so I can just open a folder on my server now, oh, I can
Craig Kerstiens 32:40
do things in balance, right? If you've got a 20 person team, how are you doing this? How much is it changing? But is this is, you know, we used to like our sync files to deploy, right? Like, like, hey, guess what, it's still worth back then. We brand websites on the internet, same as we do today. Just fine.
Carlton Gibson 33:02
Okay. Well, I wanted to ask you one more thing about crunchy data. And perhaps we can swing around to, you know, perhaps a few more Django questions that we've got lined up. But the question I had was, I saw your recent so when you first launched, I was a bit like all but it's quite a lot more expensive at small scale than an RDS because the small RDS costs, you know, 810 bucks a month, and, you know, then you can scale up as you need to, whereas country data was more like in the 30 $40, which is absolutely reasonable. But you know, it's for a smaller project. That's perhaps prohibitive. But then last month, month, two months ago, you did you went for 10 bucks a month or something for the minimum, which I just think is a power move. Coach, God, tell us about that.
Craig Kerstiens 33:43
Yeah, I mean, so initially, we were really focused on like, being a production ready and like data, absolutely. Backups, disaster recovery monitoring metric, like everything you truly need, right? Like PG bouncer is built in, like, you don't need to go and layer it in. I don't know other providers that can get it just right out of the box. I think Heroku kind of has an add on for it now. But like on rds, there's this separate RDS proxy that you go pay for, like, No, this is what it takes to run a production database. And we're running a lot of very large production databases. Now I care about having developers but we needed to nail the, the advanced stuff, like we now we're going kind of downmarket that it can be there for everyone. And it's you know, as you say, 10 bucks a month for a full Postgres database with PL Python and with, you know, post GIS and you can create more databases Heroku. You know, when they had they're kind of like lighter to your plans. It's like, here's a million rows, and I'm like, a million rows, what is like, and it's one, you get a shared multi tenant database thing. This is your own dedicated instance. And then the
Carlton Gibson 34:50
key point, the key point there for the hobby developers is I can create as many databases in that instance as I want and so you know, if I've got if I'm running 50 micro projects, hobby micro projects, I'm still running it on the same 110 Buck instance. And then that that's as close to zero as you ever gonna get like, well, technically zero, but it's effectively zero.
Craig Kerstiens 35:11
And so, you know, we were talking kind of like about pi ladies and some of those workshops. If you spend less than $5 in a month, we don't believe. So it's not a $5 credit. But if you're below that, $5 in a month, like, great, it's on us. And so you can go in, like, suspend your database and idle it. So if you're like, building and developing, right, and you're like, Hey, I'm, we've got a couple of universities now that are pointing their students at us are like, great, I only like 5pm, and on the weekends, and like, it's there. And then when you're developing and learning and building out your app, your project, turn it back on, right. And then if you want to, hey, if you want to have your portfolio live to demo and show off what you can, like, great, it's 10 bucks, right? It's manage like less than you paid for the textbook, right? But then archive it, it's sitting there, and you're just paying for the storage. And if you're below that, $5 It's It's there on us.
Carlton Gibson 36:08
So I think that's I just think that's a superpower. But I think that's, you know, that's gonna grab a lot of people's attention, I think and rightly so I think when I saw you put out I was like, Yes, that's good luck sauce.
Will Vincent 36:21
So Can I Can I ask about because I'm interested in how people learn how to code are these? Do you have more insight on like, these, these universities? Like, is this their databases course? Is this like, like an intro like CS 50? Kind of thing? Do you have like our from where you said, how are people kind of learning how to code these days? Because I'm always curious and can't it's I never get a consistent answer from people on like, how they're actually like, like building a website, right? Still, many universities don't have a course on If anything, it's an adjunct or, like it's not a core thing, web and like an undergrad CS degree.
Craig Kerstiens 36:55
I've talked to the EU, that seems to be a little more common now. And it's, you know, I think it's very much the basics of building a web app to showcase something right. One of these is actually an A, I don't know if it's an MBA or a master's program. He has a master's program. But it's typically like people out in industry. So it's not like I'm getting a CS masters like coming back and wanting to kind of level up on their technical skills to kind of advance their career, right. So in this one is like on the geospatial side and so geospatial. It's like you do anything geospatial, you're using post GIS likely. And so like how you build an app to showcase this. And it's you know, a lot of the apps are, you know, simple little apps, but it's hey, I can showcase that, hey, I'm doing this analysis here. I want to show off a website. Not deep deep on the sequel side, but they touched just enough of it right? Where, hey, the databases here. Um, I'm definitely seeing more universities teach. Here's how to build a web app. And but the problem is man, like the way they they plan out a course right? They'll work on a course and develop it and plan it and establish that curriculum for like two years. And they do it. And guess what, everything about that we talked about on web development and how to build a web app is changed, like, what does it like remix index js now or the front end frameworks? And I'm like, I don't even mean I missed jQuery. Right? Like I like I don't know,
Will Vincent 38:17
HTMX. It's HTMX. But yeah.
Craig Kerstiens 38:21
So I think it's hard for like universities. So like, how do they? What the pace at which things change? How do they keep up and do all that? Right? I don't know that I have an answer
Carlton Gibson 38:32
to Django that the answer is they should use Jaguar. But
Will Vincent 38:34
even five years ago, while like, I had like a Genk, Django con, you know, a couple professors who are using quantity Django are looking at my books. And yeah, to your point, they have to have it. They're planning now for next fall. And it's like, well, the, you know, the books gonna change like Django might change a little bit. And it's not not so much if you've been in the space, but if you're a beginner or a newcomer, it's enough to kind of break everything and you can't really plan around it.
Craig Kerstiens 39:01
Yeah. In the, for me, especially on the front end side. That the pace
Will Vincent 39:06
Oh, yeah. Well, forget about forget about that. Yeah.
Craig Kerstiens 39:10
That's the one that's like, oh, man, it's just like anxiety immediately of, I don't know anything about like the front end world anymore. Like, like it changes so so quickly,
Will Vincent 39:19
can ask you. So you're sort of unique in that you. You're not a pure Django person, though. You have a strong background Django like, are there other frameworks that the Django community should look at in terms of, hey, there's some good stuff going on there databases or otherwise? Like I'm constantly looking at Laravel impresses me in terms of being batteries included, but from where you sit, are there other ones beyond you know, rails that Django people should look
Craig Kerstiens 39:44
at? I mean, you nailed it on Laravel. That's the big one that comes to mind. I would like a similar and like, kind of modern, you know, PHP is a new thing. Like, yeah, things are different. And so I launched the Python support at Heroku. I also launched the PHP in the Gotta support so, and it was, you know, the PHP one was like What? What really like Heroku official peach. It's a different world now, right? Um,
Will Vincent 40:10
it's huge. I've got a buddy as a consultant, and he's constantly. I mean, it's, it's, you know, I don't wanna throw stereotypes, but it there's a lot of money in, in PHP and a lot of tech stacks that just work. I mean, hosting is easier, right? They've got, you know, Laravel itself has forage or whatever it's called, you know, so, whereas in Django, it's like, you know, so,
Craig Kerstiens 40:30
um, I mean, on the, you know, the front end side, I think next JX next, Jas and remix, like, we we've gotten on the remix run like bandwagon ourselves and I'm not knee deep in it, but it as a front end framework, it seems like next and remix both solve some of the problems that I think existed with vanilla react and, you know, we complicated things on the front end, and it feels like they're getting a little better, like, part of me is so confused. It's like, why do we need a state machine on the front end? Like why why doesn't like request response? Just work? I, there's, there's people in that world that will educate me at some point, like, I'm sure there's a good reason, I hope maybe, but it's just the pendulum just swings, right? Yeah, is entirely does, right. And I felt like I'm an old curmudgeonly, like, you know, yeah, just use Django and Postgres, like build your app, make your money, solve your problems, and like, you don't need anything else. But I also think that's, that's fairly true. Like, you can live in that world. And it's great, and you don't have to relearn something every 12 months. Not to say you don't learn new things in that that stack like Postgres advances, Django advances, but that stack works pretty well.
Will Vincent 41:40
I was gonna ask, are there obvious things Django could add to our official Postgres support? I mean, I think the last the last big one was, I think, full, full text search. I mean, we've, we Django has felt historically, like we couldn't pay for code or really push things apps in a Kickstarter or some things. And we found out through the Python Software Foundation, that's not necessarily the case. So this is one of the things is we're kind of looking for doable. You know, three, six month projects that Django can support and especially around Postgres. So what we are, what cool features is Django not have support for that. Maybe we should? It's a
Craig Kerstiens 42:16
great question. I feel like Django has done really well, like the the JSON support is good. The full text search support is good. I don't, I don't know right off. I mean, it's I don't know that it fits in Django, but maybe it does some of the stuff around like, constraints, like I don't like, like enums aren't a you know, there's an internal site discussion about enums of like, Hey, is anyone seen them out in the wild these days with Postgres? And the answer is, most uniformly No. It's like, use a check constraint, and instead, because then you can easily change it and relax it. And you don't have to do a table rewrite. It's not as invasive like, oh, okay, like, I don't know, if an enum in Django maps down to like, a check constraint, which is maybe a more modern way of doing that, right, or exclusion constraints with range types. How do you say like, you can't be registered for a university class that, you know, overlaps times, like range types, and check and exclusion constraints are amazing. Like, if you actually had to enforce that in code, like you're going to write wind up with, like, race condition. And if you remember back to like university scheduling, it's like, no, no, it's just like a class with 30 people how, like, there's no way two people will be registering at the same time. And I remember like, oh, yeah, my registration opens at this time. And I, I want the 1pm. I do not want the 8am class. So I'm like, there when it opens, ready to go. And yeah, did you just fill up a class? Because like, they both had their session working in you know, you know, multiple web servers and like, so like, relying on the database, I think a little bit more for like, constraints is a big one. Constraints are so valuable, and yeah, good.
Carlton Gibson 44:01
So it's good. You mentioned it, because that's one of the Good Feet that has come on quite a long way and Django over the last few versions. So we first of all unique constraints. And now, you know, to have to check exactly which you know which constraints we've got covered there. But Django 4.1. Now that has it. So when you do validation on a form, the database constraints that you've got checked for that form validation as well. So it goes back and we'll go. And so it's like, it's not, we first of all, it was like, Yeah, you get an integrity error when you save because you violated the database constraint. But now you get the nice form error, and you can go and handle it. And it's like, yes, that's, that's lovely that you say that, because it's one of the features that been pushed out over the last few versions, and there's really flourish, so that's lovely.
Craig Kerstiens 44:44
I don't I feel like Django in general does a great job. So
Carlton Gibson 44:51
one question I had on this topic was one thing that's always really interests me about PACE, Postgres is being able to take a create custom type, so you can You've got a relation, and then you can use it as a field. Yeah. In another relation. It's like, I've never sort of sat down and thought if that would be if that seemed feasible to make work at the Django ORM level, but that'd be kind of amazing.
Craig Kerstiens 45:13
Yeah. And it's one of those things where it's like, when you understand that you're like, oh, man, now what can I do with this? And it's like, wait. It's like, and like, I mean, I've seen it used out in the wild, but not as common, right? Like, it's one of those like, alright, this is a deep database hack. thing for a very specialized case. It's not one of those. I would be more scared of Django supported it like easily in the REM that it's like, oh, man, we've got people now building monstrosities. I so I mean, I could actually see a reason for Django not supporting it. Because, you know, cool, I built this table. And now I want to embed the table and embed the table inside a, you know, another column and another column. And now I built a no SQL database inside Postgres, that's also relational. Like, I could just be TV and turtles all the way down. Okay, good.
Carlton Gibson 46:07
Good. That's a good answer.
Will Vincent 46:12
I want to ask you some fun questions about personal coding setup. Like, so let's, maybe we'll go quickly. So, text editor of choice these days.
Craig Kerstiens 46:22
Usually Sublime Text?
Will Vincent 46:25
Oh, okay. Nice. Mac or Windows, Mac? Or Linux? Yeah. Okay. Good, man. Get this visual editor, what do you use to see your Git?
Craig Kerstiens 46:36
No command line all the way.
Will Vincent 46:38
Cool. Okay. Like that. What else? What's another, like, cool personal, like tool setup you have that maybe others aren't using?
Craig Kerstiens 46:47
Um, so I mean, I, you know, my, I don't spend as much time in code, right. So my stack is like, notion linear. For like project management, superhuman for email, I think my like, my default these days is like, when I pick up a new tool, if it doesn't have like a command palette, like a Command K, like, then cool, I'm not gonna like look for another tool. So it's actually a thing that we're building into crunchy bridge. I'm hoping it ships early q1 Next year, of like, command palette inside crunchy bridge, where it's like, great, you want to find slow queries, Command K, database name, slow queries, like, you can imagine all these things that like, I forgot where to go into what to do, like, queries over five minutes, right on your database, just Command K, start typing, and it'll take you to that screen where you could kill the query. So I'm, I'm stoked, I think, maybe be the first depending on you know, who listens and tries to outrun us and ship it? Yeah, I'm a big fan of tools with like a command palette, to just like, I can be a power user, but also like this take like this, as an average user turns me into a power user.
Carlton Gibson 47:58
It's discoverability, right? Because you just start typing in all the possible matches come up is oh, it's that one? Right, brilliant.
Will Vincent 48:05
How do you think so the fact that you mentioned that crunchy is like, around a third of open source contributions to Postgres, like Django doesn't have anything like that. We have massive companies that use it, that it's sort of like the bigger you get, the less you're, you contribute to Django? Like any, any thoughts on how we can do better about getting some of these massive organizations to actually, that are using Django, because what happens, I think, is that they kind of get off the versions, and then they have a custom fork, like Instagram has this. And they're kind of not really using Django itself, or I don't know, like it, we'd love to have them. That's, that's such a great thing for Postgres, right, we're always like, like Postgres had a table at Django con, like, Django has no tables anywhere, right? Like Postgres seems to have figured something out there that Django needs to?
Craig Kerstiens 48:50
Well, but I think it's, it's not the big companies, right? It's, you know, the smaller ones that are doing kind of, like, you know, we're 100 person company, right? Like, look, look at AWS, right, like, are they running a proprietary fork of Postgres, so they should be contributing things back? Yeah. Probably like, like, there are Instagram, right, or Instacart? Or, like, so. Yeah, I mean, I think there's, I don't know how to get the big ones to write. I think that's the reason of like, hey, how do you support and sustain open source? It's, you know, some smaller companies building models around this, right? We have people that are full time working on Postgres, and like, we're, we're 100 person company, it's like, you know, can we take that person and that salary and build it? I can do product or otherwise? Sure, but we're committed to Postgres right. And I think there's an opportunity for Django you know, the mix of consultancies and smaller shops like to do that right. And, and product companies, right, like had Heroku. We were for a little while headed down that path. Right. And I remember way back at Roku, there was there was chain D and there was what was Andrews A startup. I forget Oh,
Will Vincent 50:05
Lynard?
Craig Kerstiens 50:07
No, no, before that he was building a Python for Heroku. Oh, I don't know. I don't know about that. But yeah, so Heroku for Python, um, I emailed them ahead of time, there were three of them that existed kills me that I can't remember now, what the name of it was. Um, but yeah, how do you build product companies around, you know, Django that then also contribute back? That's a good question. I think. I'd say Postgres is a little lucky, maybe a little bit of skill, and then some some super intentional kind of companies doing that.
Carlton Gibson 50:38
Can I ask, is there any sort of business case to be made that it pays for itself in terms of expertise, and, you know, the ability to react to problems more quickly, and so that you're paying for these engineers to work on Postgres, but because of that you have in house expertise that's significantly higher level than you would otherwise have?
Craig Kerstiens 51:03
I think certainly, it's the pitch I give, right. Like, you can't,
Carlton Gibson 51:07
but Right, but it does does it does it? Does it cut the mustard with the business type. So they go Oh, yeah, actually, yes. Yeah.
Craig Kerstiens 51:13
I mean, for for us. Absolutely. Right. So I think it's a case by case but it's like, you know, you, you don't need a Postgres contributor on staff as a Django development shop. Right? Does that mean you can't discover a Postgres bug and need help with it? Yes. Now, guess what, even if you wanted to pay the salary of a dedicated Postgres person as a Django shop, you, you couldn't hire them. Because you don't have interesting enough problems. Like you, you uncover one bug a year, right? It's like they're bored with the work now as a Django shop, right? Man, having someone that's like knee deep and Django core, and you got, you know, five clients, simultaneous projects, like, great. Oh, yeah. Here's an interesting one. We heard this from users, right? Here's where we can go in upstream, something like that model can work really well. And what I tell our customers like, why do you pick us over someone else? Like, great when you have that Postgres bug that you could never get to the bottom of like, forget, even if you could, if you wanted to pay the person, you can't hire the DBA? Because you don't have interesting enough problems. But guess what? Here, like your problem is interesting to them. And like, let's pull them in. Right. And so when you need that one week of the year, you've got it with us.
Carlton Gibson 52:27
Yeah, I mean, I've done a lot of freelancing. And one, when I started contributing to DRF, in particular, it absolutely changed the dynamic because instead of going to shops, they'd be like, Oh, you're sure you can do this? They'd be like, Oh, you maintain that that dependency that we're building our budget? Yeah. And that just changed everything. Massively. I mean, Zapier had I had Simon charrette, who's one of our big ORM contributors, like, you know, ease, but they give him a lot of time to work on Django itself. And they must obviously see the benefit of doing that.
Craig Kerstiens 52:56
Yeah, so I mean, it's a model that I think works. And I wish I would see more of it. We did some of it at Heroku, manage, do some of that at situs. And when I was at Microsoft a little bit and crunchy, we're doing a ton of, you know, Heroku, we did a cross various languages and other areas, too. So I've seen the model work outside of the database world. It takes some buy in right at the business level. And it takes interaction on the technology side to like, I have found that like hiring someone just to work on open source off on an island isn't as good. It's like, you've got to, you got to get the communicate, like, what are you seeing out in the field? What it costs, like, what problems are your developers, right? How do you get that communication? Like, communications? Are you got to build that in and then it really shines.
Will Vincent 53:43
Okay. So what's the AI disruption for databases, right? Like, what's the copilot? Or is there one coming?
Craig Kerstiens 53:51
There? So much is already there in Postgres, right? Like, you point me at a Postgres database. And I can tell you, like, you have this query that was probably generated by this ORM. And it's like missing with like, select related, right? Like, I can look at the query from inside the database PG, stat statements already has it. And so I you know, I'm optimistic we're, we're doing a lot of r&d and investing there to be like, hey, I can tell you like, from not knowing you are a Django app or not knowing you are a Rails app, because like Django has these conventions on the tables are named this way, the join looks this way, I can probably tell you to search for this and your app code and change it. And now this will improve your performance. So
Will Vincent 54:35
that's like my favorite. Like, there's like a meme of like, you know, AI and then like, you pull back the curtain. It's like, if then if then, like, you know, but it works.
Craig Kerstiens 54:45
It works so well. So I feel like we're really close to a lot of this with Postgres. And so I'm maybe oversimplifying it, but I think it's like a lot of this simple stuff goes a really, really, really long way.
Carlton Gibson 54:58
And is that what you We're excited about coming up in the post goes, well, you know if that if I would say, you know, what's what's on the horizon that we should look out for,
Craig Kerstiens 55:05
for the postgres I mean, so for that's definitely what I'm excited about for crunchy and what we're focused on and like coming soon from us, from the postgres world, you know, the pluggable storage stuff, right? Like Postgres got pluggable storage backends couple of years ago, but we haven't seen like another predominant one emerge. So like there was C heap for a little while and Z store. There's there's a few others. We haven't seen one when yet. So I, but those sort of things move at a different timeline and pace right on like, years. So it's not like, oh, we'll see it next year, and it's gonna win. I think it's like five years before we have a another standard pluggable storage back end. Extensions, like, our awesome, like Postgres that I'm aware of is the only database that allows the latest like deep low level extension hooks that can change the query plan or change the storage engine. It's super exciting. And it's not like there's like one extension, it's like, as Postgres evolves, it's like more hooks get exposed. So you can do more things that you never thought possible, right? Like Postgres is great at full text search at geospatial JSON. I think it's like just missing like being a graph database, like it's good at time series.
Carlton Gibson 56:22
Does the Apache H project Oh, there's one thing that's
Craig Kerstiens 56:25
asked about that every, like, nine months, so it's been around a while. I don't know anyone using it out in the wild. If someone's using it on the wild, like, drop me like, I'd love to hear about it. What's the experience? Like? I'm pretty easy to find search, like Craig and Postgres, and like, shoot me a note, a DM and email. I haven't heard of cases out in the wild. So I don't know enough. Like I get asked like every 16 months, like, what about this one? And like, I still don't know. And it's been like five years now. Because it's not a new project. It's new to the Apache project. But it's, it's not a new extension. Okay, so.
Will Vincent 57:02
So we're coming up on time a little bit. Are there any questions or recommendations you could give to? To us? You know, what, what could Django do to make crunchy or Postgres is life easier? Does anything jump off here? jumped to mind,
Craig Kerstiens 57:15
I think the biggest thing is like, don't find developers. And I hate to use the term developers like I am a developer, just a really bad one that's like, Oh, my God offers developers. Don't be afraid of the database. Like, I find that way too. Like, like, a app developer. That's a really good developer shows up and apologizes, or like, I'm not a database expert. So like, don't judge me. And it's like, I'm I, like, don't judge me for all of my tear. Like, we all have terrible code, like all over our apps, right? And like, don't judge me from a year ago, because now I'm a better developer, like, just, you know, don't talk to my future self, you know, because then I'll have been a bad developer right now. Like, don't be afraid of the database. Don't be afraid of SQL, right? Like, you don't have to become an expert at it like, but like, willingness to learn. I feel like sometimes we abandon it when it comes to databases as developers, like, we'll, we'll dig into advanced caching and you know, templating, and, you know, like, how does DNS work does do any of us know, right, but we'll go and learn about that before we'll learn about the basics of SQL. And I find there's just like, fear and aversion to databases that I wish people would just kind of like, don't be afraid of it, be a little willing to dig in, and then you get so much more out of it.
Will Vincent 58:34
And there are these tools? Like there's the was, I think one of the Harvard places has these like SQL like ways to go deeper a sequel, right? Because I think part of it is you if using Django, right, like, okay, learn just like very basic SQL, and then you can use the ORM. But like, there isn't that need, unless you're like at scale to kind of dive into SQL as part of it. Like part of why it's a gap for developers, whereas maybe as you grow, like, caching, okay, like, I don't know, yeah, I don't know, it's kind of like, how do you become a database person? Right, like you said, you? Like, is there a straight? Yeah, it's just like a weird sort of, you sort of end up there as opposed to like, starting there in a career, whereas like, web, you can kind of stay in Web.
Craig Kerstiens 59:13
Yeah. So I mean, like, so one thing like crunchy, we have like a Postgres playground, which we put like Postgres in your web browser. Oh, yeah, that's
Will Vincent 59:19
right. I thought yeah, that's that was this year that came out right, like follow along like
Craig Kerstiens 59:23
it boot. It's running Postgres not on the cloud somewhere connected to it. Like it's in your web browser. And it'll load we have like exercises and guided tutorials, that it's like, yeah,
Will Vincent 59:32
I'm gonna link to the I'm sorry, I remember that. That was really exciting that came out Oh, I just
Craig Kerstiens 59:37
I just looked on YouTube like four or five new tutorials there. I didn't know about since yesterday, I think um, but it's like, you have the question like a query I write it every single job is like, like user signups or like revenue and then it's like growth week over week or month over month, right? And I bet your first thing is going to like, I'm going to jump to the Django or M and write a query, that's probably terrible. And then I'm going to iterate over stuff and do comparisons over a list. Why not just write like, try to write it in SQL and see how it comes out. Because it's going to be more performant, than anything you do an O or M for that kind of like, reporting stuff. And like, just a willingness of like, hey, if I'm asking questions of data, and it's some kind of report, like, Can I, can I try SQL? Can I give it a shot? Right? That's, I don't know that there's a, an answer for like, how do you get better at it? Except for like, when you have the question, see if SQL can answer it, some of the time doesn't have to be all the time. But at least some of the time. Yeah,
Will Vincent 1:00:42
there. I mean, there's, for me, with beginners, I often say if you're writing raw SQL, you're doing it wrong. So I think that's part of it is like, you know, yes, you can do it to see things and at scale. But I think maybe that's also part of it is there's this sense of like, there's the Simon charrettes of the world who've were smarter than you. So like, Don't even touch it. Right. I mean, that was Simon's whole talk at Django con was like, it's the ORM is not the scary thing that only, you know, three sorcerers understand.
Craig Kerstiens 1:01:06
I think it's like, if you're asking questions of data, right? If I'm not doing a quick, like, you know, the, like, quick look up. Like, if I'm asking you a question about the data, right? I'm doing some kind of analysis. sequel is the right tool for it. Right? If, um, and so like those cases, right, when you want to ask how many users did this, how many people did that? Like, what happened to this over this period of time? sequel is a great thing, right? And that's, that's when it's useful to dig into. Yeah, and the other thing is, like, when you dig into SQL, like, don't worry about it being performance, worried about it being readable. SQL is a powerful language. If you like, you know, do either of you like writing SQL? I like the results. Okay. If I ask that to 100 people, I'll probably get like five to 10 yeses, right? Like, it's not a lot of people. Any of you ever either. Have you ever liked reading someone else's sequel?
Will Vincent 1:02:02
No. Definitely not.
Craig Kerstiens 1:02:08
100. Yeah. But like, I can write SQL that it's like, you know, with like us CTE is use common table expressions, like, build, like building blocks that it's like, don't like a DBA is gonna be like, cringe. This is this is so me like unoptimized. And it's like, yeah, but developers can read it. And you can come back and see what's going on and tweak it and change it. And like, CTs are amazing for readable SQL and like, I'll indent my SQL like, I will my code that is laying it's like, like, writing readable SQL. And people are like, Oh, actually, this doesn't have to be what I thought about, which is like, three joins on one line, like what's happening, and you just cry, right? Yeah, no,
Carlton Gibson 1:02:52
entirely. I think that was that's why we ended up in this position where it's like, no, use the ORM. Don't use SQL because you'd have basically Junior mid level coders writing hand writing the worst sequel in the world. And then you come and you'd need to maintain that and it was unmaintainable. And so it was like no, no with it, the correct thing to do is to use the ORM and the abstractions it provides. But there's another there's another use cases, another category of things, which isn't like right in the middle of your application code, you're slapping some horrible, so clearly, they
Craig Kerstiens 1:03:25
should have like some block forget, it's like, thought it was some like, horribly executed joke about like, like my, like my sequel, like, like how I like my, not my SQL, but like, How dare my, like, queries and it's like, some basics of indentation and CTE. So two things will make all the difference.
Will Vincent 1:03:49
Yeah. Okay, super. Anything else, Carlton? I think we're well,
Carlton Gibson 1:03:53
I was just gonna say, do you remember Postgres weekly?
Craig Kerstiens 1:03:57
I helped start it. I do a very, very, very little bit for it. Now. It's Peter Cooper does all the heavy lifting. It's kind of like, early on, it was you know, hey, there wasn't now that you've gotten it going, right. It's kind of like crazy. It's easy to discover the things people tell us that it's there. There's there's clear kind of outlets and mediums way back then it was like, Where is Postgres content? So I did a lot more curation in the early days. Now, it's a lot less and a lot of Peter Cooper, but like, yeah, it was years ago, we were both speaking at a conference. And I'm like, so Postgres weekly, are you going to do it? And he's like, because he had the domain. And he's like, but I don't know anything about Postgres. And I can't curate it myself. I'm like, Oh, he's like, I got all the mechanics in place. I'm like, This sounds great. So I did start curating it. And like, he's got an awesome backend system that manages it ships it out. Like it's so it's great. So it's, I highly recommend it. Uh, you know, I tweet a lot about Postgres. There's the crunchy data newsletter. Postgres weekly, a number like if you're, hey, I want to dig in a little bit in Postgres Weekly has always been targeted more at app devs. Like people think, oh, a database newsletter, that's we're not targeting DBAs if you want to go like deep on like Postgres stuff during the postgres mailing list, it's a great resource, the hackers, that Postgres hackers mailing list, if you want to learn about databases, it's a wonderful place to lurk. But if you're just hey, I'm at Django Dev, and I just kind of want to know about the cool new things that I can take advantage of Postgres weekly is an awesome resource. Yeah, no,
Carlton Gibson 1:05:33
I would I subscribe to that I have done for a number of years. And it's like my little window into the postgres world night I truly appreciate. So I want to say thank you, to you and to Peter for doing that. Because it's important.
Will Vincent 1:05:42
All right. I think we're, we're at time. So I'll say thank you again, Greg, this was, this is great. This is really interesting to hear all these things. And I, you know, I think my biggest takeaway is your your point about the next generation of hosts, like, nothing's gonna replace Heroku it's gonna be like, it's enough to do a database. Well, it's enough to do a pass. Well, like that. makes all the sense in the world, even just talking to you about some of the challenges and things that you see. So I'm gonna have to think about that as I changed my recommendations to people in my various books.
Craig Kerstiens 1:06:13
Yeah, I think it'll make the world a little messier. But I think it's like it is probably the right thing for a little while. Cool. Thanks for having me. It was fun.
Carlton Gibson 1:06:19
No, thank you for coming. A real honor to have you on and we'd like to have a chat with you. It's Super, thank you.
Will Vincent 1:06:25
So we have links to everything crunchy, Craig's personal site. And thank you, Craig. Thank you, Carlton. We'll see everyone next time. Bye bye. Take care. Bye bye.