Django Chat

Django's Async Future - Tom Christie

Episode Summary

Tom is the creator of Django REST Framework, HTTPX, and a whole suite of new async Python web stack packages.

Episode Notes

Episode Transcription

Carlton Gibson  0:05  

Hi, and welcome to another episode of Django chat weekly podcast on the Django web framework. I'm Carlton Gibson joined as ever by Will Vincent, how are you? I'm great. How are

 

Will Vincent  0:13  

you, Carlton?

 

Carlton Gibson  0:14  

I am marvelous. And today we're joined by a very special guest, Tom Christie, creator of Django rest framework. And well, you know who he is. Hey, Tom, how you doing?

 

Will Vincent  0:24  

Oh, pleasure. Thanks for coming on. There's so many things to talk about. Let's just as a broad structure, I gotta ask you about Django rest framework, and then really want to talk about encode and async and the future of all that. But maybe before we get to that, so how did you learn how to code because you're doing some really cool, almost like bare metal programming type stuff with a sink and I know you you have a formal background in computer science. Is that right?

 

Tom Christie  0:51  

Yes, yeah.

 

So I started out the way I guess quite a lot people People who are my age who are in the programming industry did which was writing games for zedeck spectrum and doing a bit of, you know, basic programming there and also on on whatever think, you know, the cube basic as well, wasn't it? That was the windows dos one, I think came with snakes. And something else gorillas was another one. Okay, both of you

 

Carlton Gibson  1:30  

know, good. gorillas is where you threw the bananas, right? And you had to throw the throw up and

 

Tom Christie  1:36  

hit the building with the source code. So you could go and have a look and then try and figure out how do I turn that into our flying triangle instead? Or

 

Will Vincent  1:45  

Yeah, I thought you're gonna talk about we had Nicolas, tell me on talking about the what's Carlton what's that British thing that

 

Carlton Gibson  1:52  

you all the BBC, the BBC Micro.

 

Will Vincent  1:55  

Did you did you play around with that as well, Tom, because in the States, we didn't have anything like that. Now. No, no, that was not

 

Tom Christie  2:01  

a thing. But I but

 

you know, it was kind of by accident, trying to figure out well, what are you going to do once you get to leaving school age and just ended up doing a computer science degree and I've worked in a few different areas kind of since then. So I ended up moving into when I'd finished my degree, I then went and did a master's in computer speech and language processing. It just seemed like an interesting area. I was really interested in artificial intelligence at the time. So I went and did that. And then worked in that area of the industry for a while. And then, like most people who were on that small Master's Course with me, we eventually all ended up moving out of speech recognition. Because it was such a it was such it's such a new suit of mine. Well, yeah. Like in terms of where the money was at. Yeah, it was too soon because it was right before, you know, that was all before Siri came along and you know, a big boom and all of that. But even so, I think it's always quite a difficult space to work in a way because it's only really with the mega corporations. And then you have this little research team and it's such a specialized area that you're working in.

 

Personally, I found it quite frustrating eventually. So

 

Will Vincent  3:37  

well, there is an i sorry, I just because I know that with current options there's for doing audio to text one of the few non mega corpse is speechmatics, which is I don't think a mega Corp was based in the UK, but maybe they're bigger than I think they are. In terms of

 

Tom Christie  3:54  

I don't, I don't I don't follow that anymore. So you

 

Will Vincent  3:57  

played around with transcribing our podcasts and it's like, it's like Google Amazon, you know, Microsoft, and then there's like a UK one. And they had a nice drag and drop option, not, you know, configure an API to test it out. And yeah, sorry, I interrupted, please. Okay.

 

Tom Christie  4:14  

And then ended up slightly by chance really, but ending up working in networking. So worked with a content distribution network who were doing stuff with peer to peer, and building caches to live inside ISP is in order to save them traffic on all the peer to peer stuff that was going through their networks. Did that for a while, and then eventually, at the same time, as I moved to Brighton, decided, Okay, I just want to have a complete change of stuff and I would like to be able to, again, with both of those things, they were really, really Kind of. But they're very deep, technical kind of areas. But I didn't know how to build my own website, right? Or my, you know, or just do those sorts of things and that freedom to be able to okay, right, I know the basics of how to put stuff together and build stuff that I can do with and eventually, you know, build up the skills that you can use to potentially build a company or do something like that, rather than,

 

you know, the back end stuff. And when I started,

 

you know, when I started kind of working in the web industry, that was, you know, I guess about a year later was when I started working on rest framework.

 

Carlton Gibson  5:46  

Right. And so did you find Django at that point?

 

Tom Christie  5:49  

Yes, I guess that the company that I was doing that I was working with at the time must have been using it. No, actually, I think they weren't. I think they were using they were using Java. And that was massively frustrating.

 

But yes, it sounds

 

Carlton Gibson  6:05  

so we're looking for jank

 

Tom Christie  6:08  

into look, we should do more of this stuff because that stuff's really painful for me.

 

Will Vincent  6:14  

Well, and listen, the natural language things was that in Python, because most of it is now.

 

Tom Christie  6:20  

Um, so pipe,

 

Will Vincent  6:24  

hippos Python around on your on your degree.

 

Tom Christie  6:26  

It certainly wasn't very big on the scene, right? And I've struggled to remember at what point I switched over from bash scripting and parallel to Oh, wait a minute. This is really, really helpful. Like, yes is much but I think it probably was, yeah, I think it. I think it probably was after I'd left the speech recognition stuff, because I can remember like, some of the more impressive things that I did when I was there were very complicated as a batch script. strung together. Because for all of the, with all this stuff where you're dealing with court large corpuses of audio, you need to do so much wrangling basic wrangling with them. You guys got that stuff? And then you've got your, like very detailed his my bit of see that, you know, that actually does the signal processing and pulls the waveform out into a set of different signals that because

 

Carlton Gibson  7:33  

yeah, that's still the case, right? Because a lot of the scientific Python libraries extensions around the underlying c implementations and stuff. So it's not necessarily changed at the implementation level, even now, today,

 

Will Vincent  7:44  

possibly. So what will that background actually that makes a lot of sense. I was just saying before we recorded your your talk at Django con Europe this year, on async and future of Django was fantastic and you did a really good job of breaking down going a level deeper level deeper on The layers of abstraction that I think is often assumed because people work at their own level. But given that you've jumped about a whole bunch of those levels professionally, it makes sense how you would be able to wrap it up together, you sort of kept gone going up the abstraction layers in a way to, you know, working with web stuff. But anyways, we'll link to that talk I think for for people interested in async. You're talking to Andrews. Godwin's talk, which we'll link to from Django con this year, I think are the the two places I point people to? So I want to dive into that stuff. But did you what else with your story, so you were doing web stuff in at some point, you did? Django rest framework and then encode, right, should we finish the

 

Tom Christie  8:41  

professional? Yeah, so what's the origin was the DRF origin story was okay. Yeah, sure. So, what happened was basically there's this one big motivating factor, I have no idea kind of where it came out of, but there was, the idea was, Why on earth is there Out of all these API's that we're building, none of them are web browsable. Why on earth is it that when we're having to interact with our web API's, we're having to do that from the command line, rather than just being able to point a web browser at it. I HDCP is designed with content negotiation so that you can serve different representations to different kinds of clients, so that a programmatic client could get Jason but a web browser could request HTML instead. And I had a couple of ideas about how you would do it because there was there were things that made it awkward to do so for instance, browsers not being able to post Jason inside one of their regular forms. I mean, this was a way back. So even where JavaScript was, was a bit

 

Carlton Gibson  9:54  

different than as well. Even sending non POST requests you know to send a put or delete You know, the HTTP verbs.

 

Tom Christie  10:01  

They malfeasance themselves, don't natively support it. Right? And I had a couple of ideas about, well, look here, here are different ways that you could get around that. And it seemed to me such an obvious I'm such a beneficial idea that the Web API should be web browsable. That, you know, I looked into how can I use one of the existing frameworks, that's the time to do this? And eventually went, No, I don't think so. Let's just, you know, let's build let's build a proof of concept first, and show that you can do this thing. And then it just kind of snowballed from there. You know, it was the big like, that was kind of always the big motivation because I thought it was such a, you know, I thought was an important idea. I ended up wanting to pile more and more time into improving the framework, so I could show them People how I thought we ought to be interacting with API's.

 

Carlton Gibson  11:03  

And the browsable API is still like, you know, I don't know how many years later now, 789 years later, perhaps, the browsable API is still really cool. You know, you fire up a new API, and you go to it, and it's like, oh, yeah, great. And okay, at some point, you know, you get nested serializers. And all right, it doesn't handle loads. But for get up and running, it's like, oh, wow, this is super, and I don't have to crack open the command line. And I can play around, I can create some models. Still, it's, you know, still an awesome thing.

 

Tom Christie  11:28  

Yeah. I mean, obviously, you know, I think there are nicer things that we could be doing with it. But time and time and effort. Yes.

 

Will Vincent  11:38  

Well, there's me too great for I mean, I think of a lot is the admin for Django itself, where it just works. It's fantastic. And you get it out of the box. And then if you need more, you can use postman or something. Same way as you know, you shouldn't force her Django admin to do too much. But it's just that first time experience of like, you can see it and it works. You know, that's really really powerful. Oh, mean so for example, like in my, my book, my book on Django API's, I mean, it's all using the built in web views. And I say, you know, you can go to postman you can do all this other stuff But really most of what you need especially when you starting out it's just the first time first time I saw visually those things actually I think that's the case where many people they they you know, because the command line just doesn't do it.

 

Tom Christie  12:22  

Yeah. Well having having the immediacy and the accessibility of it

 

Will Vincent  12:25  

but that's interesting almost work backwards. So you you started with that idea rather than saying, you know, tasty pie or twisted or the other ones you know, the serializers were the issue you almost sort of started with the end goal there is that

 

Tom Christie  12:40  

Yeah, yeah, definitely. Yeah, it was, it was tasty pie and piston at the time,

 

Carlton Gibson  12:48  

I use tasty pie and if you use piston, I should

 

Tom Christie  12:50  

I will I should have done what Jesper? No the the author of piston date, which is found bit bucket and then sell that to Atlassian for Notice how much that's why he's

 

Carlton Gibson  13:06  

still sitting on it.

 

Will Vincent  13:11  

Wow, well, so um, so Django rest framework. So 311 is coming out? Well, we'll be out by the time this is released, where it's so feature complete as is where do you see? What do you see is the future for that project? And then we'll get into all these other projects. Like, I can't necessarily think of what more needs beyond polishing but you know, YouTube wrote and maintain it. So yeah. So

 

Tom Christie  13:36  

at the at the moment, I would say it's just in its mature phase. Now, if the if that is another big jump, I think that, to me, the obvious place would be, so we have the we have an admin like view in there. At the moment, right, but it's not great. It's not that great. And I think like the obvious place that you could put some improvements in would be making it so that rather than the browsable API, defaulting to look like, here's a bunch of JSON. And here's some form fields, it's like, well, here's your tab for the JSON. And here's your tab for, I want to see this data displayed in the table and be able to paginate through it with pagination controls and so on. So, yeah, if there's another, like big jump to happen there, I think that's probably where I'd be most interested in putting time in.

 

Carlton Gibson  14:41  

That's kind of interesting.

 

Tom Christie  14:42  

I mean, there's also you know, there's also I guess, yeah, that's, that's where I'd be most interested, right?

 

Carlton Gibson  14:49  

Because there's so many use uses DRF. for building projects in the commercial environment have done you know, Sal, I found it I was building iOS apps and I needed a back end with an API, no nudity. Go and I was playing with tasty pie. And I was like, This isn't working for me. And I found the RF and I was like, this is working for me. For me still in that still thinking about that environment, it's the it's the integration with the open API tooling, the schemers the client generations and all that stuff. And that can't be part of rest framework, really. But integrating with that ecosystem and making that more full fetch full, fully fleshed out, I think that that's where the growth is for the over the next short term period. Because I think that's, you know, as a jobbing programmer, that's what you need, you know, you need your API quickly, but you also need all these client libraries. And if you can produce decent docs, and, you know, code examples, and, you know, and all that stuff,

 

Tom Christie  15:44  

the other area that I'm particularly interested in as well as you know, at the moment, we have that what you have if you if you're coming to a new project, and you want to build an API, you've got two different slices of option and one of your options is you uses a completely house hosted service something like Firebase, or something like that, and you build within their service. And you have to be happy to kind of live within the set of constraints that they provided and know that you're going to stay on their platform indefinitely. Or you go the other route. And you work with a framework like Django or rails, I think, a really nice space. That's somewhere in the middle, which is, you know, what's the easiest way to get put, you know, express it like this, like, what's the easiest way to get started with Jango? Oh, well start building your admin and your API out in this hosted service here. And you can put some, you know, you can start putting data in and so on, and you can start interacting with it as an API. And when you're ready, here's this big green, let's go button and you hit that and you've got your code and you're you're ready to start running. That so that's another area that I'm interested in working in.

 

Carlton Gibson  17:03  

So the other big project that was around right at the beginning was make dogs right which is, which is an incredible that even exists because Sphinx was the big beast in the room and you create this other thing was it is my burning question. Was it just that you hated restructured text so much that you wanted to use markdown? Or

 

Will Vincent  17:20  

was there another?

 

Tom Christie  17:21  

No, it's just showed working with markdown so much.

 

You know, in the tool, and the tooling for markdown was a lot better, you know, as well, like, there's so many markdown editors and yeah, still is right and support from markdown came to GitHub before support for restructured text and, you know, all these sorts of things.

 

And yeah, it's, you know, it's a trade off. Obviously, like, you know, it's still all these years later, it doesn't have interlinking

 

just pushed out a package that does

 

You know, you pointed out your Python package and you can you can inject your, your doc strings into your, you know, your markdown.

 

Carlton Gibson  18:13  

Your price is a wrapper around ation around doc utils.

 

Tom Christie  18:19  

is a it's an it's a mock docs equivalent of is it auto doc? Right? This is equivalent of auto doc. But for mock docs,

 

Carlton Gibson  18:32  

so be like creating API documentation.

 

Tom Christie  18:36  

Placing API documentation within your pros documentation. So that came out last month. And we're using that so far with HTTP x. I can't remember if I'm using it in any of the other projects. Yeah.

 

Carlton Gibson  18:55  

Okay. What's that called? For our listeners? We'll put it in the show notes.

 

Tom Christie  19:01  

Automatic dogs maybe?

 

Carlton Gibson  19:05  

Okay, we'll find it. We'll find it in the show logs. Now.

 

Tom Christie  19:07  

I'm using it now. So I forgot. We just

 

Carlton Gibson  19:12  

abstract it away. That's pretty cool. So you've just said the big exciting thing as well that you've mentioned HTTP. So tell us what that is.

 

Tom Christie  19:19  

Yeah. So actually, I'll, I'll start from what was the motivation? And then I'll kind of come to where it is now, what what it is now. So the gap that was missing was? Well, two things in particular number one, how do you, you know, what's the async HTTP clients? for Python. So you know, with requests, you're using the threat concurrency model, and you're generally only able Well, you're only able to open as many outgoing HTTP CP requests as you have threads running, where as if you're using the asynchronous model, so and that might be, you know, say maybe 10 or, or 20 concurrent requests. Whereas in the asynchronous model, the mate the process of making a request, doesn't block all of the other little async tasks that are off running concurrently alongside your other one thread of control that you're looking at right now. And you're able to serve thousands of concurrent HTTP requests. Now, the big question was, what we've got requests at the moment. But how do we bring async functionality to requests? So I put a bunch of time into trying to look at what would be a good way of doing that. And eventually that evolved into you know what? Let's build a complete HTTP client from scratch all the way through, because we need to build one that is async, capable of front and center. And also, you know, like we're in a different landscape now than when request started. So let's start by building an HTTP client that uses type annotations all the way through. So it's very, very precise about exactly what the API is. And another big thing that it's started to provide is HTTP two supports. Right? So there's, there's one other kind of halfway incomplete, Python HTTP client that supports HTTP two. But there wasn't really anything and fully mature that had that in place. So the two big things were let's look at async support and let's look at HTTP two. Support. What it's evolved into is a an HTTP client that is largely requests compatible. So wherever possible, the API meets up with requests.

 

Carlton Gibson  22:18  

So I can come along as a longtime request user, I can get my input HTTP x, I can do my get and it will be more or less the same.

 

Tom Christie  22:25  

Yeah, that that supports async. Right now, that means two things. So we support async. io, but we've also just added properly support for the trio async libraries as well. We have a little sidestep, what's the difference there?

 

Carlton Gibson  22:44  

async IO versus trio. Let's go, you know, in 10 words, what's the difference?

 

Tom Christie  22:50  

So, so trio is an alternate implementation instead of async. io and all like the event loop. The big thing is, is that Yes, the tree Oh, people were bashing over the head because it doesn't use event loops or something but and it had. Essentially it's a more thoroughly designed, it's got this really nice core tenants called structured concurrency, which doesn't just apply to Treo, but as a general idiom for that. That they've kind of discovered for Look, here's a sensible way to deal with concurrency without having callbacks. So all over the place without potentially leaking tasks that you forget about having, you know, well defined semantics for what happens when tasks are canceled and so on. It's a really thoroughly designed alternative to async IO. It's a bit awkward because the two things are completely incompatible, but from the point of view, of HTTP acts, it's largely irrelevant. We've got these two tiny little backends, one for async. io and one for trio. And we're able to support both of those.

 

Yeah, both of those different platforms.

 

The plan is what we did have, we did have sync and async support for HTTP x. momentarily, I've dropped out the sync support, because we needed to re engineer the way that that works. Previously, the think code was kind of bridging on to the async code. So it would seem to the user like you're running synchronous code, but actually, you've got an event loop running under the hood and async code running underneath. And we're actually switching that round so that instead, the sync version of the client will be a pure sync version of the client and the agent will be a pure async one. And the folks working on worked on URL lib three, which is the thing that powers requests, and some really smart ideas about how to do that. So we've been working on that basis,

 

Carlton Gibson  25:16  

right. So but and I was lucky, I've followed that conversation always a bit. As well, I don't drop the sync support, because I just want to be able to pull in HTTP x is my replacement. But having looked at it async IO has this nice function called run, which you can just call from a synchronous context. And you can pass it your co routine, which will be your HTTP x gets. Yeah, and it will, it will go on without you having to pull up your event loop and run the event loop yourself. It will, which is only a few lines, but it'll do all that for you. So yeah,

 

Tom Christie  25:45  

well, it's not importantly, the thing support is coming back. Yeah, and I mean, I don't know. When's the podcast coming out?

 

Will Vincent  25:51  

Well, when do you want to come out? Yeah.

 

Tom Christie  25:55  

Anyway, hopefully it's already back in. Right? Yeah. By the time you This folks think and I say

 

that additionally is about blah, blah, blah.

 

Carlton Gibson  26:05  

Right. So hang on before we before we lose the thread here. You just mentioned you earlier right now. So one thing that I was sort of under the impression that the requests, one of the things about PCOS was it was built on kind of some older tech, which wasn't quite as robust. And then the underlying networking layer is kind of be redone. And URL lib three as part of that. But all of that's a bit vague to me, because I haven't dug into it and then researched it. Could you What's going on there? Like what's, you know, URL lib, three? Is that is that what we need to be using? And is is and what's going on? How would you describe that stack? Has it been rewritten? Or is it so

 

Tom Christie  26:40  

the URL lib three team are also working on a new version of that which brings synchronous and asynchronous supports.

 

Carlton Gibson  26:54  

Now that's the

 

Tom Christie  26:56  

only what I would like to do is an IDE Don't think is that difficult is provide the ability for HTTP acts to be used with that, once that's available. I mean, we've already implemented a slice of it as well ourselves. But that's the mature battle tested battle scars, you know, deals with all these different edge cases that they've hit in the wild when service costs attorney way is. So whereas we've got this kind of this is clean HTTP works like this. This is what we expect version, it would be really, really beneficial to kind of have both those options on the table and then to figure out where we're going from there.

 

Carlton Gibson  27:51  

So that's still a work in progress. Right. But yeah, that's kind of underlying level that that layer, underlying layer is being

 

Tom Christie  27:58  

Yeah, yeah, that's clean. Working in the in the exact same space as us and a whole bunch of the work that they've done. So for example, their approach on to how to build code that is both sync and async is what we're now tacking for in HTTP x, because it's smarter than the approach we have before it's gonna work out better for the project.

 

Will Vincent  28:22  

Well, I think one of the things I took away from from your talk as well, Tom, for people listening who aren't, are new to this space is that async requires rebuilding all the layers in the stack, which is sort of why, you know, why is the rest framework person working on HTTP, but it's because you sort of have to rebuild everything this async world.

 

Tom Christie  28:44  

Yes, but I, you know, that's, and that's certainly the motivation, but what I hope as well is,

 

you know, as mature and great as requests is, I think it'd be really great to have a new HTTP clients that does all the same slicer stuff as it because I'm really pleased with the work that we've done on HTTP x, like the design is, is really nice. And I think folks kind of coming to it now and digging into the code base will be able to get a handle on. Oh, right. Okay. I see how, I mean, that's kind of true for requests as well, actually. But, you know, we've kind of been stuck in this place where there's just this one entity that has backed onto your old three, it would be nice for us to freshen up and all that again,

 

Carlton Gibson  29:35  

and it's stable, right? HTTP sec, because it's like naught point eight. Now, it will stay stable if you Pinta? No, but what I mean is it's stable isn't the right word. It's the word I mean, like it's fully functional, fully. You could put it into a project does everything I need is like is the naught point what does the naught point what's left for one point higher than

 

Tom Christie  29:57  

the base, so you should pay To a median release version, so you should pin to naught point eight or naught point nine, you know, naught point nine point x or naught point, one x. And the big thing that will still change is once we add sync support back in, then you've got to go, Well, what are we calling the async? client? And what are we calling the sync client? And what are we calling async requests and model calling sync requests

 

Carlton Gibson  30:29  

to maybe an import changes?

 

Tom Christie  30:31  

Yes, you've got a namespace here, and you've got to name it, but there's not actually a massive amount more technical work required to get there. And once it hits 1.0 that's when we'll have the right here's our formal deprecation policy. Here's what you can expect to happen once we upgrade to version And he is time or so.

 

Will Vincent  31:03  

All that stuff. So I really want to ask you about starlet. Oh, cool. So let's just assume so ASCII there async was lip service gateway interface. We could talk about that. But let's assume that exists starlets really interesting to me. And you've also done some work building out. You've been putting the code, like a new demo project with it, I believe. So this is, let me tell you what I think it is. And you can correct me. So this is you're building out. So lightweight framework using ASCII, that the ideas of which could eventually roll into a larger thing like Django itself. Is that how would you describe the motivations because I look at it as like, it's really educational for me, but you have so many projects and sometimes I'm like, wait, what is Tom? Like, you've got like a dozen active projects that are all cool and all

 

Tom Christie  31:52  

I've been trying to keep that more and more neatly correct.

 

Will Vincent  31:58  

What do you I mean, what you I mean, Django rest framework HTTP acts as G. unicorn API star like,

 

Tom Christie  32:07  

yeah, I mean, there's kind of, there's kind of three projects. Okay, so there's Django rest framework as HTTP x. And there's the async web stack. async web stack includes, well, we need an async server. So here's unicorn. That's your async web server, but we need an async web framework. So his starlet, we need a way of interacting with async database drivers. That's a nice level. So here's the databases package. We need an ORM eventually, I mean, so do RM package. You know, that's kind of more on my horizon, but kind of got the basics in there. So there's that bit. Um, and yeah, and one or two other bits and pieces around the periphery there.

 

Will Vincent  33:11  

And then Carlton, what's the schedule for Django three releases for how these things are added in right? Three One is going to have or three zero has ASCII three one is,

 

Carlton Gibson  33:19  

yeah, okay, so we so we've just released 3.0. And that that can run under an ASCII web server. So unicorn or Daphne or the other one that I can remember hyper con. So you can embed and what's useful about that at this stage is you can embed Django next to your data set application or, you know, next to your style application. It's not actually async views. So hopefully 3.1 will have async view capacity and for there, you know, from from the Django perspective, Tom's work is like breaking new ground and it's showing the way to go and it's like, it's just amazing and Tom's taught from Django con Europe when it was called sketching out a Django redesign, and he spent the entire time talking about styling, not the entire time. In a basic was talking about standard, but how else do you show what Django could become looking at other than showing what an async web framework looks like. And that's, you know, from the Django perspective, it's like, it's amazing that this stuff's here, that the work that Tom's doing and seeing it and we can take some ideas and we're about to take it all and because Django has got a lot of history, and it's always gonna be, it's Django and it's gonna be backwards compatible, and it you know, so it's a really exciting area, and there's plenty of room for styling and to rob all the good ideas, or as many ideas we can and we won't be able to rob them because Django Django and it's got a history and it needs it needs to maintain that. Yeah, 3.1 phrasing views which is

 

Tom Christie  34:42  

feeling really excited, really nice has been being able to see how once we've got a standardized approach, ASCII, yeah. And I Andrew Godwin's designed with enable able to go off and I can go off and do a whole bunch of work in This kind of completely different approach and different space and he can go off and do. Right. How do we start to bring this into Django? And they end up being complimentary,

 

Carlton Gibson  35:11  

right? Yeah. Okay. And we mix and match,

 

Tom Christie  35:13  

right? Yeah. Yeah. So for example, like we've got the baseline functionality in Django 3.0 of you can. You can be running in a sink, right at the very bottom level. Well, what that means, I mean, I haven't dug into it in detail, but you'd be able to do some neat, you know, not stuff that you'll need to do all the time. But the idea is like, Okay, well, we have this one particular, this one particular case where we want this path to always just proxy requests off to some other service. And you could add in as gay middleware to let you do that.

 

Carlton Gibson  35:55  

You're using HTTP X to make the request.

 

Tom Christie  35:58  

Yeah. Before it, there's it just it starts to give you opportunities. But coming back to what Will asked earlier actually because kind of crystallized the answer to that in my mind, he said, Well, what what is starlet? I'll describe it. What do you think? Well, actually what styler is essentially how words, I start to go about designing the fundamentals of something like Django, if I was coming to it fresh now from where we are now in a landscape. So that's not to say it's any, you know, of course, it's not remotely at that level, but that's because, right, let's put these foundations here, let's put those foundations there. Let's do this bit. Let's do that bit. And, you know, because when I, especially when I started working in the async space, you know, did things like okay, well, I've got you have a call and and that's built here. But my god, you know, trying to build some production service on top of that, like, what are all the other bits of work I'd have to do to do that it seems immense and undoable, and I don't want to have to work with these async database drivers. It's not my core expertise. I don't understand how to do that. But gradually, as we fill in more and more of the blanks, you know, you're gonna start to see, you know, more and more mature framework ecosystem that

 

Carlton Gibson  37:34  

Yeah, and it's about providing the abstraction. So you don't want to handle ASCII send and receive handlers you want. A handler takes a request, and you send back a response and you want the framework to do the send and receive to ASCII, and you don't want to look. Yeah, yeah, that kind of thing.

 

Tom Christie  37:49  

Yes, yeah. Yeah. And yeah, the way that it's designed is it's kind of designed to be to be very lightweight, but without sacrificing The kinds of abstractions that you want to be working with at the framework level.

 

Carlton Gibson  38:05  

Yeah, and one thing you've talked about a lot is the importance of all this stuff, because we talk about async. And it's all very, you know, but why does it matter? How important is this to Python and Django? What's the space

 

Tom Christie  38:18  

where it's important, particularly, is pythons position within the landscape, right? So two of its biggest competitors in particular, would be JavaScript, and go. And both of those, when compared to the synchronous Python frameworks, are able to handle vastly higher levels of throughputs. So you know, for each server that you're running, you're able to handle a certain number of active users. Now. That doesn't In a vast swathes of businesses, that doesn't always matter. But for the heavy hitters it does, and, and also, you know, a lot of businesses that that they're starting out and they're hoping Well, one day, we want this to be, you know, and we don't want to have started out in a technology that we then find we're fighting against the currents with. And the intention here is to build up the kind of framework where you've got all of the wonderful productivity benefits that working with Python brings without having to make the case of it doesn't matter that it's less, you know, able to handle vast amounts of loads than JavaScript without having to make that argument, being able to say it's basically equivalent and Of course not forego, but the goalposts are shifted that much closer that it's not gonna matter to your teams.

 

Carlton Gibson  40:08  

Yeah, if you can get if you can get 95% performance. You know, that's probably good enough.

 

Will Vincent  40:13  

So anecdotally here in Boston, I hear there's a fair number of enterprise Java companies stacks that, you know, they're they're switching over to new technologies, in part because they can't hire Java developers because no one coming out of school knows Java. And so they're looking at Yeah, those three Python JavaScript go is the modern current workforce can use. And maybe because I flew, I flew back from pi con on the plane with a whole bunch of these people saying I was like, Oh, why using Python just for recruiting? So it's almost right either like, but you know, of those are the three kind of modern I need something high performing one really, yeah, people would say go and then JavaScript is mangled Siamese. It's an interesting perspective. And I think it's right that you can just check that box like again in your talk. You've had great, he showed a bunch of benchmarks and kind of why they're not the standard benchmarks people trot out don't really matter, or are, are sort of incomplete, right? I mean, Carlton and I often talk in this podcast about how Python is not with the programming language is not going to be the thing that slows down your web stack anyway. But even if you're looking language, the language, it's easy to look at one of those and think Python slow, but it's really so

 

Tom Christie  41:25  

yeah, so let's talk about some other aspects of it as well though, right, because WebSocket support, right, for instance, that's one of the big now Django has that with Jango channels. But it's had to be kind of designed, you know, into an existence that whereas with, you have a call and starlet, were able to start from a starting point of great we want to be able to handle long lived connections such as WebSockets, or streaming HTTP requests. Now, I know there's not necessarily loads and loads of good kind of tutorial level examples yet of look at the interesting functionality that you can start to easily build out with using these stacks. But that's going to start to come. And I think that when it doesn't, when you start to be able to show, look, here's a production ready framework that you can start working with now and look at those sorts of real time things that you can start to build with it. I think that will be interesting. And the and the other thing is forgetting about the async at all, but just stall it in its own right, I think is really interesting because the, you know, having had 10 years of working with Django and right How would I approach this from scratch just they The way that it's stripped everything right down. You know, it's easy to say I think that people will kind of get a feel for it when they've been working with it for a bit. But it's a very, very low depth of complexity stack. Right. And that's something that I can say nebulously, and it's difficult to. difficult to quantify. But it's a, you know, I think it's a really elegant design in and I think that as the space on top of that starts to mature, that's going to kind of become apparent in the way that it allows you to kind of, you know, the framework ends up having a low maintenance cost. So you can work on new things quickly. You can adapt new things quickly. If you get things wrong, it's easy to move stuff around. Versus, okay, cool. We're supporting you know, versus john goes completely different. thing, which is supporting 10s of thousands of businesses. And here's all the things we're carrying along with us. We do a fantastic job of continuing to evolve and support them. But it's also worth putting time into. Great. Here's this new sprinter on the scene.

 

Let's work with that for a bit. Yeah,

 

Carlton Gibson  44:24  

uh, yeah, no, totally, totally, totally. And like Django has got 10 years of edge cases to, you know, to accommodate and to maintain and rightfully so. I always say one of the reason why it's still exciting is because we don't break stuff. So you know that it's reliable. No, but you can use the new version, you can update it, and you can use the new features because you don't have you know, I always argued that unless you unless you're going to really abandon your app. Don't be on the LTS be on the latest major version and get the new feature and have fun with it. I mean, yeah, you've got an update every nine months. But that's just like taking taking your car to the garriage

 

Will Vincent  44:57  

was a hosted API is the name of the the demo app that you built out. And I guess I, I want to mention to people listening if they're more beginners Jacob Kaplan moss, one of the chango creators, his he had a PI con talk 2017, which will link to on if he were going to build Django today from scratch, what would he redo? Because I think when, at least when I try to explain to, but like I work in a co working space with a whole bunch of people using Django who aren't developers. So they're managing developers. And when I tried to describe async, I see their eyes glaze over. And I think part of it is they don't they don't understand how the pieces what is the stack, what is a web framework. So you know, what is whiskey? How does all these things stack together? So yeah, I think of this as an educator, I want to write some courses on a sink, but I think it really has to start with, well, here's all the things you were using. You don't have to think about but like, here's what the stack is, here are the choices. Because I think until you can understand why a sink beyond a real time sounds nice. You have to see where what you're dealing with and you people haven't.

 

Tom Christie  46:01  

See, I think the thing is the reason that we're having to talk about the technical aspects is because it's still in mature. Yeah, right. And what I've been working on really hard is, you know, hosted API is just like kind of the very right, let me figure out what we're still missing is until you can get to the point where you can put your technology in front of people who really don't know an awful lot about the space and say, hey, look at this cool thing. Isn't this what your business wants to be able to do? Or to be able to build or, you know, to have your team work with? You know, and I'm hoping to be able to get into that much more user facing space.

 

Will Vincent  46:46  

What What do you think that example is right? Because I think of well, maybe it's like what is you know, something you build the sink way and the async way, and you can just show it's massively more performant is there like it I mean, it's like, oh, chat seems a little bit faster, like, what would be a good, you know, killer example, right? Like, what's the like equivalent of like?

 

Tom Christie  47:07  

So a real time admin, right? So a real time admin, plus, oh, and look, here's a bit of JavaScript that we've written on the other side. And here's a website, where we've got these various kind of graphs that are updating, and we can see when we put, you know, either, when we're putting data into the admin, we can see that stuff updating over here.

 

Will Vincent  47:31  

The kind of hosted API is

 

Tom Christie  47:34  

no, you know, maybe one day I'm skirting around, you know, bigger plasm. Or here's a simpler example. Right? So being able to when you when you start up Django, right, and you get your little welcome screen, and then you've added in the admin and then you get the admin what will be, will be really nice in async framework lands will be one thing you started your new project and you go to a screen, you can see, like a real time graph of how many requests per second, are we running at the moment, and it's updating all the time, what's our throughput? How many servers are running and currently connected to the same channel at the moment to be able to see really responsive set of information like that. And have those be in your basic programmers toolbox. So you know, rather than Django debug toolbar is the thing is the kind of level that we're looking at, you know, responsive set of information that's coming back straight away about you know, those

 

Carlton Gibson  48:43  

sorts of things in in the in the Phoenix web framework, which is in which is built on the the elixir language, which is kind of like a functional thing built in the learning VM. They, they've got a thing, which is really the new hotness at the moment called light which they call Live View, which is exactly this kind of real time community. Back to the server and they can do you know, inline validation send back the response updates in line. And it's, it's like Ajax but Ajax plus, plus, plus, plus, plus, plus, plus, plus, because it's so quick and, you know, and really easy to implement. And they kind of got that nice and be nice if starlet had that be nice of Django and get can get there. And you know, for me, you you start, you've got to take a beginner, and you run them through and you create a Django model, and they start and they spin up the admin and they do imagine doing that, but just with with an async framework where, you know, they don't even know about async, they're just in there. And all of a sudden, it's like they're programming and they're learning to program but using this this really cool modern async stack.

 

Yeah, fantastic. Yeah.

 

Tom Christie  49:45  

One of the things that is difficult, though, of course, is trying to balance that, you know, trying to balance Django rest framework with. Okay, I think that the revitalizing the HTTP CP space in Python is really important. So here's HTTP x. Where, oh, I think that the async web space in Python is really important. So here's the async web stack, right? trying to balance those things and trying to make the case for you know, especially with in places where stuff is newer, trying to make the case for, yes, this is less mature. No, your business shouldn't necessarily be working on this yet, unless you know that you've got a good reason to. But yes, we think it's worth investing the time upfront on

 

Carlton Gibson  50:40  

and so we come to the final and perhaps most important topic is encode RSS and your work to build says the tagline was that the tank sustainable software development, right that's, that's how you

 

Tom Christie  50:52  

Yes, yeah.

 

Carlton Gibson  50:56  

It was one

 

Will Vincent  51:00  

collaboratively funded software development is what it is.

 

Tom Christie  51:02  

There you go. Yes. That that, that, that shines,

 

Carlton Gibson  51:06  

because like we've we've mentioned what a dozen packages they are f star H, unicorn, you know, HTTP. How on earth can you do it? And, you know,

 

Tom Christie  51:17  

well, yeah, and there's a there's a short answer to that is

 

my time is fully funded. I work on this stuff full time. Actually, I think it's interesting. Going back into the story of how that happens come about, because it's quite lucky, really. You know, Django rest framework to version two was when we very first ran a Kickstarter for it. And at the time, I was working with the web consultancy daps. And they were really on board with the idea of running a Kickstarter for this. So we kind of came to her an arrangement where we said, okay, we'll do a Kickstarter and You know we'll figure out for however much money that Kickstarter makes, here's how much of my time can be taken out of client projects and instead just be put to work on restaurant work. And we we run the Kickstarter and it was wildly successful you know for rest framework to I can't remember how many how long ago that was now,

 

Carlton Gibson  52:20  

but it was a way back because I would I got involved with breast framework before version two but it was like it was coming along and there was the Kickstarter and it was it was easy. Yeah.

 

Tom Christie  52:31  

And I think you know, the timing happens to be just right people could see you know, what we wanted to do it was kind of just about started emerging as you know, like the more mature like for a long time tasty pie had been overall, the kind of the slightly better choice in terms of if you just want to look at what features of age got, but the timing just happened to be just Right. And, you know, we got like 30,000 pounds from the Kickstarter and I worked on Rei framework to for I can't remember but you know, between six months and eight and nine months somewhere in that area, which was fantastic when we came to. Okay, here's a new slice of work that I think needs to happen and rest framework three.

 

Again, we got pretty lucky with the timing because Mozilla had just launched their grants scheme.

 

And I knew that although I wanted to run a big kind of initial fundraiser for it, what I actually really wanted was ongoing funding rather than a single Kickstarter style, drop, work, work work. Now we're done. And it wasn't really obvious to me that leaving my job would be a good idea. But because I'd applied to the mosque round the way it went was once that gronk you know, once they came back and said, Yep, here you go. Here's your grant, which I think was about 50, I think was $50,000. I think it was 30,000 pounds again. So that gave me this huge chunk of runway, you know, like best part of a year. And I said, Okay, like, tabs is a great place to work. But I can see an opportunity here. So I quit and launched the, the ongoing funding model. And I thought, well, we'll wait and see how many companies pitch in with this and what kind of level it gets to. And if it's 50% of what I need, then at least I'll know. You know, I'll have a year's worth of figuring out and Maybe I can go back to DevOps and work there and, you know, halftime or whatever. But as it happens, we just about got, we've just about got enough to keep me going full time on things. And sometimes we've had a bit more and sometimes Carlton's had contract time on it. Sometimes in the past, I've supplemented it with a little bit of contracting and so on. But on the whole, I'm trying not to do any of that at all, and just use the funded time. And, and the pitch that I, that I always go with, if I'm ever on stage talking about is, you know, I want to work for your company full time for $100 a month. Because that's, that's what we're, that's and it's so clear, then. Why collaboratively funded, open source development makes economic sense. Right. It's like an insurance and like makes business sense.

 

Carlton Gibson  56:03  

Yeah, to like one of these plans to service your boiler like, you know, you make a subscription to keep the fundamental infrastructure of your business, there isn't a there isn't a person listening to this podcast who doesn't use rest framework? Right? You know, if all those people find it a little bit of all those companies find it a little bit, then it's totally sustainable on the ongoing basis.

 

Will Vincent  56:24  

Yeah, I think it's easier for people to understand that DRF is, is you and a small team, whereas I mean, I'm constantly struck by Django is just this amorphous thing, even though it is half dozen dozen people who really do the yeoman's work. It's a little bit easier in a way when you can people can see Oh, it's it's going to Tom and his crew, right. It's not, it's a little bit of an easier ask because I think it was in the context of I'm constantly thinking for the Django Software Foundation. I think there's a lot of ways they could get more money which would help a lot of things But it's a little bit harder than when you're literally on stage saying, you know, I want to work for him or her dollars a month. And it's just like, man, if I'm using DRF for anything as a business, that's obviously a total no brainer. So

 

Tom Christie  57:13  

I think yeah, I think it's, I think it's really important that there's a very direct link between sponsorships and the person doing the work because I think it is a personal relationship. I mean, you know, I, I think there's probably more that we could be doing in that space as well. You know, so, for instance, things that are kind of toyed with recently as well. It would be great to get some more to get somebody else in with a bit of funded time on the project. Well, maybe the right sort of way to approach that is your you have a amenti a mentorship program. And you specifically asked for, you know, we want to run this program. And we we need three companies to sponsor it. Yeah, here's how much we expect you to buy. And then there's a very direct relationship, even though they can say, we're sponsoring this program. Right,

 

Will Vincent  58:18  

that I just said, I think that is what they want at a corporate level. They want to I mean, it's like giving to, you know, university, you want your name in the building versus just general fund, right, everyone's point to their specific thing. I mean, because I think again, in the in the Django context, I would posit that 95 99% of people have no idea who that a Jenga fellow exists, let alone that there are two, let alone that Carlton is one of them. I mean, it's just totally the DSF nobody knows what that is, because they don't really have to. So it's, it's I like your idea of direct relationship for mentee. I mean, what would just this just this year, the DSF with Jacob Kaplan moss is mentoring someone on Think what adding something to the funding page, they got 1000 people, you know, who wanted to be mentored? And that, and I think if you had a company that would sponsor that or something? Yeah, that's brilliant. That makes a lot of sense. It's like a badge that a company can talk about. And yeah, have, as you said, a direct relationship between their funding and what they're getting out of it.

 

Tom Christie  59:21  

Well, it's a lot easier for me and rest framework to be able to tackle that sort of thing, because I can just be very unilateral in how I want to approach funding, right. Whereas as more difficult with can be, you know, something where there's a bunch of different people who are all kind of equally responsible for the health of the project to go to make very, like direct decisions about you know what, let's put the funding up front and center and let's be very direct about this. You know, those conversations can be more complicated. Yeah.

 

Carlton Gibson  1:00:00  

JOHN has a big ship. It takes a while to turn.

 

Will Vincent  1:00:03  

Yeah, but and yet it you know, if you go to a Django con in the US or Europe or now Africa next year, I mean, it's still a small number of people who really make it happen, which I mean, this is like I, you know, being by far the newest to this community of the three of us. I keep getting struck by first it's like, how is this amorphous thing? And then I'm like, Oh, my God, it's so fragile. Like, I mean Carlton's, giving some talks to this way. Django Django isn't, you know, this big organization is it's this Marissa says, myself, there's half a dozen people who are very active, and then there's, you know, 50 people who are kind of active and then that's it. That's, that's it. And you know, people will email the Django developers saying, The Django team and it's like, really, that's you if you step up.

 

Tom Christie  1:00:50  

But it's true in so many places as well, right? It's, you know, there's a lot of places where there's these core but sort of Get in Python or any other bit of the space where something absolutely critical and it's one thing in the garbage You know, every now and again, who's working working on our love? Yeah.

 

Will Vincent  1:01:18  

Yeah. Well, as we wrap up, is there anything else you want to mention? highlight? I mean, there will be a couple thousand people listening to this and you know, it'll be there for posterity

 

Tom Christie  1:01:30  

just Hi.

 

Come back in six months time and I might have something different to say to that question. So

 

Will Vincent  1:01:42  

we'll see. Okay, great. Well, look, I'm links to all this stuff, encode everything in the show notes. So please do check out all the packages and stacks that are being worked on.

 

Tom Christie  1:01:52  

I'm also I'm hoping to go to pi con for in 2020, which is not something very often Yeah, I'm not often, I think, yeah, over in the States. So it would be really wonderful to chat with loads and loads and loads of people there because like I say, I'm not over that way much so

 

Will Vincent  1:02:16  

and and but you'll be at Django con Europe, they've publicly announced it's in Porto, Portugal. Are you going to be there?

 

Tom Christie  1:02:23  

I haven't figured that out yet.

 

Will Vincent  1:02:25  

Okay. Sorry to put you on the spot. I think I'm going to try to cross over the pond for the first time. I'm really excited to

 

Tom Christie  1:02:32  

ya know, I've been in a bit of a squish at the moment. Okay.

 

Will Vincent  1:02:37  

We can cut that down.

 

Carlton Gibson  1:02:41  

Okay, so Tom, thank you so much for coming on the show and listen from us and from the whole Django community and beyond. Thank you so much for the contributions you've given all the work you've done, you just you know, super cool. That's a pleasure. Thank you, Tom. Nice to Jango chat. Join us next time, folks.