Channels are the current way to add async support and websockets to Django. We discuss the history of the project, its current status, and how it relates to Django 3+.
Will Vincent 0:06
Hello, and welcome to another episode of Django Chat, a weekly podcast on the Django web framework. This week, we're going to talk about the very popular Jango channels package. I'm Will Vincent joined as always by Carlton Gibson. Hi, Carlton.
Carlton Gibson 0:17
Hello. Hi, well,
Will Vincent 0:18
so Jango channels. This is a very popular package with a long history and ties into bleeding edge. Django, so let's describe. So what is change channels, I'll tee that up to you. So it's been around for a couple years. And you are currently the maintainer of it, though it was originally created by Andrew Godwin. And really it was a first attempt at tackling a sink.
Carlton Gibson 0:42
Yeah, exactly. Exactly. And so it's now in like version two. version one was the first go what well, how might we might bring me Django pacing. I think Andrew talked about this a little bit when we interviewed him.
Will Vincent 0:55
Yes, and he has some you can check out that interview. So we'll as well that will link to talking specifically about channels as well as async Bs, our interview with him, he talked quite a bit about async itself modern async. And so the the first thing it does
Carlton Gibson 1:07
is it enables you to, it will wrap your existing Django application in an async handler with an ASCII handler, much like that this is going to be available in Django 3.0, almost, you know, very similar. And it will enable you to use an ASCII server to serve your Django application. And then on top of that, so that's the first thing and that's it. That's kind of interesting. That's what people do in Django 3.0. On top of that, it has
Will Vincent 1:36
WebSocket support. Yes, yeah.
Carlton Gibson 1:39
And it also has proper async view support. So you can write async def views, which will handle say HTTP but in an inappropriate way, all the way down. And it also there's another interesting bit, it has what's called a channel there which enables you to kind of talk between applicants So you've need a backing back for that. So the one that's in play is Redis. There's a channels reddish Redis package, which sits on top of that. And that enables you, for instance, to have a group chat, for instance. So you open a connection to my website. And it's we got a web socket that's enables you to do async WebSocket communication back and forward to the web server. And I opened one as well. And then the channels layer bit enables us to send messages not just back and forth to the server to ourselves, but to a group, which then fans them out to everybody in the group. So we could have a chat room or while you could do live streaming, Jason
Will Vincent 2:41
has a course on building a Uber, clone. So real time with with with taxis. We'll link to that. Yeah, these are the
Carlton Gibson 2:50
exam, you know, you could you could communicate using a mobile app, for instance, to the channel server that might be able to talk between apps or you know, I think But drive as an app and you have an app and the to communicate virus server using probably WebSockets. or similar to Yeah,
Will Vincent 3:07
yeah. WebSockets. So but that's, I guess the question for people that was, so what should I use now? Because channels has been under very active development in a way. The new async stuff is a bit of a.
Carlton Gibson 3:20
Okay, so the first Yep. So the first thing to say about that is the status of chat. So I've so Android stepped back from channels, and he's now working on async in Django, and I've said to him, I'm just maintaining it. And there's a massive difference between when Django Andrew was actively developing channels, and I'm doing maintenance and I, I have I doing this isn't isn't part of my Django fellow work. So it's just a volunteer basis. So I have an hour a couple of hours each week where I triage tickets, and I fix bugs over the course of the medium term. So we have a release now maybe two, three releases a year just fixing one or two bugs rather than Andrew actively actively developing it and pushing it forwards but it is maintained. It's great and there are there are issues and if you want to contribute I'm happy to help you contribute. And we'll get those issues fixed and we can put new releases out. But it's it's good and it's there. What should you be using Django 3.0 just about to come out if all you want to do is wrap your application in and serve it via an ASCII server, use Django 3.0. Because that's all that's that part of channels will be in Django chords, no point using channels for that. Eventually will remove that from channels, but not until 2.2. End of Life, I suppose. Because that doesn't have the ASCII support. Right? If you want to use WebSockets, then yes, channels is a great solution for that. And if you at this stage want to write async views. Then still channels because 3.1 should have that in Django core but doesn't yet. And if you want to play around with the channels, layer stuff, which is kind of exciting. It's, you know, it's kind of like enables you to build Say something. It's kind of like a promo message bus. It's not it's Got a full scale one, you know, there are some full scale applications that you have to run. But if you just want to play around with passing messages between different clients, channels will serve that purpose. And so it's a good starting point for that. And if you've run into difficulties or scaling issues or whatever, then perhaps you can move up to one of these more fully fledged message bus applications at the time. So that would be my answer on what should you be using depends what you want to do,
Will Vincent 5:24
as ever? Well, and I think the history of channels is interesting, just in terms of shedding a light on Django and how its structured because so channels is not part of Django, but it was added as an official Django project in 2016. Yes, so this is I guess, a little bit like Django rest framework is actually I don't know respironics official project, I don't think
Carlton Gibson 5:45
rescue it was framework was originally under Tom Christie, who was the creator of restaurant, his sort of GitHub account and his personal account, and then he created encode OSS software which is like, they seem to Do sustainable open source software development so you can fund the projects. So the main the biggest project, there is rest framework, but also the new HTTP x, which is a fully async requests compatible HTTP client is going to be or library. It's going to be,
Will Vincent 6:15
yeah, we must have time to talk about that. Because that's the thing you and I talk I get really excited about, but I don't think everyone else knows about it.
Carlton Gibson 6:22
Yeah, I mean, it's it uses yellow lip three, it's fully async compatible. It's got multiple backends async IO entry. Oh, it's, it's, you know, it's gonna
Will Vincent 6:31
be ubiquitous. In the future. Yes.
Carlton Gibson 6:34
Yeah. It's, it's really hot. And also, he's got them this style eight as the microphone went there. And then some other things are all under encode. So the rest frameworks under that, and you can sponsor encode to help find rest framework and make that sustainable. But wise channels, but channels is an official Django project. Why is that? I think because it was clear right from day want that async was where we wanted to go. It was clear to Andrew certainly, and you know, but it wasn't something that we could just bring straight into Django. Django is mature and stable, right? This idea that it's really exciting because it's mature, mature and stable, and you can be on the latest release, because you know, it isn't gonna break. And it's all these wonderful things that maybe would have on the new features. But that means we can't just bring something a massive architectural redesign in and put it straight into Django straight from the word go, because that's not mature, stable behavior. And, to be honest, that turned out to be very prescient, because channels one looked like it was the perfect solution. And it looked great, but it turned out it had scaling issues. And so that required the whole channels to rewrite. And if we'd have had to do that inside Django core that would have been, would it be an end of story?
Will Vincent 7:56
Yeah, well, I want to mention to a link there is a thing called Django enhanced proposals based on Python has the same things of which having official projects is one of them. The new async stuff, I'll link to that that's, that's kind of where these decisions are made, I would say right, or, you know, these big changes to Django.
Carlton Gibson 8:16
So well, so there's a whole depth on async. And that had to be discussed at length, and then that was approved, and now it's being implemented. And we're, we're at stage one now with Django 3.0, where Django speaks at Husky and then stage two will be that you can write async there for us. And stage three will be the RM and template layer and these other
Will Vincent 8:36
one I think it's worth repeating how, how these things are structured because I I'm always struck by the fact that chango seems mysterious and otherworldly, otherworldly to people when in fact, it's all out there in the in the public. And if you're curious or want to be involved in Django stuff, there is the Django Django core will not Django core, what's it called? Not Django users, Django developers and Developers Google group, which we'll link to, and there's also a separate Django users Google group though now there's a new discourse forum board, which we'll link to, which is an effort to make it a little more beginner friendly. But anyways, all this is out in the open. So the question is who decides? Everyone and no one is basically Yeah, like the answer if you're active.
Carlton Gibson 9:20
So if you want to get involved, I'd say, subscribe to Django developers, not very high traffic, you know, a few emails a week. And if you haven't got the capacity to read them, don't read them, you know, just sort of have a rule that puts them in a folder automatically. And then you can look at the folder at once every so often. We'll just go to the Google group once a week and sort of browse through it and have a quick read, it becomes more relevant as you get in you know it, the more time you spend hanging around, the more it's Oh, I wrote that, you know, okay. Make sense. It's,
Will Vincent 9:47
yeah, and I know that Andrew mentioned at Django con that the there is a section on async in the new forum, in part because he didn't want the Django developers group to just be a thing. issues. So I think I think this is great to have just a better UI around discussing these things. But again, it's it's all out there in the open, there isn't someone behind the curtain deciding if you have opinions and
Carlton Gibson 10:14
get involved, okay, and the channels projects like so if you go to GitHub, the Jango organization, there's Daphne, which is the web server, there's channels, which is the main library, there's channels Redis, which is the behind one, as well as that there's ASCII ref, which is the reference implementation of the ASCII standard, which is the async. standard for web service to talk to applications, like whiskey, but ASCII, those four repos, that's where the most of the discussion happens. And if you want to watch those repos, keep GitHub will send you notifications and then you know, you can join in the discussions and if you know if you want to help get involved in channels, I'm really happy to help get you set up help, you know, help talk you through what my understanding and we are You know, write some tests and
Will Vincent 11:02
yeah, it looks like there are there are, I wish we had better visibility on this. There are, I know, anecdotally of places using channels for real on, you know, professional projects. I just don't know how public that is. But it's you can definitely use it right now. And if you need something right now, you should use it rather than waiting. Django itself.
Carlton Gibson 11:25
Yeah, I mean, you know, the, the, even though it's, you know, the few people that help a little bit, even though it's just me and I don't have much time and I certainly haven't had much time this year for various personal reasons, but it's the number of open tickets on those three repos combined is quite low. There aren't many. It's not like there are loads of outstanding bugs or whatever, there are a few and we need to chip them away over time. But it's nothing that you know, a little bit of input from the community, we wouldn't have those all wrapped up so it is they are good quality packages and they do work for you. And you know, you can use them. You can do a Ready?
Will Vincent 12:00
Yeah. Well, and that's the interesting thing that Andrew is talking Jango Khan, he alluded to, which is that, you know, three years ago, kind of everyone thought WebSockets and all this stuff would be everywhere now. And it turns out that the need hasn't quite matched that that isn't the case yet. It's not quite as ubiquitous, you know, everything doesn't have to be real time, even though when WebSockets came out, and you know, HTTP two and all these things, it's all kind of there now. But waiting, I would say for use cases beyond a chat app, or, you know, you know, taxi riding, I think these will come. But it's, it's interesting that the demand isn't, is more of the issue than the technology generally speaking it.
Carlton Gibson 12:42
Yeah, I think it is interest. I mean, somebody there was an email to Django developers, quite recently. I can't, you know, it doesn't matter exactly when but it was saying, Well, why aren't WebSockets in Django cool. And it's like, well, exactly. This point, is it? It turns out that they're not the silver bullet people. You know, there was a lot of excitement. To have this we have to have this. They're Okay, they're great. They work. But there are no other solutions. They've got server side events where you have a slow HTTP connection, you fire updates just down the wire. So most of most of what people use WebSockets for is like, I just want an update from the sun. Yeah. Well, it turns out, you can hold an HTTP connection open. Yeah. And you can just have the server push more data when it wants. And you don't need a whole the whole WebSockets because the trouble with WebSockets is you need client libraries. And then you've got JavaScript, and you've got, you know, loading issues. And you've got Well, what's the quality of the JavaScript client? And you know, so a lot of issues we get on channels turns out to be well, I'm using a broken WebSockets library,
Will Vincent 13:41
right? Well, since I think we have a couple minutes, maybe, can you so make the case for why HTTP x is so exciting? What does it What will it solve? That's not channels at all, but it's I know, but it's but I'm related. It's a flavor. It's related ish.
Carlton Gibson 13:55
Well, for instance, I don't know. Let's say you want to Fetch webpages, you want to you know, you're writing a scraper, no one writes a scraper. But let's say you wanted to write a scrape, because you want to do want to know, but okay, but if you're using requests, which nothing requests a great live, it's been the cornerstone of Python development for, you know, many years. But you've got this the same sync versus async problem, in that you're using up a whole thread a whole worker for each request. Yeah. Whereas if you can do this using async IO, you can get that same worker can probably get throughput of 100 times.
Will Vincent 14:31
Yeah, I mean, it's just a massive change of what what's the word
Carlton Gibson 14:35
change escapes in order to
Will Vincent 14:36
change for sure you also perform, right.
Carlton Gibson 14:39
But if you but so if you wanted to do if you wanted to make web requests at scale, your your massive savings or if you even wanted to do it at small scale, so you you want to get some little Amazon micro thing you Bob, you could probably run it using HTTP X on just a single instance. And you may have to make all the web requests You ever want to make versus Oh, hang on, I need to scale up on it. Yeah, move these to get the throughput that I'm like, Yeah,
Will Vincent 15:05
no.
Carlton Gibson 15:06
So that's I mean, that's the primary difference. It's but also it's written on the new underlying networking layer.
And I don't know, new sewing, will know, I don't know all the underneath. But
anecdotally, you know, Pete, that we'd have to get a lot. Yeah, we'll get him on Patel. You know, maybe talk more sense. But that's that request was built over a whole URL. It wasn't that great. And, and so it was a nice API on top of some kind of not very nice foundations, whereas HTTPS is built on top of the new free and it's all, you know, hundred percent type annotated. And it's, you know,
Will Vincent 15:44
yeah. So
Carlton Gibson 15:46
this is these are the exciting things. It's, it's, it's async and B. It's built on better foundations. I would say those are the two take home points for me. We'll have to as I say we'll have to get on and off skin.
Will Vincent 15:56
Well, and I think one last thing I would say again, repeating Andrews talk on the state of async in Django from Django con, where he even said that, you know, async is hard to reason about and hard. You shouldn't just take it lightly. I mean, he was envisioning that even in an application or website that needs it, it should probably only be 10 to 20% of the website, it's not that something, everything needs to switch over to async. It's that for certain things, it'll have big benefits. And the way it's being written as you can toggle back and forth within an existing Django project, which is sort of
Carlton Gibson 16:30
nice. Yeah, I mean, so. Yeah, exactly. This, you want to write 90% of your views, just like you always have. And then when, where does async gonna help? It's like, if you're fetching out to a third party service, or if you're, you know, if you've got to fetch something from the database and get some other data from somewhere else, and then combine them, you know, those kind of cases where you're IO bound, and you're not tying up the whole worker for this, but your, you know, your Bob, your bog standard Quickly server response that could just might as well just be sync,
Will Vincent 17:03
right? Well, and I'm, again with three one I had, that's what I'm, you know, as a content creator, teacher, that's when I'm going to start banging out a whole bunch of tutorials, because with the combination of ASCII and the views, you'll be able to demonstrate this stuff. And then you know, oh RM and these other cases, but I think that's, that's when it really will be real. For a lot of folks, the the built in async stuff, I would say,
Carlton Gibson 17:26
Yeah, for me that like the exciting case with async views is this kind of Prop I've talked about it before is the but these kind of proxy view. So say, you've got a REST API. And you've got these endpoints, one for books, one for authors, and you want to combine them so you can get a nicely nested response that contains lots of data. So a mobile client only has a quest, well then a proxy endpoint instead of creating, you know, a whole different API, you can have a proxy endpoint, which makes these requests for you and combines them how the client needs it and then returns those That will always be IO bound. So if you can use async there for me, that's a really exciting Yeah.
Will Vincent 18:07
That's what we get excited about Carlton, things like that. That's what gets us out of bed in the morning. Well, you know, that's
alright everyone. This is a this is a tech podcast right?
Hello, everyone. Thanks for listening. If you have questions, you can reach us at chat chango on Twitter, all the new episodes are on Jango chat.com website, and we'll see you all next week.
Carlton Gibson 18:27
Take care, buh bye.
Will Vincent 18:28
Okay, so we thought we were done. But Carlton really wanted to get some extra points in go,
Carlton Gibson 18:32
you know, right. So channels. And it turns out I thought channels and HTTP were two different topics. It turns out they're not because the these proxy endpoints that we were talking about, that's the perfect use case for an async HTTP client, right? So you've got an HTTP request comes in, you want to proxy out to say, three separate endpoints, and you want to make but you want to make those requests to those three separate endpoints. In an asynchronous way, so if you were to just use requests, you block the entire worker whilst you're waiting for them to come back. But if you were using HTTP x, you could fire off all three, they don't cut, they'd all happen at the same time, they come back together, and you can get your response compiled and sent back to the client. Nice and easily. So you could do that already using channels and HTTPS. And when Django has async. there for us, you'll be able to do it in Django direct
Will Vincent 19:24
boom. Mind blown.
Carlton Gibson 19:27
That was it that was just like Yeah, but we're already there. You know, these these these. We already have the tools in place now for being able to write these nice async views.
Will Vincent 19:41
Okay, everyone, that's our last point we wanted to get in. Yeah, we really are. Okay, well, bye.