Django Chat

Third Party Packages

Episode Summary

Django 3rd Party Packages are the "secret sauce" in many Django projects. In this episode we discuss our personal top picks.

Episode Notes


Episode Transcription

Will Vincent  0:06  

Hello, welcome to another episode of Django chat. I'm Will Vincent joined by Carlton Gibson. Hello. And today we're going to talk about third party packages, which are often referred to as a secret sauce of Django. So to start things off, why do we have third party packages? Carlton, why would you say you know Django is a batteries included framework, it comes with a lot out of the box. Why not have everything in the box?


Carlton Gibson  0:29  

Because it's been possible to maintain, essentially, yeah, and for 80% of projects, they wouldn't need 80% of the functionality. So batteries included is it's the 80% cases, it's almost every project should need everything that comes with the core framework. And then in order to keep it maintainable, but also in to keep it relevant. extra functionality lives outside the core.


Will Vincent  0:53  

So we wanted to go through our favorite third party packages, and we'll listen to them because there are, let's say, thousands there's a there's a website Django packages that lists all of them that will link to in the show notes there. Tons and tons and tons, partly because Django with its projects and app structure lends itself quite well to separating out functionality for these things. So if you build it for yourself, you can make it available to other developers. And then, you know, can be community projects.


Carlton Gibson  1:21  

But it's always been encouraged. Right. So now we've got 10 year history of people being wrecked encouraged to build that body apps and then building third party apps and making them available. And so they're just there's so many almost everything you think of, there's a good app out there that does it or there's one that's at least Good, good enough for you to think uses a kind of inspiration to copy it in your own brain.


Unknown Speaker  1:44  

Yeah, yeah, I think it's often it's a great way to learn if you're to, if it doesn't do exactly what you want to look and see how someone implemented functionality because again, with the web, almost nothing is unique. It's someone's had the problem and tackled it before and Just one caveat is that you before you install a third party app, just have a look and make sure it's well maintained. Because what can happen is you install a third party app. And it works great. And that's fine, but it doesn't get updated. And over time, then when you want to upgrade that Django, you can't upgrade Django because you're tied to this third party app. So just just vet them before you install.


Will Vincent  2:22  

Yeah, don't don't go crazy. I mean, the way to, you know, how do you know something's active, I would say should have a fair number of stars. So probably a couple hundred, and you should see some action within the last three to six months. If not, they will. Okay, but no Django app.


Carlton Gibson  2:40  

Well, yes, but like be sensible, but Django app calm, which is a package I maintained. It's used by Django compressor and a few other applications. But I this week, I released a new version, which just changed the packaging metadata to say It supports Python 3.7 and the latest Django is and things like this. But it hadn't been updated for two years, but it had zero issues and zero pull requests.


Will Vincent  3:07  

Yeah. Okay. That's That's very nice. Yeah. So


Carlton Gibson  3:11  

you know, it's not that it hasn't been updated, like things which are very stable. They don't need updating every week.


Will Vincent  3:17  

Yeah. Fair enough. I know I brought it up, because it's sometimes hard to hard to tell. But anyways, all the packages we're going to talk about today are vetted. And so if they're what you need, you should feel as comfortable as one can using them. And we should note we're not going to be so there's also what's the third party app, there are additional things like wagtail, Django, CMS, which I guess they're technically apps, but they're really more like,


Carlton Gibson  3:42  

frameworks. Yeah,


Will Vincent  3:42  

frameworks, frameworks.


Carlton Gibson  3:44  

So we're not wagtail and CMS, the full CMS frameworks, and they're amazing, and they're great, but they're, they've got their own ecosystems. They've got their own dependencies.


Unknown Speaker  3:51  

Oh, wow. Yeah. So


Will Vincent  3:54  

we're not going to talk about those. Today's maybe in a future episode. So let's let's start right at the top. I mean, I think everyone would agree the number one third party package, and most people probably don't know that it's a third party package is Django rest framework, which is how one builds RESTful API is with Django. It is what the majority of Django professional Django developers, I would say use when they use Django. And yet, it is not part of Django core, which you know, you're a co maintainer on it. So perhaps you can speak to why it's not part of Django itself.


Carlton Gibson  4:25  

Ah, just historical, I guess. To create JSON responses was, you know, there's a JSON response class. And then there was a package called tasty pie, which enables you to do RESTful APIs, which was a third party package and then come along came rest framework, and it was never it was just it just was built as a third party package. And it survived as a third party package, and it continues to this day, but it's, yeah, I mean, for me Django Django rest framework right now, what else do I need?


Will Vincent  5:00  

Yeah, no, exactly. So it's um, that's probably the number one third party app that everyone will use. And we will we'll do an entire episode or episodes on on it. It has its own third party packages that we're not going to talk about today. Yeah, but that's, you know, another big one. So we'll actually in the news is Jango channels, because you've, you've taken on a new role with this, right, Carlton?


Carlton Gibson  5:28  

Yeah, so Andrew Godwin, who created channels has just stepped back from that. So hopefully, that gives him a bit of capacity to work on bringing async support into Django core itself. But he stepped back from the maintain maintenance role in Jango channels, and I've set up to do that with a couple of other people. So we're just keeping that ticking over but that gives async support to right, this is the future. Yeah, the main the main use case I guess right now is for WebSocket WebSockets. Right. If you want a WebSocket thing, then Jango channels is the way to go. But also if you want to Build async applications using Django, you can do that right now with channels. And then we'll see what happens with the work that Andrew comes in perhaps over the next six or 12 months, we'll start seeing how Django itself will provide support for async. Code, the Django channel if you want WebSockets Jango channels?


Will Vincent  6:17  

Yeah, exactly. Moving on. So I would say the one that I use on every project is Django debug toolbar, which is a way to view your, your requests, so to see how efficient you know, your, your blog list is, this is a small one. It's, I can't think of why you wouldn't want to every every project Yeah,


Carlton Gibson  6:40  

it's just super lets you view template context variables, which and which template, you know, you always, this template extends that template that includes this template, which templates actually did get rendered. And what were the context that they actually saw. This is just super, lets you view your SQL requests lets you I know you can only concrete plugins What if you want to view extra bits and bobs? You can?


Will Vincent  7:03  

Yeah. So both to, you know, hop into your own project or a new project and see, yeah, that what is the template structure and especially for performance, you can really see, you know, how many queries are involved. So that's a, that's really a must have. I can't think of a project I've worked on that didn't have that. Yep. Another one is that I especially like is Django extensions, which is a way it's a, it's a several tools, but the really nice one is you can add, where's it plus, right, so you can do Python run server plus. So if you want to go into the Django shell, it will automatically preload all the things that you'll need. It has a whole bunch of it's really a Swiss Army Knife of sort of additional tools that as you become more experienced with Django are just really nice to have.


Carlton Gibson  7:50  

Yeah, my my favorite one there is it will graph your models for you and I'll put those in. In the dot format. The graph is format which then I dragged into on omnigraffle, which is capable editor for editing knows, but you can drag that into anywhere. And then it gives you a nice kind of class diagram of your models, which is lovely to have. And you can edit that and you can. It's one I use a lot. So Django extensions, super.


Will Vincent  8:16  

Yeah, yeah, I don't use all those. All those parts of you just mentioned I should, I should look into that. Well, which ones do you have? We've got a list of things.


Carlton Gibson  8:25  

Which object. So another one, I maintains Django filter, which is awesome. If you want to create filter, if you want to allow people to filter by URL parameters. So you know, if I put in, you know, size, size equals a bit large, and I want to filter my query set, well, you don't just want to expose that raw user input, and then feed it into the ORM because that's dangerous. So Django filter will create a form which will then validate that the value was sensible. It will then take a quilter set and it will filter it and you can create these you can very easily and very declarative Create filter sets for your models which enable you to filter by the parameters that you want. And you can filter across relationships and things like this. And then Django filter integrates with Django itself, but also with Django rest framework to. So it enables you to create very easily filterable endpoints on your API's, which is kind of cool. So you know, I wonder, I've got a list of blog posts, I want to filter by those that are tagged Django Easy peasy.


Will Vincent  9:26  

Yeah. Yeah. And this, I mean, that's a separate, you know, security thing around just for you sort of alluded to it with URLs, you know, why you often probably don't want to include the primary key you know, for blog posts, because, you know, giveaway the number of blogs you have, right, like you'd use a UID or you could slugga phi it.


Carlton Gibson  9:47  

Yeah, there's a nice little thing called hash IDs, which are worth looking at, I don't know, I don't know. What they do is they take the number and they pass it through a little algorithm and it comes up with a would like to say Seven or eight digit, random string, and they came in, it's reversible and that that's great, but it's slightly longer than 123 10.


Will Vincent  10:11  

Yeah. And it's not necessarily giving away that, you know, you have 100 you know, blog posts on your site,


Carlton Gibson  10:17  

the old the also important, the order is not deducible. Right. So you can't predict what the next one will be.


Will Vincent  10:25  

Right? Yeah. I mean, you know, slugs the problem with slugs, if people haven't experienced this is, you know, what do you do if, you know your URL structure is, you know, my website comm slash blog slash the name and I write one called hello world. And then Carlton writes one called hello world, they can't exist the same URL URL endpoint, so you have to deal with complex collisions of that sort. So that's why


Carlton Gibson  10:48  

so lots of ways you will just on slugs while we're here, we might as well say I quite like using unique for month unique for years. So you put in the URL you might have 2000 1902 for February and then you might have a slug, which is unique for that month. And then it's not unique forever. Yeah, only one that month. And I kind of quite like using that sometimes. And that's an option on model fields. You can if you've got a date time field on the model you can give unique for a month and say has to be unique reference to this date time field.


Will Vincent  11:22  

Yeah, I mean, I didn't know that. I mean, there's also an argument around, not putting the, you know, not Time Stamping things in New York.


Carlton Gibson  11:31  

Like that. Now, lots of people don't like


Will Vincent  11:33  

it. Yeah, that's interesting.


Carlton Gibson  11:34  

If it is a traditional blog, like a web log, yeah, there's a difference between kind of what you want to have put out evergreen content where you don't want the date in the URL. Yeah. And a web log where it literally is saying, hey, today I did this. There's no harm then that being grouped by month or by year. Oh,


Will Vincent  11:51  

yep. Anyway, yeah.


Carlton Gibson  11:54  

So what I like I like Jenga, Watson, which is a super package which enables you to Make use of your databases text searching features is slightly perhaps viewed is slightly out of date now, because Django has got more full text search features built in, but Django Watson is very good and worth looking at. and it supports all the different backends.


Will Vincent  12:18  

Great, but I haven't used it.


Carlton Gibson  12:20  

Well, it's great if you enables you to set up an index on a model, and then you can pass in a query set. And you can just get back that you can get back a query set, which only matches which the search and then you could filter them further. So you might say, filter by a tag, then search for a string, and then filter again by you know, published Listen, whatever you wanted to, but it's it's super Django Watson that's called the other one by by the same author is called Django reversion. And this enables you to keep model histories of your models. So say you've got blog posts, and you wanted to be able to go back in here History and say, oh, what the I've edited it. And then it's been edited again, it's been added again, but actually the version two ago, two years ago, I need to go and look at again, Django reversion let you do that.


Will Vincent  13:13  

That'd be very useful. He's,


Carlton Gibson  13:14  

he's, you know, he's two good apps there that I like.


Will Vincent  13:18  

Yeah. Well, another one. Next, our list that I really like is called Django all auth, which is what you would use for authentication again, because the challenge is that Django does not come with a built in user registration signup form. So there's a couple packages out there. I like Django off the best. It does. I guess two things. One is it enable social authentication. So if you want to add Google, Facebook, Twitter, it's very straightforward to do that. The other is that it provides a whole ton of customizable features you can swap in email instead of username, has a whole bunch of additional settings. So you know, it's, it is a third package. So, you know, like, for example, with email, this is a common common one, the Django default is to have username email password, which is a little bit outdated right now, more common on websites is to have login or signup be with email and password. So you can do that yourself, you need to dive in and use the model managers, which is a little bit of an advanced feature for people. Or you can if you know that you need social or you want some of these additional features, you can just use Django all off and it's a bit faster to get up and going. So in my my first book Django for beginners, I show how to do it. We use Django all off in some of the later chapters. And I think in my, my new book I'm going to show for example, with email, I'm going to show how to do it with model managers. But we're going to end up using Django all off because if you do want to have social authentication, which a lot of people do, it's a great way to do it and not have to roll your own solution.


Carlton Gibson  14:58  

Yeah and cat will capture the The OAuth tokens you need to hit the API. So you know, if you've got twitter login and you want to add Twitter features to your website, you've got the the right credential to then make use of requests to the Twitter API on the user's behalf which


Will Vincent  15:13  

Yeah, you can even see any admin I mean, all that stuff is is is a bit of a minefield, you really don't want it really one


Carlton Gibson  15:19  

thing I was gonna say about OAuth was that it needs to do a better tutorial because it is quite a paper then you've just said it's in your book. I have one I have one


Unknown Speaker  15:25  

I really I do.


Will Vincent  15:27  

Yeah, Yes, there is. Of course, there is one you know there was one from like four or five years ago called the missing all out tutorial. Yeah, Google search for Django Allah tutorial, see what comes up?


Carlton Gibson  15:35  

Okay. Because,


Will Vincent  15:37  

because because I, you know, this is, this is how I learn. I get frustrated, and I read it up.


Carlton Gibson  15:42  

No, I remember when I started using all auth It was like, wow, this is this is powerful. It does exactly what I want. But I have no way of understanding this apart. Yeah, reading the code and working out and yeah, it's a package that I use. And I was thinking, well, he needs to Taurus, but of course, you've written one already. Well,


Will Vincent  15:57  

yeah. Well, that's I mean, and you know, that is the The thing about all these third party packages is you get a tremendous amount of power, but then you need to learn how to use the packages. And you know, as I sort of mentioned with Django, all auth, you know, I've gone full circle on it where I, you know, I didn't, I didn't initially know how to, you know, years ago how to do emails to have username, I found Django all auth I sort of, you know, Cluj together, how to do it. And then I wanted and sort of learn how to do it the wrong way. And now, most cases, I use Django all off. Because I understand what's doing and it's managing the complexity, but it's a bit of a circle in the same way. You know, I would say generic class based views are versus luxurious views. No. And the other


Carlton Gibson  16:39  

thing I like about all we've turned this into an all off podcast now but the thing I like about all of as well as it enables users to manage multiple email addresses, which is nice if you want to, if they they want to control notifications, if you need to send a user notifications, they want to say look, I want notifications to this email address, not that one and it enables them to add multiple email addresses and choose which one's the primary and Things like that. So it's something that I like about it anyway, we should move on.


Will Vincent  17:04  

Yeah. And I included in Django x, which is my a Django starter project, because I think it's, um, it's a great I think it's, it's, it's something I use in almost every project. Alright, moving on. So we have ones for the deployment, which are static files, which is adding files here. Yeah. So white noise. Do you want to talk about that?


Carlton Gibson  17:25  

Well, yeah, okay. So there's two strategies here. static files can be a pain, so the white noise,


Will Vincent  17:33  

but let me explain. So static files, you can when you're just prototyping, you can store them within Django, but you really don't want to do that. You want them on a CDN for performance reasons. Yes, yes. You can have that within Django. So you know, when I teach this, I, I just sort of show how to set up static files and stuff. And then and we use it and then later show how to, you know, deploy using Django storages or something on s3 or something, I don't know, I sort of go I go progressively with it.


Carlton Gibson  18:10  

Rather the thing. The thing is that the static files app will serve your views in development is perfectly well. And so they've got this little if debug that was, yeah, yeah, we all use and that's perfect. But you can't use that in, in, in development. So then white noise comes along, and it gives you a view, which will serve your static files and adds the right caching headers and does all these nice things for you. That's that's one strategy. That's called white noise. And then the other option is Django storages which if you want to put your static files on s3, or as your storage is or glue, Google blob store that you It gives you the tools to do that.


Will Vincent  18:46  

Yeah, I think you can use white noise in particular integrates well with Heroku. Right, okay. In general, though, you do want to put your static files on a CDN.


Carlton Gibson  18:59  

Yeah. So you put it somewhere. If you put so if you like, if you use Django storages and put your state files in s3, for example, you then put cloud front in front of it to yell speed up or whatever.


Will Vincent  19:10  

Yeah, well, that's, we can get all that later. Another big one. Those two good ones. Yeah, no, those are great. Those are ones I use all the time. environment variables. This is something you should use in your projects. And Django environment is sort of the default. Whether Yeah, there's also dot m, but Django environment sort of managing your environment variables. Do you want to I'm blanking here on how to explain the package properly other than it helps you manage


Carlton Gibson  19:39  

variables helps you control your environment. I mean, this is a fast right, we want to keep we want to use we want to do the right thing and not put sensitive variables into our into our settings file. So we keep them in the environment. But then we've got to remember that we need to export them into the the recording process before we can launch Django itself. Pain, junk environment helps you


Will Vincent  20:02  

manage, right because you'll use different variables, you know, locally versus deploy. And you don't want to put that in source control. For example, like your, your secret key or your you know, your stripe ID if you're using payments or something. Yeah, your database, you know, yeah, your database. Yeah.


Carlton Gibson  20:16  

Yeah. What's it called? I am access key thing. Like, yeah, you need to keep that secret. So, environment variables for that. Okay, fine. We'll stick with skip on that one. Django Guardian is another nice one. So if you wanted to do so Django has a nice permission system permissions system, which by default, the origin control. All the permissions will say yes, you can access blog posts. Yes, you can access the sales catalog. But if you want to restrict object access to individual objects, then Django Guardian will help you do that so you can create object permissions. For this specific blog post or this specific item in the sales catalog, and that's, that's a useful addition.


Will Vincent  21:08  

Yeah. Yeah, you know, permissions and authorizations is, you know, very much sort of intermediate level thing. But every single project, you want to lock those down and set up the hierarchy. And that's, again, every project you're doing that.


Carlton Gibson  21:24  

Yeah, I mean, you know, once you get beyond a certain size of team, it will always be the case that there's a category which people can the people can only access a subset of, and as soon as you need that you need object permissions. In Django Guardian is the go to solution for that.


Will Vincent  21:40  

Yep. So another one, I'll tee this up. Django compressor is one I really like that you maintain, which is for compressing your static files.


Carlton Gibson  21:49  



Will Vincent  21:52  

you shouldn't do that. You should use Django compress or something like that on all your projects.


Carlton Gibson  21:56  

Yeah, I mean, so what's really good about it as well as if you use pre pre processes like sass last or post CSS or, you know, I use it for Elm that other people can use it for react. And what you do is you just create a template block, which has got you with the compressed tag and then you put your script tags in them as normal. So you might put your, your your link tags be your CSS, or your script tags via JavaScript, but you give them a special media type, which will be like for sass, so content is stylesheet, slash slash. And then when it sees that it will run it through the preprocessor and convert your sass into CSS so that you can load it straight into the browser. And then development it keeps them separate so that you can trace it back to the individual stylesheets. But then for production, you run a compressed command and it compiles them all into one file and you can minimize them and you can choose it them and you can run it whatever pipeline you want to. And then you put them into the static. static files folders for white noise or Django storage is to deploy and job done


Will Vincent  23:00  

Yeah, you know, it all ties in very nicely in the end. You know, there's a number of local versus production things with Django that, you know, we talked about a couple of that these packages help you with, because, you know, you want to get up and running quickly and locally, but then for production, it's a different set of requirements, often.


Carlton Gibson  23:17  

Yeah. And so anyway, I really love that. And I'm not a front end developer. And so if I work in, you know, a team where there's a, you know, professional front end developer, they, they will replace Django compressor with their own build system and their own, whatever model will have the deployment. But for me, as a Django developer, I can do something that's semi respectable, quite more than semi respectable, just using Django compressor. And then, when people need to go that bridge further, they can do that.


Will Vincent  23:45  

Yep. No, it's a really nice optimization. You know, maybe. So one more this is sort of a fun one, Django admin honeypot. So best practices that one will notice that every Django site has the admin at slash admin. So if you were a malicious actor, you could write a script and find out if a site was a Django site. And then you could try to hack it. There's a couple ways you can safeguard that. Number one is change the URL. So don't change, don't have admin at admin. Django admin honeypot lets you see people trying to break into your admin. So it's a, you know, it's sort of a fun one, it's a good reminder that you should harden your admin. There's actually a number of steps to do on that. But that's sort of a fun one, if you want to see someone in particular trying to break in your site versus just random bots on the internet.


Carlton Gibson  24:36  

Yeah, right. Brilliant. All right. Well, that's a whole long list, right? But check the show notes. For more on those. This is just a quick run through of our favorites. There are, as we said, millions or thousands of others. There's a group on GitHub called jazz band, which contain a whole group. So the danger with a third party app is it's difficult to maintain over time. And so jazz band takes They call packages which need help maintenance and then they maintain them as a group. And so they keep these packages going in their library. So anything in jazz band you can rely on as being well maintained. Check out Django packages as well said,


Will Vincent  25:12  

if you think there's any packages that we haven't covered or should go in depth on, please let us know. You can do that at Jango There's a way to contact us. We're also on chat Django, and we'll see you the next episode.


Carlton Gibson  25:24  

Okay, bye bye.