We discuss why web security is so important, how to implement it properly with Django, common web attacks, social engineering, the principle of least privilege, and more.
Carlton Gibson 0:05
Hi, and welcome to another episode of Django Chat. I'm Carlton Gibson joined by Will Vincent. Hi, Will.
Will Vincent 0:09
Hi, Carlton, how are you? Good, I'm good. I'm This week, we're gonna talk about security, which I'm kind of excited about because it is a deep and important topic. And I think it's interesting when I'm teaching beginners and I sort of say, and then show them, the web is a really, really dangerous place. And it's why frameworks like Django that takes a lot of effort and a lot of precautions. It really helps you out. So I think we're going to talk about what are these classic dangers? And then what is Django do to help you with them? And then how do you configure it? Yep. So let's start off. So why is the web dangerous? The web is dangerous, because there's lots of bad actors, but really bad bots out there trying to hack your site for whatever reason. We've talked about this. In other episodes. If you have a slash admin page as a default, you should change that because you or I could create a little bot that goes around to every website out there and looks for slash admin and tries to as much as much worse than that because
Carlton Gibson 1:03
it's not that we create that bot. It's that there are scripts down there kits downloadable off the internet that have had scripts to check every known web vulnerability in every known framework and how to set that to every addressable i p IP address that your computer has, you can reach so it's like you just have to set up your thing and fire off this kit. And this will go and find every server out there and try every vulnerability against that server.
Will Vincent 1:28
right this is this was called there's probably still a script kiddies back in the day where you can download and run something you don't know how it works, but you can still cause a lot of damage.
Carlton Gibson 1:36
So this is why it's important to keep updated and people think oh, you know, it's just not gonna affect me. I've got a small site I don't you know, this security. No, this security your is if you have a weakness in your application. It's not if your application will get hijacked, it's when
Will Vincent 1:49
it's right. Well, let's talk about you know, so what are the just thinking of I just saw a headline Athena, which is JP Morgan's trading platform that does billions and billions of dollars of training, they've publicly announced they're not going to make it over to Python three by January 1. So that's a big invitation to hackers. Just on the Python side, not to mention changose updates coming up. You want to, you want to stay up to date, let's talk about so the biggest risk to so.
Carlton Gibson 2:17
Okay. But that's slightly because they'll pay people to mentor them to backport security fixes to 2.7 for as long as they have, right?
Will Vincent 2:24
Yes, but still, there will be Yeah, that's perhaps an unfairness, it's
Carlton Gibson 2:28
going to be this example. It's just for them. For them. It's just going to be disproportionately expensive because they'll have to pay you know, super consultant rates for every, you know, thing to make sure that they get pocket pass compliance that they've done it, whereas everybody else who can't afford that budget is just going to be stuffed, right.
Will Vincent 2:46
So what so what before we get into the technical things, so the biggest risk is what's called social engineering, which is people. So that's some individual who has access to things is compromised, either they're disgruntled, but more likely, they click on Phishing links. So someone sends them an email that looks like a real email. They click on a link. I mean, this is what happened. In the US in our elections. Some people in Hillary Clinton's staff clicked on phishing email links, click on one link, and it goes to a site that looks like a site, but it's not. We'll get into how that kind of works, but and then that person because that person is their level of permissions are now owned by Russia, or whomever. So the biggest risk is always people. So you want to make sure that you set permissions you don't give permissions are set appropriately, you won't have a lot of permissions levels in your application, you also probably two factor authentication is a good idea. I know there's some talk about eventually rolling that into Django itself. But permissions I would say is the biggest one, like for example, yeah. And engineers need right access. No.
Carlton Gibson 3:47
Yeah, I mean, this is kind of principle of least privilege, right? Yeah. That you know, and it might sometimes be a pain or Wouldn't it be nice if I could just do this thing well do you need to do that is it not just for the sake of security bear that you can't do that in the back? Have to go passes through this other person who can and
Will Vincent 4:02
yeah, and it's me it's a tough job to be the chief security officer, whatever the title is, especially at larger companies, in some ways, you're, you're the whipping post, because it is, it seems like it's expensive upfront to do all these things until something bad happens. And if you don't do these things, they will happen. So it's, it's, you know, it's important to do I mean, the number two risk, I would say, I'm curious what you think, Carlton after people, so social engineering would be updates, just keep up to date, keep up to date with Django. Most of the minor releases are security related. Especially third party packages, for example, Django crispy forms, just went from up to 1.1 point 8.0. But the one from 1.71 to 1.72. There was a security issue there, right that I know, Adam Johnson and you were mentioning, I should update my books to fix that
Carlton Gibson 4:55
bias. The key the key point is key. Like the minor fixes that Probably you know, that's where you'll find Oh, there was this little injection vulnerability that's
Will Vincent 5:05
patched up, right? The minor fixes are almost always a security thing. May I guess what else would they be maybe like some egregious bug fix. But, wolf basically.
Carlton Gibson 5:13
Yeah, well, depends on the policy. Right. So Django itself, like the main, the main, the current major version. So currently 2.2 will receive bug fixes for the current for that current version, but bug fixes for new features. So if it was a new feature introduced in 2.2, then you'll get a bug fix in 2.2 point four say, but beyond that, the extended release or you know, the one point 11 point 25.
Unknown Speaker 5:37
That's the LTS
Carlton Gibson 5:40
Yeah, the LTS all those out those long year and a half to three years of whatever it is of security releases, they are, you know, they are security releases that we they would also get data loss bugs, but those are so few and far between is to, you know, almost never happen. So the majority of patches that are security fixes, and most of them are public in that, you know, there's a, there's a a database, a CVE database of security exploits, and you publish those so that people know about them and know what the mitigation is. And you know, people who are interested in security, subscribe to that those mailing lists and they know about them, so they know what to update, it's really important that you keep up to date. And as I said, it was we were talking about being it's because for every vulnerability, someone will write the and it's not someone evil is be a penetration tester or a pen tester, someone whose job it is, is to find out whether an application is secure, so that they can secure it, but they rot internally,
Will Vincent 6:37
internally at a company
Carlton Gibson 6:38
Yeah, but the kit can be used for good or for evil, you know, it can be used for penetration testing, which is a legitimate use, or it can be used for penetration, which is obviously not
Will Vincent 6:50
less legitimate. And we've we've talked to another episode about how to update Django. There's all these great deprecation warnings and what you want to do is you want to go through And run, I'm forgetting off the top my head. What is it? It's with the W flag. What's the Yeah, it would be so you would to Python and then cap dash,
Carlton Gibson 7:06
capital W for warnings, right? And if you put warnings all wall, it's because it reads like all the warnings all then all these deprecation warnings, or pending that. So there's kind of levels a pending deprecation warning means it's not quite deprecated, but it will be soon. And then the deprecation warning means Hey, you really better fix this now because it's going to disappear very shortly. But by default, pending deprecation warnings are silent. So with it without the all flag, yes, they don't show up. But you want to run that at least time to time or at least in perhaps in your, in your ci as a different build. So you can you can see these deprecation warnings and then you've got ages and ages to fix them. So yeah, like you know, if there's if there's an deprecation introduced in Django 3.0 it will be it won't be removed until Jenga 4.0. So you've got like three whole versions to
Will Vincent 7:58
or how to fix it.
Carlton Gibson 7:59
Yeah, like 3.3 point oh 3.1 3.2 and then 4.0.
Will Vincent 8:03
Is that is that the agreed timeline? There's only 3.2.
Carlton Gibson 8:07
Yeah, so we've so the way it works now as we have the new, the point two version will be the LTS and then the major version will follow that. So 3.2 will be followed by 4.0.
Will Vincent 8:18
Okay, good. What else? Love if you're curious on I'll just slightly plug myself if you're curious on how to actually step through all this. I have a whole chapter actually couple chapters, in my book, doing it for professionals on doing this on making something production ready, because the information is out there, but it may not be in the most concise form. So what's another step environment variables, this is another must have for security. And basically anything you don't want to put secrets in source control. So you don't want something, for example, your secret key in your settings, file API keys. You don't want that floating around. So anyone who has access to your code base, any employee of any level can see them so you create an environment variable which is Basically live separately. And how would you describe it, Carlton? What's homeless? Okay, so verbal,
Carlton Gibson 9:06
well, okay. So in the Unix environment, every every process which is launched gets an environment dictionary basically right as with keys to strings, which the launching process can set and it will be inherited from the launching process. Hence, you know, when you're in bashing what you might have, you might have put export, Django settings model the export says and process down to the Java processes.
Will Vincent 9:34
Yeah, so you can store those environment variables and a whole bunch of access, but
Carlton Gibson 9:39
you can access them in Python with OS by the OS environment environment or the OS module dot and then get environment or the environment dictionary, you can access directly, you can set environment and you can use those to inject these environment variables into your settings file. So so when you first start Django start project, you get a settings file, and it's got a generated secret key for you. And that's fine for when you start. But before you deploy, you want to change that to use an environment variable. And then you can keep that in a dot m five file people use a lot. And then you know, there's various ways of launching with, with that environment bear with those environment variables in your, in your process environment,
Will Vincent 10:21
right? And you probably want to generate a new secret key because chances are you've done a command. And actually, there's a there's a, I'll put the command as an eat command, you can use changose generator, but basically it's a 50 character, 50 character, long string, random string
Carlton Gibson 10:36
that's generated. That's that it's, it's really important. Yeah, I mean, if someone has a secret key, that's that session, session keys, all the encryption, like everything, it's all uses this as kind of like a seed. And if that seeds known, then you know, then all of that can be broken because the algorithms are a node, and it's good that they're known because then they can be audited, but the seed has to be kept secret. Otherwise they don't work.
Will Vincent 11:02
Right. And again, any commit ever, that just one command of 1000 that happens to have the secret key or some other secret there is available to anyone who hacks into your GitHub, get lab, whatever, anyone who has access to the code base, so change it. Yeah.
Carlton Gibson 11:18
But yeah, and the but these things are cheap as well in the you know, like you IDs or whatever secret keys, you can just generate new.
Will Vincent 11:25
Yeah, right. There's not much cost to doing it. And I would mention, so what else? I mean, the main ones we've talked about this in a deployment episode, we go on at length, but you turned a bug off. You want to set you're allowed hosts so you can't just let any old person come in. And if you run the there's a deployment checklist built into Django and there's Doc's on it Python manage.py, check, dash dash deploy, it will say, Hey, here's all the things you want to make sure are configured properly for production.
Carlton Gibson 11:56
You know, again, it was headed well that had a strict transport all these kinds Things that you might do. And then for me, the thing that you need to do more than, you know, the sort of general rule is handling user input. So yeah, that's the goal here is you must always filter input. And that always means always pass it through a form with whatever validators, a sensible or you know, rest framework serializers, or something like that. So, you filter all input, and then you escape all output. So if you know you take if you're going to take user generated markdown and render it to HTML and put it on a page, you must you must run that through a sanitizer like bleach, or you must escape it right now obviously, you have to if you're generating HTML, you have to run it to sanitize it like bleach because if you escape it, you'll just see you know, HTML in your HTML page. What does that mean to escape something escaped Well, okay, so to escape something. So Django gives you the escape utility, which will for instance, take all opening angle brackets and convert them into a And lt semicolon, HTML entity so instead of it looked so if you put, if you had angle bracket p, close angle bracket, instead of it appearing as an HTML tag, a paragraph tag a p tag in HTML, it would appear as the HTML entity less than the HTML empty greater than right here in between so that it would it would be viewed. You'd see it as as the HTML on the other rendered page.
Will Vincent 13:27
Yeah, it just makes it really hard to describe that. Yeah, it well, I pass. I partly toss it to you because a little hard to describe. So I actually have I have a blank in HTML escape tool that I built just with JavaScript that on my website to be Espenson comm slash HTML escape tool so you can play around and you can escape or what is it
unengaging code? Yeah, I was gonna say it's a D escape. Anyways, these are gonna be on
Carlton Gibson 13:53
on escape.
Will Vincent 13:57
Alright, so yeah, user input is dangerous. You Can't trust someone out there and you want to do all these you know, so if you use use Django,
Carlton Gibson 14:06
so sorry that the escape function is in Django dot utils. Great, great folder, do go and rummage through that HTML dot html. And then there's escape. It's an escape is conditional escape format. html. There's all sorts of good stuff in it. Yeah, strip tags. All sorts of extra.
Will Vincent 14:22
Yeah. So we could associate. So what are the common attacks? I don't think we've covered this in a deployment episode, but just just very briefly, common web attacks you should be aware of. So the top one is, or one of the top ones is SQL injection. So someone has your forum and tries to put some SQL in there like delete database, or you hear about the school where they had little Johnny tables who work now there's a
Carlton Gibson 14:46
there's a joke about a it's just a stupid programmer joke but about how some parents name their son.
Will Vincent 14:53
You know, something, something, code on delete from or semicolon and delete from, you know, where it would be, it'd be fun. I know these exist. But it'd be fun to build a hackable Django site, you know, maybe just like in SQL lite, and just, you know, pull off the protections and just let someone actually do this. Because the problem is, it's, you don't want to play around with this on real sites. And a lot of real sites are just shockingly susceptible to these things for one reason or another.
Carlton Gibson 15:25
Okay, but it's remarkably hard to these days. Fortunately, to get SQL into your database directly. You have to like, you know, say you're using Django, you have to get the connection, then you have to pull out the cursor, then you have to concatenate your string and then you have to, like, you know, that stuff's known, but it's not in any of the books you have to you know, dig down and work out how to use that. So you wouldn't be able to get user concatenate SQL into your database. If you're using the DRM unless you try it. You'd have to really
Will Vincent 15:57
write another reason to use have a good friend Like Django instead of doing it raw, among others. So what else? xS s cross site scripting, someone injects a little bit of JavaScript into your page. This is why you want to automatically escape. user input. CSRF is a big one cross site request forgery, that just basically exploits the trust in the users browser. So this is why whenever you're doing a forum that has a post in Django, you want to put in those CSRF tag to load in the CSRF middleware, which will puts in a random secret key, basically is a cookie that doesn't let you so so what why is this work? It's like if you're logged into Today was a classic example. If you log into a banking site, a server sends back a session token. Again, this is because HTTP is stateless. So each request is kind of independent. But what happens if I get in a separate tab, I have an email open, someone sends me a link, and it sends me to something that looks like my banking site. But it's not actually the banking site, but my browser already has loaded my token. So basically that person on this fake banking site because I'm logged in and another tab, in this new tab can act like me and can do all sorts of nefarious things like take out money, because I'm still logged in. So this is sort of the fundamental trade off of the statelessness of the web is this security problem of how do you know that? another tab open? is actually the correct one?
Carlton Gibson 17:21
Yeah. And so what you do, what the CSRF does is it injects this CSRF token in it, you know, if standardly, it's required for all post or POST requests, which are going to change something. And if the post data doesn't include the correct CSRF token, then that the post request will be rejected, even if all the other fields were bad,
Will Vincent 17:42
but you still you have to add the CSRF tag.
Carlton Gibson 17:45
Yeah, so right. So if you're rendering a form, you have to add that Yeah.
Will Vincent 17:49
Yeah. I mean, that's, I wonder if almost never would you not want to have that? There automatically.
Carlton Gibson 17:55
Yeah, no, I mean, like, This used to be a big issue and it's almost not an issue at all anymore. Because the modern right web frameworks, not just Django. But all the modern web frameworks handle this in more or less the same way. And it's kind of as long as you follow the practice and you use CSRF. Yes, you really are protected.
Yes, it's nice. You know,
Will Vincent 18:19
what else, there are some others that lie can go through. So click jacking. That's another one. There's a middleware to protect against this. That's if someone puts a hidden iframe. So an iframe could be when you put another site in a site. So for example, if you have a Google Maps embedded on a site or a YouTube video, but what happens is someone can just because it looks like something doesn't mean that's what it is. So someone could show an image of a kitten. And you think, Oh, that's a cute kitten, but actually goes to Amazon and tries to make an order or something on your site. So Django has clickjacking middleware checks, basically, whether a resource can be loaded within a frame iframe on the page, is that right? I see one.
Carlton Gibson 18:56
Yeah, yeah. And you basically want to say no,
Will Vincent 18:59
yeah, yeah. Almost turn it off, but almost never Do you want to?
Carlton Gibson 19:03
Yeah, I mean, I can't think I've wanted to actually embed my own site in a wireframe. Or if you'll if you'll if you'll discuss, right where they've got that comment widget thing. Yes, you embed it, then Okay, then you need people to be able to embed your site in an iframe. But if you're not discuss
Will Vincent 19:21
that, discuss, discuss, yeah, it's D is that every time you pronounce it? Yeah,
Carlton Gibson 19:26
I don't know. It's comment. So I thought it was discussed. But yeah,
Will Vincent 19:28
Bella Django? I believe so.
Carlton Gibson 19:30
Okay, well, they all go discuss. But I you know, if you're not then when have you ever wanted to put your site? And I feel like,
Will Vincent 19:38
yeah, I mean, the only you know that. I think that classic example. It's just like a static site where you want to, I don't know, like a WordPress site for a client, we want to put in the map so that people can find it, something like that. But on a Django site, you can still do it. But that's the only example. I
Carlton Gibson 19:54
don't know. I mean, there are you know, if you, I don't know, like these little widgets where you want to embed it in Some of the site then yeah, okay, you might use an iframe there, because they're still the most unique. They're still the most powerful way of being able to do stuff on your site.
Will Vincent 20:07
Right. But it's also the the danger of a widget is you're trusting that widget and all their friends to be good actors. And they may not be or maybe they are, but then you're still using the widget and someone takes control of it, or, you know, so this is so the problem is like this pattern of you
Carlton Gibson 20:24
have suggested this pattern a few times on the podcast. But if you're in this kind of situation, then what you actually want to do is have two Django apps, you want to have your main Django app, which is sensible and doesn't let you know, it's got all your on site functionality. And it doesn't let anybody embed an iframe. And then with the same project, you just want another with the.py file somewhere else that serves a an optimized and minimized Django app, which just does the iframe necessary bit and it doesn't have access to all the other stuff because you don't want that you don't need to have all that in all of it in one application. The same with async stuff, you might, you might need only a couple of endpoints which need the new async stuff or just save those from us with a, with a different worker with a different, you know, server and have most of it going through the, you know, tried and tested whiskey pipeline, and then just have, you know, your WebSockets coming into a different work a different server input or different process.
Will Vincent 21:22
Yeah, that seems like good advice to me. So. So what's the takeaway? So security's important Django basically has you covered, but you have to use Django appropriately. And that would means I would say, definitely run the deployment checklist yet, make sure you're using environment variables. If you do those two things, Django will hold your hand. There's a host of smaller things, but that gets you most of the way there if you pass. I'd say one more that you got it filled filter input and escape output. And if you do that, in the major attacks are coming. Yeah. So pretty thing. So
Carlton Gibson 21:54
because it's a it's cross site scripting, right? If you don't, that's the big dangerous. Suddenly but Django
Will Vincent 22:01
using Django templates, they automatically auto auto escape this. Yes for you to do and forms also do check validation and our forms will also protect you.
Carlton Gibson 22:13
Right But the danger the difficulty comes when people want to, for instance, allow user submitted URLs. Okay, so then you've got a you've there are filters like you are IRI to Ura, which you should run any user submitted URLs through to make sure they're incorrectly URL encoded, and, like, escaping a bit more complicated than Django does it all for you? Yes. If you, if you if you just have the if you just take the user submitted content and you put it into a template, Django will auto escape it, but sometimes you don't want it escaped because it an escaped URL won't work. So you need then to run it through the appropriate filters to make sure it's safe to present that you're
Will Vincent 22:57
right. And you mentioned to just the Django bleach package which we'll link to, I wonder, which is mosaic is in Mozilla package. So this is
Carlton Gibson 23:03
say you want to you, you actually want to present user submitted HTML. So you can't escape it because it ceases to function as HTML once it's escaped. So you you have to sanitize it and bleach is a sanitizer and it will enable you to provide a very limited set of tags and a limited set of attributes and anything which doesn't match those get stripped out. So for instance, GitHub have a markdown editor field, right? They you can't put any old HTML in there. There's a very limited set of HTML use true try putting your script script tag in there, it just won't appear. But yeah, they filter that out.
Will Vincent 23:41
I was just wondering if so there is that jenko bleach package if if that if you would recommend using that versus Oh, I've never used
Carlton Gibson 23:49
I didn't even saw I didn't even know there was a giant bleach package. I don't know. Yeah.
Will Vincent 23:52
Yeah, they may not bleach. And I guess it was this last tangent. I mean, that's something I find more and more as I'm perfect. In my Django journey is that I see, well, specifically with markdown, I'm building something that I'm using markdown and a whole bunch of Django, markdown packages. But I mean, I can also just pull in, what is it markdown to whatever the Python library is and create a custom template, custom filter. So it's sort of a nice place to get when I look at an app and can think, do I really want all that functionality and all that trust that comes along with it? I mean, a lot of times it's helpful, but if I can just do it manually, more and more, I would rather roll it myself if I can do it concisely. Okay, so this
Carlton Gibson 24:35
Yeah, and this comes back in the neatly full circle, the number two risk we said was not updating, and why don't people update? They don't update because there's a third party dependency which isn't compatible with the new version. Yes. So don't take on third party dependencies is is a big issue. We just had one with Django today, Python 3.8 has been released. And there's this trace back there that we use with the test runner to format format tray specs. And that's great. But it's got a bug that's fixed in master against Django variants python 3.8. But it's not been released yet. So we now have, we've had to insert a conditional skip this test, if this version is lower than because in order to get around that, and Django doesn't take on dependencies for exactly this reason, because then we were tied on them. And okay, that will be released soon. And then, you know, we'll be able to drop that skip if but projects really get stuck. Or I'm using, you know, this third party package, which hasn't been updated in three years, and it's not compatible with any supported version of Django. So I'm on an end of life Django, because of some third party package that I took on, because it saved me half an hour on the first day.
Will Vincent 25:45
Oh, that is so true. That's so true. I think I think half our listeners are gonna go in half are gonna like that's an interesting idea.
Carlton Gibson 25:52
But, but so don't take on third party dependencies. I mean, not not unless, you know, it's just like don't do it lightly. Yeah, don't don't don't do it like easy. What's it's up to So like, you know, okay, you mentioned crispy forms that hadn't been updated hadn't been updated for a year and a half, it hadn't need to be because it was 2.2 compatible A long time ago. And we've just updated a release, which is 3.0 compatible. And since my talk at Django con Europe, I've had loads of new contributors to help me get the bootstrap stuff going much better. So I'm really pleased with that. Thank you, everybody. Right? Right. But the whole point is crispy forms is stable is whatever a year year and a half between releases is not a problem for crispy forms. But a package where it's new, you know, it's only had 30 commits, it's only got one contributor in it. B, you got to be really cautious about taking that on. But can you can you get what you need from it without having a third party dependency? If you can, I would recommend,
Will Vincent 26:43
right, I almost wonder it'd be nice if there was a, like a checkmark that you could see Oh, something hasn't been updated in a year, but that's because it's fine. Or because it's super stable, right? What like a stability thing because even crispy forms or you know, whatever package is stable. There's still going to be a whole ton of issues and NP Rs. And if you're not knowledgeable, you may well there may be some and you may look and see, well, this hasn't been updated for a year and a half, and I see a whole bunch of prs. I don't know if I can trust it. And turns out those are minor things versus a boo like there isn't Python three support that? Yeah.
Carlton Gibson 27:19
I don't know the answer to that. Be honest. That that's difficult. I mean, that took
me a long time. I used to be like, Oh, you know, just bring in this third party dependency and bang some stuff out. And then you just find yourself getting into a spider's nips, is that a
spider's nest? spiderweb. I don't know.
But tangled up, you get tangled up in these dependencies, which don't really giving you much value.
Will Vincent 27:41
Okay, that's a lot to say insecurity. Hopefully that helps everyone. We'll have links to everything. as ever. We are at Jango chat, calm chat, Django on Twitter, and we'll see you all next week. Bye.
Carlton Gibson 27:51
Bye bye. Join us next time.