Adam is a Django Technical Board member, longtime contributor, and author of the new book Speed Up Your Django Tests. We talk all things testing in Django, performance tips, continuous integration, pytest, easy wins, optimal test structure, and more.
Carlton Gibson 0:05
Hi, welcome to another episode of Django Chat, a weekly podcast on the Django web framework. I'm Carlton Gibson joined us up by Will Vincent. Hello, Will. How are you? I'm good. Hi, Carlton. Hello world. And today we have Adam Johnson back with us as well, longtime contributor, my SQL expert came on the show before just technical board member just released a new book, speeding up your Django test. So we're going to talk about how Adam, how are you? Hello? Hello from sunny London, Sunny London. Is it sunny today?
Adam Johnson 0:34
No, no.
Will Vincent 0:36
Evening. So what's the difference? Right?
Carlton Gibson 0:38
It's allowed to rain at night.
Will Vincent 0:39
So you're you're on for a second time we'll try to focus the discussion on your new book on testing since that's a topic that we can always talk more about. Do you want to quickly mention the book and why you wrote it? Sure. So the book is speed up your Django test.
Adam Johnson 0:56
The title is like it sounds quite nice, but I guess I ended up writing quite long other stuff in there, just like through the lens of like, you've got tests, but you want it to be fast. And so it covers a lot of like Easy, easy ish techniques to like flip some configuration or like run your tests in parallel, that will make them faster. But that also dives into like, what is the test? How can you write them to be fast? And how can you
Will Vincent 1:25
be accurate? And we should mention your quick bonafide, I mean, you've worked with Django for well over eight years, you've worked on the Django testing library as well as a number of pi test plugins. Is that correct?
Adam Johnson 1:38
Is that nearly eight years with Django, so you
Will Vincent 1:41
don't need to be unduly modest when you talk about talking about testing in Django? Just want to get that out there. Well, maybe just a quick question. So. So why performance? At what point does performance become an issue? Because for a lot of people, just having tests as an issue, so it seems a little bit secondary to focus on performance, and I know that you talk about testing in general through that lens. But what's when do people realize their tests are slow? Like At what point in the process or the size of the codebase? Does that happen for people?
Adam Johnson 2:08
Yeah, I think this is something I didn't touch upon much in the book on the app update about, like, how fast should test be. And I think like, maybe one reason people don't write test is because they're perceived as slow. And normally more like it's slow to write, like, I don't know what I'm doing. It's a whole different paradigm of coding. But also, it can get quite tiresome to run this thing if it takes one minute, even each time. And that can be quite a noticeable difference to a flow. But I know a lot of companies like they wait until that has take half an hour before they even consider something. This is what happened to me in my previous position a few years ago, why plan tests were taking 20 minutes, 25 minutes. That's like a very slow feedback cycle to make a typo or realize that the future doesn't work in this situation. and not get the feedback.
Carlton Gibson 3:01
Can I just ask that? So okay, the whole test suite takes 20 minutes. Okay, that's too long. But you don't run the whole test suite when you're writing some new code, right? You You run one or two tests individually, and they only take you a couple of seconds. And then the last part
Will Vincent 3:17
of a karratha. Not everyone.
Adam Johnson 3:18
Yeah, I guess that's one thing. But yeah, and that's definitely something you should do. And but it's also like, generally, poor requests are gated on the whole test suite passing. Yes. So if you have, like a bug in production, you don't want to be waiting an extra 25 minutes before you get that fix out. He can easily become the thing that everyone skips even like, oh, the tests have been slow. Let's just merge the code. And then you just go.
Carlton Gibson 3:43
It's Friday. It's lunchtime. Ah,
Will Vincent 3:46
yes. So in terms of writing tests, Python has built in unit test framework. Django has its own test. Was it based off that test? I should know off the top my
Adam Johnson 3:56
head we call it Django testing framework. It's not really good. something great.
Will Vincent 4:02
But you know you and basically everyone I know who works with Python or Django uses pi test. Could you just talk about why that is? And I know you've noted that, you know, flask also recommends pi test. How do you think of those stages of unit test to Django test to pi test?
Adam Johnson 4:19
Well, I guess unit test I see is like something slightly historical and the Python context. And as Python was getting started, unit testing was also getting popularized. And through j unit, which was this Java framework, and then it's clones. So unit test is effectively a part of the Java testing framework. And that's why you might consider it quite an pythonic. And the way it means things like there's all these assert functions, and they're all in camel case, and you have to put everything in classes, which is something that's not necessarily pythonic. And so, we've got unit test, Django just built on top of what was there Python, and and there have been like a bunch of attempts to make a more pythonic testing framework. I think pi test is the one that has stuck with the community has a lot of effort put behind it. And so yeah, I think that's why we tend to stick with it. That said, like, when it comes to Django, like there's really a balance, we've got all these testing features in the Django testing framework. And, like Database Control, like extra assertion functions, and things like the setup test data, which I have a whole section on, because that's a real speed boost your tests that you can easily replicate, reproduce in PI tests, or at least haven't been. And so I think you really have to mix them at this point to get the best of both worlds. So you use like the Django test case classes. And with PI test,
Carlton Gibson 5:52
I mean, the thing I really liked about pi test, weather first came across it was the plane asserts, right because Even now I have every time I'd like self assert, what if it's not assert equal? Or assert raises messes with us all the time? I have to go and look it up because it's just, I can't remember those methods but assert
Adam Johnson 6:14
assert equal or equals. That wasn't even something that
Will Vincent 6:18
I still I still, yeah. Well, honestly, off the top my head, I can't recall which of the one is the one you're supposed to use now, except it's one of them, not the other.
Adam Johnson 6:29
Exactly. It's equal. And you just you get a warning if you use a equals, but it still works. Just add extra warnings, tests.
Will Vincent 6:37
Yeah, I think I've stripped I think I've updated all three of my books, because they've been brought to my attention that that, you know, past ones wasn't always like that. Well as I did want to. So speaking of these three different testing things, one thing I've seen with total beginners is if they're new to Python, they're new to Django. they're new to unit test and new to Django tests. A lot of times, you know, everything is Magic. So were the fact that the Django testing things, mimics unit tests. Often they don't know what's Python versus what's Django let alone what's pi test? So I really am glad that you have your book to go further steps because I see people, I mean, probably I myself to have confused, you know, what is a Python thing versus what is it Django specific thing until I look it up? So I find it interesting that there's, it's convenient, in makes sense that it follows those Maxim's but also means if you just kind of drop into a project or blindly follow along with some tutorials, you really don't know what's going on with these different layers of testing things that are using a modern Django stack.
Adam Johnson 7:43
Yeah, it can be quite hard to figure out which piece of documentation I'm going to go look at, even
Will Vincent 7:48
while it's so separate from, you know, the official docs and in your book, how, how do you think about, you know, so if I came to you with a Django project that, you know, maybe didn't have tests. How would you Where would you start? How would you structure that? And how would you educate a client who maybe you know, has a code base and their engineer quit or something and they hire you? How do you do that? And?
Adam Johnson 8:10
Well, I guess we're testing is boys depends on what the aim is. And I think we know, in general, like, you can test exhaustively, but it's not always worth it. Like in the hardware world, they draw the they have a much higher bar, they draw the line way further out, they will test things exhaustively. Every every CPU design will just be stressed tested for days at a time. And it's not something we tend to do in the web community because it's quite easy to push a fix. But I like to try and keep a higher bar, I guess, than most projects and aim for something like 100% coverage. And but it really depends on what we're aiming to do because if you've got a website that doesn't support margin of importance. Like if it's a Jeff sharing site for you and a few friends, maybe you don't even need tests, you can just fix what has problems. If it's like a voting website, then I hope it has like very good tests.
Carlton Gibson 9:15
Somebody had asked you about coverage, because like, there's one way to get 100% coverage quite easily is if you test do write big, broad, broad integration tests that sort of I know, they test the total outside interface of the project. And by doing that, they call through every layer and you can get quite close to 100% really quickly, but they're not good tests right now.
Adam Johnson 9:37
Yeah. That's another thing. I think the branch option A coverage like restricts you a bit more from doing that, because that means that every f has to branch both ways. So it can be very hard from the outside of the project to run exhaustive tests before they become slow. And yeah, that's in coverage five, that's the new format and ability to build plugins that would like to assert that this file is covered by that test file. And so hopefully we'll see something for Django, where you know, you've got your models file that must be covered exactly hundred percent in a test models file, you've got your views file must be covered exactly by the test views.
Carlton Gibson 10:17
Yeah, that's a real help, right? Because, well, it's easy to cheat.
Adam Johnson 10:24
And not always a bad thing, right? Like, if you just have that one integration test, That's way better than nothing.
Carlton Gibson 10:30
Hey, and like those tests, those tests work, right? Like so you need to check your login flow, like that kind of high level integration check. It does this does that does that that brilliant, and it'll catch things. But for me, one of the advantages of one of the things I really like why do I tell the reason I really love testing is because it enables me to make changes. I know I didn't break anything. And if, if I've got small unit test, covering little bits of functionality, I can make those changes knowing that didn't break, didn't break anything in adjusting stuff. And without that you can't do it.
Adam Johnson 11:06
Yeah, if you have those things that just test your log info, you have no idea if this particular thing in the login view that you're tweaking is actually working on.
Will Vincent 11:17
So the book is on performance wins. But in terms of talking about it, there's a number of kind of easy ones are things that you have whole chapters on what what comes to mind if someone says, well, Adam, what are some performance things I can do to improve my tests and
Adam Johnson 11:31
so like, I have this easy wins chapter that is basically a bunch of configuration or things to install. And some of them are quite context dependent, and but some of them apply to nearly every project. So you've got like changing authorization password hashes, that something was in the Django Doc's, I just try and explain it a little bit better. Basically, they're like, Django goes off module uses a deliberately slow password hashing. So every time you Check a password, which in tests will be every time you fake a login through the test client. And it takes a few hundred milliseconds of processing power this, this adds up. So in tests, you can swap to one that basically does nothing. And it's very, very insecure, and should never be deployed in production. But next it has fast.
Will Vincent 12:20
Well, sort of like the built in server among
Adam Johnson 12:23
Exactly, yeah. And then there's like a bunch of packages, you could install or configuration to change because Jang has some built in for in memory usage. So like, disk is still like 40 times slower than memory. Even with a fast SSD, you're not going to get memory level performance. So just swapping your test use in memory. That's normally a big win. And I guess that final final thing, that'd be quite an easy thing to switch over to, and especially when your product is young, is parallelizing your tests. So this is built into Django test framework as a very good point. Again, for pi tests, but you know, your computer has many cores these days, you should use all of them when writing tests. It's no good just burning through one of them when you could be using 12.
Carlton Gibson 13:13
Or to come back to the sector because you call this like, easy, easy, quick wins. And yeah, just issues in memory. But like, that chapter is amazing, right? Because you're like, Oh, well, you know, you can use it in memory, you think of an obvious one file storage or something. I don't know. Okay, that's, that's, yeah, that's not then you're like, oh, but you know, you could configure my SQL and Postgres to use your memory, you know, in memory file system. And then, you know, you've got the memory performance that you get a SQL lite, but for Postgres SQL, you're like, Oh, wow. Hang on. That's pretty cool.
Will Vincent 13:49
Yeah, that was one I also was like, Whoa, I hadn't thought of that one.
Adam Johnson 13:52
Yeah, that's just like one line for Docker. If you're using Docker for your database, you can tell it to mount the database on a temp Fs And like I put it in the Django mice code test suite. And I took 20% of the total runtime.
Will Vincent 14:08
And also for those playing Django chat, you know, drinking game Docker was mentioned. So
Adam Johnson 14:15
Doctor, doctor, Dr.
Will Vincent 14:18
Carlson, please, I have a whole bunch of questions. But Carl tonight I tend to
Carlton Gibson 14:21
okay. So, you know, one thing that comes up with tests all the time is, and this is slightly orthogonal to the topic of the book, but his database, because that's, you know, database is going to be one of the slowest aspects, but no matter what you do, and Django, you know, very much tied to the database through the ORM. So, I mean, what's the question? Now, I guess, it's really important to optimize there for your Django project, because the nature of Django the way you build your develop with Django, because we're model based because the arms they're optimizing. That's important, I guess. What would you say? I mean,
Will Vincent 14:55
well, that also ties into migrations, too, which I know you have a whole thing about in the book.
Adam Johnson 14:59
Yeah. But tackle just like using the ORM first. And so so the ORM is basically a translated objects to SQL and then calling the database with SQL. And there have been a number of attempts to build like a mock ORM, which keeps the objects in Python. And I think, like this, this is something that could work for like a very limited subset of the API. But SQL has so many oddities to it, especially the variant of SQL you're using, like, there's so many differences between Postgres and SQL lite, even, that I wouldn't want to test against, like a fake version of my database, and there's just too much of a mismatch. And so I think it's really important there to focus on like, how can we optimize the actual database we're connecting to like in memory is one way paralyzation and just use multiple databases that can use multiple cores on your computer. That's really good way of doing it. upgrading to latest version also, generally, you'll see a small or notable improvement in performance. And migrations are a bit of a different kettle of fish. Because they're like this whole bunch of SQL and Python operations that you have to run at the start of tests to get a fresh database that your project expects. There's not really like a way around that. You have to get that database into the state that your program expects it like, as soon as you deploy your app to production, that's the state the database is going to be in. And so there's like the keep DB flag in Django test runner and an equivalent in PI test Django that keeps the database around between test runs. So if you rerun the same set of tests against the same thing, you can avoid that have a massive overhead. And the other thing is like squash migrations regularly. So combine like All these long sets of operations into one slimmer set.
Carlton Gibson 17:04
Yeah. So you in the book, you've been very follow the party line, you're like, Yeah, yes, the squash tech, I mean, what's what's your off the record view on, you know, just sort of deleting or your migrations and clearing out migration history table and starting again with one initial.
Adam Johnson 17:22
I've never actually done that. I guess because the biggest Django project I worked on was the Y plan, and it didn't have Django migrations. And it did for third party apps, because you can't get away from it for third party apps, even like Django controls. But it kind of predated even South being good. So there was a custom migration runner that just ran a bunch of SQL files in a directory. And it wasn't the easiest to work with. And, but I guess it was faster than Django migrations, especially the time As to like, actually just deleting all your migrations replacing them? Yeah, I think it works. I think there could be even some documentation on how to do that. Because there's so many caveats. Like if you're deleting run SQL operations, you're not gonna have a database in the same state,
Carlton Gibson 18:20
run Python. I mean, but you the key point is that all your databases need to be migrated to the exact same point. And, you know, then Okay, maybe, maybe, maybe,
Adam Johnson 18:32
yeah, and I mean, maybe it's safer at that point to just take a schema dump of production and load that and then write your new migrations on top of that. But
Carlton Gibson 18:43
the other chapter I really liked was some profiling because I think profiling is kind of something that we will know in theory we should do, but that you very rarely seen a good guide to it. And I think your chapter on profiling system mating isn't yet the It says here ladies, here are the tools. Here's how they fit together. Here's the you eyes that you felt. And I'm like, yeah, that's, that's almost perfect. And that could be a book in itself.
Adam Johnson 19:07
Now maybe I open it up. And, yeah, profiling is something that has helped me spot a bunch of stuff in January even like, so the two tools are really covered in the book, our C profile, which is built into Python, and pi spy, which has this rust based profiler, that it knows how to read pythons memory layer. It's quite magical. It works with like Python to seven up to three, eight, maybe three, nine now. It could just read what is happening inside of Python. And it does this like 100 times a second it pauses your program checks what functions are running records that and keeps going. And, and like I've been able to spot like, Oh, this Jenga function is in the top 20 functions in my test suite. It could definitely be cached inside of j Go and things like that. I think we should be doing more profiling in general, as a community. I've definitely had some, like optimization pull requests made my projects. I have a sense that this is in no way a bottleneck, if you read the code, you might suspect it's a bottleneck. But if you did a profile, you might not even be able to spot it. So
Carlton Gibson 20:19
yeah. I mean, that's one of the sort of golden rules right is don't don't guess what you're spending your time measuring.
Adam Johnson 20:27
I did originally start that chapter with the old Donald Knuth. Quote, premature optimization is the root of all evil, although I changed it when I was rebalancing the agendas of the quotes.
Will Vincent 20:40
Fair enough. So So actually, so caching we just mentioned, you talked about that in your continuous integration chapter. And I wanted to ask more broadly, so there's a lot of options for that and I hear anecdotally, almost everyone kind of uses something different for that. Like what What is your preferred choice? And kind of what is the landscape of good and bad options you've seen for for ci and Django projects?
Adam Johnson 21:08
Right. So what ci should you be using? And yeah, I don't know. That's something of a moral question. Alongside technical Austin,
Will Vincent 21:17
you have a Greenfield project. It's all Adam. What are you gonna use? Well, if they're repos on GitHub these days, I would use GitHub actions. Okay, but I heard a number of people very much in favor.
Adam Johnson 21:31
Yeah, it works great. And it's fast. Microsoft thrown a lot of money getting a lot of runners, they're ready to run your open source projects or your closed source projects. A lot of community work. But I moved to it because Travis CI was being slow. And then I discovered that it's because GitHub introduced a new rate limits. This was after I'd moved to GitHub actions, but because GitHub introduced a rate limit on On the way Travis was integrated legacy wise, so Travis just couldn't push the results back and you needed to go and re integrate your app with the new, like, GitHub apps rather than GitHub legacy integrations. So maybe I wish I was still on Travis had some benefits.
Will Vincent 22:20
Yeah, cuz that was I mean, Travis and circle and Travis in particular was kind of the had like a good default for open source packages. So are you saying so does that is that something malicious on Microsoft's part, or more just to change and they didn't decide to update it?
Adam Johnson 22:37
I mean, I guess I don't really have a read there. I never received a notification as a Travis user. And probably Travis could have detected that. But I never received a notification on GitHub either. So get up to the detected them.
Will Vincent 22:49
I mean, it could be all the above too hard to know.
Carlton Gibson 22:52
My little two cents on the same question is the like, get ABS get vaccines is nice, because it's right there. And it's it's awesome. But you know whether you use circle ci, that's great. That's fine, Travis. Good as well be if you've been using that for a long time with the open source stuff. Use a hosted one right? Don't Don't lay unless you've got a real reason don't spin up a Jenkins and try and manage that because that's hard work.
Adam Johnson 23:16
Okay, yeah, that would be my one rule. I like the last time I touch Jenkins. I was like, I'm never ever going to use Jenkins. Again,
Will Vincent 23:24
it seems like it's the most common legacy thing, right? Because it's free as in, not developer time free, but free. But I see that the most. I mean, I've used that. And when I ask people about this, often, if they're not using a hosted one, they're, you know, they're some version of Jenkins.
Adam Johnson 23:43
It's not that it's not good as content. It's just it's just so much time keeping it up to date and running in sync with what stuff you need. Yeah,
Carlton Gibson 23:52
I remember the first time I came to hosted ci and you know, it was just like the scales falling from my eyes. I've been, you know, we've got to do this so that I would go around to come companies current clients and be like, yeah, we're gonna use hosted so yeah, that was like, you know, my hammer of the year.
Adam Johnson 24:11
Yeah, like anytime you save, managing your ci environment is time you get for testing or feature development.
Will Vincent 24:17
I have another question. But Carlton, get yours in.
Carlton Gibson 24:20
Okay. Well, the other thing that I said, what I really liked about the book is there's so many things that aren't testing related at all. So we've talked about profiling, we're talking about ci now, I mean, I see is testing but you know, kind of also is, it's not speeding up your unit tests. It's the whole development process. But then there's a whole little section on queues, which, you know, configure your queues and it's like, you might be using this queue. And so you configure that and it's like, it's like,
Adam Johnson 24:46
going for tests. But yes,
Carlton Gibson 24:48
Yeah, I know. But it's like, you know, it's like a sort of mini essay on you know, the different cue options that are available and everything Yeah, super all my favorites are all in there. And it's like, yes. So that's what it's kind of what I like I like, like, how you've gone over the entire spectrum of stuff. And it's not just, you know, it's not just tests in the narrow sense. Like, I've read a lot of books on unit testing, it's like this the whole process. So it's not really a question. It's just,
Adam Johnson 25:18
I guess it's a bit discursive in that way. But I hope the titles let you like skim the bits that you don't need to know about right now. And come
Carlton Gibson 25:26
back to them when you think you do. Yeah, but they still occur. I mean, it's anything but it's like a massively awesome resource. It's only 160 pages, right? It's not like it's like this distraction, distracting, but it's just massively dense. And what else is in there that I really liked? Like advanced trip, tips on configuring the test case sub classes, and you know, using test case sub classes and I'm trying to think take one but you know, that's that's hard knowledge to gain So, you know, thanks for putting it all down.
Adam Johnson 26:03
Thank you. First of all for
Will Vincent 26:04
Well, maybe this is the the segue, I did want to ask at one point about the writing process for you. So I know that you've, you've been writing almost weekly, what for months now? And so the book panic kind of came out of that. Could you just talk about upping your, you know, output as a blogger, and then what the book process was like, because I think a lot of people think about potentially doing a book, but most people don't actually do it.
Adam Johnson 26:30
Right. And well, I guess, like, it looks like I write every week, but I don't like to queue up unpublished posts, and then when it comes time to publish them, I look at the ones that like, I'm like, this is this is nearly ready, and then I push it. And I guess that is one thing that I've learned is that like, you don't need to be like perfectly ready to publish. And like take that MVP approach that we put in startups and put it in writing because normally, it's Better that something is out there, then it never gets out. And, and you learn more by doing the whole cycle not just by like writing one complete perfect post a month, but by writing like four different tutorials of different clients and things like that. So the book came out of me sitting down to write a blog post, which was like, gonna be like top 10 tips to speed up your Django test suite. And like, I wrote a bit of this, that and then I had like a 4000 word draft or something. And that wasn't even like in prose that was just like bullet points and like, do this thing copied some text from elsewhere? And yeah, and I guess all that stuff was like, already in my head and my notes from like, years of working with Django tests, and I made drafts like, this could be a blog post on this one little thing and I realized that just bringing them all together would actually be more powerful. Yeah, I guess locked down was really the biggest motivating factor for getting this dunk. Send it home all the time.
Carlton Gibson 28:03
only so much Netflix I can watch. I'm gonna have to run my book now.
Will Vincent 28:06
Yes. Yeah, well, kudos for writing it and publishing it is, you know, it's kind of like until you're on the other side. It's easy to hear people say that but then you realize it though, I will say if you I don't know if you're planning to put it into paperback, but that's a whole another fun thing to deal with doing and updating.
Adam Johnson 28:23
That's maybe another future program. And I've got a bunch of updates to print because, like, polished a library this week for mocking time because it was barely actually I wanted a section on how to mock the current time but I can't find a library that would be like perfectly recommend I knew of the trade offs between two of them so I wrote the third one. And so yeah, I think that ebook form will be me for a little while. While Sally iterate, add a few more sections and maybe rearrange stuff. And
Carlton Gibson 28:57
I think because it's technical, then you know, maybe like to take examples of GitHub actions or whatever, you know that that's liable to change. And so to have it in ebook format where you can
Adam Johnson 29:09
update it, and the good thing is like I have all these Django projects for the code, they get included. So I can like upgrade my requirements to Django, three, one and October, rerun all of those tests, although some of them are meant to fail for examples, and then it will definitely update process. I don't know how often a new update will and
Will Vincent 29:30
too often it can be very burnout inducing for me at least once a month. You know, some version of something I did wrong or Django changed or Docker changed or Python changed or Heroku changed or stripe changed. It is important to do it is also important not to jump on everything that comes in because I have an endless supply of stuff. I mean, I'm currently updating My Django for professionals book because there's stuff around stripe and then they actually Well, we could talk about this. I asked on the Django forum, there's a thing around. What is the thing I asked the question, you helped? You helped answer this both of you. Give us a clue. The thing that was one of the in the deployment checklists, something was added for three Oh, that was flipped. And then in three, one, I think you did it Adam is going to be switched again, I forget which secure refer. So people noted for me that that had changed because in two to the two two version, there were nine things that if you just ran the deployment checklist failed, and then it's now seven and then yes, secure refers one that changed. And then also the default will change is there currently what isn't a default but then for three one, it's going to be set to something else is it stuff like that times 1000, where it's educational and I'm really glad people do it, but it's Yeah, so I I try to get better about devoting specific time to updates, rather than just having it constantly weigh on me.
Adam Johnson 31:08
Yeah, I was an upgrade. I was reading a post fool's book a beta testing goat. And yes, and he actually like deliberately pins it Django 111. And a specific Python version. It says, use these throughout the book. I'm not going to support the others if you see random errors, it's probably because of that come back here and and install the versions I'm working with. Yeah, and he'll updated at some point but he's like very tightly integrates a specific versions. I think that what
Will Vincent 31:39
we have to pin it, I guess I would say the two things I would say is, one is beginners need it to be up to date or need and want it to be up to date and they will still install the latest version of Django even though you tell them not to, and they will get tripped up. So for maybe you know, Harry's book is a little more advanced. You can get away with that for a beginner book. I think you have to The other thing is anything that's non Django can and will change a million more times. Like I mentioned the stripe API. I would love to have nothing but Django and Python in my books. I'm kind of getting there. Instead of waving my hands and saying, Yeah, Heroku Yeah, stripe, I think I'm probably actually going to remove the stripe chapter, for example, because it just change, it's going to change again. It's more than I can handle, I probably will keep up Heroku because I do want to show easy in quotes deployment. But anytime you step outside of Python, Django, you're just opening yourself up sendgrid sendgrid changed, you know, in April, all of a sudden change the config. So chapter 12 in my beginners book was broken. So that was fun. You know, so, you know, I'm hoping to, I think either you have to charge a lot or just not do it to show integrations with non core Django things even if it's things that a modern Django stack probably uses as a content creator and updater it's really, really hard to do.
Adam Johnson 33:03
Well, one of the things I deliberately dropped from my book was, well didn't put in was that Docker. There it is, again. Jeff Triplett found posted about it. There's like a new build kit integration in Docker, like this new way of building images. And it's like, I've know, a bazillion times faster. I was thinking, Okay, I'll put this and then I was like, No, this is gonna change in like one month's time because they'll make a default or something.
Will Vincent 33:30
Yeah, yeah. Well, I didn't even mention Docker in that list. They, yeah, there was some there was another bug, like a month ago where people you know, dozen people are writing me and it's like, I don't know, I got to put on my thinking hat. And there was something specific to Windows Docker. Not Mac Docker, not Linux that was breaking. Which, you know, the whole point of Docker is to not have to worry about systems. And honestly in my professionals book, which I know you've gone through with, with someone you were teaching I I may well remove Docker from it in future additions, in part because it's just another layer of complexity even though it is quite common. But I don't know it's a trade off because it you know, because these are modern tools, right? Like, you know, pi test right pi test is not Django. I mean, it kind of is, but it really isn't your ci, it's hard to strike that balance. I think it's a little bit easier, maybe in more advanced things, or it's maybe more easier for things that are you charge a lot for, but you to update stuff for, you know, Docker and whomever it's just makes people not update their stuff.
Adam Johnson 34:33
Yeah, with Docker, I've, I've tended now towards just using it for running the database. And like Redis or memcached. And, and don't use an image for the Python stuff. I feel like it's, it's quite a bit easier. Just have Python on your system with pi m or something and you've got it there. You know, Python is working on your terminal. You don't need to build images again and again, update
Will Vincent 34:59
your team. Setting as well that works out. Okay.
Adam Johnson 35:01
And yeah, it has felt like one client laughs
Unknown Speaker 35:04
What were you gonna say Carlton?
Carlton Gibson 35:06
Well, I kind of in the same sort of boat, like, you know, for running, I don't know, some Postgres enabled, you know, database. So I'm not installing that locally because it will destroy my entire system when I get it wrong. Yes, to have to have, you know, various versions of Python setup locally, and, you know, the, the core set of dynamic libraries, I need to compile the core dependencies. That's not difficult to do. And this idea that you you don't need to have a known environment, you know, a reproducible, containerized environment for that kind of local development. Yeah, brilliant. When other cases, Yes, totally. But
Adam Johnson 35:50
I think when you hit a certain size, that's definitely savings. Like, if you're using five different external libraries, and you've got 20 team members, you're going to have one real pain of a time getting or 100 of those installs in sync.
Will Vincent 36:04
So you one of the things you have that I want to ask about is this triple A test structure. Could you explain what what that is?
Adam Johnson 36:12
Sure. Like, I know he invented this and came to me to a friend James Cook at his Python presentation one year pike on UK that is, and, and he's written a blog post on it, which I find myself pointing people to quite a lot. I guess it's something that's implicit in testing, that this is the general structure of how to write a test. And often tests like when they tend away from it, they become more difficult to use, or run or less performant. So the triple A, stands for arrange, act, assert. So you always think of your test as like arranging the environment. So putting that pre existing user in the database or campus collating the URL to go out and fetch or something like that. And the act is like actually using the system under test the component that you're trying to test and storing its result results. And then the assert is when you make some assertions on what happened, like, did it actually update that user? Or did it and return the correct response? And sometimes you have multiple sessions, like, did it both send an email and return the correct response from the view? And But yeah, I tried to recover that in the book as a way of like, here's how you should think about tests and structuring them better
Carlton Gibson 37:40
what I liked about your discussion, there was the way you but this bought into their will, you know, can I group asserts together? Well, you know, think about it. If you're arranging the same stuff, and then you're doing the same actions, then you can make the set of related asserts as though you don't have to duplicate that because that's obviously going to run time into time. But yes, you know, this is why they There's so much more than just speeding up your tests in this book is it's like, Yeah, okay. You've you've sort of tied it back to speeding up your tests. But it's general principles for how you arrange it unit test. So how you might think about the structure of the code as well, I think
Adam Johnson 38:15
that's that perhaps dogmatically applied idea of like a single assertion per test. And then you can end up with tests that like, one fetches a view and checks, it returns a 200 status code and other factors of view and checks that it has a certain header, another factor is the view again, and you've just tripled the work for like fetching the view. Whereas you could really think of all those asserts it's like, they're just asserting different parts of the HTTP response, which is ultimately like a text file.
Carlton Gibson 38:43
One. One thing I'd like to do in that circumstance, where I combine assertions is I'd like to be able to sort of conditionally assert, you know, even if that one fails, run the other assertions Tell me what else went wrong, but the tests stop at the first one that fails, and it's a bit like what but You know, the actual informative one is four down. Sometimes Yeah. What do you think about that kind of scenario? Is there? Is there a magic key that
Adam Johnson 39:09
there are there are different attempts to solve this? And yeah, like maybe if it's only like an HTTP response. So you could imagine some kind of magic objects that you say, you know, they're so culturally, this header, this should be there, this should be in the body. You compare that object with the actual response, and it figures out all the differences then displays an error if there are any. But more generally, there's like a tool in unit test, and therefore Django test framework called sub test. And this is for like a context manager when you mark a block of code is like, this is like part of this test. If it fails still continue. And they capture and show all of the errors at the end. And pi test this PI test support for sub test or there's a plugin called I think pi test check. We have all these check functions that do the same I don't think there's like a perfect solution. Out of all of these options right now, sub test if you want to use it parser that's like doubling up your typing and make your tests much less verbose. And I guess it'd be really nice if just, you could have pi tests, like automatically keep running through all the assert statements. It's not always possible though, sometimes you have an assert statement that does another action implicitly. And that's part of using the a patent. If you have your act like right there in the middle before you do any asserts, then it's going to be easier to make all those decisions.
Carlton Gibson 40:38
Yeah, it's in Django test suite use sub test a lot for parameterization. You know, we've got four or five different versions of the same test, you know, pi test does it differently with parameterize fixtures, but, you know, we use it a lot there for that. But I like the idea of using it for keeping your your tests running. I hadn't really thought
Will Vincent 40:59
was that I mean, so we're around 40 minute mark. But there's two questions I definitely want to get in. And I don't know. Carlton, I want to ask you about mocking. And then I'd like to ask you about the Django technical board and who was on that and what that entails, because it's important, but I'm not sure how aware people are. So either one of those two, I mean, in particular, you have a thing called the danger with mocking, which sounds sort of exciting.
Adam Johnson 41:25
Okay, and was that the question? This isn't like some part, I just be caught up just
Will Vincent 41:30
speaking out loud. So I wanted to ask what the danger of mocking and then also, both of you are on the technical board over technical? Well, let's talk about that job.
Adam Johnson 41:43
Yeah. So the technical board is five of us who have been elected to manage changes technical direction. And this doesn't mean we're involved in every single decision. But the idea is that for bigger decisions, we are the Voting body that can say yay or nay to any particular suggestion. And with the recent implementation of DEP 10. We're meant to be taking a more active role as well. So we'll start seeking projects and like evaluating them, and together as a board, and another column even made a GitHub team for us today so that we can help review Marius prs was cutting us on holiday. And so the fibers are myself, Andrew Godwin, Emmerich, Augustine, Marcus Holtzman and Simon share it. And yeah, Does that answer the question?
Will Vincent 42:44
Yeah, I think so. Just I think, you know, it's a very important board team that I know that not many people are necessarily aware of. So we've had Andrew on the show. We're actually interviewing Eric tomorrow. We've had Marcus on And we hope to have Simon on. But, you know, when I talked to Carlton about who's really with their sleeves up doing things in Django all five of you are I mean, the Mount, you know, not what is it? mount? That's Zeus. Anyways? Mount Olympus Madalyn does thank you on the mat on the Mount Olympus of of active chango things.
Adam Johnson 43:24
I have no idea which God demigod I'd be then probably not a very important one. Yeah.
Carlton Gibson 43:31
Yeah. Like everyone on the technical board. Yeah, I'm impressed with all my fellow technical board members like contributions. I think the two things that I'd say is one that to be on the technical board you you have to be active, but you have to have a show in history to even be a candidate to be, you know, on the technical board, you have to have a shown recent history of active contributions. So that's one thing. The second thing is that the technical Was there there? Is this decision making most normally, most decisions are reached by sort of consensus in the discussion on Django developers, you know, not often. I mean, recently, there's a discussion about adding type into the Django. And though there was there's been lots of discussion over several years. And, you know, lots of proposals and lots of pros and lots of cons. And at that point, there wasn't a consensus that came out. And so, you know, we asked the technical board to, you know, make that decision. But that's quite rare. Normally, you know, there's a bit of debate, a bit of conversation, but people don't Yeah, no, actually, that's the way forward. Which is kind of nice, I think.
Adam Johnson 44:41
Yes. I think we work pretty well as a community and it's good that we aren't needed to make any big decisions.
Will Vincent 44:47
Good. Thank you. Yeah, I mean, we'll link to there. There's a team section on the docks and it is with DEP 10. There'll be a little bit of changes but so mocking what is mocking and what makes it tick. Not the danger with
Adam Johnson 45:02
mocking Yes. And I guess there's just generally a trend to overuse mocking in the Python world because it's so easy. Python as a language, let's do it well, and then unit test mark is like, ready in there to go. And it can just mock pretty much anything. And so I called my mocking chapter targeted marketing, because I think this is really the key point with mocking is that you should make them as targeted as possible, and know what role they're supposed to be fulfilling when you meet them. Whether it's, they're just to check that your system calls another system, or is there to stop you from calling the other system. And then like making them really targeted is is the is the key. If you use a unit test dot mock object, it's like very untargeted by default, it's just this weird, magical Python object that you can call any function on it and it's there and it returns another mock And then that you can compare with anything. And like, I think for beginners, it's very hard to get your head around private, allowing you to do this. Like, there's an object that compares equal to every single number, every single string, every single dictionary. No type errors ever. It can't really like break your program. But in tests, it tends to just like, pass every test, because it just kind of allows everything to happen without actually testing anything. At the end of the day.
Carlton Gibson 46:33
Also, I see people or projects where there's been mocking, and it's mocked out something. But what you end up the unit tests are testing the market, not the not the system. And it's like, but this Why is why did it break in production? And he's got all these unit tests. It's like, Yeah, but in the end, they're not serving anything.
Adam Johnson 46:53
Yeah, right. You draw a boundary, you put mocks at that boundary for the other system. And then actually What you tested against this market is completely different. So again, it's like the mock should be targeted, that should be known to be like a good replacement for that other system. So one of the libraries I mentioned is requests mock. This is a good mock for the pipe Python requests library. It, it actually injects itself inside part of requests with a known public API. And so like quite a lot of actual requests code gets tested when you run this, it's just not the third party HTTP requests. And yeah, rather than like mocking out with unit tests Don't mock the entire interface and you didn't even check for typos with like request dot get.
Carlton Gibson 47:43
Did you pass the right did you call the right method name? Yes.
Will Vincent 47:48
Thank you, Carlton. I don't know if you had any other things you want to ask Adam.
Carlton Gibson 47:51
No, I mean, no, I mean, I'm just I'm just reading the book last night food. I've just, you know, blown away. I was just like, what really good You know, really good thing. It's not for novices. But it's, you know,
Will Vincent 48:05
yeah, it's a Jenga developer. I
Carlton Gibson 48:08
think you use Django, you write unit tests this, like, just get it even if you don't write loads of other stuff in it. It's really good
Adam Johnson 48:19
to write unit tests.
Will Vincent 48:21
Do you personally, Adam, I mean, you have any projects or things you want to call out that you've been working on?
Adam Johnson 48:27
Well, that's the book as on this time machine library. I mentioned earlier, that I really, that's, that's actually my first Python library using C under the hood. And, and I'd like to see it tested a bit more. It's like a somewhat of a drop in replacement for a popular freeze gun. So if you're using that and you're listening, then maybe try it out. And have Matt Just follow my blog. But I guess it's not hard to come across if you're reading things like Django news.
Will Vincent 48:58
Yeah, I mean, Pretty much every week. It's just like, well, there's a couple from Adam and, you know, it's Yeah, along with this podcast. It's one of the weekly things. So good. Well, thank you so much for taking the time. Thank you for writing the book. I hope more people will read it. I know, many people have bought it. But again, I think it's a core part of educational canon of Django stuff. So
Adam Johnson 49:22
well, thank you. Thanks for coming on. Thank you very much for having me again. Brilliant to chat.
Will Vincent 49:28
Okay, everyone, we are at ChatDjango on Twitter. And we'll see you next time. Bye bye. Join us next time. Bye.