This book is my attempt to share with the world the journey I’ve taken from "hacking" to "software engineering". It’s mainly about testing, but there’s a lot more to it, as you’ll soon see.
I want to thank you for reading it.
If you bought a copy, then I’m very grateful. If you’re reading the free online version, then I’m still grateful that you’ve decided it’s worth spending some of your time on. Who knows, perhaps once you get to the end, you’ll decide it’s good enough to buy a real copy for yourself or for a friend.
If you have any comments, questions, or suggestions, I’d love to hear from you. You can reach me directly via [email protected], or on Mastodon + Twitter @hjwp. You can also check out the website and my blog, and there’s a mailing list.
I hope you’ll enjoy reading this book as much as I enjoyed writing it.
If you can see this, you are reading an early release of the third edition, either via www.obeythetestinggoat.com, or via the O’Reilly Learning site. Congratulations!
At the time of writing, all of the code listings in the main book (the chapters up to 25, but not the appendices) have been updated to Python 3.13 and Django 5.
We’re still in tech review, and many chapters still need a little work, but the core of the book is there.
Thanks for reading, and please do send any and all feedback! At this early release stage, feedback is more important than ever. You can reach me via [email protected]
``Who are you, why have you written this book, and why should I read it?'' I hear you ask.
I was lucky enough, early on in my career, to fall in with a bunch of TDD fanatics, and it made such a big impact on my programming that I was burning to share it with everyone. You might say I had the enthusiasm of a recent convert, and the learning experience was still a recent memory for me, so that’s what led to the first edition, back in 2014.
When I first learned Python (from Mark Pilgrim’s excellent Dive Into Python), I came across the concept of TDD, and thought ``Yes. I can definitely see the sense in that.'' Perhaps you had a similar reaction when you first heard about TDD? It sounds like a really sensible approach, a really good habit to get into—like regularly flossing your teeth.
Then came my first big project, and you can guess what happened—there was a client, there were deadlines, there was lots to do, and any good intentions about TDD went straight out of the window.
And, actually, it was fine. I was fine.
At first.
At first I knew I didn’t really need TDD because it was a small website, and I could easily test whether things worked by just manually checking it out. Click this link 'here', choose that drop-down item 'there', and 'this' should happen. Easy. This whole writing tests thing sounded like it would have taken 'ages', and besides, I fancied myself, from the full height of my three weeks of adult coding experience, as being a pretty good programmer. I could handle it. Easy.
Then came the fearful goddess Complexity. She soon showed me the limits of my experience.
The project grew. Parts of the system started to depend on other parts. I did
my best to follow good principles like DRY (Don’t Repeat Yourself), but that
just led to some pretty dangerous territory. Soon I was playing with multiple
inheritance. Class hierarchies eight levels deep. eval statements.
I became scared of making changes to my code. I was no longer sure what depended on what, and what might happen if I changed this code 'over here', oh gosh, I think that bit over there inherits from it—no, it doesn’t, it’s overriden. Oh, but it depends on that class variable. Right, well, as long as I override the override it should be fine. I’ll just check—but checking was getting much harder. There were lots of sections to the site now, and clicking through them all manually was starting to get impractical. Better to leave well enough alone, forget refactoring, just make do.
Soon I had a hideous, ugly mess of code. New development became painful.
Not too long after this, I was lucky enough to get a job with a company called Resolver Systems (now PythonAnywhere), where Extreme Programming (XP) was the norm. They introduced me to rigorous TDD.
Although my previous experience had certainly opened my mind to the possible benefits of automated testing, I still dragged my feet at every stage. ``I mean, testing in general might be a good idea, but 'really'? All these tests? Some of them seem like a total waste of time… What? Functional tests 'as well as' unit tests? Come on, that’s overdoing it! And this TDD test/minimal-code-change/test cycle? This is just silly! We don’t need all these baby steps! Come on, we can see what the right answer is, why don’t we just skip to the end?''
Believe me, I second-guessed every rule, I suggested every shortcut, I demanded justifications for every seemingly pointless aspect of TDD, and I came out seeing the wisdom of it all. I’ve lost count of the number of times I’ve thought ``Thanks, tests'', as a functional test uncovers a regression we would never have predicted, or a unit test saves me from making a really silly logic error. Psychologically, it’s made development a much less stressful process. It produces code that’s a pleasure to work with.
So, let me tell you all about it!
My main aim is to impart a methodology—a way of doing web development, which I think makes for better web apps and happier developers. There’s not much point in a book that just covers material you could find by Googling, so this book isn’t a guide to Python syntax, or a tutorial on web development 'per se'. Instead, I hope to teach you how to use TDD to get more reliably to our shared, holy goal: clean code that works.
With that said: I will constantly refer to a real practical example, by building a web app from scratch using tools like Django, Selenium, jQuery, and Mocks. I’m not assuming any prior knowledge of any of these, so you should come out of the other end of this book with a decent introduction to those tools, as well as the discipline of TDD.
In Extreme Programming we always pair-program, so I’ve imagined writing this book as if I was pairing with my previous self, having to explain how the tools work and answer questions about why we code in this particular way. So, if I ever take a bit of a patronising tone, it’s because I’m not all that smart, and I have to be very patient with myself. And if I ever sound defensive, it’s because I’m the kind of annoying person that systematically disagrees with whatever anyone else says, so sometimes it takes a lot of justifying to convince myself of anything.
I’ve split this book into three parts.
- [part1] (Chapters #chapter_01–#chapter_07_working_incrementally): The basics
-
Dives straight into building a simple web app using TDD. We start by writing a functional test (with Selenium), and then we go through the basics of Django—models, views, templates—with rigorous unit testing at every stage. I also introduce the Testing Goat.
- [part2] (Chapters #chapter_08_prettification–#chapter_18_second_deploy): Web development essentials
-
Covers some of the trickier but unavoidable aspects of web development, and shows how testing can help us with them: static files, deployment to production, form data validation, database migrations, and the dreaded JavaScript.
- [part3] (Chapters #chapter_19_spiking_custom_auth–#chapter_27_hot_lava): More advanced testing topics
-
Mocking, integrating a third-party system, test fixtures, Outside-In TDD, and Continuous Integration (CI).
On to a little housekeeping…
The following typographical conventions are used in this book:
- Italic
-
Indicates new terms, URLs, email addresses, filenames, and file extensions.
Constant width-
Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.
- *Constant width bold*
-
Shows commands or other text that should be typed literally by the user.
Occasionally I will use the symbol:
[...]
to signify that some of the content has been skipped, to shorten long bits of output, or to skip down to a relevant section.
|
Tip
|
This element signifies a tip or suggestion. |
|
Note
|
This element signifies a general note or aside. |
|
Warning
|
This element indicates a warning or caution. |
Spotted a mistake or a typo? The sources for this book are available on GitHub, and I’m always very happy to receive issues and pull requests: https://github.com/hjwp/Book-TDD-Web-Dev-Python/.
Code examples are available at https://github.com/hjwp/book-example/; you’ll find branches for each chapter there (e.g., https://github.com/hjwp/book-example/tree/chapter_03_unit_test_first_view). You’ll find a full list, and some suggestions on ways of working with this repository, in [appendix_github_links].
This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.
We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Test-Driven Development with Python, 3rd edition, by Harry J.W. Percival (O’Reilly). Copyright 2024 Harry Percival, 978-1-098-14871-3.”
If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at [email protected].
|
Note
|
For more than 40 years, O’Reilly Media has provided technology and business training, knowledge, and insight to help companies succeed. |
Our unique network of experts and innovators share their knowledge and expertise through books, articles, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, visit https://oreilly.com.
Please address comments and questions concerning this book to the publisher:
- O’Reilly Media, Inc.
- 1005 Gravenstein Highway North
- Sebastopol, CA 95472
- 800-889-8969 (in the United States or Canada)
- 707-829-7019 (international or local)
- 707-829-0104 (fax)
- [email protected]
- https://www.oreilly.com/about/contact.html
We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://learning.oreilly.com/library/view/~/9781098148706.
For news and information about our books and courses, visit https://oreilly.com.
Find us on LinkedIn: https://linkedin.com/company/oreilly-media
Follow us on Twitter: https://twitter.com/oreillymedia
Watch us on YouTube: https://youtube.com/oreillymedia
If you’re reading the free edition of this book hosted at http://www.obeythetestinggoat.com, then the license is Creative Commons Attribution-NonCommercial-NoDerivatives [1]. I want to thank O’Reilly for their fantastic attitude towards licensing, most publishers aren’t so forward-thinking.
I see this as a "try-before-you-buy" scheme really. If you’re reading this book it’s for professional reasons, so I hope that if you like it, you’ll buy a copy—if not for yourself, then for a friend! O’Reilly have been great, they deserve your support. You’ll find links to buy back on the homepage.