Python unit tests have assertIn/assertNotIn for strings. Django has assertContains/assertNotContains with HTML-aware processing of responses. Django also has assertInHTML – but somehow there is no assertNotInHTML.
This creates confusion when writing Django unit tests – if I use assertContains and assertNotContains to test a response, I should be able to switch my test cases to assertInHTML / assertNotInHTML if I refactor my code so the tests are no longer for a whole response.
There are two sources of confusion for me here:
- Since
assertContains has assertNotContains and assertIn has assertNotIn – given assertInHTML, I tend to assume assertNotInHTML exists, try to use it, and realise my mistake.
- Sometimes I work on projects that have a
assertNotInHTML. That’s defined in a base test case class, so when I write unit tests I don’t realise this is a project-specific method – and when I go looking for its HTML-processing logic on assertions docs, I’m surprised not to see it.
So how about a new assertNotInHTML?
3 Likes
For ref, I mistakenly reported this on the issue tracker (#34658, sorry) and got two replies,
I fully agree with the rationale, but for this, perhaps, Thibaud could you please create a forum post to allow for people to present any counter argument we may not be considering?
– Natalia Bidart
I’m skeptic. Personally, I find the use of *NotIn* assertions really rare and not the best practice. There can many many reasons why something is not in something else, and it becomes even more unreliable when we do this in HTML. It’s much more stable and bulletproof to use *In* assertions. I don’t think it’s worth adding.
Mariusz Felisiak
@felixxm I’m 100% with you on *NotIn* assertions often being problematic. I try to avoid them in new code whenever possible. Working on an existing project I still find that assertNotInHTML has its place though:
- If I have to switch a test from
assertNotContains(…, html=True) to something that’s not a response – it’s really valuable to be able to change the assertion and move on. Sometimes there’s a place for refactoring tests, sometimes I just want my changes’ diff to be as to-the-point as possible.
- If a test uses
assertNotIn and there is HTML in it – there’s a fair chance the assertion will be more correct as an HTML-aware assertNotInHTML. Here again, sometimes there’s a place for making the tests more sound, sometimes it’s better to keep to the existing patterns.
2 Likes
I’m in favour of adding this. It seems like a straightforward gap in Django to me. If we think there’s a risk of misuse, we should add a warning in the documentation. I think being able to use it for scenarios like moving from assertNotContains outweighs the downsides.
As mentioned in the ticket, I’m +1 to have assertNotInHTML as a counterpart of assertNotContains. I have use these assertNot* in the past when, for example, a view returned a filtered list of items, and the test needs to ensure that the proper filtering is done.
I’m aware there are other options (arguably more robust) to test this kind of logic (like using a proper data-test attribute for each item and grabbing all occurrences with PyQuery to assert over the final list), but I still find these assert helpers quite useful for creating simpler and more straightforward tests.
Makes a lot of sense to have this test method for symmetry. Just because not everyone uses it is not a great reason to exclude it. Refactoring tests is a great example, or as a precondition to a later assertInHtml
I wanted this not the other day…
I was checking that a correct template partial was rendered, rather than the whole template, and wanted to check that a header from another part of the page wasn’t there. I ended up using assertNotIn on the exact string, which was OK…
I should have used assertNotContains, but at that moment I’d forgotten it. 
If I have to switch a test from assertNotContains(…, html=True) to something that’s not a response…
I think that’s a good point.
I’d probably lean towards a +1 myself.
I’ve accepted the ticket based on this discussion.
2 Likes