Using assert outside tests

Help Wanted
1

There are a few cases where it is pointed out that using assert outside tests are valid.
Example: This PR.

Clarify this?

1 Like
2

Python has an option to compile the optimized bytecode and create the respective .pyo files by using the options -O and -OO.

During this, these basic optimizations are done:

  • All the assert statements are removed
  • All docstrings are removed (when -OO is selected)
  • Value of the __debug__ built-in variable is set to False

Now, if someone never intends to run the optimized bytecode, they can ignore this issue. But, it is still recommended not to have it in non-test files.
A better way for internal self-checks is to explicitly check and raise respective Error using an if statement.

Here’s an example to show why the issue is raised for assert statement in non-test files:

Let’s call this script foo.py

import sys


assert len(sys.argv) == 5
print("Argument variables are: ", args)

This is supposed to work fine if the length of the argument variables is 5. But, would raise an AssertionError otherwise.

When the optimized bytecode is compiled using:
python -O -m compileall foo.py
A new file foo.pyo is created.

Here’s the traceback when we run both the file:

foo.py:
$ python foo.py 1 2 3 4 5
Traceback (most recent call last):
  File "foo.py", line 4, in <module>
    assert len(sys.argv) == 5
AssertionError
foo.pyo:
$ python foo.pyo 1 2 3 4 5 6
sys.args are:  ['foo.pyo', '1', '2', '3', '4', '5', '6']

Note: python -O foo.py would work the same way as of running the .pyo file.

Here, all the internal self-checks using the assert statements are removed, as we can see.
Therefore, there’s a chance for an application to behave strangely in this case.

4 Likes
3

This is one of the reasons I disabled deepsource.io again (side note: lets us enable a project without having to drop in a config file first to see if your project is going to be helpful).

assert also is used in type-hinted code-bases, see the mypy documentation on casts and asserts for an example. Several IDEs make use of asserts for similar code-analysis use cases.

Last but not least, assertions are added in code specifically to help catch corner cases during development; ‘optimised’ code in production settings should have gone through dev cycles that would have tripped the assert. That asserts are neutralized when using-O is not a reason to relegate them to tests only.

For my test project, deepsource.io found just 1 potential issue (an unused variable in a test), and a large pile of false positives (assert outside of a test, list comprehension in a str.join() call (a gen expression is slower there as str.join() has to process the sequence passed in twice, an important exception), complaining about __init__.py being empty module in a package, were some of the others).

4

DeepSource raises security issues for assert statements when it is used in any file other than tests. These occurrences may be valid or invalid depending upon the use cases.
Since our main concern with security issues is to avoid false-negatives, this issue is always raised for non-test files.

DeepSource also has an option to ignore such issues across multiple levels to reduce the noise.

ban_b101
ban_b1011054×344 2.39 MB

We will support running analysis on a project to preview the issues without adding the config soon.
Also, thank you for letting us know about the false positives you came across. We have fixed them in our last release.

1 Like
6

Hi Srijan, can you explain how the test are recognized? I name my tests with postfix *_test. That is a valid pytest naming convention (besides prefixing). DS reports Assert statement used outside of tests which is wrong. Thank you.

7

Hey @groundf – Have you configured test_patterns in the .deepsource.toml file? More info on test_patterns here - Concepts - DeepSource Docs

8

@jai Thank you, exactly what I need.

1 Like