Python gotchas to look out for

Python gotchas to look out for

1. Changes in yield in Python 3.8

With the release of Python 3.8, there were some major additions and breaking changes made to the language. There is a change in behavior while using yield within generator expressions and comprehensions.

Take this example:

MAGIC_NUMBERS = {(yield num) for num in SECRET_NUMBERS if can_amaze(num)}

This was permitted till Python 3.7 (although it would still have thrown a DeprecationWarning) — and became a SyntaxError in Python 3.8.

There are some more inconsistencies to watch out for when using yield. In Python 2.7, for instance, this won't throw any errors:

# Using `yield` in a generator expression:
even_numbers = ((yield num) for num in range(100) if num % 2 == 0)

# Using ‘yield` in a set comprehension:
odd _numbers = {(yield num) for num in range(100) if num % 2 == 0}

But, if yield is used in a list comprehension like this, Python 2.7 throws a SyntaxError saying 'yield' outside function:

odd _numbers =[(yield num) for num in range(100) if num % 2 == 0]

odd _numbers =[(yield num) for num in range(100) if num % 2 == 0]

In Python 3.x (till Python 3.7), all of the above mentioned operations were allowed. From Python 3.8 onwards, this has become illegal. To read more about this issue check out this bug report.

2. Raising another exception when assert fails is ineffective

When an assert condition is not satisfied, it always raises an AssertionError. Even if another built-in exception is explicitly raised, it will be ineffective.

Example:

assert isinstance(num_channels, int), ValueError(
    'Number of image channels needs to be an integer'
)

Here, when assert fails, the user would be expecting a ValueError like this:

ValueError: Number of image channels needs to be an integer

But instead, the above code will throw an AssertionError when the condition fails:

AssertionError: Number of image channels needs to be an integer

3. Using the sqrt method for Pythagorean calculations

Calculating the length of the hypotenuse using the standard formula sqrt(a**2 + b**2) may lead to an OverflowError if either of a or b or both are very large.

For example:

def get_hypotenuse(a, b):
    return math.sqrt(a**2 + b**2)

get_hypotenuse(2e154, 1e154)

Running this gives the output:

OverflowError: (34, 'Numerical result out of range')

To avoid this, use the hypot method from the math module.

hypotenuse = math.hypot(2e154, 1e514)  # Gives the desired output without overflowing

Even if one (or both) side provided to the hypot method is too big to be stored (say 2e308 and 1e154), there won't be any runtime error and the return value would be inf.

4. Empty modules

Having empty modules lying around is never a good idea, as it populates the project unnecessarily. If the file is necessary for some reason, the file should be documented explaining why it is there.

✨ Readability counts ✨

5. Flask app running with debug as True

Debuggers are meant to be used locally, and should never be enabled in production.

When a flask app is run with its debug mode enabled, an attacker can gain access to the service running the application if the application uses an interactive debugger like werkzeug. In such case, the application becomes vulnerable to remote code execution as any arbitrary code can be run through the interactive debugger.

Here's an example of the vulnerability:

from flask import Flask

app = Flask(__name__)

@app.route('/crash')
def main():
    ...
    raise Exception()

app.run(debug=True)

The recommended way is to remove the statement enabling the debugger, or to set environment variable to disable this in the production environment. Here is an article about how Patreon got hacked because of this vulnerability.

After a recent update to our Python analyzer, DeepSource automatically detects all of the above issues in your code. What other issues would you like to add to the list? Let us know on DeepSource Discuss.