Effective Python: 59 Specific Ways to Write Better Python

Code Sample of Book "Effective Python: 59 Specific Ways to Write Better Python" by Brett Slatkin.

Chapter 1: Pythonic thinking

Item 1: Know which version of python you're using

• 1. There are two major version of Python still in active use: Python 2 and
     Python 3.
• 2. There are multiple popular runtimes for Python: CPython, Jython,
     IronPython, PyPy, etc.
• 3. Be sure that the command-line for running Python on your system is the
     version you expect it to be.
• 4. Prefer Python 3 for your next project because that is the primary focus
     of the Python community.

Item 2: Follow the PEP 8 style guide

• 1. Always follow the PEP 8 style guide when writing Python code.
• 2. Sharing a common style with the larger Python community facilitates
     collaboration with others.
• 3. Using a consistent style makes it easier to modify your own code later.

Item 3: Know the difference between bytes, str, and unicode

• 1. In Python 3, bytes contains sequences of 8-bit values, str contains
     sequences of Unicode characters. bytes and str instances can't be
     used together with operators (like > or +).
• 2. In Python 2, str contains sequences of 8-bit values, unicode contains
     sequences of Unicode characters. str and unicode can be used together
     with operators if the str only contains 7-bit ASCII characters.
• 3. Use helper functions to ensure that the inputs you operate on are the
     type of character sequence you expect (8-bit values, UTF-8 encoded
     characters, Unicode characters, etc.)
• 4. If you want to read or write binary data to/from a file, always open the
     file using a binary mode (like 'rb' or 'wb').

Item 4: Write helper functions instead of complex expressions

• 1. Python's syntax makes it all too easy to write single-line expressions
     that are overly complicated and difficult to read.
• 2. Move complex expressions into helper functions, especially if you need to
     use the same logic repeatedly.
• 3. The if/else expression provides a more readable alternative to using
     Boolean operators like or and adn in expressions.

Item 5: Know hot to slice sequences

• 1. Avoid being verbose: Don't supply 0 for the start index or the length of
     the sequence for the end index.
• 2. Slicing is forgiving of start or end indexes that are out of bounds,
     making it easy to express slices on the front or back boundaries of a
     sequence (like a[:20] or a[-20:]).
• 3. Assigning to a list slice will replace that range in the original
     sequence with what's referenced even if their lengths are different.

Item 6: Avoid using start, end and stride in a single slice

• 1. Specifying start, end, and stride in a slice can be extremely confusing.
• 2. Prefer using positive stride values in slices without start or end
     indexes. Avoid negative stride values if possible.
• 3. Avoid using start, end and stride together in a single slice. If you need
     all three parameters, consider doing two assignments (one to slice,
     another to stride) or using islice form itertools built-in module.

Item 7: Use list comprehensions instead of map and filter

• 1. List comprehensions are clearer than the map and filter built-in
     functions because they don't require extra lambda expressions.
• 2. List comprehensions allow you easily skip items from the input list, a
     behavior map doesn't support without help from filter.
• 3. Dictionaries and sets also support comprehension expressions.

Item 8: Avoid more than two expressions in list comprehensions

• 1. List comprehensions support multiple levels of loops and multiple
     conditions per loop level.
• 2. List comprehensions with more than two expressions are very difficult to
     read and should be avoided.

Item 9: Consider generator expressions for large comprehensions

• 1. List comprehensions can cause problems for large inputs by using too much
     memory.
• 2. Generator expressions avoid memory issues by producing outputs one at a
     time as an iterator.
• 3. Generator expressions can be composed by passing the iterator from one
     generator expression into the for subexpression of another.
• 4. Generator expressions execute very quickly when chained together.

Item 10: Prefer enumerate over range

• 1. enumerate provides concise syntax for looping over an iterator and
     getting the index of each item from the iterator as you go.
• 2. Prefer enumerate instead of looping over a range and indexing into a
     sequence.
• 3. You can supply a second parameter to enumerate to specify the number from
     which to begin counting (zero is default).

Item 11: Use zip to process iterators in parallel

• 1. The zip built-in function can be used to iterate over multiple iterators
     in parallel.
• 2. In Python 3, zip is a lazy generator that produces tuples. In Python 2,
     zip returns the full result as a list of tuples.
• 3. zip truncates its outputs silently if you supply it with iterators of
     different lengths.
• 4. The zip_longest function from the itertools built-in module lets you
     iterate over multiple iterators in parallel regardless of their
     lengths (see Item 46: Use built-in algorithms and data structures).

Item 12: Avoid else blocks after for and while loops

• 1. Python has special syntax that allows else blocks to immediately follow
     for and while loop interior blocks.
• 2. The else block after a loop only runs if the loop body did not encounter
     a break statement.
• 3. Avoid using else blocks after loops because their behavior isn't
     intuitive and can be confusing.

Item 13: Take advantage of each block in try/except/else/finally

• 1. The try/finally compound statement lets you run cleanup code regardless
     of whether exceptions were raised in the try block.
• 2. The else block helps you minimize the amount of code in try blocks and
     visually distinguish the success case from the try/except blocks.
• 3. An else block can be used to perform additional actions after a
     successful try block but before common cleanup in a finally block.

Chapter 2: Functions

Item 14: Prefer exceptions to returning None

• 1. Functions that return None to indicate special meaning are error prone
     because None and other values (e.g., zero, the empty string) all
     evaluate to False in conditional expressions.
• 2. Raise exceptions to indicate special situations instead of returning
     None. Expect the calling code to handle exceptions properly when they
     are documented.

item 15: Know how closures interact with variable scope

• 1. Closure functions can refer to variables from any of the scopes in which
     they were defined.
• 2. By default, closure can't affect enclosing scopes by assigning variables.
• 3. In Python 3, use the nonlocal statement to indicate when a closure can
     modify a variable in its enclosing scopes.
• 4. In Python 2, use a mutable value (like a single-item list) to work around
     the lack of the nonlocal statement.
• 5. Avoid using nonlocal statements for anything beyond simple functions.

Item 16: Consider generators instead of returning lists

• 1. Using generators can be clearer than the alternative of returning lists
     of accumulated results.
• 2. The iterator returned by a generator produces the set of values passed to
     yield expressions within the generator function's body.
• 3. Generators can produce a sequence of outputs for arbitrarily large inputs
     because their working memory doesn't include all inputs and outputs.

Item 17: Be defensive when iterating over arguments

• 1. Beware of functions that iterate over input arguments multiple times. If
     these arguments are iterators, you may see strange behavior and missing
     values.
• 2. Python's iterator protocol defines how containers and iterators interact
     with the iter and next built-in functions, for loops, and related
     expression.
• 3. You can easily define your own iterable container type by implementing
     the iter method as a generator.
• 4. You can detect that a value is an iterator (instead of a container) if
     calling iter on it twice produces the same result, which can then be
     progressed with the next built-in function.

Item 18: Reduce visual noise with variable positional arguments

• 1. Functions can accept a variable number of positional arguments by using
     *args in the def statement.
• 2. You can use the items from a sequence as the positional arguments for a
     function with the * operator.
• 3. Using the * operator with a generator may cause your program to run out
     of memory and crash.
• 4. Adding new positional parameters to functions that accept *args can
     introduce hard-to-find bugs.

Item 19: Provide optimal behavior with keyword arguments

• 1. Function arguments can be specified by position or by keyword.
• 2. Keywords make it clear what the purpose of each arguments is when it
     would be confusing with only positional arguments.
• 3. Keywords arguments with default values make it easy to add new behaviors
     to a function, especially when the function has existing callers.
• 4. Optional keyword arguments should always be passed by keyword instead of
     by position.

Item 20: Use None and Docstrings to specify dynamic default arguments

• 1. Closure functions can refer to variables from any of the scopes in which
     they were defined.
• 2. By default, closure can't affect enclosing scopes by assigning variables.
• 3. In Python 3, use the nonlocal statement to indicate when a closure can
     modify a variable in its enclosing scopes.
• 4. In Python 2, use a mutable value (like a single-item list) to work around
     the lack of the nonlocal statement.
• 5. Avoid using nonlocal statements for anything beyond simple functions.

Item 21: Enforce clarity with key-word only arguments

• 1. Keyword arguments make the intention of a function call more clear.
• 2. Use keyword-only arguments to force callers to supply keyword arguments
     for potentially confusing functions, especially those that accept
     multiple Boolean flags.
• 3. Python 3 supports explicit syntax for keyword-only arguments in
     functions.
• 4. Python 2 can emulate keyword-only arguments for functions by using
     **kwargs and manually raising TypeError exceptions.

Chapter 3: Classes and Inheritance

Item 22: Prefer helper classes over bookkeeping with dictionaries and tuples

• 1. Avoid making dictionaries with values that are other dictionaries or
     long tuples.
• 2. Use namedtuple for lightweight, immutable data containers before you need
     the flexibility of a full class.
• 3. Move your bookkeeping code to use multiple helper classes when your
     internal state dictionaries get complicated.

Item 23: Accept functions for simple interfaces instead of classes

• 1. Instead of defining and instantiating classes, functions are often all
     you need for simple interfaces between components in Python.
• 2. References to functions and methods in Python are first class, meaning
     they can be used in expressions like any other type.
• 3. The call special method enables instances of a class to be called
     like plain Python functions.
• 4. When you need a function to maintain state, consider defining a class
     that provides the call method instead of defining a stateful closure
     (see Item 15: "Know how closures interact with variable scope").

Item 24: Use @classmethod polymorphism to construct objects generically

• 1. Python only supports a single constructor per class, the init method.
• 2. Use @classmethod to define alternative constructors for your classes.
• 3. Use class method polymorphism to provide generic ways to build and
     connect concrete subclasses.

Item 25: Initialize parent classes with super

• 1. Python's standard method resolution order (MRO) solves the problems to
     superclass initialization order and diamond inheritance.
• 2. Always use the super built-in function to initialize parent classes.

Item 26: Use multiple inheritance only for mix-in utility classes

• 1. Avoid using multiple inheritance if mix-in classes can achieve the same
     outcome.
• 2. Use pluggable behaviors at the instance level to provide per-class
     customization when mix-in classes may require it.
• 3. Compose mix-ins to create complex functionality from simple behaviors.

Item 27: Prefer public attributes over private ones

• 1. Private attributes aren't rigorously enforced by the Python compiler.
• 2. Plan from the beginning to allow subclass to do more with your internal
     APIs and attributes instead of locking them out by default.
• 3. Use documentation of protected fields to guide subclass instead of trying
     to force access control with private attributes.
• 4. Only consider using private attributes to avoid naming conflicts with
     subclasses that are out of your control.

Item 28: Inherit from collections.abc for custom container types

• 1. Inherit directly from Python's container types (like list or dict) for
     simple use cases.
• 2. Beware of the large number of methods required to implement custom
     container types correctly.
• 3. Have your custom container types inherit from the interface defined in
     collections.abc to ensure that your classes match required interfaces
     and behaviors.

Chapter 4: Metaclasses and Attributes

Item 29: Use plain attributes instead of get and set methods

• 1. Define new class interfaces using simple public attributes, and avoid set
     and get methods.
• 2. Use @property to define special behavior when attributes are accessed on
     your objects, if necessary.
• 3. Follow the rule of least surprise and void weird side effects in your
     @property methods.
• 4. Ensure that @property methods are fast; do slow or complex work using
     normal methods.

Item 30: Consider @property instead of refactoring attributes

• 1. Use @property to give existing instance attributes new functionality.
• 2. Make incremental progress toward better data models by using @property.
• 3. Consider refactoring a class and all call sites when you find yourself
     using @property too heavily.

Item 31: Use descriptors for reusable @property methods

• 1. Reuse the behavior and validation of @property methods by defining your
     own descriptor classes.
• 2. Use WeakKeyDictionary to ensure that your descriptor classes don't cause
     memory leaks.
• 3. Don't get bogged down trying to understand exactly how getattribute
     uses the descriptor protocol for getting and setting attributes.

Item_32_Use getattr, getattribute, and setattr for lazy attributes

• 1. Use getattr and setattr to lazily load and save attributes for an
     object.
• 2. Understand that getattr only gets called once when accessing a
     missing attribute, whereas getattribute gets called every time an
     attribute is accessed.
• 3. Avoid infinite recursion in getattribute and setattr by using
     methods from super() (i.e., the object class) to access instance
     attributes directly.

Item 33: Validate subclass with metaclass

• 1. Use metaclasses to ensure that subclass are well formed at the time they
     are defined, before objects of their type are constructed.
• 2. Metaclass have slightly different syntax in Python 2 vs. Python 3.
• 3. The new method of metaclasses is run after the class statement's
     entire body has been processed.

Item 34: Register class existence with metaclass

• 1. Class registration is a helpful pattern for building modular Python
     programs.
• 2. Metaclass let you run registration code automatically each time your
     base class is subclassed in a program.
• 3. Using metaclass for class registration avoids errors by ensuring that
     you never miss a registration call.

Item 35: Annotate class attributes with metaclass

• 1. Metaclass enable you to modify a class's attributes before the class is
     fully defined.
• 2. Descriptors and metaclasses make a powerful combination for declarative
     behavior and runtime introspection.
• 3. You can avoid both memory leaks and the weakref module by using
     metaclasses along with descriptors.

Chapter 5: Concurrency and parallelism

Item 36: use subprocess to manage child processes

• 1. Use the subprocess to run child processes and manage their input and
     output streams.
• 2. Child processes run in parallel with the Python interpreter, enabling you
     to maximize your CPU usage.
• 3. Use the timeout parameter with communicate to avoid deadlocks and hanging
     child processes.

Item 37: Use threads for blocking I/O, avoid for parallelism

• 1. Python threads can't bytecode in parallel on multiple CPU cores because
     of the global interpreter lock (GIL).
• 2. Python threads are still useful despite the GIL because they provide an
     easy way to do multiple things at seemingly the same time.
• 3. Use Python threads to make multiple system calls in parallel. This allows
     you to do blocking I/O at the same time as computation.

Item 38: Use lock to prevent data races in threads

• 1. Even though Python has a global interpreter lock, you're still
     responsible for protecting against objects without locks.
• 2. Your programs will corrupt their data structures if you allow multiple
     threads to modify the same objects without locks.
• 3. The lock class in the threading built-in module is Python's standard
     mutual exclusion lock implementation.

Item 39: Use queue to coordinate work between threads

• 1. Pipelines are a great way to organize sequences of work that run
     concurrently using multiple Python threads.
• 2. Be aware of the many problems in building concurrent pipelines: busy
     waiting, stopping workers, and memory explosion.
• 3. The Queue class has all of the facilities you need to build robust
     pipelines: blocking operations, buffer sizes, and joining.

Item 40: Consider coroutines to run many functions concurrently

• 1. Coroutines provide an efficient way to run tens of thousands of functions
     seemingly at the same time.
• 2. Within a generator, the value of the yield expression will be whatever
     value was passed to the generator's send method from the exterior code.
• 3. Coroutines give you a powerful tool for separating the core logic of your
     program from its interaction with the surrounding environment.
• 4. Python 2 doesn't support yield from or returning values from generators.

Item 41: Consider concurrent.futures for true parallelism

• 1. Moving CPU bottlenecks to C-extension modules can be an effective way to
     improve performance while maximizing your investment in Python code.
     However, the cost of doing so is high and may introduce bugs.
• 2. The multiprocessing module provides powerful tools that can parallelize
     certain types of Python computation with minimal effort.
• 3. The power of multiprocessing is best accessed through the
     concurrent.futures built-in module and its simple ProcessPoolExecutor
     class.
• 4. The advanced parts of the multiprocessing module should be avoided
     because they are so complex.

Chapter 6: Built-in Modules

Item 42: Define function decorators with functools.wraps

• 1. Decorators are Python syntax for allowing one function to modify another
     function at runtime.
• 2. Using decorators can cause strange behaviors in tools that do
     introspection, such as debuggers.
• 3. Use the wraps decorator from the functools built-in module when you
     define your own decorators to avoid any issues.

Item 43: Consider contextlib and with statements for reusable try/finally behavior

• 1. The with statement allows you to reuse logic from try/finally blocks and
     reduce visual noise.
• 2. The contextlib built-in module provides a contextmanager decorator that
     makes it easy to use your own functions in with statements.
• 3. The value yielded by context managers is supplied to the as part of the
     with statement. It's useful for letting your code directly access the
     cause of the special context.

Item 44: Make pickle reliable with copyreg

• 1. The pickle built-in module is only useful for serializing and
     de-serializing objects between trusted programs.
• 2. The pickle module may break down when used for more than trivial use
     cases.
• 3. Use the copyreg built-in module with pickle to add missing attributes
     values, allow versioning of classes, and provide stable import paths.

Item 45: Use datetime instead of time for local clocks

• 1. Avoid using the time module for translating between different time zones.
• 2. Use the datetime built-in module along with the pytz module to reliably
     convert between times in different time zones.
• 3. Always represent time in UTC and do conversations to local time as the
     final step before presentation.

Item 46: Use built-in algorithms and data structures

• 1. Use Python's built-in modules for algorithms and data structures.
• 2. Don't re-implement this functionality yourself. It's hard to get right.

Item 47: Use decimal when precision ia paramount

• 1. Python has built-in types and classes in modules that can represent
     practically every type of numerical value.
• 2. The Decimal class is ideal for situations that require high precision and
     exact rounding behavior, such as computations of monetary values.

Item 48: Know where to find community built modules

• 1. The Python Package Index (PyPI) contains a wealth of common packages
     that are built and maintained by the Python community.
• 2. pip is the command-line to use for installing packages from PyPI.
• 3. pip is installed by default in Python 3.4 and above; you must install it
     yourself for older versions.
• 4. The majority of PyPI modules are free and open source software.

Chapter 7: Collaboration

Item 49: Write docstrings for every function, class and module

• 1. Write documentation for every module, class and function using
     docstrings. Keep them up to date as your code changes.
• 2. For modules: introduce the contents of the module and any important
     classes or functions all users should know about.
• 3. For classes: document behavior, important attributes, and subclass
     behavior in the docstring following the class statement.
• 4. For functions and methods: document every argument, returned value,
     raised exception, and other behaviors in the docstring following the
     def statement.

Item 50: Use packages to organize modules and provide stable APIs

• 1. Packages in Python are modules that contain other modules. Packages allow
     you to organize your code into separate, non-conflicting namespaces with
     unique absolute module names.
• 2. Simple package are defined by adding an init.py file to a directory
     that contains other source files. These files become that child modules
     of the directory's package. Package directories may also contain other
     packages.
• 3. You can provide an explict API for a module by listing its publicly
     visible name in its all special attribute.
• 4. You can hide a package's internal implementation by only importing public
     names in the package's init.py file or by naming internal-only
     members with a leading underscore.
• 5. When collaborating within a single team or on a single codebase, using
     all for explicit APIs is probably unnecessary.

Item 51: Define a root exception to insulate callers from APIs

• 1. Defining root exceptions for your modules allows API consumers to
     insulate themselves from your API.
• 2. Catching root exceptions can help you find bugs in code that consumes an
     API.
• 3. Catching the Python Exception base class can help you find bugs in API
     implementations.
• 4. Intermediate root exceptions let you add more specific types of
     exceptions in the future without breaking your API consumers.

Item 52: Know how to break circular dependencies

• 1. Circular dependencies happen when two modules must call into each other
     at import time. They can cause your program to crash at startup.
• 2. The best way to break a circular dependency is refactoring mutual
     dependencies into a separate module at the bottom of the dependency tree.
• 3. Dynamic imports are the simplest solution for breaking a circular
     dependency between modules while minimizing refactoring and complexity.

Item 53: Use virtual environments for isolated and reproducible dependencies

• 1. Virtual environment allow you to use pip to install many different
     versions of the same package on the same machine without conflicts.
• 2. Virtual environments are created with pyvenv, enabled with source
     bin/activate, and disabled with deactivate.
• 3. You can dump all of the requirements of an environment with pip freeze.
     You can reproduce the environment by supplying the requirements.txt file
     to pip install -r.
• 4. In versions of Python before 3.4, the pyvenv tool must be downloaded and
     installed separately. The command-line tool is called virtualenv instead
     of pyvenv.

Chapter 8: Production

Item 54: Consider module-scoped code to configure deployment environments

• 1. Programs often need to run in multiple deployment environments that each
     have unique assumptions and configurations.
• 2. You can tailor a module's contents to different deployment environments
     by using normal Python statements in module scope.
• 3. Module contents can be the product of any external condition, including
     host introspection through the sys and os modules.

Item 55: Use repr strings for debugging output

• 1. Calling print on built-in Python types will produce the human-readable
     string version of a value, which hides type information.
• 2. Calling repr on built-in Python types will produce the printable string
     version of a value. These repr strings could be passed to the eval
     built-in function to get back the original value.
• 3. %s in format strings will produce human-readable strings like str.%r will
     produce printable strings like repr.
• 4. You can define the repr method to customize the printable
     representation of a class and provide more detailed debugging
     information.
• 5. You can reach into any object's dict attribute to view its internals.

Item 56: Test everything with unittest

• 1. The only way to have confidence in a Python program is to write tests.
• 2. The unittest built-in module provides most of the facilities you'll need
     to write good tests.
• 3. You can define tests by subclassing TestCase and defining one method per
     behavior you'd like to test. Test methods on TestCase classes must start
     with the word test.
• 4. It's important to write both unit tests (for isolated functionality) and
     integration tests (for modules that interact).

Item 57： Consider interactive debugging with pdb

1. You can initiate the Python interactive debugger at a point of interest
   directly in your program with the import pdb; pdb.set_trace() statements.
2. The Python debugger prompt is a full Python shell that lets you inspect
   and modify the state of a running program.
3. pdb shell commands let you precisely control program execution, allowing
   you to alternate between inspecting program state and progressing program
   execution.

item 58: Profile before optimizing

• 1. It's import to profile Python programs before optimizing because the
     source of slowdowns is often obscure.
• 2. Use the cProfile module instead of the profile module because it provides
     more accurate profiling information.
• 3. The Profile object's runcall method provides everything you need to
     profile a tree of function calls in isolation.
• 4. The Stats object lets you select and print the subset of profiling
     information you need to see to understand your program's performance.

Item 59: Use tracemalloc to understand memory usage and leaks

• 1. It can be difficult to understand how Python programs use and leak
     memory.
• 2. The gc module can help you understand which objects exist, but it has no
     information about how they were allocated.
• 3. The tracemalloc built-in module provides powerful tools for understanding
     the source of memory usage.
• 4. tracemalloc is only available in Python 3.4 and above.