Chapter 17 - Pythonic OOP: Properties and Dunder Methods

Many languages have OOP features, but Python has some unique OOP features, including properties and dunder methods. Learning how to use these Pythonic techniques can help you write concise and readable code.

Properties allow you to run some specific code each time an object’s attribute is read, modified, or deleted to ensure the object isn’t put into an invalid state. In other languages, this code is often called getters or setters. Dunder methods allow you to use your objects with Python’s built-in operators, such as the + operator. This is how you can combine two datetime.timedelta objects, such as datetime.timedelta(days=2) and datetime.timedelta(days=3), to create a new datetime.timedelta(days=5) object.

In addition to using other examples, we’ll continue to expand the WizCoin class we started in Chapter 15 by adding properties and overloading operators with dunder methods. These features will make WizCoin objects more expressive and easier to use in any application that imports the wizcoin module.

Properties

The BankAccount class that we used in Chapter 15 marked its _balance attribute as private by placing an underscore at the start of its name. But remember that designating an attribute as private is only a convention: all attributes in Python are technically public, meaning they’re accessible to code outside the class. There’s nothing to prevent code from intentionally or maliciously changing the _balance attribute to an invalid value.

But you can prevent accidental invalid changes to these private attributes with properties. In Python, properties are attributes that have specially assigned getter, setter, and deleter methods that can regulate how the attribute is read, changed, and deleted. For example, if the attribute is only supposed to have integer values, setting it to the string '42' will likely cause bugs. A property would call the setter method to run code that fixes, or at least provides early detection of, setting an invalid value. If you’ve thought, “I wish I could run some code each time this attribute was accessed, modified with an assignment statement, or deleted with a del statement,” then you want to use properties.

Turning an Attribute into a Property

First, let’s create a simple class that has a regular attribute instead of a property. Open a new file editor window and enter the following code, saving it as regularAttributeExample.py:

class ClassWithRegularAttributes:
    def __init__(self, someParameter):
        self.someAttribute = someParameter

obj = ClassWithRegularAttributes('some initial value')
print(obj.someAttribute)  # Prints 'some initial value'
obj.someAttribute = 'changed value'
print(obj.someAttribute)  # Prints 'changed value'
del obj.someAttribute  # Deletes the someAttribute attribute.

This ClassWithRegularAttributes class has a regular attribute named someAttribute. The __init__() method sets someAttribute to 'some initial value', but we then directly change the attribute’s value to 'changed value'. When you run this program, the output looks like this:

some initial value
changed value

This output indicates that code can easily change someAttribute to any value. The downside of using regular attributes is that your code can set the someAttribute attribute to invalid values. This flexibility is simple and convenient, but it also means someAttribute could be set to some invalid value that causes bugs.

Let’s rewrite this class using properties by following these steps to do this for an attribute named someAttribute:

1. Rename the attribute with an underscore prefix: _someAttribute.
2. Create a method named someAttribute with the @property decorator. This getter method has the self parameter that all methods have.
3. Create another method named someAttribute with the @someAttribute.setter decorator. This setter method has parameters named self and value.
4. Create another method named someAttribute with the @someAttribute.deleter decorator. This deleter method has the self parameter that all methods have.

Open a new file editor window and enter the following code, saving it as propertiesExample.py:

class ClassWithProperties:
    def __init__(self):
        self.someAttribute = 'some initial value'

    @property
    def someAttribute(self): # This is the "getter" method.
        return self._someAttribute

    @someAttribute.setter
    def someAttribute(self, value): # This is the "setter" method.
        self._someAttribute = value

    @someAttribute.deleter
    def someAttribute(self): # This is the "deleter" method.
        del self._someAttribute

obj = ClassWithProperties()
print(obj.someAttribute) # Prints 'some initial value'
obj.someAttribute = 'changed value'
print(obj.someAttribute) # Prints 'changed value'
del obj.someAttribute # Deletes the _someAttribute attribute.

This program’s output is the same as the regularAttributeExample.py code, because they effectively do the same task: they print an object’s initial attribute and then update that attribute and print it again.

But notice that the code outside the class never directly accesses the _someAttribute attribute (it’s private, after all). Instead, the outside code accesses the someAttribute property. What this property actually consists of is a bit abstract: the getter, setter, and deleter methods combined make up the property. When we rename an attribute named someAttribute to _someAttribute while creating getter, setter, and deleter methods for it, we call this the someAttribute property.

In this context, the _someAttribute attribute is called a backing field or backing variable and is the attribute on which the property is based. Most, but not all, properties use a backing variable. We’ll create a property without a backing variable in “Read-Only Properties” later in this chapter.

You never call the getter, setter, and deleter methods in your code because Python does it for you under the following circumstances:

• When Python runs code that accesses a property, such as print(obj.someAttribute), behind the scenes, it calls the getter method and uses the returned value.
• When Python runs an assignment statement with a property, such as obj.someAttribute = 'changed value', behind the scenes, it calls the setter method, passing the 'changed value' string for the value parameter.
• When Python runs a del statement with a property, such as del obj.someAttribute, behind the scenes, it calls the deleter method.

The code in the property’s getter, setter, and deleter methods acts on the backing variable directly. You don’t want the getter, setter, or deleter methods to act on the property, because this could cause errors. In one possible example, the getter method would access the property, causing the getter method to call itself, which makes it access the property again, causing it to call itself again, and so on until the program crashes. Open a new file editor window and enter the following code, saving it as badPropertyExample.py:

class ClassWithBadProperty:
    def __init__(self):
        self.someAttribute = 'some initial value'

    @property
    def someAttribute(self):  # This is the "getter" method.
        # We forgot the _ underscore in `self._someAttribute here`, causing
        # us to use the property and call the getter method again:
        return self.someAttribute  # This calls the getter again!

    @someAttribute.setter
    def someAttribute(self, value):  # This is the "setter" method.
        self._someAttribute = value

obj = ClassWithBadProperty()
print(obj.someAttribute)  # Error because the getter calls the getter.

When you run this code, the getter continually calls itself until Python raises a RecursionError exception:

Traceback (most recent call last):
  File "badPropertyExample.py", line 16, in <module>
    print(obj.someAttribute)  # Error because the getter calls the getter.
  File "badPropertyExample.py", line 9, in someAttribute
    return self.someAttribute  # This calls the getter again!
  File "badPropertyExample.py", line 9, in someAttribute
    return self.someAttribute  # This calls the getter again!
  File "badPropertyExample.py", line 9, in someAttribute
    return self.someAttribute  # This calls the getter again!
  [Previous line repeated 996 more times]
RecursionError: maximum recursion depth exceeded

To prevent this recursion, the code inside your getter, setter, and deleter methods should always act on the backing variable (which should have an underscore prefix in its name), never the property. Code outside these methods should use the property, although as with the private access underscore prefix convention, nothing prevents you from writing code on the backing variable anyway.

Using Setters to Validate Data

The most common need for using properties is to validate data or to make sure it’s in the format you want it to be in. You might not want code outside the class to be able to set an attribute to just any value; this could lead to bugs. You can use properties to add checks that ensure only valid values are assigned to an attribute. These checks let you catch bugs earlier in code development, because they raise an exception as soon as an invalid value is set.

Let’s update the wizcoin.py file from Chapter 15 to turn the galleons, sickles, and knuts attributes into properties. We’ll change the setter for these properties so only positive integers are valid. Our WizCoin objects represent an amount of coins, and you can’t have half a coin or an amount of coins less than zero. If code outside the class tries to set the galleons, sickles, or knuts properties to an invalid value, we’ll raise a WizCoinException exception.

Open the wizcoin.py file that you saved in Chapter 15 and modify it to look like the following:

1 class WizCoinException(Exception):
2     """The wizcoin module raises this when the module is misused."""
    pass

class WizCoin:
    def __init__(self, galleons, sickles, knuts):
        """Create a new WizCoin object with galleons, sickles, and knuts."""
3         self.galleons = galleons
        self.sickles  = sickles
        self.knuts    = knuts
        # NOTE: __init__() methods NEVER have a return statement.

--snip--

    @property
4     def galleons(self):
        """Returns the number of galleon coins in this object."""
        return self._galleons

    @galleons.setter
5     def galleons(self, value):
6         if not isinstance(value, int):
7             raise WizCoinException('galleons attr must be set to an int, not a ' + value.__class__.__qualname__)
8         if value < 0:
            raise WizCoinException('galleons attr must be a positive int, not ' + value.__class__.__qualname__)
        self._galleons = value

--snip--

The new changes add a WizCoinException class 1 that inherits from Python’s built-in Exception class. The class’s docstring describes how the wizcoin module 2 uses it. This is a best practice for Python modules: the WizCoin class’s objects can raise this when they’re misused. That way, if a WizCoin object raises other exception classes, like ValueError or TypeError, this will mostly likely signify that it’s a bug in the WizCoin class.

In the __init__() method, we set the self.galleons, self.sickles, and self.knuts properties 3 to the corresponding parameters.

At the bottom of the file, after the total() and weight() methods, we add a getter 4 and setter method 5 for the self._galleons attribute. The getter simply returns the value in self._galleons. The setter checks whether the value being assigned to the galleons property is an integer 6 and positive 8. If either check fails, WizCoinException is raised with an error message. This check prevents _galleons from ever being set with an invalid value as long as code always uses the galleons property.

All Python objects automatically have a __class__ attribute, which refers to the object’s class object. In other words, value.__class__ is the same class object that type(value) returns. This class object has an attribute named __qualname__ that is a string of the class’s name. (Specifically, it’s the qualified name of the class, which includes the names of any classes the class object is nested in. Nested classes are of limited use and beyond the scope of this book.) For example, if value stored the date object returned by datetime.date(2021, 1, 1), then value.__class__.__qualname__ would be the string 'date'. The exception messages use value.__class__.__qualname__ 7 to get a string of the value object’s name. The class name makes the error message more useful to the programmer reading it, because it identifies not only that the value argument was not the right type, but what type it was and what type it should be.

You’ll need to copy the code for the getter and setter for _galleons to use for the _sickles and _knuts attributes as well. Their code is identical except they use the _sickles and _knuts attributes, instead of _galleons, as backing variables.

Read-Only Properties

Your objects might need some read-only properties that can’t be set with the assignment operator =. You can make a property read-only by omitting the setter and deleter methods.

For example, the total() method in the WizCoin class returns the value of the object in knuts. We could change this from a regular method to a read-only property, because there is no reasonable way to set the total of a WizCoin object. After all, if you set total to the integer 1000, does this mean 1,000 knuts? Or does it mean 1 galleon and 493 knuts? Or does it mean some other combination? For this reason, we’ll make total a read-only property by adding the code in bold to the wizcoin.py file:

    @property
    def total(self):
        """Total value (in knuts) of all the coins in this WizCoin object."""
        return (self.galleons * 17 * 29) + (self.sickles * 29) + (self.knuts)

    # Note that there is no setter or deleter method for `total`.

After you add the @property function decorator in front of total(), Python will call the total() method whenever total is accessed. Because there is no setter or deleter method, Python raises AttributeError if any code attempts to modify or delete total by using it in an assignment or del statement, respectively. Notice that the value of the total property depends on the value in the galleons, sickles, and knuts properties: this property isn’t based on a backing variable named _total. Enter the following into the interactive shell:

>>> import wizcoin
>>> purse = wizcoin.WizCoin(2, 5, 10)
>>> purse.total
1141
>>> purse.total = 1000
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: can't set attribute

You might not like that your program immediately crashes when you attempt to change a read-only property, but this behavior is preferable to allowing a change to a read-only property. Your program being able to modify a read-only property would certainly cause a bug at some point while the program runs. If this bug happens much later after you modify the read-only property, it would be hard to track down the original cause. Crashing immediately allows you to notice the problem sooner.

Don’t confuse read-only properties with constant variables. Constant variables are written in all uppercase and rely on the programmer to not modify them. Their value is supposed to remain constant and unchanging for the duration of a program’s run. A read-only property is, as with any attribute, associated with an object. A read-only property cannot be directly set or deleted. But it might evaluate to a changing value. Our WizCoin class’s total property changes as its galleons, sickles, and knuts properties change.

When to Use Properties

As you saw in the previous sections, properties provide more control over how we can use a class’s attributes, and they’re a Pythonic way to write code. Methods with names like getSomeAttribute() or setSomeAttribute() signal that you should probably use properties instead.

This isn’t to say that every instance of a method beginning with get or set should immediately be replaced with a property. There are situations in which you should use a method, even if its name begins with get or set. Here are some examples:

• For slow operations that take more than a second or two—for example, downloading or uploading a file
• For operations that have side effects, such as changes to other attributes or objects
• For operations that require additional arguments to be passed to the get or set operation—for example, in a method call like emailObj.getFileAttachment(filename)

Programmers often think of methods as verbs (in the sense that methods perform some action), and they think of attributes and properties as nouns (in the sense that they represent some item or object). If your code seems to be performing more of an action of getting or setting rather than getting or setting an item, it might be best to use a getter or setter method. Ultimately, this decision depends on what sounds right to you as the programmer.

The great advantage of using Python’s properties is that you don’t have to use them when you first create your class. You can use regular attributes, and if you need properties later, you can convert the attributes to properties without breaking any code outside the class. When we make a property with the attribute’s name, we can rename the attribute using a prefix underscore and our program will still work as it did before.

Python’s Dunder Methods

Python has several special method names that begin and end with double underscores, abbreviated as dunder. These methods are called dunder methods, special methods, or magic methods. You’re already familiar with the __init__() dunder method name, but Python has several more. We often use them for operator overloading—that is, adding custom behaviors that allow us to use objects of our classes with Python operators, such as + or >=. Other dunder methods let objects of our classes work with Python’s built-in functions, such as len() or repr().

As with __init__() or the getter, setter, and deleter methods for properties, you almost never call dunder methods directly. Python calls them behind the scenes when you use the objects with operators or built-in functions. For example, if you create a method named __len__() or __repr__() for your class, they’ll be called behind the scenes when an object of that class is passed to the len() or repr() function, respectively. These methods are documented online in the official Python documentation at https://docs.python.org/3/reference/datamodel.html.

As we explore the many different types of dunder methods, we’ll expand our WizCoin class to take advantage of them.

String Representation Dunder Methods

You can use the __repr_() and __str__() dunder methods to create string representations of objects that Python typically doesn’t know how to handle. Usually, Python creates string representations of objects in two ways. The repr (pronounced “repper”) string is a string of Python code that, when run, creates a copy of the object. The str (pronounced “stir”) string is a human-readable string that provides clear, useful information about the object. The repr and str strings are returned by the repr() and str() built-in functions, respectively. For example, enter the following into the interactive shell to see a datetime.date object’s repr and str strings:

>>> import datetime
1 >>> newyears = datetime.date(2021, 1, 1)
>>> repr(newyears)
2 'datetime.date(2021, 1, 1)'
>>> str(newyears)
3 '2021-01-01'
4 >>> newyears
datetime.date(2021, 1, 1)

In this example, the 'datetime.date(2021, 1, 1)' repr string of the datetime.date object 2 is literally a string of Python code that creates a copy of that object 1. This copy provides a precise representation of the object. On the other hand, the '2021-01-01' str string of the datetime.date object 3 is a string representing the object’s value in a way that’s easy for humans to read. If we simply enter the object into the interactive shell 4, it displays the repr string. An object’s str string is often displayed to users, whereas an object’s repr string is used in technical contexts, such as error messages and logfiles.

Python knows how to display objects of its built-in types, such as integers and strings. But it can’t know how to display objects of the classes we create. If repr() doesn’t know how to create a repr or str string for an object, by convention the string will be enclosed in angle brackets and contain the object’s memory address and class name: '<wizcoin.WizCoin object at 0x00000212B4148EE0>'. To create this kind of string for a WizCoin object, enter the following into the interactive shell:

>>> import wizcoin
>>> purse = wizcoin.WizCoin(2, 5, 10)
>>> str(purse) 
'<wizcoin.WizCoin object at 0x00000212B4148EE0>'
>>> repr(purse) 
'<wizcoin.WizCoin object at 0x00000212B4148EE0>'
>>> purse
<wizcoin.WizCoin object at 0x00000212B4148EE0>

These strings aren’t very readable or useful, so we can tell Python what strings to use by implementing the __repr__() and __str__() dunder methods. The __repr__() method specifies what string Python should return when the object is passed to the repr() built-in function, and the __str__() method specifies what string Python should return when the object is passed to the str() built-in function. Add the following to the end of the wizcoin.py file:

--snip--
    def __repr__(self):
        """Returns a string of an expression that re-creates this object."""
        return f'{self.__class__.__qualname__}({self.galleons}, {self.sickles}, {self.knuts})'

    def __str__(self):
        """Returns a human-readable string representation of this object."""
        return f'{self.galleons}g, {self.sickles}s, {self.knuts}k'

When we pass purse to repr() and str(), Python calls the __repr__() and __str__() dunder methods. We don’t call the dunder methods in our code.

Note that f-strings that include the object in braces will implicitly call str() to get an object’s str string. For example, enter the following into the interactive shell:

>>> import wizcoin
>>> purse = wizcoin.WizCoin(2, 5, 10)
>>> repr(purse)  # Calls WizCoin's __repr__() behind the scenes.
'WizCoin(2, 5, 10)'
>>> str(purse)  # Calls WizCoin's __str__() behind the scenes.
'2g, 5s, 10k'
>>> print(f'My purse contains {purse}.')  # Calls WizCoin's __str__().
My purse contains 2g, 5s, 10k.

When we pass the WizCoin object in purse to the repr() and str() functions, behind the scenes Python calls the WizCoin class’s __repr__() and __str__() methods. We programmed these methods to return more readable and useful strings. If you entered the text of the 'WizCoin(2, 5, 10)' repr string into the interactive shell, it would create a WizCoin object that has the same attributes as the object in purse. The str string is a more human-readable representation of the object’s value: '2g, 5s, 10k'. If you use a WizCoin object in an f-string, Python uses the object’s str string.

If WizCoin objects were so complex that it would be impossible to create a copy of them with a single constructor function call, we would enclose the repr string in angle brackets to denote that it’s not meant to be Python code. This is what the generic representation strings, such as '<wizcoin.WizCoin object at 0x00000212B4148EE0>', do. Typing this string into the interactive shell would raise a SyntaxError, so it couldn’t possibly be confused for Python code that creates a copy of the object.

Inside the __repr__() method, we use self.__class__.__qualname__ instead of hardcoding the string 'WizCoin'; so if we subclass WizCoin, the inherited __repr__() method will use the subclass’s name instead of 'WizCoin'. In addition, if we rename the WizCoin class, the __repr__() method will automatically use the updated name.

But the WizCoin object’s str string shows us the attribute values in a neat, concise form. I highly recommended you implement __repr__() and __str__() in all your classes.

Numeric Dunder Methods

The numeric dunder methods, also called the math dunder methods, overload Python’s mathematical operators, such as +, -, *, /, and so on. Currently, we can’t perform an operation like adding two WizCoin objects together with the + operator. If we try to do so, Python will raise a TypeError exception, because it doesn’t know how to add WizCoin objects. To see this error, enter the following into the interactive shell:

>>> import wizcoin
>>> purse = wizcoin.WizCoin(2, 5, 10)
>>> tipJar = wizcoin.WizCoin(0, 0, 37)
>>> purse + tipJar
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'WizCoin' and 'WizCoin'

Instead of writing an addWizCoin() method for the WizCoin class, you can use the __add__() dunder method so WizCoin objects work with the + operator. Add the following to the end of the wizcoin.py file:

--snip--
1     def __add__(self, other):
        """Adds the coin amounts in two WizCoin objects together."""
2         if not isinstance(other, WizCoin):
            return NotImplemented

3         return WizCoin(other.galleons + self.galleons, other.sickles + self.sickles, other.knuts + self.knuts)

When a WizCoin object is on the left side of the + operator, Python calls the __add__() method 1 and passes in the value on the right side of the + operator for the other parameter. (The parameter can be named anything, but other is the convention.)

Keep in mind that you can pass any type of object to the __add__() method, so the method must include type checks 2. For example, it doesn’t make sense to add an integer or a float to a WizCoin object, because we don’t know whether it should be added to the galleons, sickles, or knuts amount.

The __add__() method creates a new WizCoin object with amounts equal to the sum of the galleons, sickles, and knuts attributes of self and other 3. Because these three attributes contain integers, we can use the + operator on them. Now that we’ve overloaded the + operator for the WizCoin class, we can use the + operator on WizCoin objects.

Overloading the + operator like this allows us to write more readable code. For example, enter the following into the interactive shell:

>>> import wizcoin
>>> purse = wizcoin.WizCoin(2, 5, 10)  # Create a WizCoin object.
>>> tipJar = wizcoin.WizCoin(0, 0, 37)  # Create another WizCoin object.
>>> purse + tipJar  # Creates a new WizCoin object with the sum amount.
WizCoin(2, 5, 47)

If the wrong type of object is passed for other, the dunder method shouldn’t raise an exception but rather return the built-in value NotImplemented. For example, in the following code, other is an integer:

>>> import wizcoin
>>> purse = wizcoin.WizCoin(2, 5, 10)
>>> purse + 42  # WizCoin objects and integers can't be added together.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'WizCoin' and 'int'

Returning NotImplemented signals Python to try calling other methods to perform this operation. (See “Reflected Numeric Dunder Methods” later in this chapter for more details.) Behind the scenes, Python calls the __add__() method with 42 for the other parameter, which also returns NotImplemented, causing Python to raise a TypeError.

Although we shouldn’t be able to add integers to or subtract them from WizCoin objects, it would make sense to allow code to multiply WizCoin objects by positive integer amounts by defining a __mul__() dunder method. Add the following to the end of wizcoin.py:

--snip--
    def __mul__(self, other):
        """Multiplies the coin amounts by a non-negative integer."""
        if not isinstance(other, int):
            return NotImplemented
        if other < 0:
            # Multiplying by a negative int results in negative
            # amounts of coins, which is invalid.
            raise WizCoinException('cannot multiply with negative integers')

        return WizCoin(self.galleons * other, self.sickles * other, self.knuts * other)

This __mul__() method lets you multiply WizCoin objects by positive integers. If other is an integer, it’s the data type the __mul__() method is expecting and we shouldn’t return NotImplemented. But if this integer is negative, multiplying the WizCoin object by it would result in negative amounts of coins in our WizCoin object. Because this goes against our design for this class, we raise a WizCoinException with a descriptive error message.

Enter the following into the interactive shell to see the __mul__() dunder method in action:

>>> import wizcoin
>>> purse = wizcoin.WizCoin(2, 5, 10)  # Create a WizCoin object.
>>> purse * 10  # Multiply the WizCoin object by an integer.
WizCoin(20, 50, 100)
>>> purse * -2  # Multiplying by a negative integer causes an error.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\Al\Desktop\wizcoin.py", line 86, in __mul__
    raise WizCoinException('cannot multiply with negative integers')
wizcoin.WizCoinException: cannot multiply with negative integers

Table 17-1 shows the full list of numeric dunder methods. You don’t always need to implement all of them for your class. It’s up to you to decide which methods are relevant.

Some of these methods are relevant to our WizCoin class. Try writing your own implementation of the __sub__(), __pow__(), __int__(), __float__(), and __bool__() methods. You can see an example of an implementation at https://autbor.com/wizcoinfull. The full documentation for the numeric dunder methods is in the Python documentation at https://docs.python.org/3/reference/datamodel.html#emulating-numeric-types.

The numeric dunder methods allow objects of your classes to use Python’s built-in math operators. If you’re writing methods with names like multiplyBy(), convertToInt(), or something similar that describes a task typically done by an existing operator or built-in function, use the numeric dunder methods (as well as the reflected and in-place dunder methods described in the next two sections).

Reflected Numeric Dunder Methods

Python calls the numeric dunder methods when the object is on the left side of a math operator. But it calls the reflected numeric dunder methods (also called the reverse or right-hand dunder methods) when the object is on the right side of a math operator.

Reflected numeric dunder methods are useful because programmers using your class won’t always write the object on the left side of the operator, which could lead to unexpected behavior. For example, let’s consider what happens when purse contains a WizCoin object, and Python evaluates the expression 2 * purse, where purse is on the right side of the operator:

1. Because 2 is an integer, the int class’s __mul__() method is called with purse passed for the other parameter.
2. The int class’s __mul__() method doesn’t know how to handle WizCoin objects, so it returns NotImplemented.
3. Python doesn’t raise a TypeError just yet. Because purse contains a WizCoin object, the WizCoin class’s __rmul__() method is called with 2 passed for the other parameter.
4. If __rmul__() returns NotImplemented, Python raises a TypeError.

Otherwise, the returned object from __rmul__() is what the 2 * purse expression evaluates to.

But the expression purse * 2, where purse is on the left side of the operator, works differently:

1. Because purse contains a WizCoin object, the WizCoin class’s __mul__() method is called with 2 passed for the other parameter.
2. The __mul__() method creates a new WizCoin object and returns it.
3. This returned object is what the purse * 2 expression evaluates to.

Numeric dunder methods and reflected numeric dunder methods have identical code if they are commutative. Commutative operations, like addition, have the same result backward and forward: 3 + 2 is the same as 2 + 3. But other operations aren’t commutative: 3 – 2 is not the same as 2 – 3. Any commutative operation can just call the original numeric dunder method whenever the reflected numeric dunder method is called. For example, add the following to the end of the wizcoin.py file to define a reflected numeric dunder method for the multiplication operation:

--snip--
    def __rmul__(self, other):
        """Multiplies the coin amounts by a non-negative integer."""
        return self.__mul__(other)

Multiplying an integer and a WizCoin object is commutative: 2 * purse is the same as purse * 2. Instead of copying and pasting the code from __mul__(), we just call self.__mul__() and pass it the other parameter.

After updating wizcoin.py, practice using the reflected multiplication dunder method by entering the following into the interactive shell:

>>> import wizcoin
>>> purse = wizcoin.WizCoin(2, 5, 10)
>>> purse * 10  # Calls __mul__() with 10 for the `other` parameter.
WizCoin(20, 50, 100)
>>> 10 * purse  # Calls __rmul__() with 10 for the `other` parameter.
WizCoin(20, 50, 100)

Keep in mind that in the expression 10 * purse, Python first calls the int class’s __mul__() method to see whether integers can be multiplied with WizCoin objects. Of course, Python’s built-in int class doesn’t know anything about the classes we create, so it returns NotImplemented. This signals to Python to next call WizCoin class’s __rmul__(), and if it exists, to handle this operation. If the calls to the int class’s __mul__() and WizCoin class’s __rmul__() both return NotImplemented, Python raises a TypeError exception.

Only WizCoin objects can be added to each other. This guarantees that the first WizCoin object’s __add__() method will handle the operation, so we don’t need to implement __radd__(). For example, in the expression purse + tipJar, the __add__() method for the purse object is called with tipJar passed for the other parameter. Because this call won’t return NotImplemented, Python doesn’t try to call the tipJar object’s __radd__() method with purse as the other parameter.

Table 17-2 contains a full listing of the available reflected dunder methods.

The full documentation for the reflected dunder methods is in the Python documentation at https://docs.python.org/3/reference/datamodel.html#emulating-numeric-types.

In-Place Augmented Assignment Dunder Methods

The numeric and reflected dunder methods always create new objects rather than modifying the object in-place. The in-place dunder methods, called by the augmented assignment operators, such as += and *=, modify the object in-place rather than creating new objects. (There is an exception to this, which I’ll explain at the end of this section.) These dunder method names begin with an i, such as __iadd__() and __imul__() for the += and *= operators, respectively.

For example, when Python runs the code purse *= 2, the expected behavior isn’t that the WizCoin class’s __imul__() method creates and returns a new WizCoin object with twice as many coins, and then assigns it the purse variable. Instead, the __imul__() method modifies the existing WizCoin object in purse so it has twice as many coins. This is a subtle but important difference if you want your classes to overload the augmented assignment operators.

Our WizCoin objects already overload the + and * operators, so let’s define the __iadd__() and __imul__() dunder methods so they overload the += and *= operators as well. In the expressions purse += tipJar and purse *= 2, we call the __iadd__() and __imul__() methods, respectively, with tipJar and 2 passed for the other parameter, respectively. Add the following to the end of the wizcoin.py file:

--snip--
    def __iadd__(self, other):
        """Add the amounts in another WizCoin object to this object."""
        if not isinstance(other, WizCoin):
            return NotImplemented

        # We modify the `self` object in-place:
        self.galleons += other.galleons
        self.sickles += other.sickles
        self.knuts += other.knuts
        return self  # In-place dunder methods almost always return self.

    def __imul__(self, other):
        """Multiply the amount of galleons, sickles, and knuts in this object
        by a non-negative integer amount."""
        if not isinstance(other, int):
            return NotImplemented
        if other < 0:
            raise WizCoinException('cannot multiply with negative integers')

        # The WizCoin class creates mutable objects, so do NOT create a
        # new object like this commented-out code:
        #return WizCoin(self.galleons * other, self.sickles * other, self.knuts * other)

        # We modify the `self` object in-place:
        self.galleons *= other
        self.sickles *= other
        self.knuts *= other
        return self  # In-place dunder methods almost always return self.

The WizCoin objects can use the += operator with other WizCoin objects and the *= operator with positive integers. Notice that after ensuring that the other parameter is valid, the in-place methods modify the self object in-place rather than creating a new WizCoin object. Enter the following into the interactive shell to see how the augmented assignment operators modify the WizCoin objects in-place:

>>> import wizcoin
>>> purse = wizcoin.WizCoin(2, 5, 10)
>>> tipJar = wizcoin.WizCoin(0, 0, 37)
1 >>> purse + tipJar
2 WizCoin(2, 5, 46)
>>> purse
WizCoin(2, 5, 10)
3 >>> purse += tipJar
>>> purse
WizCoin(2, 5, 47)
4 >>> purse *= 10
>>> purse
WizCoin(20, 50, 470)

The + operator 1 calls the __add__() or __radd__() dunder methods to create and return new objects 2. The original objects operated on by the + operator remain unmodified. The in-place dunder methods 3 4 should modify the object in-place as long as the object is mutable (that is, it’s an object whose value can change). The exception is for immutable objects: because an immutable object can’t be modified, it’s impossible to modify it in-place. In that case, the in-place dunder methods should create and return a new object, just like the numeric and reflected numeric dunder methods.

We didn’t make the galleons, sickles, and knuts attributes read-only, which means they can change. So WizCoin objects are mutable. Most of the classes you write will create mutable objects, so you should design your in-place dunder methods to modify the object in-place.

If you don’t implement an in-place dunder method, Python will instead call the numeric dunder method. For example, if the WizCoin class had no __imul__() method, the expression purse *= 10 will call __mul__() instead and assign its return value to purse. Because WizCoin objects are mutable, this is unexpected behavior that could lead to subtle bugs.

Comparison Dunder Methods

Python’s sort() method and sorted() function contain an efficient sorting algorithm that you can access with a simple call. But if you want to compare and sort objects of the classes you make, you’ll need to tell Python how to compare two of these objects by implementing the comparison dunder methods. Python calls the comparison dunder methods behind the scenes whenever your objects are used in an expression with the <, >, <=, >=, ==, and != comparison operators.

Before we explore the comparison dunder methods, let’s examine six functions in the operator module that perform the same operations as the six comparison operators. Our comparison dunder methods will be calling these functions. Enter the following into the interactive shell.

>>> import operator
>>> operator.eq(42, 42)        # "EQual", same as 42 == 42
True
>>> operator.ne('cat', 'dog')  # "Not Equal", same as 'cat' != 'dog'
True
>>> operator.gt(10, 20)        # "Greater Than ", same as 10 > 20
False
>>> operator.ge(10, 10)        # "Greater than or Equal", same as 10 >= 10
True
>>> operator.lt(10, 20)        # "Less Than", same as 10 < 20
True
>>> operator.le(10, 20)        # "Less than or Equal", same as 10 <= 20
True

The operator module gives us function versions of the comparison operators. Their implementations are simple. For example, we could write our own operator.eq() function in two lines:

def eq(a, b):
    return a == b

It’s useful to have a function form of the comparison operators because, unlike operators, functions can be passed as arguments to function calls. We’ll be doing this to implement a helper method for our comparison dunder methods.

First, add the following to the start of wizcoin.py. These imports give us access to the functions in the operator module and allow us to check whether the other argument in our method is a sequence by comparing it to collections.abc.Sequence:

import collections.abc
import operator

Then add the following to the end of the wizcoin.py file:

--snip--
1     def _comparisonOperatorHelper(self, operatorFunc, other):
        """A helper method for our comparison dunder methods."""

2         if isinstance(other, WizCoin):
            return operatorFunc(self.total, other.total)
3         elif isinstance(other, (int, float)):
            return operatorFunc(self.total, other)
4         elif isinstance(other, collections.abc.Sequence):
            otherValue = (other[0] * 17 * 29) + (other[1] * 29) + other[2]
            return operatorFunc(self.total, otherValue)
        elif operatorFunc == operator.eq:
            return False
        elif operatorFunc == operator.ne:
            return True
        else:
            return NotImplemented

    def __eq__(self, other):  # eq is "EQual"
5         return self._comparisonOperatorHelper(operator.eq, other)

    def __ne__(self, other):  # ne is "Not Equal"
6         return self._comparisonOperatorHelper(operator.ne, other)

    def __lt__(self, other):  # lt is "Less Than"
7         return self._comparisonOperatorHelper(operator.lt, other)

    def __le__(self, other):  # le is "Less than or Equal"
8         return self._comparisonOperatorHelper(operator.le, other)

    def __gt__(self, other):  # gt is "Greater Than"
9        return self._comparisonOperatorHelper(operator.gt, other)

    def __ge__(self, other):  # ge is "Greater than or Equal"
a        return self._comparisonOperatorHelper(operator.ge, other)

Our comparison dunder methods call the _comparisonOperatorHelper() method 1 and pass the appropriate function from the operator module for the operatorFunc parameter. When we call operatorFunc(), we’re calling the function that was passed for the operatorFunc parameter—eq()5, ne()6, lt()7, le()8, gt()9, or ge()a—from the operator module. Otherwise, we’d have to duplicate the code in _comparisonOperatorHelper() in each of our six comparison dunder methods.

Our WizCoin objects can now be compared with other WizCoin objects 2, integers and floats 3, and sequence values of three number values that represent the galleons, sickles, and knuts 4. Enter the following into the interactive shell to see this in action:

>>> import wizcoin
>>> purse = wizcoin.WizCoin(2, 5, 10)  # Create a WizCoin object.
>>> tipJar = wizcoin.WizCoin(0, 0, 37) # Create another WizCoin object.
>>> purse.total, tipJar.total # Examine the values in knuts.
(1141, 37)
>>> purse > tipJar # Compare WizCoin objects with a comparison operator.
True
>>> purse < tipJar
False
>>> purse > 1000 # Compare with an int.
True
>>> purse <= 1000
False
>>> purse == 1141
True
>>> purse == 1141.0 # Compare with a float.
True
>>> purse == '1141' # The WizCoin is not equal to any string value.
False
>>> bagOfKnuts = wizcoin.WizCoin(0, 0, 1141)
>>> purse == bagOfKnuts
True
>>> purse == (2, 5, 10) # We can compare with a 3-integer tuple.
True
>>> purse >= [2, 5, 10] # We can compare with a 3-integer list.
True
>>> purse >= ['cat', 'dog'] # This should cause an error.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\Al\Desktop\wizcoin.py", line 265, in __ge__
    return self._comparisonOperatorHelper(operator.ge, other)
  File "C:\Users\Al\Desktop\wizcoin.py", line 237, in _comparisonOperatorHelper
    otherValue = (other[0] * 17 * 29) + (other[1] * 29) + other[2]
IndexError: list index out of range

Our helper method calls isinstance(other, collections.abc.Sequence) to see whether other is a sequence data type, such as a tuple or list. By making WizCoin objects comparable with sequences, we can write code such as purse >= [2, 5, 10] for a quick comparison.

There are no reflected comparison dunder methods, such as __req__() or __rne__(), that you’ll need to implement. Instead, __lt__() and __gt__() reflect each other, __le__() and __ge__() reflect each other, and __eq__() and __ne__() reflect themselves. The reason is that the following relationships hold true no matter what the values on the left or right side of the operator are:

• purse > [2, 5, 10] is the same as [2, 5, 10] < purse
• purse >= [2, 5, 10] is the same as [2, 5, 10] <= purse
• purse == [2, 5, 10] is the same as [2, 5, 10] == purse
• purse != [2, 5, 10] is the same as [2, 5, 10] != purse

Once you’ve implemented the comparison dunder methods, Python’s sort() function will automatically use them to sort your objects. Enter the following into the interactive shell:

>>> import wizcoin
>>> oneGalleon = wizcoin.WizCoin(1, 0, 0) # Worth 493 knuts.
>>> oneSickle = wizcoin.WizCoin(0, 1, 0)  # Worth 29 knuts.
>>> oneKnut = wizcoin.WizCoin(0, 0, 1)    # Worth 1 knut.
>>> coins = [oneSickle, oneKnut, oneGalleon, 100]
>>> coins.sort() # Sort them from lowest value to highest.
>>> coins
[WizCoin(0, 0, 1), WizCoin(0, 1, 0), 100, WizCoin(1, 0, 0)] 

Table 17-3 contains a full listing of the available comparison dunder methods and operator functions.

You can see the implementation for these methods at https://autbor.com/wizcoinfull. The full documentation for the comparison dunder methods is in the Python documentation at https://docs.python.org/3/reference/datamodel.html#object.__lt__.

The comparison dunder methods let objects of your classes use Python’s comparison operators rather than forcing you to create your own methods. If you’re creating methods named equals() or isGreaterThan(), they’re not Pythonic, and they’re a sign that you should use comparison dunder methods.

Summary

Python implements object-oriented features differently than other OOP languages, such as Java or C++. Instead of explicit getter and setter methods, Python has properties that allow you to validate attributes or make attributes read-only.

Python also lets you overload its operators via its dunder methods, which begin and end with double underscore characters. We overload common mathematical operators using the numeric and reflected numeric dunder methods. These methods provide a way for Python’s built-in operators to work with objects of the classes you create. If they’re unable to handle the data type of the object on the other side of the operator, they’ll return the built-in NotImplemented value. These dunder methods create and return new objects, whereas the in-place dunder methods (which overload the augmented assignment operators) modify the object in-place. The comparison dunder methods not only implement the six Python comparison operators for objects, but also allow Python’s sort() function to sort objects of your classes. You might want to use the eq(), ne(), lt(), le(), gt(), and ge() functions in the operator module to help you implement these dunder methods.

Properties and dunder methods allow you to write classes that are consistent and readable. They let you avoid much of the boilerplate code that other languages, such as Java, require you to write. To learn more about writing Pythonic code, two PyCon talks by Raymond Hettinger expand on these ideas: “Transforming Code into Beautiful, Idiomatic Python” at https://youtu.be/OSGv2VnC0go/ and “Beyond PEP 8—Best Practices for Beautiful, Intelligible Code” at https://youtu.be/wf-BqAjZb8M/ cover some of the concepts in this chapter and beyond.

There’s much more to learn about how to use Python effectively. The books Fluent Python (O’Reilly Media, 2021) by Luciano Ramalho and Effective Python (Addison-Wesley Professional, 2019) by Brett Slatkin provide more in-depth information about Python’s syntax and best practices, and are must-reads for anyone who wants to continue to learn more about Python.