Python

Affirming your PSF Membership voting status

Every PSF Voting Member (Supporting, Managing, Contributing, and Fellow) needs to affirm their membership in order to vote in this year’s election.This year’s Board Election begins June 20th, and voter eligibility cut-off is June 15th. You should have ...
Python

The Python Language Summit 2023: Burnout is Real


Guido van Rossum, creator of the Python programming language, spoke at the 2023 Python Language Summit on the subject of open-source burnout, strategies for tackling it, and how to avoid it.The first known case of burnout in the field of open-sou...
Python

The Python Language Summit 2023: Python on Mobile


At the Python Language Summit 2023, Russell Keith-Magee presented on the ongoing efforts of BeeWare, a project that aims to make it easier to run Python on mobile platforms such as Android and iOS.The BeeWare logoRussell Keith-Magee is one busy beeImp...
Python

The Python Language Summit 2023

Every year, just before the start of PyCon US, core developers, triagers, and special guests gather for the Python Language Summit: an all-day event of talks where the future direction of Python is discussed. The Language Summit 2023 included three ba...
Python

The Python Language Summit 2023: Lightning Talks


The Python Language Summit 2023 closed off with a trio of lightning talks from Dong-hee Na, Carl Meyer and Amethyst Reese.Dong-hee Na: Let’s support LLVM-BOLT as an official featureCPython Core Developer Dong-hee Na gave a short presentation on the LL...
Python

The Python Language Summit 2023: Pattern Matching, __match__, and View Patterns

One of the most exciting new features in Python 3.10 was the introduction of pattern matching (introduced in PEPs 634, 635 and 636). Pattern matching has a wide variety of uses, but really shines in situations where you need to undergo complex destructurings of tree-like datastructures.That’s a lot of words which may or may not mean very much to you – but consider, for example, using the ast module to parse Python source code. If you’re unfamiliar with the ast module: the module provides tools that enable you to compile Python source code into an “abstract syntax tree” (AST) representing the code’s structure. The Python interpreter itself converts Python source code into an AST in order to understand how to run that code – but parsing Python source code using ASTs is also a common task for linters, such as plugins for flake8 or pylint. In the following example, ast.parse() is used to parse the source code x = 42 into an ast.Module node, and ast.dump() is then used to reveal the tree-like structure of that node in a human-readable form:

>>> import ast
>>> source = "x = 42"
>>> node = ast.parse(source)
>>> node
<ast.Module object at 0x000002A70F928D80>
>>> print(ast.dump(node, indent=2))
Module(
body=[
Assign(
targets=[
Name(id='x', ctx=Store())],
value=Constant(value=42))],
type_ignores=[])


How does working with ASTs relate to pattern-matching? Well, a function to determine whether (to a reasonable approximation) an arbitrary AST node represents the symbol collections.deque might have looked something like this, before pattern matching…

import ast

# This obviously won't work if the symbol is imported with an alias
# in the source code we're inspecting
# (e.g. "from collections import deque as d").
# But let's not worry about that here :-)

def node_represents_collections_dot_deque(node: ast.AST) -> bool:
"""Determine if *node* represents 'deque' or 'collections.deque'"""
return (
isinstance(node, ast.Name) and node.id == "deque"
) or (
isinstance(node, ast.Attribute)
and isinstance(node.value, ast.Name)
and node.value.id == "collections"
and node.value.attr == "deque"
)



But in Python 3.10, pattern matching allows an elegant destructuring syntax:

import ast

def node_represents_collections_dot_deque(node: ast.AST) -> bool:
"""Determine if *node* represents 'deque' or 'collections.deque'"""
match node:
case ast.Name("deque"):
return True
case ast.Attribute(ast.Name("collections"), "deque"):
return True
case _:
return False


I know which one I prefer.For some, though, this still isn’t enough – and Michael “Sully” Sullivan is one of them. At the Python Language Summit 2023, Sullivan shared ideas for where pattern matching could go next.
Playing with matches (without getting burned)
Sullivan’s contention is that, while pattern matching provides elegant syntactic sugar in simple cases such as the one above, our ability to chain destructurings using pattern matching is currently fairly limited. For example, say we want to write a function inspecting Python AST that takes an ast.FunctionDef node and identifies whether the node represents a synchronous function with exactly two parameters, both of them annotated as accepting integers. The function would behave so that the following holds true:

>>> import ast
>>> source = "def add_2(number1: int, number2: int): pass"
>>> node = ast.parse(source).body[0]
>>> type(node)
<class 'ast.FunctionDef'>
>>> is_function_taking_two_ints(node)
True


With pre-pattern-matching syntax, we might have written such a function like this:

def is_int(node: ast.AST | None) -> bool:
"""Determine if *node* represents 'int' or 'builtins.int'"""
return (
isinstance(node, ast.Name) and node.id == "int"
) or (
isinstance(node, ast.Attribute)
and isinstance(node.value, ast.Name)
and node.value.id == "builtins"
and node.attr == "int"
)

def is_function_taking_two_ints(node: ast.FunctionDef) -> bool:
"""Determine if *node* represents a function that accepts two ints"""
args = node.args.posonlyargs + node.args.args
return len(args) == 2 and all(is_int(node.annotation) for node in args)



If we wanted to rewrite this using pattern matching, we could possibly do something like this:

def is_int(node: ast.AST | None) -> bool:
"""Determine if *node* represents 'int' or 'builtins.int'"""
match node:
case ast.Name("int"):
return True
case ast.Attribute(ast.Name("builtins"), "int"):
return True
case _:
return False

def is_function_taking_two_ints(node: ast.FunctionDef) -> bool:
"""Determine if *node* represents a function that accepts two ints"""
match node.args.posonlyargs + node.args.args:
case [ast.arg(), ast.arg()] as arglist:
return all(is_int(arg.annotation) for arg in arglist)
case _:
return False


That leaves a lot to be desired, however! The is_int() helper function can be rewritten in a much cleaner way. But integrating it into the is_function_taking_two_ints() is… somewhat icky! The code feels harder to understand than before, whereas the goal of pattern matching is to improve readability.Something like this, (ab)using metaclasses, gets us a lot closer to what it feels pattern matching should be like. By using one of Python’s hooks for customising isinstance() logic, it’s possible to rewrite our is_int() helper function as a class, meaning we can seamlessly integrate it into our is_function_taking_two_ints() function in a very expressive way:

import abc
import ast

class PatternMeta(abc.ABCMeta):
def __instancecheck__(cls, inst: object) -> bool:
return cls.match(inst)

class Pattern(metaclass=PatternMeta):
"""Abstract base class for types representing 'abstract patterns'"""
@staticmethod
@abc.abstractmethod
def match(node) -> bool:
"""Subclasses must override this method"""
raise NotImplementedError

class int_node(Pattern):
"""Class representing AST patterns signifying `int` or `builtins.int`"""
@staticmethod
def match(node) -> bool:
match node:
case ast.Name("int"):
return True
case ast.Attribute(ast.Name("builtins"), "int"):
return True
case _:
return False

def is_function_taking_two_ints(node: ast.FunctionDef) -> bool:
"""Determine if *node* represents a function that accepts two ints"""
match node.args.posonlyargs + node.args.args:
case [
ast.arg(annotation=int_node()),
ast.arg(annotation=int_node()),
]:
return True
case _:
return False


This is still hardly ideal, however – that’s a lot of boilerplate we’ve had to introduce to our helper function for identifying int annotations! And who wants to muck about with metaclasses?
A slide from Sullivan's talk

A __match__ made in heaven?
Sullivan proposes that we make it easier to write helper functions for pattern matching, such as the example above, without having to resort to custom metaclasses. Two competing approaches were brought for discussion.The first idea – a __match__ special method – is perhaps the easier of the two to immediately grasp, and appeared in early drafts of the pattern matching PEPs. (It was eventually removed from the PEPs in order to reduce the scope of the proposed changes to Python.) The proposal is that any class could define a __match__ method that could be used to customise how match statements apply to the class. Our is_function_taking_two_ints() case could be rewritten like so:

class int_node:
"""Class representing AST patterns signifying `int` or `builtins.int`"""
# The __match__ method is understood by Python to be a static method,
# even without the @staticmethod decorator,
# similar to __new__ and __init_subclass__
def __match__(node) -> ast.Name | ast.Attribute:
match node:
case ast.Name("int"):
# Successful matches can return custom objects,
# that can be bound to new variables by the caller
return node
case ast.Attribute(ast.Name("builtins"), "int"):
return node
case _:
# Return `None` to indicate that there was no match
return None

def is_function_taking_two_ints(node: ast.FunctionDef) -> bool:
"""Determine if *node* represents a function that accepts two ints"""
match node.args.posonlyargs + node.args.args:
case [
ast.arg(annotation=int_node()),
ast.arg(annotation=int_node()),
]:
return True
case _:
return False


The second idea is more radical: the introduction of some kind of new syntax (perhaps reusing Python’s -> operator) that would allow Python coders to “apply” functions during pattern matching. With this proposal, we could rewrite is_function_taking_two_ints() like so:

def is_int(node: ast.AST | None) -> bool:
"""Determine if *node* represents 'int' or 'builtins.int'"""
match node:
case ast.Name("int"):
return True
case ast.Attribute(ast.Name("builtins"), "int"):
return True
case _:
return False

def is_function_taking_two_ints(node: ast.FunctionDef) -> bool:
"""Determine if *node* represents a function that accepts two ints"""
match node.args.posonlyargs + node.args.args:
case [
ast.arg(annotation=is_int -> True),
ast.arg(annotation=is_int -> True),
]
case _:
return False




Match-maker, match-maker, make me a __match__…

A slide from Sullivan's talk

The reception in the room to Sullivan’s ideas was positive; the consensus seemed to be that there was clearly room for improvement in this area. Brandt Bucher, author of the original pattern matching implementation in Python 3.10, concurred that this kind of enhancement was needed. Łukasz Langa, meanwhile, said he’d received many queries from users of other programming languages such as C#, asking how to tackle this kind of problem.The proposal for a __match__ special method follows a pattern common in Python’s data model, where double-underscore “dunder” methods are overridden to provide a class with special behaviour. As such, it will likely be less jarring, at first glance, to those new to the idea. Attendees of Sullivan’s talk seemed, broadly, to slightly prefer the __match__ proposal, and Sullivan himself said he thought it “looked prettier”.Jelle Zijlstra argued that the __match__ dunder would provide an elegant symmetry between the construction and destruction of objects. Brandt Bucher, meanwhile, said he thought the usability improvements weren’t significant enough to merit new syntax.Nonetheless, the alternative proposal for new syntax also has much to recommend it. Sullivan argued that having dedicated syntax to express the idea of “applying” a function during pattern matching was more explicit. Mark Shannon agreed, noting the similarity between this idea and features in the Haskell programming language. “This is functional programming,” Shannon argued. “It feels weird to apply OOP models to this.”

Addendum: pattern-matching resources and recipes
In the meantime, while we wait for a PEP, there are plenty of innovative uses of pattern matching springing up in the ecosystem. For further reading/watching/listening, I recommend:
“A perfect match: The history, design and future of Python’s structural pattern matching” – A talk by Brandt Bucher at PyCon 2022
“Structural Pattern Matching in the Real World” – A talk by Raymond Hettinger at Pycon Italia 2022
RegexMatcher: a class integrating pattern matching with Python’s re module. A 2022 Advent of Code solution by Ned Batchelder.
approximately: A way to compare float and complex numbers using pattern matching, while avoiding the perils of floating-point arithmetic. A StackOverflow Q&A by Raymond Hettinger.
“A few related schemes for implementing view patterns in Python”: A gist by Michael Sullivan (from February 2023)
Python

Python 3.12.0 beta 1 released

I'm pleased to announce the release of Python 3.12 beta 1 (and feature freeze for Python 3.12).https://www.python.org/downloads/release/python-3120b1/This is a beta preview of Python 3.12Python 3.12 is still in development. This release, 3.12.0b1, is t...
Python

PSF Board Election Dates for 2023

Board elections are a chance for the community to choose representatives to help the PSF create a vision for and build the future of the Python community. This year there are 4 seats open on the PSF board. You can see who is on the board currently here...
Python

PEP 713: Callable Modules

Modules are currently not directly callable. Classes can define a __call__ method that makes instance objects callable, but defining a similarly named function in the global module scope has no effect, and that function can only be called by importing ...
Python

Thank You for Many Years of Service, Van!

We are wishing farewell to Van Lindberg after 16 years of service to the PSF, who has decided to step down from his Board Director and General Counsel roles. He helped us grow from a small volunteer organization with no staff, to an organization with 9...
Python

Protecting the Python Trademarks

Who is the Trademarks Working Group?The Python Software Foundation Trademarks Working Group was created by the PSF Board of Directors to  monitor and authorize (or prohibit) use of trademarks held by the PSF.  The WG—initially dubbe...
Python

Python 3.12.0 alpha 6 released

I'm pleased to announce the release of Python 3.12 alpha 6.https://www.python.org/downloads/release/python-3120a6/This is an early developer preview of Python 3.12.Major new features of the 3.12 series, compared to 3.11Python 3.12 is still in developme...
Python

PEP 709: Inlined comprehensions

Comprehensions are currently compiled as nested functions, which provides isolation of the comprehension's iteration variable, but is inefficient at runtime. This PEP proposes to inline list, dictionary, and set comprehensions into the code where they ...
Python

PEP 706: Filter for tarfile.extractall

The extraction methods in :external+py3.11:mod:`tarfile` gain a filter argument, which allows rejecting files or modifying metadata as the archive is extracted. Three built-in named filters are provided, aimed at limiting features that might be surpris...
Python

Python 3.12.0 alpha 4 released

I'm pleased to announce the release of Python 3.12 alpha 4.https://www.python.org/downloads/release/python-3120a4/This is an early developer preview of Python 3.12.Major new features of the 3.12 series, compared to 3.11Python 3.12 is still in developme...
Python

Starting 2023 with momentum, thanks to you!

We are grateful to each of you who shared or donated to our year-end fundraiser and membership drive. Over 300 individual donations plus new Supporting Memberships, renewals, and JetBrains’ generous match came together to raise $61,868 total for our wo...
Python

Introducing a New Sliding Scale Membership

The Python Software Foundation (PSF) has made a sliding scale option available for Supporting Membership, because we want to make membership more accessible for more people and to increase the size and diversity of our voting membership. New Supporting...
Python

Where is the PSF?

Where to Find the PSF OnlineOne of the main ways we reach people for news and information about the PSF and Python is Twitter. There’s been a lot of uncertainty around that platform recently, so we wanted to share a brief round up of other places you c...
Python

Python 3.12.0 alpha 2 released

I'm pleased to announce the release of Python 3.12 alpha 2.https://www.python.org/downloads/release/python-3120a2/This is an early developer preview of Python 3.12.Major new features of the 3.12 series, compared to 3.11Python 3.12 is still in developme...
Python

PEP 701: Syntactic formalization of f-strings

This document proposes to lift some of the restrictions originally formulated in PEP 498 and to provide a formalized grammar for f-strings that can be integrated into the parser directly. The proposed syntactic formalization of f-strings will have some...
Python

Thank You for Making PyCon US amazing, Jackie!

Jackie Augustine is moving on from her role of Director of Events at the PSF. Please join me in thanking Jackie for all of her amazing work with our volunteers, vendors, and staff. Her energy, heart, and dedication will be keenly missed. Jackie says, “...
Python

Python 3.12.0 alpha 1 released

 As Pablo released Python 3.11.0 final earlier today, now it's my turn to release Python 3.12.0 alpha 1.https://www.python.org/downloads/release/python-3120a1/This is an early developer preview of Python 3.12Major new features of the 3.12 series, ...
Python

Python 3.11.0 is now available

 This is the release of Python 3.11.0Python 3.11 is finally released. In the CPython release team, we have put a lot of effort into making 3.11 the best version of Python possible. Better tracebacks, faster Python, exception groups and except*, ty...
Python

Join the Python Developers Survey 2022: Share and learn about the community

This year we are conducting the sixth iteration of the official Python Developers Survey. The goal is to capture the current state of the language and the ecosystem around it. By comparing the results with last year’s, we can identify and share with everyone the hottest trends in the Python community and the key insights into it. We encourage you to contribute to our community’s knowledge. The survey should only take you about 10-15 minutes to complete.

Contribute to the Python Developers Survey 2022!








The survey is organized in partnership between the Python Software Foundation and JetBrains. After the survey is over, we will publish the aggregated results and randomly choose 20 winners (among those who complete the survey in its entirety), who will each receive a $100 Amazon Gift Card or a local equivalent.

Python

Python 3.11.0rc2 is now available

This is the second release candidate of Python 3.11https://www.python.org/downloads/release/python-3110rc2/This release, 3.11.0rc2, is the last preview before the final release of Python 3.11.0 on 2022-10-24.Entering the release candidate phase, only r...
Python

Python 3.11.0rc1 is now available

This is the first release candidate of Python 3.11https://www.python.org/downloads/release/python-3110rc1/This release, **3.11.0rc1**, is the penultimate release preview.  Entering the release candidate phase, only reviewed code changes which are ...
Python

Python 3.10.6 is available

Here you have a nice package of 200 commits of bugfixes and documentation improvements freshly made for Python 3.10. Go and download it when is still hot:https://www.python.org/downloads/release/python-3106/This is the sixth maintenance release of Pyth...
Python

Python 3.11.0b5 is now available

Here we are. The universe. The vastness of spacetime. At the edge. The last frontier. The last beta*(conditions apply) for Python 3.11.We have defied the powerful gods of release blockers and we have won by using the required amount of ruse and subterf...
Python

Distinguished Service Award Granted to Naomi Ceder

Naomi Ceder, a longtime Python activist and organizer has been recognized with the PSF’s Distinguished Service Award. Naomi served on the PSF Board from 2015-2020, and as Chair from 2017-2020, supported the search for a new Executive Director last year and keynoted the most recent PyCon US in Salt Lake City.

The PSF’s Distinguished Service Award (DSA) is granted to individuals who make sustained exemplary contributions to the Python community. Each award is voted on by the PSF Board and they are looking for people whose impact has positively and significantly shaped the Python world. Naomi’s work with the Python community very much exemplifies the ethos of “build the community you want to see.” She seems to particularly enjoy taking on the hardest parts, getting a new initiative started and figuring out how to take an idea from the drawing board to a regular activity that we can’t imagine leaving out.

After receiving the award Naomi shared, "I'm so grateful for the recognition, and even more grateful for all of the support that our community has given me over the years. I'm excited to see a new generation of Python volunteers continue the work to make our community and the PSF more global and inclusive, and I'm looking forward to working with smaller communities as they grow and develop."

Over the years Naomi has taken on many leadership roles to make PyCon US successful and welcoming. She served as Chair of the Hatchery Program and she helped found PyCon Charlas, our Spanish language track. At different points in time, she’s also been the Co-chair of Sprints, an Organizer of the PyCon Education Summit and Chair for poster sessions at PyCon US. She also co-founded Trans*Code, an on-going series of hackdays (mostly) in the UK, which aims to build community and foster tech education and skills for transgender and non-binary folks. PyCon US and the global Python community would not look like it does without her tireless, largely behind the scenes work. Her deep thoughtfulness coupled with her energy is an immeasurable gift to the Python community. 

Curious about previous recipients of the DSA or wondering how to nominate someone? We got you.
Python