Protocols

docs/source/class_basics.rst

@@ -151,7 +151,174 @@ concrete. As with normal overrides, a dynamically typed method can
 implement a statically typed abstract method defined in an abstract
 base class.
 
+.. _protocol-types:
+
+Protocols and structural subtyping
+**********************************
+
+.. note::
+
+ The support for structural subtyping is still experimental. Some features
+ might be not yet implemented, mypy could pass unsafe code or reject
+ working code.
+
+There are two main type systems with respect to subtyping: nominal subtyping
+and structural subtyping. The *nominal* subtyping is based on class hierarchy,
+so that if class ``D`` inherits from class ``C``, then it is a subtype
+of ``C``. This type system is primarily used in mypy since it allows
+to produce clear and concise error messages, and since Python provides native
+``isinstance()`` checks based on class hierarchy. The *structural* subtyping
+however has its own advantages. In this system class ``D`` is a subtype
+of class ``C`` if the former has all attributes of the latter with
+compatible types.
+
+This type system is a static equivalent of duck typing, well known by Python
+programmers. Mypy provides an opt-in support for structural subtyping via
+protocol classes described in this section.
+See `PEP 544 <https://www.python.org/dev/peps/pep-0544/>`_ for
+specification of protocols and structural subtyping in Python.
+
+User defined protocols
+**********************
+
+To define a protocol class, one must inherit the special
+``typing_extensions.Protocol`` class:
+
+.. code-block:: python
+
+ from typing import Iterable
+ from typing_extensions import Protocol
+
+ class SupportsClose(Protocol):
+ def close(self) -> None:
+ ...
+
+ class Resource: # Note, this class does not have 'SupportsClose' base.
+ # some methods
+ def close(self) -> None:
+ self.resource.release()
+
+ def close_all(things: Iterable[SupportsClose]) -> None:
+ for thing in things:
+ thing.close()
+
+ close_all([Resource(), open('some/file')]) # This passes type check
+
+.. note::
+
+ The ``Protocol`` base class is currently provided in ``typing_extensions``
+ package. When structural subtyping is mature and
+ `PEP 544 <https://www.python.org/dev/peps/pep-0544/>`_ is accepted,
+ ``Protocol`` will be included in the ``typing`` module. As well, several
+ types such as ``typing.Sized``, ``typing.Iterable`` etc. will be made
+ protocols.
+
+Defining subprotocols
+*********************
+
+Subprotocols are also supported. Existing protocols can be extended
+and merged using multiple inheritance. For example:
+
+.. code-block:: python
+
+ # continuing from previous example
+
+ class SupportsRead(Protocol):
+ def read(self, amount: int) -> bytes: ...
+
+ class TaggedReadableResource(SupportsClose, SupportsRead, Protocol):
+ label: str
+
+ class AdvancedResource(Resource):
+ def __init__(self, label: str) -> None:
+ self.label = label
+ def read(self, amount: int) -> bytes:
+ # some implementation
+ ...
+
+ resource = None # type: TaggedReadableResource
+
+ # some code
+
+ resource = AdvancedResource('handle with care') # OK
+
+Note that inheriting from existing protocols does not automatically turn
+a subclass into a protocol, it just creates a usual (non-protocol) ABC that
+implements given protocols. The ``typing_extensions.Protocol`` base must always
+be explicitly present:
+
+.. code-block:: python
+
+ class NewProtocol(SupportsClose): # This is NOT a protocol
+ new_attr: int
+
+ class Concrete:
+ new_attr = None # type: int
+ def close(self) -> None:
+ ...
+ # Below is an error, since nominal subtyping is used by default
+ x = Concrete() # type: NewProtocol # Error!
+
+.. note::
+
+ The `PEP 526 <https://www.python.org/dev/peps/pep-0526/>`_ variable
+ annotations can be used to declare protocol attributes. However, protocols
+ are also supported on Python 2.7 and Python 3.3+ with the help of type
+ comments and properties, see
+ `backwards compatibility in PEP 544 <https://www.python.org/dev/peps/pep-0544/#backwards-compatibility>`_.
+
+Recursive protocols
+*******************
+
+Protocols can be recursive and mutually recursive. This could be useful for
+declaring abstract recursive collections such as trees and linked lists:
+
+.. code-block:: python
+
+ from typing import TypeVar, Optional
+ from typing_extensions import Protocol
+
+ class TreeLike(Protocol):
+ value: int
+ @property
+ def left(self) -> Optional['TreeLike']: ...
+ @property
+ def right(self) -> Optional['TreeLike']: ...
+
+ class SimpleTree:
+ def __init__(self, value: int) -> None:
+ self.value = value
+ self.left: Optional['SimpleTree'] = None
+ self.right: Optional['SimpleTree'] = None
+
+ root = SimpleTree(0) # type: TreeLike # OK
+
+Using ``isinstance()`` with protocols
+*************************************
+
+To use a protocol class with ``isinstance()``, one needs to decorate it with
+a special ``typing_extensions.runtime`` decorator. It will add support for
+basic runtime structural checks:
+
+.. code-block:: python
+
+ from typing_extensions import Protocol, runtime
+
+ @runtime
+ class Portable(Protocol):
+ handles: int
+
+ class Mug:
+ def __init__(self) -> None:
+ self.handles = 1
+
+ mug = Mug()
+ if isinstance(mug, Portable):
+ use(mug.handles) # Works statically and at runtime.
+
 .. note::
+ ``isinstance()`` is with protocols not completely safe at runtime.
+ For example, signatures of methods are not checked. The runtime
+ implementation only checks the presence of all protocol members
+ in object's MRO.
 
- There are also plans to support more Python-style "duck typing" in
- the type system. The details are still open.

docs/source/common_issues.rst

@@ -226,6 +226,48 @@ Possible strategies in such situations are:
  return x[0]
  f_good(new_lst) # OK
 
+Covariant subtyping of mutable protocol members is rejected
+-----------------------------------------------------------
+
+Mypy rejects this because this is potentially unsafe.
+Consider this example:
+
+.. code-block:: python
+
+ from typing_extensions import Protocol
+
+ class P(Protocol):
+ x: float
+
+ def fun(arg: P) -> None:
+ arg.x = 3.14
+
+ class C:
+ x = 42
+ c = C()
+ fun(c) # This is not safe
+ c.x << 5 # Since this will fail!
+
+To work around this problem consider whether "mutating" is actually part
+of a protocol. If not, then one can use a ``@property`` in
+the protocol definition:
+
+.. code-block:: python
+
+ from typing_extensions import Protocol
+
+ class P(Protocol):
+ @property
+ def x(self) -> float:
+ pass
+
+ def fun(arg: P) -> None:
+ ...
+
+ class C:
+ x = 42
+ fun(C()) # OK
+
 Declaring a supertype as variable type
 --------------------------------------
 

docs/source/faq.rst

@@ -101,35 +101,38 @@ Is mypy free?
 Yes. Mypy is free software, and it can also be used for commercial and
 proprietary projects. Mypy is available under the MIT license.
 
-Why not use structural subtyping?
-*********************************
+Can I use structural subtyping?
+*******************************
 
-Mypy primarily uses `nominal subtyping
-<https://en.wikipedia.org/wiki/Nominative_type_system>`_ instead of
+Mypy provides support for both `nominal subtyping
+<https://en.wikipedia.org/wiki/Nominative_type_system>`_ and
 `structural subtyping
-<https://en.wikipedia.org/wiki/Structural_type_system>`_. Some argue
-that structural subtyping is better suited for languages with duck
-typing such as Python.
-
-Here are some reasons why mypy uses nominal subtyping:
+<https://en.wikipedia.org/wiki/Structural_type_system>`_.
+Support for structural subtyping is considered experimental.
+Some argue that structural subtyping is better suited for languages with duck
+typing such as Python. Mypy however primarily uses nominal subtyping,
+leaving structural subtyping opt-in. Here are some reasons why:
 
 1. It is easy to generate short and informative error messages when
  using a nominal type system. This is especially important when
  using type inference.
 
-2. Python supports basically nominal isinstance tests and they are
- widely used in programs. It is not clear how to support isinstance
- in a purely structural type system while remaining compatible with
- Python idioms.
+2. Python provides built-in support for nominal ``isinstance()`` tests and
+ they are widely used in programs. Only limited support for structural
+ ``isinstance()`` exists for ABCs in ``collections.abc`` and ``typing``
+ standard library modules.
 
 3. Many programmers are already familiar with nominal subtyping and it
  has been successfully used in languages such as Java, C++ and
  C#. Only few languages use structural subtyping.
 
-However, structural subtyping can also be useful. Structural subtyping
-is a likely feature to be added to mypy in the future, even though we
-expect that most mypy programs will still primarily use nominal
-subtyping.
+However, structural subtyping can also be useful. For example, a "public API"
+may be more flexible if it is typed with protocols. Also, using protocol types
+removes the necessity to explicitly declare implementations of ABCs.
+As a rule of thumb, we recommend using nominal classes where possible, and
+protocols where necessary. For more details about protocol types and structural
+subtyping see :ref:`protocol-types` and
+`PEP 544 <https://www.python.org/dev/peps/pep-0544/>`_.
 
 I like Python and I have no need for static typing
 **************************************************

docs/source/generics.rst

@@ -1,6 +1,8 @@
 Generics
 ========
 
+.. _generic-classes:
+
 Defining generic classes
 ************************
 
@@ -489,6 +491,73 @@ restrict the valid values for the type parameter in the same way.
 A type variable may not have both a value restriction (see
 :ref:`type-variable-value-restriction`) and an upper bound.
 
+Generic protocols
+*****************
+
+Generic protocols (see :ref:`protocol-types`) are also supported, generic
+protocols mostly follow the normal rules for generic classes, the main
+difference is that mypy checks that declared variance of type variables is
+compatible with the class definition. Examples:
+
+.. code-block:: python
+
+ from typing import TypeVar
+ from typing_extensions import Protocol
+
+ T = TypeVar('T')
+
+ class Box(Protocol[T]):
+ content: T
+
+ def do_stuff(one: Box[str], other: Box[bytes]) -> None:
+ ...
+
+ class StringWrapper:
+ def __init__(self, content: str) -> None:
+ self.content = content
+
+ class BytesWrapper:
+ def __init__(self, content: bytes) -> None:
+ self.content = content
+
+ do_stuff(StringWrapper('one'), BytesWrapper(b'other')) # OK
+
+ x = None # type: Box[float]
+ y = None # type: Box[int]
+ x = y # Error, since the protocol 'Box' is invariant.
+
+ class AnotherBox(Protocol[T]): # Error, covariant type variable expected
+ def content(self) -> T:
+ ...
+
+ T_co = TypeVar('T_co', covariant=True)
+ class AnotherBox(Protocol[T_co]): # OK
+ def content(self) -> T_co:
+ ...
+
+ ax = None # type: AnotherBox[float]
+ ay = None # type: AnotherBox[int]
+ ax = ay # OK for covariant protocols
+
+See :ref:`variance-of-generics` above for more details on variance.
+Generic protocols can be recursive, for example:
+
+.. code-block:: python
+
+ T = TypeVar('T')
+ class Linked(Protocol[T]):
+ val: T
+ def next(self) -> 'Linked[T]': ...
+
+ class L:
+ val: int
+ def next(self) -> 'L': ...
+
+ def last(seq: Linked[T]) -> T:
+ ...
+
+ result = last(L()) # The inferred type of 'result' is 'int'
+
 .. _declaring-decorators:
 
 Declaring decorators