Crate pyo3_stub_gen

This crate creates stub files in following three steps using inventory crate:

Define type information in Rust code (or by proc-macro)

The first step is to define Python type information in Rust code. type_info module provides several structs, for example:

• type_info::PyFunctionInfo stores information of Python function, i.e. the name of the function, arguments and its types, return type, etc.
• type_info::PyClassInfo stores information for Python class definition, i.e. the name of the class, members and its types, methods, etc.

For better understanding of what happens in the background, let’s define these information manually:

use pyo3::*;
use pyo3_stub_gen::type_info::*;

// Usual PyO3 class definition
#[pyclass(module = "my_module", name = "MyClass")]
struct MyClass {
    #[pyo3(get)]
    name: String,
    #[pyo3(get)]
    description: Option<String>,
}

// Submit type information for stub file generation to inventory
inventory::submit!{
    // Send information about Python class
    PyClassInfo {
        // Type ID of Rust struct (used to gathering phase discussed later)
        struct_id: std::any::TypeId::of::<MyClass>,

        // Python module name. Since stub file is generated per modules,
        // this helps where the class definition should be placed.
        module: Some("my_module"),

        // Python class name
        pyclass_name: "MyClass",

        members: &[
            MemberInfo {
                name: "name",
                r#type: <String as ::pyo3_stub_gen::PyStubType>::type_output,
            },
            MemberInfo {
                name: "description",
                r#type: <Option<String> as ::pyo3_stub_gen::PyStubType>::type_output,
            },
        ],
        doc: "Docstring used in Python",
    }
}

Roughly speaking, the above corresponds a following stub file my_module.pyi:

class MyClass:
    """
    Docstring used in Python
    """
    name: str
    description: Optional[str]

We want to generate this type_info::PyClassInfo section automatically from MyClass Rust struct definition. This is done by using #[gen_stub_pyclass] proc-macro:

use pyo3::*;
use pyo3_stub_gen::{type_info::*, derive::gen_stub_pyclass};

// Usual PyO3 class definition
#[gen_stub_pyclass]
#[pyclass(module = "my_module", name = "MyClass")]
struct MyClass {
    #[pyo3(get)]
    name: String,
    #[pyo3(get)]
    description: Option<String>,
}

Since proc-macro is a converter from Rust code to Rust code, the output must be a Rust code. However, we need to gather these type_info::PyClassInfo definitions to generate stub files, and the above inventory::submit is for it.

Gather type information into StubInfo

inventory crate provides a mechanism to gather inventory::submitted information when the library is loaded. To access these information through inventory::iter, we need to define a gather function in the crate. Typically, this is done by following:

use pyo3_stub_gen::{StubInfo, Result};

pub fn stub_info() -> Result<StubInfo> {
    let manifest_dir: &::std::path::Path = env!("CARGO_MANIFEST_DIR").as_ref();
    StubInfo::from_pyproject_toml(manifest_dir.join("pyproject.toml"))
}

There is a helper macro to define it easily:

pyo3_stub_gen::define_stub_info_gatherer!(sub_info);

Generate stub file from StubInfo

StubInfo translates type_info::PyClassInfo and other information into a form helpful for generating stub files while gathering.

generate module provides structs implementing std::fmt::Display to generate corresponding parts of stub file. For example, generate::MethodDef generates Python class method definition as follows:

use pyo3_stub_gen::{TypeInfo, generate::*};

let method = MethodDef {
    name: "foo",
    args: vec![Arg { name: "x", r#type: TypeInfo::builtin("int") }],
    signature: None,
    r#return: TypeInfo::builtin("int"),
    doc: "This is a foo method.",
    is_static: false,
    is_class: false,
};

assert_eq!(
    method.to_string().trim(),
    r#"
    def foo(self, x:int) -> int:
        r"""
        This is a foo method.
        """
        ...
    "#.trim()
);

generate::ClassDef generates Python class definition using generate::MethodDef and others, and other *Def structs works as well.

generate::Module consists of *Def structs and yields an entire stub file *.pyi for a single Python (sub-)module, i.e. a shared library build by PyO3. generate::Modules are created as a part of StubInfo, which merges type_info::PyClassInfos and others submitted to inventory separately. StubInfo is instantiated with pyproject::PyProject to get where to generate the stub file, and StubInfo::generate generates the stub files for every modules.

Re-exports

• pub use generate::StubInfo;
• pub use inventory;
• pub use pyo3_stub_gen_derive as derive;

Modules

• exception
• generate
  Generate Python typing stub file a.k.a. *.pyi file.
• pyproject
  pyproject.toml parser for reading [tool.maturin] configuration.
• stub_type 🔒
• type_info
  Store of metadata for generating Python stub file

Macros

• create_exception
• define_stub_info_gatherer
  Create a function to initialize StubInfo from pyproject.toml in CARGO_MANIFEST_DIR.
• impl_stub_type
  Implement PyStubType
• module_variable

Structs

• TypeInfo
  Type information for creating Python stub files annotated by PyStubType trait.

Traits

• PyStubType
  Annotate Rust types with Python type information.

Type Aliases

• Result