pyo3_stub_gen

Module readme

Source
Expand description

§pyo3-stub-gen

DeepWiki

Python stub file (*.pyi) generator for PyO3 with maturin projects.

crate namecrates.iodocs.rsdoc (main)
pyo3-stub-gencratedocs.rsdoc (main)
pyo3-stub-gen-derivecratedocs.rsdoc (main)

§Design

Our goal is to create a stub file *.pyi from Rust code, however, automated complete translation is impossible due to the difference between Rust and Python type systems and the limitation of proc-macro. We take semi-automated approach:

  • Provide a default translator which will work most cases, not all cases
  • Also provide a manual way to specify the translation.

If the default translator does not work, users can specify the translation manually, and these manual translations can be integrated with what the default translator generates. So the users can use the default translator as much as possible and only specify the translation for the edge cases.

pyo3-stub-gen crate provides the manual way to specify the translation, and pyo3-stub-gen-derive crate provides the default translator as proc-macro based on the mechanism of pyo3-stub-gen.

§Usage

If you are looking for a working example, please see the examples directory.

Here we describe basic usage of pyo3-stub-gen crate based on examples/pure example.

§Annotate Rust code with proc-macro

This crate provides a procedural macro #[gen_stub_pyfunction] and others to generate a Python stub file. It is used with PyO3’s #[pyfunction] macro. Let’s consider a simple example PyO3 project:

use pyo3::prelude::*;

#[pyfunction]
fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
    Ok((a + b).to_string())
}

#[pymodule]
fn your_module_name(m: &Bound<PyModule>) -> PyResult<()> {
    m.add_function(wrap_pyfunction!(sum_as_string, m)?)?;
    Ok(())
}

To generate a stub file for this project, please modify it as follows:

use pyo3::prelude::*;
use pyo3_stub_gen::{derive::gen_stub_pyfunction, define_stub_info_gatherer};

#[gen_stub_pyfunction]  // Proc-macro attribute to register a function to stub file generator.
#[pyfunction]
fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
    Ok((a + b).to_string())
}

#[pymodule]
fn your_module_name(m: &Bound<PyModule>) -> PyResult<()> {
    m.add_function(wrap_pyfunction!(sum_as_string, m)?)?;
    Ok(())
}

// Define a function to gather stub information.
define_stub_info_gatherer!(stub_info);

[!NOTE] The #[gen_stub_pyfunction] macro must be placed before #[pyfunction] macro.

§Generate a stub file

And then, create an executable target in src/bin/stub_gen.rs to generate a stub file:

use pyo3_stub_gen::Result;

fn main() -> Result<()> {
    // `stub_info` is a function defined by `define_stub_info_gatherer!` macro.
    let stub = pure::stub_info()?;
    stub.generate()?;
    Ok(())
}

and add rlib in addition to cdylib in [lib] section of Cargo.toml:

[lib]
crate-type = ["cdylib", "rlib"]

This target generates a stub file pure.pyi when executed.

cargo run --bin stub_gen

The stub file is automatically found by maturin, and it is included in the wheel package. See also the maturin document for more details.

§Advanced: #[gen_stub(xxx)] Attributes

§#[gen_stub(default=xx)]

For getters, setters, and classattr functions, you can specify the default value of it. e.g.

use pyo3::prelude::*;
use pyo3_stub_gen::derive::*;

#[gen_stub_pyclass]
#[pyclass]
struct A {
    #[pyo3(get,set)]
    #[gen_stub(default = A::default().x)]
    x: usize,
    y: usize,
}

impl Default for A {
    fn default() -> Self {
        A { x: 0, y: 0 }
    }
}

#[gen_stub_pymethods]
#[pymethods]
impl A {
    #[gen_stub(default = A::default().y)]
    fn get_y(&self) -> usize {
        self.y
    }
}

§#[gen_stub(skip)]

For classattrs or functions in pymethods, ignore it in .pyi file. e.g.

use pyo3::prelude::*;
use pyo3_stub_gen::derive::*;

#[gen_stub_pyclass]
#[pyclass]
struct A;

#[gen_stub_pymethods]
#[pymethods]
impl A {
    #[gen_stub(skip)]
    fn need_skip(&self) {}
}

§Contribution

To be written.

§License

© 2024 Jij Inc.

This project is licensed under either of

at your option.

  • MusicalNinjas/pyo3-stubgen
    • Same motivation, but different approach.
    • This project creates a stub file by loading the compiled library and inspecting the __text_signature__ attribute generated by PyO3 in Python side.
  • pybind11-stubgen
    • Stub file generator for pybind11 based C++ projects.