Running from within python? (no CLI)

[tjohnston-cd]

Hi! I'm writing a python-based application that wants to use pyinfra under the hood.

I would like to call pyinfra as a library, so everything is self-contained in the python side of my application.

Is there a recommended way to replace the pyinfra inventory.py deploy.py CLI step with something in my python code? Or do I need to subprocess out to the CLI in order to run things?

[ccrvlh]

Hi @tjohnston-cd,

There's an API that you can use to use Pyinfra programmatically, but it's not considered stable, and not in the official docs at the moment (you can find an example on older docs versions here).

As I understand @Fizzadar is assessing whether this will be an official implementation (stable and supported) or an (non-supported) internal API. For the first case, there's still some work to be done, for the latter there's the eventual internal changes that may affect it.

Regardless of how things evolve, I risk to say that you could easily do things programmatically if you are willing to tinker a bit with the specific formats, shouldn't be much of an issue. If the decision is to support Python API, I just reinforce I would be happy to contribute to a stable API.

[tjohnston-cd]

Thanks for the info!

Generally speaking, it would make sense for the internal API to be "supported" in that it can be used as an alternative to the CLI (and vice versa):

• Easier maintenance of both over the long term (cli isn't strongly coupled to the internals, it isn't trying to "be the internals")
  • cli implementation becomes a typical example of how to use the internal API
• Means users don't have to add complexity to their project (in the form of e.g. subprocess management) to do things this way

Take that with a grain of salt, of course. I'm not familiar enough yet to know why it is this way..
I believe ansible has roughly the same situation? Could be wrong..

[ccrvlh]

Yeah, I would say the organization is just a result of the library goals from the start (CLI-first in this case). Ansible is pretty much exactly the same, you can use the internal objects (example here), but they made the decision to make it internal (and won't make it external anytime soon), and the internal API has changed quite a bit in Ansible. You can also take a look at Suitable which is sort of a wrapper to Ansible, really cool project as well.

Salt is a bit different in the sense that it was made to be more generic from the start, so they have renderers and a lot of flexibility to interact with hosts, including a REST API. You might take a look at that as well.

At last, I do agree with you, I think it would be positive to have a stable API, and having the CLI just as an entrypoint. But this is not always trivial, specially when considering backwards compatibility.

[v3ss0n]

This is important for building a system that is reproducable.
I would like to compile into binary using nuitka so users can just run and everything setup.
That way , beign standalone is essential. THis seems to be working , going to test . https://docs.pyinfra.com/en/2.x/api/index.html

[v3ss0n]

@Fizzadar can we consider this one stable , and link it in official doc?

[Fizzadar]

I've been meaning to add it (back) to the docs! Not committing to stable yet, but at some point ideally.

[maxervo]

+1 on this, would be super super useful for projects at work as well as personal projects 🙂

Had been looking for a code-first Ansible alternative for a while (e.g. that can be configured and called programmatically all from within Python), so when I stumbled upon Pyinfra v0/1 a long time ago I had assumed Pyinfra's philosophy followed this idea and that is was seen as a core differentiator or maybe not?

I was a bit confused going through the documentation in v2 today because I could not find any mention of a Python API and all the examples were using the CLI only. Adding it back to the docs would be great (with a comment about the API being unstable but that a stable API is planned)

[v3ss0n]

It seems maintainer are quite busy with other things but this works. https://docs.pyinfra.com/en/2.x/api/index.html
Not sure it willl break in future tho.

[v3ss0n]

also , i had packaged python+pyinfra into Appimage , i am not sure it is going to be part of pyinfra since @Fizzadar seems to have no interest and no comment on it at all, but its is working fine in my production setup.

#957

All you need to do is : write your deploy script , and package it using that AppImage builder and then ./your-pyinfra-binary eg:
pyinfra-yourporject-deploy . you can send this to your client and let them run it. Works great for me and my clients.