Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support in Windows Terminal for Kitty's Graphics Protocol #8389

Closed
WSLUser opened this issue Nov 24, 2020 · 10 comments
Closed

Add support in Windows Terminal for Kitty's Graphics Protocol #8389

WSLUser opened this issue Nov 24, 2020 · 10 comments
Labels
Issue-Feature Complex enough to require an in depth planning process and actual budgeted, scheduled work. Needs-Tag-Fix Doesn't match tag requirements Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting Resolution-Won't-Fix We're just really obstinate about this. There's probably a good reason.

Comments

@WSLUser
Copy link
Contributor

WSLUser commented Nov 24, 2020

Description of the new feature/enhancement

Support Kitty's graphics rendering protocol. See https://sw.kovidgoyal.net/kitty/graphics-protocol.html. This would provide even more support than Sixel (but would be an additive sequence, not replacement.)

Proposed technical implementation details (optional)

kovidgoyal/kitty#33 provides some discussion and an implementation spec.

@WSLUser WSLUser added the Issue-Feature Complex enough to require an in depth planning process and actual budgeted, scheduled work. label Nov 24, 2020
@ghost ghost added Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting Needs-Tag-Fix Doesn't match tag requirements labels Nov 24, 2020
@DHowett
Copy link
Member

DHowett commented Nov 24, 2020

Thanks for the request! I'd rather we not plan support for a third graphics protocol when we don't even support one.

@DHowett DHowett closed this as completed Nov 24, 2020
@DHowett DHowett added the Resolution-Won't-Fix We're just really obstinate about this. There's probably a good reason. label Nov 24, 2020
@WSLUser
Copy link
Contributor Author

WSLUser commented Nov 24, 2020

Could it be a possible extension idea then? Though based on what I saw from the extension PR, not sure if this is something that could be extended under current spec.

@WSLUser WSLUser changed the title Add support in Windows Terminal for Kitty's NCBLIT_PIXEL Add support in Windows Terminal for Kitty's Graphics Protocol Jul 9, 2021
@WSLUser
Copy link
Contributor Author

WSLUser commented Jul 9, 2021

@j4james Since Sixel is looking like it's not currently feasible, would Kitty's protocol be more easily adopted/implemented?

@zadjii-msft
Copy link
Member

@WSLUser What makes you believe that sixel isn't feasible?

@WSLUser
Copy link
Contributor Author

WSLUser commented Jul 9, 2021

#448 (comment) and also seems to be more to it here: hackerb9/lsix#41

@zadjii-msft
Copy link
Member

That doesn't sound like "not feasible" to me. That sounds like "more complicated to be fully compatible than you might think". I'm pretty sure the entire saga of PRs (e.g. #10011) that @j4james has been working on has been a long steady march towards sixel support (he can correct me if I'm wrong).

@WSLUser
Copy link
Contributor Author

WSLUser commented Jul 9, 2021

Yeah I said currently. I'm hoping for us to have at least something to use and seems like getting Sixel support will be a long way aways as we're waiting for these bugs in other terminals and Sixel applications to be resolved. Since he made it clear he's not really interested in trying to support it further at this time, I thought/hoping he may be interested in trying to support this other protocol instead, which technically is superior to Sixel anyways.

@j4james
Copy link
Collaborator

j4james commented Jul 10, 2021

There are two issues involved here:

  1. Propagating images over conpty. The conclusion I reached was that the most practical long-term solution would be a pure pass-through implementation of conpty. And this would likely be necessary for any image protocol. I'm not saying it's impossible to achieve with the current conpty implementation, but that would require buffering the images on the conhost side, and then regenerating partial image slices on the fly as the screen was updated. It's complicated, inefficient, and would probably depend on proprietary VT extensions.

    Another approach would be to pass just the image protocol through directly, and hope it doesn't get erased by a subsequent refresh of the text buffer. This might work some of the time, e.g. if you're just outputting an image from the command line, but any reasonably complex application involving a mix of text and images is almost certain to break at some point.

  2. The sixel image protocol itself. Personally I would like our sixel implementation to be a true emulation of the original hardware terminals, just with simple backward-compatible extensions (like support for a larger palette size, and optional per-image palettes). Most open source terminals, however, implement a format that (as Douglas Adams would describe it) is almost, but not quite, entirely unlike sixel. And the problem is that at least some modern sixel applications rely on the behaviour of these pseudo-sixel terminals.

    I was hoping there might be a way we could implement the protocol that was both compatible with the original hardware, and also capable of supporting modern sixel applications, but that's not looking promising at the moment. And while I'd personally be quite happy with just a standard version of sixel, I don't think the rest of the WT userbase would necessarily agree, and our users tend to be abusive when they don't get what they want, so I'd rather avoid that fight.

So right now I'm working on point 1, since that I think is a necessity for any image protocol. On point 2, I've put the coding side of things on hold for now, but I'm still working in the background to try and nudge other terminals an apps in the right direction. And even if I don't get my way, there's no reason someone else with different priorities couldn't put together a sixel implementation that is good enough for modern usage.

As for the Kitty Graphics Protocol, I have zero interest in implementing that myself, but if someone did want to take that on, we still need to resolve point 1 first, so it's in no way a quick fix.

@smprather
Copy link

@j4james
Copy link
Collaborator

j4james commented Dec 22, 2024

@smprather FYI, gnuplot also supports Sixel, so you can already use it with the preview version of Windows Terminal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Issue-Feature Complex enough to require an in depth planning process and actual budgeted, scheduled work. Needs-Tag-Fix Doesn't match tag requirements Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting Resolution-Won't-Fix We're just really obstinate about this. There's probably a good reason.
Projects
None yet
Development

No branches or pull requests

5 participants